Data balancing method and device for Ceph cluster, equipment and medium

Abstract

The embodiment of the invention provides a data balancing method, device and equipment of a Ceph cluster and a medium, a plurality of placement groups are deployed in the Ceph cluster, each placement group comprises a master copy and a slave copy, the master copy and the slave copy of each placement group are placed on different storage devices in the Ceph cluster, and the method comprises the steps that data distribution information of the Ceph cluster is acquired; according to the data distribution information, performing a first balance operation on a primary copy of a placement group borne on each storage device in the Ceph cluster; and according to the data distribution information, carrying out second balance operation on the slave copies of the placement groups borne on the storage devices in the Ceph cluster. According to the embodiment of the invention, the code of the Ceph does not need to be modified, the number of the slave copies is balanced after the number of the master copies is balanced according to the data distribution information of the Ceph cluster, and the overall performance and the resource utilization rate of the cluster are improved and the risk caused by upgrading of the old version Ceph cluster is also avoided by ensuring the double-sided balance of the placement group in the Ceph cluster.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data balancing, and in particular to a data balancing method of a Ceph cluster and a data balancing device of a Ceph cluster. BACKGROUND

[0002] As a huge distributed system, data balancing is extremely important for Ceph. In order to ensure the stability of system performance, the amount of data carried on each storage device (OSD) should be as same as possible. However, the CRUSH algorithm used by Ceph completely relies on the randomness of Hash, which may cause the capacity usage of OSDs in the cluster to be obviously unbalanced, resulting in space waste. SUMMARY

[0003] In view of the above problems, the embodiments of the present application are proposed in order to provide a data balancing method, device, equipment and medium of a Ceph cluster which overcomes the above problems or at least partially solves the above problems.

[0004] According to a first aspect of the embodiments of the present application, a data balancing method of a Ceph cluster is provided, wherein a plurality of placement groups are deployed in the Ceph cluster, the placement groups include primary replicas and secondary replicas, the primary replicas and the secondary replicas of each placement group are placed on different storage devices in the Ceph cluster, and the method comprises: obtaining data distribution information of the Ceph cluster; performing a first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the first balancing operation is used to balance the number of the primary replicas of the placement groups on the each storage device; performing a second balancing operation on the secondary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the second balancing operation is used to balance the number of the secondary replicas of the placement groups on the each storage device.

[0005] Optionally, the first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information comprises: judging whether there is a placement group in an abnormal working state in the Ceph cluster; in the case where there is no placement group in the abnormal working state, performing the first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information.

[0006] Optionally, the first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information comprises: determining a primary replica distribution score of the placement groups on the storage devices in the Ceph cluster according to the data distribution information; the primary replica distribution score is inversely proportional to a variance of the number of primary replicas of the placement groups carried on the respective storage devices; when the primary replica distribution score is lower than a preset primary replica distribution score threshold, performing a first balancing operation on the primary replicas of the placement groups carried on the respective storage devices in the Ceph cluster.

[0007] Optionally, the first balancing operation on the primary replicas of the placement groups carried on the respective storage devices in the Ceph cluster comprises: determining a primary replica average value on the storage devices; traversing the respective placement groups in the Ceph cluster; if the number of primary replicas on the storage device where the primary replica of the current placement group is located is greater than the primary replica average value, migrating the primary replica of the current placement group to a storage device with the least number of primary replicas and not carrying a replica of the current placement group; calculating a variance of the number of primary replicas of the placement groups carried on the respective storage devices after the migration, and stopping the migration operation on the primary replicas in the case that the variance is zero.

[0008] Optionally, the second balancing operation on the replicas of the placement groups carried on the respective storage devices in the Ceph cluster according to the data distribution information comprises: judging whether the first balancing operation is completed; in the case that the first balancing operation is completed, performing a second balancing operation on the replicas of the placement groups carried on the respective storage devices in the Ceph cluster according to the data distribution information.

[0009] Optionally, the second balancing operation on the replicas of the placement groups carried on the respective storage devices in the Ceph cluster according to the data distribution information comprises: determining a replica distribution score of the placement groups on the storage devices in the Ceph cluster according to the data distribution information; the replica distribution score is inversely proportional to a variance of the number of replicas of the placement groups carried on the respective storage devices; when the replica distribution score is lower than a preset primary replica distribution score threshold, performing a second balancing operation on the replicas of the placement groups carried on the respective storage devices in the Ceph cluster.

[0010] Optionally, the second balancing operation on the replicas of the placement groups carried on the respective storage devices in the Ceph cluster comprises: determining a replica average value on the storage devices; traversing each placement group in the Ceph cluster; if the number of the slave copies on the storage device where the slave copy of the current placement group is located is greater than the average number of the slave copies, migrating the slave copy of the current placement group to the storage device with the least number of the slave copies and not carrying the slave copy of the current placement group; calculating the variance of the number of the slave copies of the placement groups carried by each storage device after the migration, and stopping the migration operation of the slave copies in the case that the variance is zero.

[0011] According to a second aspect of the embodiments of the present application, a data balancing device of a Ceph cluster is provided, the Ceph cluster is deployed with a plurality of placement groups, the placement groups include a master copy and a slave copy, the master copy and the slave copy of each placement group are placed on different storage devices in the Ceph cluster, and the device includes: a data distribution information acquisition module, configured to acquire data distribution information of the Ceph cluster; a first balancing module, configured to perform a first balancing operation on the master copies of the placement groups carried by each storage device in the Ceph cluster according to the data distribution information; the first balancing operation is used to balance the number of the master copies of the placement groups on the storage devices; a second balancing module, configured to perform a second balancing operation on the slave copies of the placement groups carried by each storage device in the Ceph cluster according to the data distribution information; the second balancing operation is used to balance the number of the slave copies of the placement groups on the storage devices.

[0012] Optionally, the first balancing module includes: a state judgment sub-module, configured to judge whether there is a placement group in an abnormal working state in the Ceph cluster; a first balancing sub-module, configured to perform a first balancing operation on the master copies of the placement groups carried by each storage device in the Ceph cluster according to the data distribution information in the case that there is no placement group in the abnormal working state.

[0013] Optionally, the first balancing sub-module includes: a first score determination unit, configured to determine a master copy distribution score of the placement groups on the storage devices in the Ceph cluster according to the data distribution information; the master copy distribution score is inversely proportional to the variance of the number of the master copies of the placement groups carried by the storage devices; a first balancing unit, configured to perform a first balancing operation on the master copies of the placement groups carried by each storage device in the Ceph cluster when the master copy distribution score is lower than a preset master copy distribution score threshold.

[0014] Optionally, the first balancing unit includes: The primary replica average value determination subunit is used to determine the primary replica average value on the storage device; The first traversal subunit is used to traverse each placement group in the Ceph cluster; The primary replica migration subunit is used to migrate the primary replica of the current placement group to the storage device with the fewest primary replicas and which does not carry the secondary replicas of the current placement group if the number of primary replicas on the storage device where the primary replica of the current placement group is located is greater than the average number of primary replicas. The first variance calculation subunit is used to calculate the variance of the number of primary replicas of the placement group carried by each storage device after migration, and to stop the migration operation of the primary replicas when the variance is zero.

[0015] Optionally, the second balancing module includes: The balance judgment submodule is used to determine whether the first balance operation has been completed; The second balancing submodule is used to perform a second balancing operation on the slave replicas of the placement group carried on each storage device in the Ceph cluster according to the data distribution information after the first balancing operation is completed.

[0016] Optionally, the second balancing submodule includes: The second scoring determination unit is used to determine the slave replica distribution score of the placement group on the storage device in the Ceph cluster based on the data distribution information; the slave replica distribution score is inversely proportional to the variance of the number of slave replicas of the placement group carried on each storage device; The second balancing unit is used to perform a second balancing operation on the slave replicas of the placement group carried on each storage device in the Ceph cluster when the slave replica distribution score is lower than the preset master replica distribution score threshold.

[0017] Optionally, the second balancing unit includes: Sub-units are determined from the replica average value for determining the replica average value on the storage device; The second traversal subunit is used to traverse each placement group in the Ceph cluster; The replica migration subunit is used to migrate the replicas of the current placement group to the storage device with the fewest replicas and which does not carry the replicas of the current placement group if the number of replicas on the storage device where the replicas of the current placement group are located is greater than the average number of replicas. The second variance calculation subunit is used to calculate the variance of the number of slave replicas of the placement group carried by each storage device after migration, and to stop the migration operation of the slave replicas when the variance is zero.

[0018] According to a third aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the data balancing method for a Ceph cluster as described in any of the preceding claims.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the data balancing method for a Ceph cluster as described in any of the preceding claims.

[0020] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention provides a data balancing method, apparatus, device, and medium for a Ceph cluster. The Ceph cluster deploys multiple placement groups, each including primary and secondary replicas. The primary and secondary replicas of each placement group are placed on different storage devices within the Ceph cluster. The method involves acquiring data distribution information of the Ceph cluster; performing a first balancing operation on the primary replicas of the placement groups on each storage device in the Ceph cluster based on the data distribution information; and performing a second balancing operation on the secondary replicas of the placement groups on each storage device in the Ceph cluster based on the data distribution information. This invention does not require modification of the Ceph code. By balancing the number of primary replicas based on the data distribution information of the Ceph cluster, and then balancing the number of secondary replicas, it ensures a two-sided balance for the placement groups in the Ceph cluster, improving the overall performance and resource utilization of the cluster, and avoiding the risks associated with upgrading older versions of Ceph clusters. Attached Figure Description

[0021] Figure 1 This is a flowchart of the steps of a data balancing method for a Ceph cluster provided in an embodiment of the present invention; Figure 2 This is a flowchart of another data balancing method for Ceph clusters provided in an embodiment of the present invention; Figure 3 This is a flowchart of another data balancing method for Ceph clusters provided in an embodiment of the present invention; Figure 4 This is a flowchart of another data balancing method for Ceph clusters provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of a data balancing device for a Ceph cluster provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] One of the core concepts of this invention is that, without modifying the Ceph code, the number of slave replicas is balanced after balancing the number of primary replicas based on the data distribution information of the Ceph cluster. By ensuring the two-sided balance of placement groups in the Ceph cluster, the overall performance and resource utilization of the cluster are improved, and the risks brought about by upgrading old versions of Ceph clusters are avoided.

[0024] Reference Figure 1 The diagram illustrates a flowchart of a data balancing method for a Ceph cluster provided by an embodiment of the present invention. The method may specifically include the following steps: Step 101: Obtain the data distribution information of the Ceph cluster; For example, Ceph distributed storage system is an open-source, unified software-defined storage platform that can simultaneously provide block storage, file storage, and object storage services. It achieves high availability, high reliability, and horizontal scaling by partitioning data into objects and organizing them into logical placement groups (OSDs). These OSDs are then intelligently distributed across multiple storage nodes in the cluster using the CRUSH (Controlled Replication Under Scalable Hashing) algorithm. A Ceph cluster consists of multiple storage devices (OSDs, object storage devices) used to actually store data. The data in the cluster is organized as objects and logically divided into multiple placement groups (PGs). Each placement group contains one primary replica and several secondary replicas. These replicas are distributed across different storage devices according to a high availability strategy, and replicas within the same placement group must reside in different fault domains to ensure data security and service continuity. Because the CRUSH algorithm relies on hash randomness, in actual operation, it can easily lead to uneven distribution of the number of PGs or primary replicas across OSDs, resulting in wasted storage space or read / write load imbalance. Therefore, before performing any balancing operation, it is essential to first obtain the current data distribution information of the cluster, including key indicators such as the total number of PGs on each OSD, the number of primary replicas, PG status, and fault domain topology. This information is the foundation for assessing the cluster's balance, identifying hot OSDs, and determining whether health conditions (such as all PGs being active and having consistent data) are met. Only based on accurate and real-time distribution data can we make scientific decisions about whether balancing is necessary, whether to prioritize balancing primary replicas or secondary replicas, and generate effective and safe mapping adjustment and migration instructions, avoiding blind operations that could cause data oscillations or high availability risks.

[0025] Step 102: Perform a first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the first balancing operation is used to balance the number of primary replicas of the placement groups on each storage device. For example, Ceph read operations (especially strongly consistent reads) must be handled by the primary replicas of a Group Spectrum (PG). Therefore, the distribution of primary replicas directly determines the read load pressure on each OSD. If some OSDs have too many primary replicas, even if their total number of PGs is comparable to other OSDs, they will still become read performance bottlenecks, leading to increased overall cluster response latency and uneven resource utilization. To address this, this method first independently balances the number of primary replicas based on data distribution information: by calculating the variance of the number of primary replicas for each OSD, the degree of imbalance is assessed, and primary replicas are preferentially migrated from overloaded OSDs to target OSDs with the fewest primary replicas and no fault domain conflicts. Primary replica balancing is performed before total PG balancing, which avoids misjudging the cluster state when the total PGs appear balanced but the primary replicas are severely skewed. This truly achieves fine-grained scheduling for read load, significantly improving system performance stability and user experience.

[0026] Step 103: Perform a second balancing operation on the slave replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the second balancing operation is used to balance the number of slave replicas of the placement groups on each storage device.

[0027] For example, after balancing the number of primary replicas, a secondary balancing of secondary replicas (i.e., non-primary replicas of PGs) is required to achieve a more even distribution of overall storage capacity and write load. Although secondary replicas do not directly handle client read requests, they participate in critical processes such as data synchronization, fault recovery, and data reconstruction during write operations. Uneven distribution of these secondary replicas can lead to disk space exhaustion, increased write amplification, or overloaded recovery bandwidth for some OSDs, thereby affecting cluster reliability and scalability. The second balancing operation analyzes the total number of PGs (including primary and secondary replicas) carried by each OSD based on the updated data distribution information, migrating secondary replicas on overloaded OSDs to OSDs with the fewest total number of PGs and meeting fault domain isolation requirements. This process also aims to minimize variance, ensuring a more even distribution of overall data after migration. Through this two-stage strategy of prioritizing primary replicas over secondary replicas, read performance is guaranteed while also ensuring fair use of storage resources and long-term system stability. This avoids the "pseudo-balancing" problem caused by balancing only the total number of PGs while ignoring primary replica skew, comprehensively improving the overall service capabilities of the Ceph cluster.

[0028] For example, the above steps do not require modification of Ceph's source code. Instead, through an external scheduling mechanism, without altering its internal data structure and core logic, Ceph's native upmap mapping adjustment function dynamically generates and issues balancing instructions, achieving a dual balance between the number of primary replicas in the placement group and the total number of replicas. This "plug-and-play" approach effectively alleviates read load hotspots caused by uneven distribution of primary replicas while also ensuring overall storage capacity balance, thus significantly improving cluster performance and resource utilization. Furthermore, because it is fully compatible with existing Ceph interfaces and protocols, it avoids the compatibility risks, data migration costs, and system stability issues associated with upgrading to new versions (such as the R version which introduces a read load balancing module). This makes it particularly suitable for large-scale production environments with high requirements for stability and maintainability.

[0029] Reference Figure 2 This document illustrates a flowchart of another data balancing method for a Ceph cluster provided by an embodiment of the present invention. During the usage phase, data balancing can be performed according to a preset time period. First, the current status information of the Ceph cluster is obtained to determine whether all PGs are in normal working condition. If not, the cluster sleeps for t seconds and then re-obtains the information. If normal, the cluster further determines whether the number of primary replicas of PGs is balanced. If not, a PG primary replica balancing instruction is generated and executed. If balanced, the cluster determines whether the total number of replicas of PGs is balanced. If not, a PG balancing instruction is generated and executed, which is used to balance the secondary replicas of PGs. After execution, the cluster sleeps for t seconds, and the above process is repeated cyclically to achieve continuous, low-interference optimization of the cluster's read load.

[0030] Reference Figure 3 This document illustrates a flowchart of another data balancing method for a Ceph cluster provided by an embodiment of the present invention. During the deployment phase, the status information of the Ceph cluster is first obtained to determine whether the Group Provider (PG) is in a normal state. If not, the cluster sleeps for t seconds and then re-obtains the information. If the status is normal, a PG primary replica balancing instruction and a PG replica total balancing instruction are generated sequentially. Subsequently, all balancing instructions are sent to the Ceph cluster for execution, and the data recovery speed is increased during execution to accelerate the balancing process. After balancing is completed, the data recovery speed is restored to the default value, completing the data balancing operation for the entire deployment phase.

[0031] This invention provides a data balancing method for Ceph clusters. The Ceph cluster deploys multiple placement groups, each including primary and secondary replicas. The primary and secondary replicas of each placement group are placed on different storage devices within the Ceph cluster. The method involves acquiring data distribution information from the Ceph cluster; performing a first balancing operation on the primary replicas of the placement groups on each storage device based on this data distribution information; and performing a second balancing operation on the secondary replicas of the placement groups on each storage device based on this data distribution information. This invention, without modifying the Ceph code, balances the number of secondary replicas after balancing the number of primary replicas based on the data distribution information of the Ceph cluster. By ensuring a two-sided balance within the placement groups of the Ceph cluster, it improves the overall performance and resource utilization of the cluster and avoids the risks associated with upgrading older Ceph cluster versions.

[0032] Reference Figure 4 The diagram illustrates a flowchart of another data balancing method for a Ceph cluster provided by an embodiment of the present invention. The method may specifically include the following steps: Step 201: Obtain the data distribution information of the Ceph cluster; For example, Ceph distributed storage is an open-source, unified software-defined storage platform that provides block storage, file storage, and object storage services simultaneously. It achieves high availability, high reliability, and horizontal scalability by partitioning data into objects and organizing them into logical placement groups. Using the CRUSH (Controlled Replication Under Scalable Hashing) algorithm, these placement groups are intelligently distributed across multiple storage nodes in the cluster. Because the CRUSH algorithm relies on hash randomness, in practice, it can easily lead to uneven distribution of PGs or primary replicas across OSDs, resulting in wasted storage space or read / write load imbalance. Therefore, before performing any balancing operation, it is essential to obtain the current data distribution information of the cluster, including key indicators such as the total number of PGs on each OSD, the number of primary replicas, PG status, and fault domain topology. This information is fundamental for assessing the cluster's balance, identifying hot OSDs, and determining whether health conditions (such as all PGs being active and data consistent) are met. Only by basing decisions on accurate and real-time distribution data can we make scientific decisions on whether to balance, prioritize balancing primary replicas or secondary replicas, and generate effective and secure mapping adjustment and migration instructions to avoid data oscillations or high availability risks caused by blind operations.

[0033] Step 202: Determine whether there are any placement groups in the Ceph cluster that are not in a normal working state; For example, during Ceph cluster operation, if there are placement groups in an abnormal state (such as inactive, recovering, incomplete data, or unreachable), it indicates that the cluster is currently experiencing instability due to failure, scaling, disk damage, or network anomalies. Forcibly performing data balancing operations in this situation may interfere with ongoing data repair or migration processes, increase system load, and even lead to data loss or service interruption. Therefore, it is essential to first check whether all placement groups are active and have consistent data; balancing operations should only be allowed if the cluster as a whole is healthy and the data distribution is stable.

[0034] Step 203: In the absence of any abnormally functioning placement groups, perform a first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the first balancing operation is used to balance the number of primary replicas of the placement groups on each storage device. For example, Ceph read requests (especially strongly consistent reads) must be handled by the primary replicas of placement groups. Therefore, the distribution of primary replicas directly determines the read load pressure on each storage device. Even if the total number of placement groups on each device is similar, if primary replicas are concentrated on a few devices, these nodes will still become performance bottlenecks, causing increased read latency and uneven resource utilization. Therefore, after confirming cluster health, a primary replica balancing operation is performed first, aiming to migrate primary replicas from overloaded devices to less loaded devices, thereby distributing the read request pressure. This operation not only optimizes the load balancing of read paths but also improves overall throughput and response speed.

[0035] In one embodiment, step 203 includes the following sub-steps: Sub-step S11: Determine the primary replica distribution score of the placement group on the storage device in the Ceph cluster based on the data distribution information; the primary replica distribution score is inversely proportional to the variance of the number of primary replicas of the placement group carried on each storage device; For example, the primary replica distribution score is a quantitative indicator of the load balancing degree of the primary replicas in a cluster. By counting the number of primary replicas on each storage device, the variance of their distribution is calculated: the larger the variance, the more concentrated the primary replicas and the more unbalanced the load; the smaller the variance, the more even the distribution. Setting the score to be inversely proportional to the variance intuitively reflects the level of balance—the score increases when the variance decreases, and vice versa. This design allows the system to objectively determine whether intervention is needed based on a single numerical value, avoiding subjective thresholds or complex rules. Furthermore, using variance as a basis aligns with statistical principles, sensitively capturing local hotspots and providing a scientific basis for generating optimal migration instructions. Through this scoring mechanism, the system can adaptively evaluate the quality of primary replica distribution in clusters of different sizes and configurations, laying the foundation for automated and intelligent balancing decisions.

[0036] Sub-step S12: When the primary replica distribution score is lower than the preset primary replica distribution score threshold, a first balancing operation is performed on the primary replicas of the placement group carried on each storage device in the Ceph cluster.

[0037] For example, setting a primary / replica distribution score threshold is to avoid ineffective or over-balancing. If the primary / replica distribution is already sufficiently even (score above the threshold), performing a migration not only yields minimal benefits but also triggers unnecessary data movement, consuming network bandwidth, disk I / O, and CPU resources, thus impacting user service performance. Only when the score is below the threshold (i.e., excessive variance, severe primary / replica skew) does it indicate a significant read load imbalance, at which point balancing operations have a clear performance improvement value. This threshold mechanism enables on-demand balancing, minimizing operational overhead while ensuring effectiveness. Furthermore, combining sleep intervals and single-step execution strategies further ensures a smooth and low-interference balancing process. This conditionally triggered design balances efficiency and stability, enabling the system to respond promptly to load skewing without falling into frequent oscillations, thereby maintaining a balance between high performance and high availability during long-term operation.

[0038] In one embodiment, sub-step S12 includes the following sub-steps: Sub-step S121: Determine the average value of the primary replicas on the storage device; For example, the average number of primary replicas serves as a benchmark for measuring the balance of primary replica distribution across a cluster. By calculating the arithmetic mean of the number of primary replicas on all storage devices, it's possible to clearly identify which devices are carrying more than average primary replicas (i.e., potential hotspots) and which devices have lower loads (potential migration targets). This average provides an objective threshold for subsequent judgments on "whether migration is needed": only when the number of primary replicas on a device exceeds the average is it considered unbalanced and optimization operations are triggered. Without this benchmark, it's impossible to quantify "overload" or "idleness," leading to a lack of basis for migration decisions and potentially resulting in invalid or reverse migrations. Therefore, the average number of primary replicas is a crucial prerequisite for achieving accurate and efficient load balancing.

[0039] Sub-step S122: Traverse each placement group in the Ceph cluster; For example, since the primary replica location of each placement group independently determines the read load of its storage device, checking only some placement groups may lead to local optimization while the overall load remains unbalanced. By checking each placement group one by one, the system can dynamically assess whether the device where its primary replica resides is overloaded and select the optimal migration scheme based on the global state. Although this full traversal strategy has a slightly higher computational overhead, it is perfectly acceptable during the deployment phase or low-frequency usage phase, and it can ensure the integrity and fairness of the balancing operation, avoiding new load skew caused by sampling bias.

[0040] Sub-step S123: If the number of primary replicas on the storage device where the primary replica of the current placement group is located is greater than the average number of primary replicas, migrate the primary replica of the current placement group to the storage device with the fewest primary replicas and which does not carry the secondary replica of the current placement group. For example, this migration strategy balances the dual goals of load balancing and high availability. First, migrating primary replicas from overloaded devices to those with the fewest primary replicas directly reduces variance and quickly improves overall load balancing. Second, requiring the target device not to host any secondary replicas in the placement group strictly adheres to Ceph's fault domain isolation principle—all replicas (including primary and secondary replicas) within the same placement group must be distributed across different storage devices and even different fault domains to prevent single points of failure from causing data unavailability. Allowing primary replicas to migrate to devices with existing secondary replicas would violate the replica isolation rule and weaken the system's fault tolerance. Therefore, this condition ensures that migration effectively alleviates hotspots while guaranteeing data redundancy, making it a core mechanism for the coordinated optimization of performance and reliability.

[0041] Sub-step S124: Calculate the variance of the number of primary replicas of the placement group carried by each storage device after migration, and stop the migration operation of the primary replicas if the variance is zero.

[0042] For example, variance is a mathematical metric that measures the uniformity of primary and replica distribution: the smaller the variance, the closer the load on each device; zero variance indicates that all devices have exactly the same number of primary and replicas, reaching the theoretically optimal equilibrium. Recalculating the variance after each migration verifies whether the operation truly improved the distribution and prevents ineffective or even destabilizing migrations from being adopted. When the variance drops to zero, it indicates that perfect balance has been achieved; continuing migration is not only unhelpful but also wastes system resources and increases data disturbance. Furthermore, migration can be stopped when the variance increases to avoid executing ineffective or harmful operations that further unbalance the load distribution, ensuring that each migration moves towards optimization.

[0043] Step 204: Determine whether the first balancing operation has been completed; For example, determining whether the first balancing operation is complete ensures that the distribution of primary replicas has reached a balanced state before initiating the balancing process for secondary replicas. Since the primary replicas directly handle read requests, their load balancing has a more critical impact on system performance and must be prioritized. If secondary replica adjustments are performed prematurely while the primary replicas are still unbalanced, it may cause further disturbances to the secondary replica distribution during subsequent primary replica migrations, resulting in duplicate data migrations, resource waste, or even new imbalances. By establishing clear completion criteria (such as primary replica distribution scores meeting standards or variance stability), a clear two-phase execution sequence can be established, ensuring an orderly and efficient optimization process and avoiding operational conflicts and system instability.

[0044] Step 205: After the first balancing operation is completed, a second balancing operation is performed on the slave replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the second balancing operation is used to balance the number of slave replicas of the placement groups on each storage device.

[0045] For example, after the primary replicas are evenly distributed, the cluster's read load has stabilized. At this point, balancing the secondary replicas allows for a focus on optimizing storage capacity utilization and write load distribution. While secondary replicas do not directly handle client read requests, they participate in critical processes such as write synchronization, fault recovery, and data reconstruction. Uneven distribution of these secondary replicas can lead to some storage devices running out of space or experiencing excessive I / O pressure. Placing secondary replica balancing after primary replica balancing avoids mutual interference and allows for more accurate migration strategies based on the updated stable state. This achieves a tiered optimization goal of "prioritizing performance, then optimizing resources," comprehensively improving the overall efficiency and reliability of the cluster.

[0046] In one embodiment, step 205 includes the following sub-steps: Sub-step S21: Determine the slave replica distribution score of the placement group on the storage device in the Ceph cluster based on the data distribution information; the slave replica distribution score is inversely proportional to the variance of the number of slave replicas of the placement group carried on each storage device; For example, a replica distribution score is used to quantify the balance of replica load across storage devices. By calculating the variance of the number of replicas, the degree of distribution dispersion can be objectively reflected: a larger variance indicates that some devices are carrying too many replicas, resulting in capacity or write load hotspots; a smaller variance indicates a more uniform distribution. Setting the score to be inversely proportional to the variance ensures that a higher score represents better balance, facilitating the setting of a uniform threshold for automated judgment. This mechanism enables the system to dynamically assess whether intervention is needed based on actual data, avoiding subjective experience-based decisions and ensuring that the second balancing operation is targeted, scientific, and effective.

[0047] Sub-step S22: When the slave replica distribution score is lower than the preset master replica distribution score threshold, a second balancing operation is performed on the slave replicas of the placement group carried on each storage device in the Ceph cluster.

[0048] For example, setting a scoring threshold is to prevent unnecessary migration operations. If the replica distribution is already sufficiently even (score above the threshold), performing balancing not only yields minimal benefits but also consumes network bandwidth, disk I / O, and CPU resources, impacting normal business operations. Only when the score is below the threshold (i.e., large variance and significantly uneven distribution) does it indicate a significant resource utilization imbalance, at which point migration has practical optimization value. This conditional triggering mechanism achieves "on-demand balancing," ensuring fair allocation of storage resources while minimizing system disturbances, ensuring that the second balancing operation is efficient, safe, and low-overhead.

[0049] In one embodiment, sub-step S22 includes the following sub-steps: Sub-step S221: Determine the average value of the replicas on the storage device; For example, determining the average number of slave replicas is to establish an objective baseline for balancing storage devices, identifying which devices are handling too many or too few slave replicas. This average reflects the number of slave replicas each device should handle under an ideal, uniform distribution, and is a key indicator of load skew. Only when a device's number of slave replicas is significantly higher than the average does it indicate potential storage space pressure or excessive write / recovery load, necessitating the migration of some of its slave replicas. Conversely, devices with a number below the average can be considered for migration. This baseline allows the system to accurately locate hotspot nodes, avoid blind migration, and ensure that the second balancing operation is targeted, efficient, and feasible.

[0050] Sub-step S222: Traverse each placement group in the Ceph cluster; For example, traversing all placement groups is to comprehensively assess the actual distribution of replicas, ensuring that no optimizable migration opportunities are missed. Each placement group contains multiple replicas located on different storage devices, and their distribution directly affects the overall resource utilization balance. Checking only some placement groups may lead to local adjustments while the global balance remains, or even introduce new skew. By traversing each group, the system can dynamically analyze the load of each replica's device and select the optimal migration path based on the global objective. Although this full-scan strategy incurs some computational overhead, it is performed under the premise that the primary replicas are balanced, ensuring both optimization completeness and system stability, and is a necessary step for achieving fine-grained resource scheduling.

[0051] Sub-step S223: If the number of slave replicas on the storage device where the slave replicas of the current placement group are located is greater than the average number of slave replicas, migrate the slave replicas of the current placement group to the storage device with the fewest slave replicas and which does not carry the slave replicas of the current placement group. For example, the replica distribution score is a quantitative indicator of the balance of cluster storage resources. By counting the number of replicas on each storage device and calculating its variance, the dispersion of the distribution can be objectively measured: the larger the variance, the more replicas are piled up on some devices, indicating capacity or IO bottlenecks; the smaller the variance, the more uniform the distribution. Setting the score to be inversely proportional to the variance means that a higher score represents better balance, facilitating the setting of a unified threshold for automated decision-making. This design not only conforms to statistical principles but also sensitively captures subtle load differences, providing a scientific basis for whether to initiate a second balancing mechanism, avoiding subjective judgment or hard-coded rules, and improving the system's adaptability and intelligence.

[0052] Sub-step S224: Calculate the variance of the number of slave replicas of the placement group carried by each storage device after migration, and stop the migration operation of the slave replicas if the variance is zero.

[0053] For example, zero variance means that all storage devices carry exactly the same number of replicas, achieving the theoretically optimal balance. Continuing migration at this point not only fails to further improve the distribution but also triggers unnecessary data movement, consuming network bandwidth, disk I / O, and system resources, and increasing the risk of data disturbance. By recalculating the variance after each migration, the effectiveness of the operation can be verified; once the variance drops to zero, the goal has been achieved, and subsequent migrations should be terminated immediately. Alternatively, migration can be stopped when the variance increases to avoid performing ineffective or harmful operations that further unbalance the load distribution, ensuring that each migration proceeds in an optimized direction.

[0054] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0055] This invention provides a data balancing method for Ceph clusters. The Ceph cluster deploys multiple placement groups, each including primary and secondary replicas. The primary and secondary replicas of each placement group are placed on different storage devices within the Ceph cluster. The method involves acquiring data distribution information from the Ceph cluster; performing a first balancing operation on the primary replicas of the placement groups on each storage device based on this data distribution information; and performing a second balancing operation on the secondary replicas of the placement groups on each storage device based on this data distribution information. This invention, without modifying the Ceph code, balances the number of secondary replicas after balancing the number of primary replicas based on the data distribution information of the Ceph cluster. By ensuring a two-sided balance within the placement groups of the Ceph cluster, it improves the overall performance and resource utilization of the cluster and avoids the risks associated with upgrading older Ceph cluster versions.

[0056] Reference Figure 3 The diagram illustrates a structural block diagram of a data balancing device for a Ceph cluster provided in an embodiment of the present invention, which may specifically include the following modules: The data distribution information acquisition module 301 is used to acquire the data distribution information of the Ceph cluster. The first balancing module 302 is used to perform a first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the first balancing operation is used to balance the number of primary replicas of the placement groups on each storage device. The second balancing module 303 is used to perform a second balancing operation on the slave replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the second balancing operation is used to balance the number of slave replicas of the placement groups on each storage device.

[0057] In one embodiment, the first balancing module includes: The status judgment submodule is used to determine whether there are any abnormally functioning placement groups in the Ceph cluster; The first balancing submodule is used to perform a first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information, in the absence of placement groups in abnormal working state.

[0058] In one embodiment, the first balancing submodule includes: The first scoring unit is used to determine the primary replica distribution score of the placement group on the storage device in the Ceph cluster based on the data distribution information; the primary replica distribution score is inversely proportional to the variance of the number of primary replicas of the placement group carried on each storage device; The first balancing unit is used to perform a first balancing operation on the primary replicas of the placement group carried on each storage device in the Ceph cluster when the primary replica distribution score is lower than a preset primary replica distribution score threshold.

[0059] In one embodiment, the first balancing unit includes: The primary replica average value determination subunit is used to determine the primary replica average value on the storage device; The first traversal subunit is used to traverse each placement group in the Ceph cluster; The primary replica migration subunit is used to migrate the primary replica of the current placement group to the storage device with the fewest primary replicas and which does not carry the secondary replicas of the current placement group if the number of primary replicas on the storage device where the primary replica of the current placement group is located is greater than the average number of primary replicas. The first variance calculation subunit is used to calculate the variance of the number of primary replicas of the placement group carried by each storage device after migration, and to stop the migration operation of the primary replicas when the variance is zero.

[0060] In one embodiment, the second balancing module includes: The balance judgment submodule is used to determine whether the first balance operation has been completed; The second balancing submodule is used to perform a second balancing operation on the slave replicas of the placement group carried on each storage device in the Ceph cluster according to the data distribution information after the first balancing operation is completed.

[0061] In one embodiment, the second balancing submodule includes: The second scoring determination unit is used to determine the slave replica distribution score of the placement group on the storage device in the Ceph cluster based on the data distribution information; the slave replica distribution score is inversely proportional to the variance of the number of slave replicas of the placement group carried on each storage device; The second balancing unit is used to perform a second balancing operation on the slave replicas of the placement group carried on each storage device in the Ceph cluster when the slave replica distribution score is lower than the preset master replica distribution score threshold.

[0062] In one embodiment, the second balancing unit includes: Sub-units are determined from the replica average value for determining the replica average value on the storage device; The second traversal subunit is used to traverse each placement group in the Ceph cluster; The replica migration subunit is used to migrate the replicas of the current placement group to the storage device with the fewest replicas and which does not carry the replicas of the current placement group if the number of replicas on the storage device where the replicas of the current placement group are located is greater than the average number of replicas. The second variance calculation subunit is used to calculate the variance of the number of slave replicas of the placement group carried by each storage device after migration, and to stop the migration operation of the slave replicas when the variance is zero.

[0063] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0064] This invention provides a data balancing device for a Ceph cluster. The Ceph cluster deploys multiple placement groups, each including primary and secondary replicas. The primary and secondary replicas of each placement group are placed on different storage devices within the Ceph cluster. The device obtains data distribution information from the Ceph cluster and performs a first balancing operation on the primary replicas of the placement groups on each storage device based on this information. Then, it performs a second balancing operation on the secondary replicas of the placement groups on each storage device based on the same data distribution information. This invention, without modifying the Ceph code, balances the number of secondary replicas after balancing the number of primary replicas based on the data distribution information of the Ceph cluster. By ensuring a two-sided balance within the placement groups of the Ceph cluster, it improves the overall performance and resource utilization of the cluster and avoids the risks associated with upgrading older Ceph cluster versions.

[0065] This invention also provides an electronic device, comprising: It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When executed by the processor, the computer program implements the various processes of the above-described Ceph cluster data balancing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0066] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described Ceph cluster data balancing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0067] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0068] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0073] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0074] The above provides a detailed description of a Ceph cluster data balancing method and a Ceph cluster data balancing device provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data balancing method for a Ceph cluster, characterized in that, The Ceph cluster deploys multiple placement groups, each including a primary replica and a secondary replica. The primary and secondary replicas of each placement group are placed on different storage devices within the Ceph cluster. The method includes: Obtain the data distribution information of the Ceph cluster; A first balancing operation is performed on the primary replicas of the placement groups carried on each storage device in the Ceph cluster based on the data distribution information; the first balancing operation is used to balance the number of primary replicas of the placement groups on each storage device. A second balancing operation is performed on the slave replicas of the placement groups carried on each storage device in the Ceph cluster based on the data distribution information; the second balancing operation is used to balance the number of slave replicas of the placement groups on each storage device.

2. The data balancing method for a Ceph cluster according to claim 1, characterized in that, The first balancing operation on the primary replicas of the placement group carried on each storage device in the Ceph cluster according to the data distribution information includes: Determine whether there are any placement groups in the Ceph cluster that are not in a normal working state; In the absence of any placement groups in abnormal working condition, a first balancing operation is performed on the primary replicas of the placement groups carried on each storage device in the Ceph cluster based on the data distribution information.

3. The data balancing method for a Ceph cluster according to claim 2, characterized in that, The first balancing operation on the primary replicas of the placement group carried on each storage device in the Ceph cluster according to the data distribution information includes: The primary replica distribution score of the placement group on the storage device in the Ceph cluster is determined based on the data distribution information; the primary replica distribution score is inversely proportional to the variance of the number of primary replicas of the placement group carried on each storage device; When the primary replica distribution score is lower than the preset primary replica distribution score threshold, a first balancing operation is performed on the primary replicas of the placement group carried on each storage device in the Ceph cluster.

4. The data balancing method for a Ceph cluster according to claim 3, characterized in that, The first balancing operation on the primary replicas of the placement group hosted on each storage device in the Ceph cluster includes: Determine the average number of primary replicas on the storage device; Iterate through each placement group in the Ceph cluster; If the number of primary replicas on the storage device where the primary replica of the current placement group is located is greater than the average number of primary replicas, the primary replica of the current placement group will be migrated to the storage device with the fewest primary replicas and which does not carry any secondary replicas of the current placement group. Calculate the variance of the number of primary replicas of the placement group carried by each storage device after migration, and stop the migration operation of the primary replicas when the variance is zero.

5. The data balancing method for a Ceph cluster according to claim 1, characterized in that, The second balancing operation, performed on the slave replicas of the placement group on each storage device in the Ceph cluster based on the data distribution information, includes: Determine whether the first balancing operation has been completed; After the first balancing operation is completed, a second balancing operation is performed on the slave replicas of the placement group carried on each storage device in the Ceph cluster according to the data distribution information.

6. The data balancing method for a Ceph cluster according to claim 5, characterized in that, The second balancing operation, performed on the slave replicas of the placement group on each storage device in the Ceph cluster based on the data distribution information, includes: The slave replica distribution score of the placement group on the storage device in the Ceph cluster is determined based on the data distribution information; the slave replica distribution score is inversely proportional to the variance of the number of slave replicas of the placement group carried on each storage device; When the slave replica distribution score is lower than the preset master replica distribution score threshold, a second balancing operation is performed on the slave replicas of the placement group carried on each storage device in the Ceph cluster.

7. The data balancing method for a Ceph cluster according to claim 6, characterized in that, The second balancing operation on the slave replicas of the placement group hosted on each storage device in the Ceph cluster includes: Determine the average value of the replicas on the storage device; Iterate through each placement group in the Ceph cluster; If the number of slave replicas on the storage device where the slave replica of the current placement group is located is greater than the average number of slave replicas, the slave replica of the current placement group will be migrated to the storage device with the fewest slave replicas and which does not carry the slave replica of the current placement group. Calculate the variance of the number of slave replicas in the placement group carried by each storage device after migration, and stop the migration operation on the slave replicas if the variance is zero.

8. A data balancing device for a Ceph cluster, characterized in that, The Ceph cluster deploys multiple placement groups, each including a primary replica and secondary replicas. The primary and secondary replicas of each placement group are placed on different storage devices within the Ceph cluster. The device includes: The data distribution information acquisition module is used to acquire the data distribution information of the Ceph cluster; The first balancing module is used to perform a first balancing operation on the primary replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the first balancing operation is used to balance the number of primary replicas of the placement groups on each storage device. The second balancing module is used to perform a second balancing operation on the slave replicas of the placement groups carried on each storage device in the Ceph cluster according to the data distribution information; the second balancing operation is used to balance the number of slave replicas of the placement groups on each storage device.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the data balancing method for a Ceph cluster as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the data balancing method for a Ceph cluster as described in any one of claims 1-7.

Images