A data balancing method and system for a ceph distributed storage system

Abstract

The present invention belongs to the field of cloud storage data balancing and discloses a data balancing method and system for a Ceph distributed storage system. The present invention sets a state space, an action space and a reward function, and uses a neural network composed of an actor network and two critic networks to output the state-action values in the state space and the action space. The object storage device of the Ceph system in the optimization model is adjusted through the reward function, and the data balancing problem is converted into a weight optimization problem of the OSD in the Ceph system. The data balancing of the Ceph system is completed through the interaction between the intelligent agent and the environment. In addition, the present invention can achieve a better distribution effect in a multi-copy situation without destroying the original algorithm logic.

Description

Technical Field

[0001] The present invention belongs to the field of cloud storage data balancing, and in particular relates to a data balancing method and system for a Ceph distributed storage system. Background Art

[0002] With the rapid development of the internet, new demands are constantly emerging, leading to an explosive growth in user data. This has led to an increasing demand for storage. Traditional centralized storage can no longer meet the needs of large-scale storage applications, becoming a bottleneck for data storage. Consequently, distributed storage has emerged. The Ceph system is a unified distributed file system with high performance, high reliability, and high scalability. The Ceph system uses the CRUSH algorithm for data mapping and storage, which can lead to uneven data distribution across devices. The CRUSH algorithm is a pseudo-random hashing algorithm that, given the same input, uniquely determines the list of devices storing the data. However, the CRUSH algorithm does not achieve satisfactory data balance. In practice, the capacity utilization of Object Storage Devices (OSDs) is significantly unbalanced. This not only wastes storage resources but also causes heavily utilized OSDs to become system bottlenecks during read and write operations, reducing the QoS of the entire storage device. Therefore, balanced data distribution is crucial for improving storage system performance.

[0003] To address the issue of unbalanced data distribution, a B-CRUSH algorithm has been proposed. This algorithm optimizes data distribution by designing a new hash algorithm and an adaptive model. Another approach proposes an improved CRUSH algorithm based on a temperature factor, using the designed temperature factor to dynamically increase node weights to improve the original CRUSH algorithm.

[0004] To address the issue of unbalanced data distribution, the aforementioned B-CRUSH algorithm significantly alters the original algorithm's logic, undermining its advantage of minimizing data migration during cluster expansion. Another CRUSH algorithm improvement, based on a temperature factor, can select nodes with lower loads as storage nodes. However, this algorithm cannot achieve the same distribution effect in scenarios with multiple replicas of data. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a data balancing method and system for the Ceph distributed storage system, which achieves interaction between the intelligent agent and the environment by adjusting the weight of the object storage device and completes the data balancing of the Ceph system.

[0006] To achieve the above object, a data balancing method for a Ceph distributed storage system includes the following steps:

[0007] Build a Ceph system simulation environment and map the preset placement groups to the preset object storage devices;

[0008] In the Ceph system simulation environment, a state space including all states is generated based on the placement group, object storage device, and the maximum weight value of the object storage device;

[0009] Generate an action space including all actions based on the changes in the weights of different object storage devices at the current moment;

[0010] The reward function is formed by multiplying the weight of the action taken at a certain moment by the difference between the number of placement groups in a certain object storage device at that moment and the number of placement groups in each object storage device when the data is completely evenly distributed. This is combined with the reward obtained at the next moment.

[0011] The state in the state space is input into the neural network. The neural network is composed of an actor network and two critic networks. The two critic networks output two state-action values including the state and the action, and the smallest value is selected as the final state-action value.

[0012] The critic network uses the gradient descent algorithm to determine the minimized loss function and adjusts the weight coefficient of the critic network according to the minimized loss function;

[0013] The actor network sends the output weight change value to the critic network to calculate the gradient of the final state action value, and updates the corresponding parameters of the actor network according to the gradient of the final state action value;

[0014] Add zero mean to the weight change value output by the actor network to complete the construction of the optimization model;

[0015] The optimization model is trained in conjunction with the reward function, and ultimately the weight that minimizes the standard deviation of the object storage device is obtained.

[0016] The preset placement group is mapped to the preset object storage device through the CRUSH algorithm.

[0017] State S in state space S t The expression is as follows:

[0018] S t =(N i ,N s ,W max )

[0019] Among them, t is the time, N i is the number of placement groups of the i-th object storage device at the current moment, N sis the number of placement groups in each object storage device when data is completely evenly distributed, W max The maximum weight of the object storage device at the current moment.

[0020] The expression of action a in action space A is as follows:

[0021]

[0022] Where t is the time, Indicates the change in weight of the i-th object storage device at the current moment.

[0023] The expression of the reward function is r(S t ,a t ,S t+1 ), where S t is the state in the state space S at time t, a t is the action in the action space A at time t, S t+1 is the state in the state space S at time t+1;

[0024] If the evaluation indicator is positive, the reward function is -1; if the evaluation indicator is negative, the reward function is 1; if the difference between the weight of the action taken at a certain moment and the number of placement groups in a certain object storage device at the corresponding moment and the number of placement groups in each object storage device when the data is completely evenly distributed exceeds the preset threshold, the reward function is -2.

[0025] The actor network outputs the i-th weight change value to the critic network, and the critic network calculates the gradient of the final state action value And according to Update the corresponding parameters of the actor network;

[0026] in, is the gradient of Q value relative to action a, Q is the final state action value, s is the state in state space S, a is the action in action space A, is the gradient, N is the number of cycles in the algorithm, θ is the neural network parameter, s i is the state at time i.

[0027] The specific method of training the optimization model in combination with the reward function is as follows:

[0028] Initialize the optimization model S0=(N0,N s ,W max ), N0 is the number of placement groups in the object storage device at the initial moment, N s is the number of placement groups in each object storage device when data is completely evenly distributed, W max The maximum weight value of the object storage device at the current moment;

[0029] Using actor network to select action a t , and add random noise;

[0030] Update the final state action value and obtain the corresponding reward to generate experience (S t ,a t ,r t ,S t+1 ) is added to the experience pool; among them, S t is the state in the state space S at time t, a t is the action in the action space A at time t, r t is the reward at time t, S t+1 is the state in the state space S at time t+1;

[0031] If the amount of experience in the experience pool exceeds the threshold, experience data is extracted from the experience pool to update the weight coefficients of the critic network and the corresponding parameters of the actor network;

[0032] If the amount of experience in the experience pool does not exceed the threshold, the state S t+1 =(N i ,N s ,W max ), if satisfied Then i=(i+1)mod N_OSD, where N_OSD is the number of object storage devices; if not satisfied Then i remains unchanged; among them, is the number of PGs in the i-th object storage device at time t+1, and mod is the modulo function;

[0033] Initialize state S after reaching the specified step size t =(N0,N s ,W max ), repeat the training multiple times, and save the weight that minimizes the standard deviation of the object storage device.

[0034] A data balancing method for a Ceph distributed storage system includes:

[0035] The mapping module is used to build a Ceph system simulation environment and map the preset placement group to the preset object storage device;

[0036] A state space generation module is used to form a state space including all states according to the placement group, the object storage device and the maximum weight value of the object storage device in the Ceph system simulation environment;

[0037] An action space generation module, used to form an action space including all actions based on the changes in the weights of different object storage devices at the current moment;

[0038] A reward function generation module is configured to form a reward function based on the product of the weight of the action taken at a certain moment and the difference between the number of placement groups in a certain object storage device at that moment and the number of placement groups in each object storage device when data is completely evenly distributed, as an evaluation indicator, combined with the reward obtained at the next moment;

[0039] The state-action value generation module is used to input the state in the state space into the neural network. The neural network uses a neural network composed of an actor network and two critic networks. The two critic networks output two state-action values including the state and action, and the smallest value is selected as the final state-action value;

[0040] A critic network weight coefficient adjustment module is used to determine the minimized loss function of the critic network using a gradient descent algorithm, and adjust the weight coefficient of the critic network according to the minimized loss function;

[0041] The actor network parameter update module is used to send a certain weight change value output by the actor network to the critic network to calculate the gradient of the final state action value, and update the corresponding parameters of the actor network according to the gradient of the final state action value;

[0042] The optimization model construction module is used to add zero mean to the weight change value output by the actor network to complete the construction of the optimization model;

[0043] The training module is used to train the optimization model in combination with the reward function, and ultimately obtain the weight that minimizes the standard deviation of the object storage device.

[0044] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of a data balancing method for a Ceph distributed storage system.

[0045] A storage medium stores a computer program, which, when executed by a processor, implements the steps of a data balancing method for a Ceph distributed storage system.

[0046] Compared with the existing technology, the present invention sets up a state space, an action space and a reward function, and adopts a neural network composed of an actor network and two critic networks to output the state-action value in the state space and action space. The object storage device of the Ceph system in the optimization model is adjusted through the reward function, and the data balancing problem is converted into the weight optimization problem of the OSD in the Ceph system. The data balancing of the Ceph system is completed through the interaction between the intelligent agent and the environment, and better distribution effect can be obtained in the case of multiple copies without destroying the original algorithm logic. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Flowchart of the present invention;

[0048] Figure 2 This is the network structure diagram of actors and critics in the present invention;

[0049] Figure 3 This is a flow chart of network training of the present invention;

[0050] Figure 4 This is a comparison chart of the number of PGs in OSD before and after data balancing in the present invention;

[0051] Figure 5 This is a comparison chart of the standard deviation of the number of PGs in OSD before and after data balancing in the present invention;

[0052] Figure 6 It is a system diagram of the present invention. DETAILED DESCRIPTION

[0053] The present invention will be further described below with reference to the accompanying drawings.

[0054] A data balancing method for a Ceph distributed storage system includes the following steps:

[0055] S1: Build a Ceph system simulation environment and map the preset placement group to the preset object storage device through the CRUSH algorithm.

[0056] In the ceph system simulation environment, S2 generates a state space including all states according to the placement group, object storage device and the maximum weight value of the object storage device; the state S in the state space S t The expression is as follows:

[0057] S t =(N i ,N s ,W max )

[0058] Among them, t is the time, N i is the number of placement groups of the i-th object storage device at the current moment, N s is the number of placement groups in each object storage device when data is completely evenly distributed, W max The maximum weight of the object storage device at the current moment.

[0059] S3, based on the changes in the weights of different object storage devices at the current moment, generates an action space that includes all actions; the expression of action a in action space A is as follows:

[0060]

[0061] Where t is the time, Indicates the change in weight of the i-th object storage device at the current moment.

[0062] S4, the product of the weight of the action taken at a certain moment and the difference between the number of placement groups in a certain object storage device at the corresponding moment and the number of placement groups in each object storage device when the data is completely evenly distributed is used as the evaluation index, combined with the reward obtained at the next moment, to form a reward function; the expression of the reward function is r(S t ,a t ,S t+1 ), where S t is the state in the state space S at time t, a t is the action in the action space A at time t, S t+1 is the state in the state space S at time t+1;

[0063] If the evaluation indicator is positive, the reward function is -1; if the evaluation indicator is negative, the reward function is 1; if the difference between the weight of the action taken at a certain moment and the number of placement groups in a certain object storage device at the corresponding moment and the number of placement groups in each object storage device when the data is completely evenly distributed exceeds the preset threshold, the reward function is -2.

[0064] S5, inputs the state in the state space into the neural network. The neural network adopts a neural network composed of an actor network and two critic networks. The two critic networks output two state-action values including state and action, and select the smallest value as the final state-action value.

[0065] S6, the critic network uses the gradient descent algorithm to determine the minimized loss function, and adjusts the weight coefficient of the critic network according to the minimized loss function.

[0066] S7, the actor network sends a certain weight change value output to the critic network to calculate the gradient of the final state action value, and updates the corresponding parameters of the actor network according to the gradient of the final state action value; the actor network outputs the i-th weight change value to the critic network, and the critic network calculates the gradient of the final state action value And according to Update the corresponding parameters of the actor network;

[0067] in, is the gradient of the value relative to action a, Q is the final state action value, s is the state in the state space S, and a is the action in the action space A. is the gradient, N is the number of cycles in the algorithm, θ is the neural network parameter, s i is the state at time i.

[0068] S8, add zero mean to the weight change value output by the actor network to complete the construction of the optimization model; the specific method of training the optimization model in combination with the reward function is as follows:

[0069] Initialize the optimization model S0=(N0,N s ,W max ), N0 is the number of placement groups in the object storage device at the initial moment, N s is the number of placement groups in each object storage device when data is completely evenly distributed, W max The maximum weight value of the object storage device at the current moment;

[0070] Using actor network to select action a t , and add random noise;

[0071] Update the final state action value and obtain the corresponding reward to generate experience (S t ,a t ,r t ,S t+1 ) is added to the experience pool; among them, S t is the state in the state space S at time t, a t is the action in the action space A at time t, r t is the reward at time t, S t+1 is the state in the state space S at time t+1;

[0072] If the amount of experience in the experience pool exceeds the threshold, experience data is extracted from the experience pool to update the weight coefficients of the critic network and the corresponding parameters of the actor network;

[0073] If the amount of experience in the experience pool does not exceed the threshold, the state S t+1 =(N i ,N s ,W max ), if satisfied Then i=(i+1)mod N_OSD, where N_OSD is the number of object storage devices; if not satisfied Then i remains unchanged; among them, is the number of PGs in the i-th storage device at time (t+1), and mod is the remainder function;

[0074] Initialize state S after reaching the specified step size t =(N0,N s ,W max ), repeat the training multiple times, and save the weight that minimizes the standard deviation of the object storage device.

[0075] S9, combines the reward function to train the optimization model and finally obtains the weight that minimizes the standard deviation of the object storage device.

[0076] See also Figure 6 , a data balancing method for a Ceph distributed storage system, including:

[0077] The mapping module is used to build a Ceph system simulation environment and map the preset placement group to the preset object storage device;

[0078] A state space generation module is used to form a state space including all states according to the placement group, the object storage device and the maximum weight value of the object storage device in the Ceph system simulation environment;

[0079] An action space generation module, used to form an action space including all actions based on the changes in the weights of different object storage devices at the current moment;

[0080] A reward function generation module is configured to form a reward function based on the product of the weight of the action taken at a certain moment and the difference between the number of placement groups in a certain object storage device at that moment and the number of placement groups in each object storage device when data is completely evenly distributed, as an evaluation indicator, combined with the reward obtained at the next moment;

[0081] The state-action value generation module is used to input the state in the state space into the neural network. The neural network uses a neural network composed of an actor network and two critic networks. The two critic networks output two state-action values including the state and action, and the smallest value is selected as the final state-action value;

[0082] A critic network weight coefficient adjustment module is used to determine the minimized loss function of the critic network using a gradient descent algorithm, and adjust the weight coefficient of the critic network according to the minimized loss function;

[0083] The actor network parameter update module is used to send a certain weight change value output by the actor network to the critic network to calculate the gradient of the final state action value, and update the corresponding parameters of the actor network according to the gradient of the final state action value;

[0084] The optimization model construction module is used to add zero mean to the weight change value output by the actor network to complete the construction of the optimization model;

[0085] The training module is used to train the optimization model in combination with the reward function, and ultimately obtain the weight that minimizes the standard deviation of the object storage device.

[0086] Example:

[0087] This example provides a method for balancing data in a Ceph system. Using a deep reinforcement learning algorithm, the method employs a well-designed state, action, and reward structure, along with a neural network update strategy and hyperparameters. The agent interacts with a Ceph system simulation environment to adjust OSD weights and balance data. This example simulates a Ceph system with 1024 PGs, 50 OSDs, and two backups.

[0088] See also Figure 1 , the present invention comprises the following steps:

[0089] Step S10: The built ceph system simulation environment includes placement groups (PGs), object storage devices (OSDs), etc. PGs are mapped to OSDs using the CRUSH algorithm.

[0090] Step S20: Use the Twin Delayed Deep Deterministic policy gradient algorithm (TD3) deep reinforcement learning algorithm to build an optimization model.

[0091] Step S201, set the state space S: at time t, the state is S t =(N i ,N s ,W max ), where N i Represents the number of PGs of the i-th OSD at the current moment, N s Represents the number of PGs in each OSD when data is completely and evenly distributed, W max Indicates the maximum weight of the OSD at the current moment.

[0092] Step S202, set the action space A: at time t, the action is in Indicates the change in weight of the i-th OSD at the current moment.

[0093] Step S203: Set the reward function r(S t ,a t ,S t+1 ): The product of the action weight taken at time t and the difference between the number of PGs in the i-th OSD at time t and the number of PGs in each OSD when the data is completely balanced is used as the evaluation index. t+1 Rewards at time t: If the product at time t is positive, then r = -1, otherwise r = 1. If the action weight value is too large and the number of PGs in the OSD exceeds a certain threshold, r = -2. The formula is as follows:

[0094]

[0095] Step S204: Select three independent neural networks, one actor and two critics. The actor and critic networks have four layers: an input layer, an output layer, and two hidden layers, each containing 128 neurons. At time t, the actor network sets the state S t =(N i ,N s ,W max ) as input and output a continuous action: the change value a of the i-th OSD weight t The two critic networks respectively transform the state S t and a t As input, the output state action value Q1(S t ,a t ),Q2(S t ,a t ), select the smaller Q value as the state action value Q(S t ,a t ). The network structure diagram is as follows Figure 2 shown.

[0096] Step S205: Set the neural network update strategy and hyperparameters: The weight coefficients of the critic network are updated using the gradient descent algorithm to minimize the loss function. Parameter update of the actor network: The actor network outputs the i-th weight change value to the critic network, and the critic network calculates the gradient Passed to the actor network, the actor network according to Update parameters. Hyperparameter learning rate l r =0.001, and the future reward discount rate γ = 0.99.

[0097] Step S206: Set the exploration strategy: add a noise with zero mean and variance of 0.1 to the weight change value output by the actor network to allow the model to fully explore the action space.

[0098] Step S30: train the model and save the weight that minimizes the OSD standard deviation.

[0099] Step 301, initialization state S0=(N0,N s ,W max );

[0100] Step S302: The agent selects action a using the actor network. t , and add random noise.

[0101] Step S303, update the status, get rewards, and convert the experience (S t ,a t ,rt ,S t+1 ) added to the experience pool.

[0102] Step S304: If the number of experiences in the experience pool exceeds 200, extract 32 historical experience data from the experience pool and update the network according to the network update strategy of step S205; otherwise, proceed to step 305.

[0103] Step S305, state S t+1 =(N i ,N s ,W max ), if satisfied Then i=(i+1)modN_OSD, where N_OSD is the number of OSDs, otherwise i remains unchanged. After each round reaches the specified step size, the initial state is S t =(N0,N s ,W max ), repeat the training multiple times, and save the weight that minimizes the OSD standard deviation.

[0104] See also Figure 4 ,According to the comparison data of the number of PGs in OSD before and after balancing, it can be seen that the data distribution result after optimization is more balanced.

[0105] See also Figure 5 According to the comparison chart of the standard deviation of the number of PGs in OSD before and after data balancing, taking the standard deviation of the number of PGs in OSD as the standard, the standard deviation of the number of PGs in OSD before data balancing is close to 8, and the standard deviation of the number of PGs in OSD after data balancing is reduced to about 3.7, and the data distribution is more balanced.

[0106] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A data balancing method for a ceph distributed storage system, characterized in that: The following steps are involved: Build a Ceph system simulation environment and map the preset placement groups to the preset object storage devices; In the Ceph system simulation environment, a state space including all states is generated based on the placement group, object storage device, and the maximum weight value of the object storage device; Generate an action space including all actions based on the changes in the weights of different object storage devices at the current moment; The reward function is formed by multiplying the weight of the action taken at a certain moment by the difference between the number of placement groups in a certain object storage device at that moment and the number of placement groups in each object storage device when the data is completely evenly distributed. This is combined with the reward obtained at the next moment. The state in the state space is input into the neural network. The neural network is composed of an actor network and two critic networks. The two critic networks output two state-action values including the state and the action, and the smallest value is selected as the final state-action value. The critic network uses the gradient descent algorithm to determine the minimized loss function and adjusts the weight coefficient of the critic network according to the minimized loss function; The actor network sends the output weight change value to the critic network to calculate the gradient of the final state action value, and updates the corresponding parameters of the actor network according to the gradient of the final state action value; Add zero mean to the weight change value output by the actor network to complete the construction of the optimization model; The optimization model is trained in conjunction with the reward function, and ultimately the weight that minimizes the standard deviation of the object storage device is obtained.

2. A data balancing method for a ceph distributed storage system according to claim 1, characterized in that: The preset placement group is mapped to the preset object storage device through the CRUSH algorithm.

3. The data balancing method of a ceph distributed storage system according to claim 1, characterized in that: State S in state space S t The expression is as follows: S t =(N i ,N s ,W max ) Among them, t is the time, N i is the number of placement groups of the i-th object storage device at the current moment, N s is the number of placement groups in each object storage device when data is completely evenly distributed, W max The maximum weight of the object storage device at the current moment.

4. The data balancing method of a ceph distributed storage system according to claim 1, characterized in that: The expression of action a in action space A is as follows: Where t is the time, Indicates the change in weight of the i-th object storage device at the current moment.

5. The data balancing method of a ceph distributed storage system according to claim 1, characterized in that: The expression of the reward function is r(S t ,a t ,S t+1 ), where S t is the state in the state space S at time t, a t is the action in the action space A at time t, S t+1 is the state in the state space S at time t+1; If the evaluation indicator is positive, the reward function is -1; if the evaluation indicator is negative, the reward function is 1; if the difference between the weight of the action taken at a certain moment and the number of placement groups in a certain object storage device at the corresponding moment and the number of placement groups in each object storage device when the data is completely evenly distributed exceeds the preset threshold, the reward function is -2.

6. The data balancing method of a ceph distributed storage system according to claim 1, characterized in that: The actor network outputs the i-th weight change value to the critic network, and the critic network calculates the gradient of the final state action value And according to Update the corresponding parameters of the actor network; in, is the gradient of Q value relative to action a, Q is the final state action value, s is the state in state space S, a is the action in action space A, is the gradient to be sought, N is the number of cycles in the algorithm, θ is the neural network parameter, s i is the state at time i.

7. The data balancing method of a ceph distributed storage system according to claim 1, characterized in that: The specific method of training the optimization model in combination with the reward function is as follows: Initialize the optimization model S0=(N0,N s ,W nax ), N0 is the number of placement groups in the object storage device at the initial moment, N s is the number of placement groups in each object storage device when data is completely evenly distributed, W max The maximum weight value of the object storage device at the current moment; Using actor network to select action a t , and add random noise; Update the final state action value and obtain the corresponding reward to generate experience (S t ,a t ,r t ,S t+1 ) is added to the experience pool; among them, S t is the state in the state space S at time t, a t is the action in the action space A at time t, r t is the reward at time t, S t+1 is the state in the state space S at time t+1; If the amount of experience in the experience pool exceeds the threshold, experience data is extracted from the experience pool to update the weight coefficients of the critic network and the corresponding parameters of the actor network; If the amount of experience in the experience pool does not exceed the threshold, the state S t+1 =(N i ,N s ,W max ), if satisfied Then i=(i+1)mod N_OSD, where N_OSD is the number of object storage devices, N i is the number of placement groups of the i-th object storage device at the current moment; if it does not meet Then i remains unchanged; among them, is the number of placement groups in the i-th object storage device at time t+1, and mod is the modulo function; Initialize state S after reaching the specified step size t =(N0,N s ,W max ), repeat the training multiple times, and save the weight that minimizes the standard deviation of the object storage device.

8. A data balance of a ceph distributed storage system, characterized in that: include: The mapping module is used to build a Ceph system simulation environment and map the preset placement group to the preset object storage device; A state space generation module is used to form a state space including all states according to the placement group, the object storage device and the maximum weight value of the object storage device in the Ceph system simulation environment; An action space generation module, used to form an action space including all actions based on the changes in the weights of different object storage devices at the current moment; A reward function generation module is configured to form a reward function based on the product of the weight of the action taken at a certain moment and the difference between the number of placement groups in a certain object storage device at that moment and the number of placement groups in each object storage device when data is completely evenly distributed, as an evaluation indicator, combined with the reward obtained at the next moment; The state-action value generation module is used to input the state in the state space into the neural network. The neural network uses a neural network composed of an actor network and two critic networks. The two critic networks output two state-action values including the state and action, and the smallest value is selected as the final state-action value; A critic network weight coefficient adjustment module is used to determine the minimized loss function of the critic network using a gradient descent algorithm, and adjust the weight coefficient of the critic network according to the minimized loss function; The actor network parameter update module is used to send a certain weight change value output by the actor network to the critic network to calculate the gradient of the final state action value, and update the corresponding parameters of the actor network according to the gradient of the final state action value; The optimization model construction module is used to add zero mean to the weight change value output by the actor network to complete the construction of the optimization model; The training module is used to train the optimization model in combination with the reward function, and ultimately obtain the weight that minimizes the standard deviation of the object storage device.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the processor implements the steps of a data balancing method for a Ceph distributed storage system according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a data balancing method for a Ceph distributed storage system according to any one of claims 1 to 7 are implemented.

Images