Method for backing up data, electronic equipment and computer program product

Abstract

The embodiment of the invention provides a method for backing up data, electronic equipment and a computer program product. The method can comprise the following steps: acquiring a system state of a storage system according to a data backup task; the method further includes generating, by the reinforcement learning model, a plurality of candidate backup policies based on the system state. The method further comprises the step of determining a plurality of reward values of a plurality of candidate backup strategies based on resource quantities occupied by IO tasks in a plurality of time periods corresponding to the data backup task. The method further includes selecting a target backup policy based on the plurality of reward values. The method further includes performing a data backup task using the target backup policy. According to the method, the target backup strategy used for executing the data backup task can be determined based on the reward value through the reinforcement learning model, on one hand, it is ensured that the data backup task does not have resource conflicts with the IO task, and it is ensured that the IO task is not affected while the data backup task is completed; and on the other hand, a more accurate target backup strategy can be obtained at low cost.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a method for backing up data, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Data backup is a technology that prevents data loss due to operational errors or storage system failures. It can be used to protect data in disaster recovery situations. In the daily work of storage systems, data backup is a common function, so how to achieve data backup through better methods is an important research area.

[0003] Disaster Recovery (DR) is an important function of storage systems. To avoid accidental data loss, users usually create a backup copy of important data (such as volumes, files, etc.) in the backup server in the data backup system. This function can be called replication. Like snapshots, the replication function also requires administrators to arrange replication of different objects according to different needs to achieve different data recovery time objectives and data recovery point objectives. Summary of the invention

[0004] According to an example embodiment of the present disclosure, a technical solution for backup is provided, which can determine a target backup strategy for executing a data backup task based on a reward value through a reinforcement learning model. On the one hand, it can ensure that the data backup task does not conflict with the IO task in terms of resources, so as to ensure that the IO task is not affected while the data backup task is completed. On the other hand, a more accurate target backup strategy can be obtained at a low cost.

[0005] In a first aspect of the present disclosure, a method for backup is provided, which may include obtaining a system state of a storage system according to a data backup task for backing up data from a first storage device to a second storage device. The system state may include a state of a client, a state of a first storage device, and a state of a second storage device. The method may also include generating, by a reinforcement learning model, a plurality of candidate backup strategies for the data backup task based on the system state. The method may also include determining a plurality of reward values ​​for a plurality of candidate backup strategies based on the amount of resources occupied by IO tasks in a plurality of time periods corresponding to the data backup task. The method also includes selecting a target backup strategy from a plurality of candidate backup strategies based on the plurality of reward values. The method also includes using the target backup strategy to execute the data backup task.

[0006] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes: a processor, and a memory coupled to the processor, the memory having instructions stored therein, the instructions causing the electronic device to perform operations when executed by the electronic device, the operations may include obtaining a system state of a storage system according to a data backup task for backing up data from a first storage device to a second storage device. The system state may include a state of a client, a state of a first storage device, and a state of a second storage device. The operation may also include generating, by a reinforcement learning model, a plurality of candidate backup strategies for the data backup task based on the system state. The operation may also include determining a plurality of reward values ​​for a plurality of candidate backup strategies based on the amount of resources occupied by IO tasks in a plurality of time periods corresponding to the data backup task. The operation also includes selecting a target backup strategy from a plurality of candidate backup strategies based on a plurality of reward values. The method also includes using the target backup strategy to perform the data backup task.

[0007] In a third aspect of the present disclosure, there is provided a computer program product, which is tangibly stored on a computer-readable medium and comprises computer-executable instructions which, when executed, cause a computer to perform the method according to the first aspect of the present disclosure.

[0008] From the above description, it can be seen that according to the scheme of each embodiment of the present disclosure, the target backup strategy for executing the data backup task can be determined based on the reward value through the reinforcement learning model. On the one hand, it ensures that the data backup task does not conflict with the IO task in terms of resources, so as to ensure that the IO task is not affected while completing the data backup task. On the other hand, a more accurate target backup strategy can be obtained at a low cost. It should be understood that the invention content section is provided to introduce the selection of concepts in a simplified form, which will be further described in the specific implementation methods below. The invention content section is not intended to identify the key features or main features of the present disclosure, nor is it intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 An architectural diagram of a data backup system according to some embodiments of the present disclosure is shown;

[0011] Figure 2 shows a working sequence diagram of a data backup system according to some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram showing candidate backup strategies according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A schematic diagram of a reinforcement learning model according to some embodiments of the present disclosure is shown;

[0014] Figure 5 Another schematic diagram of a reinforcement learning model according to some embodiments of the present disclosure is shown;

[0015] Figure 6 Another schematic diagram of a reinforcement learning model according to some embodiments of the present disclosure is shown;

[0016] Figure 7 A flowchart showing backing up data according to some embodiments of the present disclosure; and

[0017] Figure 8 A schematic structural diagram of a device that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] In a data backup system, in order to ensure that data is not lost due to accidents, data can be backed up through the replication function. For example, in some cases, the data backup system can include both a local backup server and a remote backup server. The data backup system can replicate the data in the local backup server to the remote backup server to back up the remote backup copy of the data. However, while the replication task is being executed, the business tasks between the client and the local backup server are also running, so there is a possibility of resource conflicts between the replication task and the business task (such as the hardware resources and network bandwidth of the data backup system). However, the administrator of the data backup system may not fully understand the actual situation of the data backup system. Therefore, if the replication task is not scheduled reasonably, the scheduling of the replication task may cause a lot of performance degradation or even business task errors.

[0021] Therefore, in order to ensure that the replication task is completed while ensuring that the business task is not affected, it is necessary to arrange the replication task reasonably. It should be noted that, for the sake of ease of understanding, the following text uses a data backup system that includes both a local backup server and a remote backup server as an example for explanation. However, it is understandable that the local backup server can be replaced by a server for other purposes, and the remote backup server can also be replaced by other types of backup servers (such as a local backup server).

[0022] The present disclosure proposes a method, an electronic device, and a computer program product for backing up data. The method for setting tracking information includes obtaining a system state of a storage system according to a data backup task for backing up data from a first storage device to a second storage device. The system state may include a state of a client, a state of a first storage device, and a state of a second storage device. The method may also include generating multiple candidate backup strategies for the data backup task based on the system state by a reinforcement learning model. The method may also include determining multiple reward values ​​of multiple candidate backup strategies based on the amount of resources occupied by IO tasks in multiple time periods corresponding to the data backup task. The method also includes selecting a target backup strategy from multiple candidate backup strategies based on multiple reward values. The method also includes using the target backup strategy to perform the data backup task. According to the scheme of the embodiment of the present disclosure, the target backup strategy for performing the data backup task can be determined based on the reward value by a reinforcement learning model. On the one hand, it is ensured that the data backup task does not conflict with the IO task in terms of resources, so as to ensure that the IO task is not affected while completing the data backup task, and on the other hand, a more accurate target backup strategy can be obtained at a low cost. .

[0023] Reference below Figure 1 The method proposed in the present disclosure is described. Figure 1An exemplary framework of a data backup system (also referred to as a storage system) 100 is shown in FIG. The data backup system includes a reinforcement learning model 102, a client 104, a local backup server (also referred to as a first storage device) 106, and a remote backup server (also referred to as a second storage device) 108. The client 104 can communicate with the local backup server 106 to perform IO, thereby executing a business task (also referred to as an IO task). For example, a business task may refer to a task of backing up data of the client 104 in the local backup server 106. The remote backup server 108 can communicate with the local backup server 106 to copy the data in the local backup server 106 to the remote backup server 108, thereby implementing the data backup task. The reinforcement learning model 102 is used to optimize the data backup task to ensure that the business task is not affected while ensuring that the data backup task is completed. When the reinforcement learning model 102 is separated from the local backup server 106, the reinforcement learning model 102 can be hosted by other computing devices in the data backup system. It should be noted that although in Figure 1 The reinforcement learning model 102 is separated from the local backup server 106 , but in some embodiments, the reinforcement learning model 102 may also be integrated into the local backup server 106 .

[0024] The following is based on the timing Figure 2 The workflow of the data backup system is described. At a specific time point, the reinforcement learning model 102 may be triggered. Exemplarily, the reinforcement learning model 102 may be triggered when one or more data backup tasks appear in the local backup server 106, and the data backup task is used to back up data from a first storage device to a second storage device. For example, when the local backup server 106 receives an instruction from the client 104 input 202 to initiate a data backup task, the reinforcement learning model 102 may be triggered. For another example, when the local backup server is set to automatically generate a data backup task at a preset time point, the reinforcement learning model 102 may be triggered at the preset time point. The triggered reinforcement learning model 102 may obtain the system operating state (also referred to as the system state) of the data backup system. For example, the triggered reinforcement learning model 102 obtains the respective states 204, 206, and 208 from the client 104, the local backup server 106, and the remote backup server 108, and normalizes them as the obtained system operating state. Figure 2 The order of 204, 206, and 208 in the above description should not be understood as a limitation on obtaining the system operation status. For example, 204, 206, and 208 may be performed in a different order than the above. Figure 2It should be noted that the device status may include historical information about the device's execution of tasks, and may also include resource information about the device. Additionally or alternatively, the triggered reinforcement learning model 102 may obtain the network status of the data backup system as the system operation status. The system operation status may characterize the operation status of the data backup system, which will affect the execution result of the data backup task.

[0025] For ease of understanding, refer to Table 1. Table 1 shows various states that the system operation state may include.

[0026] Table 1 System operation status table

[0027]

[0028]

[0029] First, the historical execution duration in Table 1 is explained. The historical execution duration may refer to the duration required for the data backup system to complete the corresponding task (business task or data backup task) within a specific time period in the past. In some embodiments, the specific time period may be associated with the current time, that is, the data backup system determines the specific time period based on the current time. For example, the specific time period may refer to several hours before the current time, etc.

[0030] It can be understood that all the history-related states in Table 1 can be understood as the states obtained by statistics in the above-mentioned specific time period. For example, the historical execution success rate of the data backup task can refer to the success rate in a specific time period, and the historical execution result of the data backup task can refer to the result (success or failure) of the last execution of the data backup task in a specific time period. These historical data can provide statistical results for the reinforcement learning model 102 to help divide the data backup tasks. In contrast, the data volume and object number can refer to the data volume and object number of the current task, which are used to represent the situation of the current task, so as to help divide the data backup tasks from another perspective. In addition, the reinforcement learning model 102 can also obtain the device resource status and network status of the device itself, and help divide the data backup tasks from the perspective of resources.

[0031] In addition to the system state, the reinforcement learning model 102 also obtains the amount of resources occupied by the current business task (hereinafter referred to as the business task), so that there will be no resource conflict between the data backup task and the business task. Exemplarily, the reinforcement learning model 102 can obtain the bandwidth occupied by the business task as the amount of resources. For example, the reinforcement learning model 102 can determine the business task and the bandwidth occupied by the business task by querying the task table or predicting based on historical situations.

[0032] Subsequently, the reinforcement learning model 102 may generate 210 a candidate backup strategy based on the obtained system state. In some embodiments, the candidate backup strategy may include a plurality of subtasks executed in different time periods and the amount of resources (e.g., bandwidth) occupied by each subtask. By dividing the data backup task into subtasks executed in different time periods, the local backup server 106 may avoid resource conflicts between the data backup task and the business task. Figure 3 , Figure 3 , four cases of candidate backup strategies are shown in the figure when the resource amount is bandwidth in actual application or when training the reinforcement learning model 102. For candidate backup strategy 1, no additional subtasks are divided, but data backup is continuously performed between 13 o'clock and 16 o'clock with a bandwidth share of 100%. From the bandwidth share of the business task, it can be seen that if candidate backup strategy 1 is adopted, there will be an obvious bandwidth conflict between the data backup task and business task 1 between 14 o'clock and 15 o'clock, which may cause the data backup business and / or business task 1 to fail. However, this problem will not occur for candidate backup strategies 2 to 4. Similarly, for candidate backup strategy 1 and candidate backup strategy 2, there will be an obvious bandwidth conflict between the data backup task and business task 2, and the same is true for other business tasks. It can be seen that it is necessary to reasonably determine from multiple candidate backup strategies an ideal backup strategy (also called a target backup strategy) that will not cause resource conflicts between the data backup task and any business task. For the convenience of explanation below, the resource amount is explained as bandwidth, but it should not be understood as a limitation of the present disclosure.

[0033] The candidate backup strategy may include the time period in which the subtask is located, the amount of data to be copied by the subtask, and the bandwidth allocated to the subtask. It is understandable that the bandwidth allocated to the subtask should be related to the length of the time period and the amount of data that needs to be backed up within the time period, so that all data that needs to be backed up can be copied to the remote backup server 108, that is, the data backup task is completed. Therefore, in some embodiments, the reinforcement learning model can determine the time period for dividing the subtask and the amount of data to be backed up by the subtask when generating the candidate backup strategy, and then determine the bandwidth allocated to the subtask based on the length of the time period and the amount of data. For example, the minimum bandwidth required to back up these data can be allocated to the subtask. By generating a candidate backup strategy including subtasks, all replication tasks of the data can be triggered at one time, while ignoring the overall state of the data backup system. In addition, the method for allocating bandwidth provided by the present disclosure is more sophisticated and will not affect the RTO required by the user.

[0034] In some embodiments, the reinforcement learning model 102 can generate multiple candidate backup strategies in sequence. For example, the reinforcement learning model can first obtain the current initial system state, and generate the first candidate backup strategy and the next state corresponding to the first candidate backup strategy based on the system state. At the same time, the reinforcement learning model calculates the reward value of the first candidate backup strategy. If the reward value is not higher than the preset threshold, the reinforcement learning model generates the second candidate backup strategy based on the next state... until the reward value of the generated candidate backup strategy is higher than the preset threshold, the generated candidate backup strategy is output as the ideal backup strategy. Or in some embodiments, the reinforcement learning model can generate N candidate backup strategies, and memorize the reward value of each candidate backup strategy during the generation process. When N candidate backup strategies have been generated, the reinforcement learning model selects the one with the highest reward value as the ideal backup strategy output. That is, both of the above methods illustrate that the reinforcement learning model 102 can select the ideal backup strategy with the largest reward value based on the reward value of each candidate backup strategy.

[0035] In some embodiments, the reinforcement learning model may adopt Figure 4 The model architecture in . The reinforcement learning model includes a critic network (critic) 402 and an action network (actor) 404. The action network is used to output actions (i.e., generate candidate backup strategies) through a policy gradient function (policy_grad), and the critic network determines the target reward (Target_Q) of the candidate backup strategy based on the candidate backup strategy (and the system reward value (system reward)), and then determines the critic network parameters that minimize the loss (TD Error) based on the target reward, and updates the critic network. Afterwards, the reinforcement learning model updates the action network and the policy gradient function through the output "action gradient (action_grad)" of the updated critic network. To update the action network in the direction of making the reward value larger. The updated action network can continue to generate the next candidate backup strategy based on the next state until the reward value of the generated candidate backup strategy is higher than the preset threshold, and the generated candidate backup strategy is output as the ideal backup strategy.

[0036] By inputting the influential and real-time system status in Table 1 Figure 4 The reinforcement learning model in can obtain an effective ideal backup strategy based on the sensitive parameters in the reinforcement learning model, thereby improving the performance of data backup. Figure 4 The final output of the reinforcement learning model can be a 1*I array, where I represents the number of data backup tasks, and each element in the array is the configuration of each data backup task (i.e., the ideal backup strategy).

[0037] by Figure 3For example, candidate backup strategies 1 to 4 may be generated sequentially by the reinforcement learning model 102 until an ideal backup strategy (ie, candidate backup strategy 4) is obtained.

[0038] In some embodiments, the reward value can be calculated based on the bandwidth occupied by the business tasks in multiple time periods, so that the reward value can represent the bandwidth conflict between the business tasks, and then the backup strategy with the least bandwidth conflict between the business tasks can be selected. For example, the candidate time periods in which the business tasks do not occupy any bandwidth in the multiple time periods corresponding to the data backup tasks can be first determined, and then the candidate backup strategy that only arranges the subtasks in these candidate time periods is used as the ideal backup strategy. Alternatively, it can be determined whether the candidate backup strategy fails based on the bandwidth occupied by the business tasks in each time period and the bandwidth occupied by the subtasks in each candidate backup strategy in the corresponding time period, and then the reward value is determined based on the determination result. For example, it can be determined whether there is a time period with a sum of bandwidth greater than a preset threshold. If it exists, it can be determined that the candidate backup strategy causes the data backup task to fail, and a value of "0" is set for it. If it does not exist, it can be determined that the candidate backup strategy will not cause the data backup task to fail, and a value of "1" is set for it, and then the reward value is calculated based on the set value.

[0039] In other embodiments, since the historical execution time included in the next system state indicates the execution time of the business task caused by the candidate backup strategy, the reinforcement learning model 102 can also determine the execution time of the business task based on the next system state, and then determine the reward value of the candidate backup strategy based on the execution time and the determination result of the data backup task. Alternatively, the reinforcement learning model 102 can further determine the execution time of the data backup task, and determine the reward value of the candidate backup strategy in combination with the execution time of the business task, the execution time of the backup task, and the determination result of the data backup task.

[0040] For example, the reward value can be calculated by formula (1):

[0041]

[0042] Among them, R is the reward value, N is the number of business tasks, P n The execution result of the nth business task. The value "1" represents success and the value "0" represents failure. Since business tasks usually do not fail, the default value P can be n Always 1. T n It refers to the execution time of the nth business task. M is the number of data backup tasks, PR m is the execution result of the mth candidate backup task, TR m is the execution time of the mth candidate backup task.

[0043] After the reinforcement learning model 102 outputs 212 the ideal backup strategy to the local backup server 106, the local backup server 106 can execute the multiple subtasks included in the ideal backup strategy to complete the data backup 214. Figure 3 In this case, the local backup server 106 performs data backup in the form of candidate backup strategy 4, that is, data backup is performed with 100% bandwidth from 13:00 to 14:00, data backup is performed with 75% bandwidth from 16:00 to 18:00, and data backup is performed with 50% bandwidth from 19:00 to 20:30.

[0044] refer to Figure 5 and Figure 6 In some embodiments, the action network and the critic network in the reinforcement learning model 102 may include an action evaluation network (actor evaluation network) 502 and an action target network (actor target network) 504, as well as a critic evaluation network 602 and a critic target network (critic target network) 604. The evaluation network updates its own parameters to the target network only when the reward value of the candidate backup strategy generated later (also called the second candidate backup strategy) is higher than that of the candidate backup strategy generated previously (also called the first candidate backup strategy), and the work of the evaluation network can refer to Figure 4 Description of the action network and the critic network in . Figure 5 The action evaluation network 502 and the policy gradient function in , and Figure 6 The loss, target reward and critic evaluation network 602 in the training will be updated. Through this soft update method, on the one hand, it can ensure that the parameters in the target network can always be used to obtain the current optimal candidate backup strategy, and on the other hand, it is more suitable for the continuous control scenario of data backup, avoiding affecting the model performance during the continuous parameter update process.

[0045] For a more complete explanation, the following is the process of updating a round of networks by the reinforcement learning model. First, the criticism evaluation network and the action evaluation network are randomly initialized, and the criticism target network, the action target network and the initial experience replay pool R are initialized. Subsequently, a random process N is initialized for action exploration, which may refer to exploration noise. The reinforcement learning model then receives an initial system state and generates a candidate backup strategy based on the current strategy and exploration noise. The candidate backup strategy is then executed and the reward and the next system state are observed, while the initial system state, the candidate backup strategy, the reward and the next system state (the combination is called a transition) are stored in R. A batch of transitions are sampled in R, and the Q reference value is calculated. The Q reference value is input into the loss, the criticism evaluation network is updated by minimizing the loss, and the action strategy is updated using the sampled policy gradient. Finally, the target network is updated to achieve a round of updates.

[0046] In order to better illustrate the present disclosure, a method 700 for backing up data is also provided. First, at box 702, according to the data backup task for backing up data from the first storage device to the second storage device, the system state of the storage system is obtained. The system state may include the state of the client, the state of the first storage device, and the state of the second storage device. The respective states of the client, the first storage device, and the second storage device can refer to those shown in Table 1. In some embodiments, the system state of the storage system can be obtained in response to determining that there is a data backup task. Taking 702 being executed by the reinforcement learning model 102 as an example, the reinforcement learning model 102 can be triggered when it is determined that there is a data backup task in the local backup server 106 to obtain the states of the client 104, the local backup server 106, and the remote backup server 108.

[0047] Next, at block 704, the reinforcement learning model generates multiple candidate backup strategies for the data backup task based on the system state. The candidate backup strategy refers to a strategy for dividing the data to be copied to the second storage device. For example, for 100G data to be transmitted, the candidate backup strategy may include how much data to transmit in what time period, and how much bandwidth to use for transmission in what time period. In some embodiments, the reinforcement learning model may generate multiple candidate backup strategies in sequence. Figure 4 Take the reinforcement learning model execution 704 as an example, Figure 4 The reinforcement learning model in can generate candidate backup strategies based on the action network, and use the critic network to update the action network to generate the next candidate backup strategy based on the observed next system state.

[0048] Then at box 706, multiple reward values ​​for multiple candidate backup strategies are determined based on the amount of resources occupied by the IO task in multiple time periods corresponding to the data backup task. The reward value can be used to characterize the ideality of the candidate backup strategy. For example, by constructing a reasonable reward function, a higher reward value can represent a more ideal candidate backup strategy. The reward function can take the completion status of the data backup task, the execution time of the data backup task, and / or the execution time of the IO task as input to calculate a reward value that can characterize the quality of the candidate backup strategy. It can be understood that the better the completion status of the data backup task, the shorter the execution time of the data backup task, and the shorter the execution time of the IO task, the more ideal the candidate backup strategy is.

[0049] Then, at box 708, a target backup strategy is selected from multiple candidate backup strategies based on multiple reward values. Since the reward value can be used to characterize the ideality of the candidate backup strategy, the ideal target backup strategy can be selected based on the reward value to effectively complete the data backup task without causing resource conflicts with the business task. Exemplarily, when the convergence condition is to generate several candidate backup strategies at box 704, the reward value of each candidate backup strategy can be memorized, and then the one with the highest reward value can be selected from the candidate backup strategies as the target backup strategy. Alternatively, when the convergence condition is to generate a candidate backup strategy with a reward value greater than a preset threshold at box 704, the most recently generated candidate backup strategy, that is, the candidate backup strategy with a reward value greater than the preset threshold, can be selected as the target backup strategy.

[0050] Finally, at box 710, the target backup strategy is used to perform the data backup task. By using the target backup strategy, the resources of the storage system can be optimized while ensuring the completion of the data backup task and the IO task. In some embodiments, the target backup strategy output to the first storage device (e.g., the local backup server 106) can be generated by the second network in the reinforcement learning model, and the reinforcement learning model also includes a first network for generating multiple candidate backup strategies at 704. When the first network determines that the reward value of the candidate backup strategy generated later is greater than the reward value of the candidate backup strategy generated previously, the first network parameters corresponding to the candidate backup strategy generated later are updated to the second network, so that the second network can always generate the candidate backup strategy with the highest reward value. For 702-710, if the administrator finds that the action output by the model (i.e., the candidate backup strategy) is continuously punished, it can reflect that there are serious problems in the current external network that need to be repaired, or that the current hardware cannot support the current business task, thereby providing more information for the system administrator.

[0051] According to the various embodiments of the present disclosure described above and their combined embodiments, the efficiency of configuring the functional state can be improved. Figures 1 to 7The implementation method shown in the figure is only for reference. Figures 1 to 7 The architecture or process illustrated in the figure may have other different forms, and may also include more or fewer functional modules and / or units. These modules and / or units may be partially or fully implemented as hardware modules, software modules, firmware modules or any combination thereof, and the embodiments of the present disclosure are not limited to this.

[0052] It is to be understood that the specific names and / or protocols of the various components in the system described herein are only to help those skilled in the art to better understand the concept of the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Moreover, in other embodiments, more or better components may be included, and alternative components with the same or similar functions may also be included.

[0053] Figure 8 800 is a schematic structural diagram of an example device 800 that can be used to implement some embodiments of the present disclosure. The device 800 can be implemented as a server or a PC, etc. The embodiments of the present disclosure do not limit the specific implementation type of the device 800. Figure 8 As shown, the device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or computer program instructions loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0054] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, an update modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0055] The processing unit 801 can execute the various methods and / or processes described above, for example Figure 7For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the CPU 801, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the CPU 801 may be configured to perform the method in any other suitable manner (e.g., by means of firmware, etc.).

[0056] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0057] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0058] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0059] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0060] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, and conventional procedural programming languages. The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server.

[0061] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer or other programmable data processing device, thereby producing a machine so that when these instructions are executed by the processing unit of a computer or other programmable data processing device, a device for realizing the functions / actions specified in one or more boxes in the flow chart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions make the computer, programmable data processing device and / or other equipment work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for realizing various aspects of the functions / actions specified in one or more boxes in the flow chart and / or block diagram. Computer-readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device, so that a series of operating steps are performed on the computer, other programmable data processing device or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing device, or other device realize the functions / actions specified in one or more boxes in the flow chart and / or block diagram.

[0062] In the information disclosed herein, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0063] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous frames can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart, and the combination of frames in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0064] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the information of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the information of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0065] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0066] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for backing up data, comprising: Acquire a system state of a storage system according to a data backup task for backing up data from a first storage device to a second storage device, wherein the system state includes a state of a client, a state of the first storage device, and a state of the second storage device; The reinforcement learning model generates a plurality of candidate backup strategies for the data backup task based on the system state; Determining multiple reward values ​​of the multiple candidate backup strategies based on the amount of resources occupied by the IO tasks in multiple time periods corresponding to the data backup tasks; selecting a target backup strategy from the plurality of candidate backup strategies based on the plurality of reward values; as well as The data backup task is executed using the target backup strategy. 2 . The method according to claim 1 , wherein each of the plurality of candidate backup strategies comprises a plurality of subtasks executed in different time periods and an amount of resources occupied by each of the subtasks.

3. The method of claim 2, wherein determining the plurality of reward values ​​for the plurality of candidate backup strategies comprises: Determining whether each candidate backup strategy causes the data backup task to fail based on the amount of resources occupied by the IO task in each of the multiple time periods and the amount of resources occupied by the subtasks in each of the multiple candidate backup strategies in the corresponding time period; as well as The plurality of reward values ​​for the plurality of candidate backup strategies are determined based on the determination result.

4. The method according to claim 3, wherein determining the plurality of reward values ​​for the plurality of candidate backup strategies further comprises: Determine the execution time of the IO task corresponding to each candidate backup strategy based on the next system state corresponding to each candidate backup strategy; as well as The multiple reward values ​​of the multiple candidate backup strategies are determined based on the execution time of the IO task and the determination result of the data backup task.

5. The method according to claim 2, wherein generating the candidate backup strategy comprises: Determine the time period for dividing the subtasks and the amount of data to be backed up for each subtask; as well as The amount of resources occupied by each subtask is determined based on the amount of data to be backed up by each subtask and the length of the time period corresponding to each subtask.

6. The method according to claim 1, wherein generating a plurality of candidate backup strategies for the data backup task comprises: The first network in the reinforcement learning model generates the multiple candidate backup strategies for the data backup task based on the system state.

7. The method according to claim 6, wherein the target backup strategy is a candidate backup strategy with the largest reward value, and the reinforcement learning model further comprises a second network, wherein the second network is used to synchronize the parameters of the first network when generating the candidate backup strategy with the largest reward value, so as to output the target backup strategy to the first storage device based on the system state.

8. The method of claim 1, wherein the system status further comprises network status related to available network bandwidth and round-trip latency.

9. The method according to claim 8, wherein The state of the client device includes at least one of the following: an IO object number of the IO task, a data volume of the IO task, a historical execution time of the IO task, or a historical execution result of the IO task; The state of the first storage device includes at least one of the following: a processor usage rate of the first storage device, a memory usage rate of the first storage device, an IO usage rate of the first storage device, a historical execution time of the data backup task, a historical execution success rate of the data backup task, an object number of the data backup task, a data volume of the data backup task, a historical redundancy deletion rate of the data backup task, or a historical execution result of the data backup task; The state of the second storage device includes at least one of the following: a processor usage rate of the second storage device, a memory usage rate of the second storage device, or an IO usage rate of the second storage device.

10. An electronic device comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, the instructions causing the electronic device to perform operations when executed by the processor, the operations comprising: Acquire a system state of a storage system according to a data backup task for backing up data from a first storage device to a second storage device, wherein the system state includes a state of a client, a state of the first storage device, and a state of the second storage device; The reinforcement learning model generates a plurality of candidate backup strategies for the data backup task based on the system state; Determining multiple reward values ​​of the multiple candidate backup strategies based on the amount of resources occupied by the IO tasks in multiple time periods corresponding to the data backup tasks; selecting a target backup strategy from the plurality of candidate backup strategies based on the plurality of reward values; and The data backup task is executed using the target backup strategy. 11 . The electronic device according to claim 10 , wherein each of the plurality of candidate backup strategies comprises a plurality of subtasks executed in different time periods and an amount of resources occupied by each of the subtasks.

12. The electronic device of claim 11, wherein determining the plurality of reward values ​​for the plurality of candidate backup strategies comprises: Determining whether each candidate backup strategy causes the data backup task to fail based on the amount of resources occupied by the IO task in each of the multiple time periods and the amount of resources occupied by the subtasks in each of the multiple candidate backup strategies in the corresponding time period; as well as The plurality of reward values ​​for the plurality of candidate backup strategies are determined based on the determination result.

13. The electronic device of claim 12, wherein determining the plurality of reward values ​​for the plurality of candidate backup strategies further comprises: Determine the execution time of the IO task corresponding to each candidate backup strategy based on the next system state corresponding to each candidate backup strategy; as well as The multiple reward values ​​of the multiple candidate backup strategies are determined based on the execution time of the IO task and the determination result of the data backup task.

14. The electronic device of claim 11, wherein generating the candidate backup strategy comprises: Determine the time period for dividing the subtasks and the amount of data to be backed up for each subtask; as well as The amount of resources occupied by each subtask is determined based on the amount of data to be backed up by each subtask and the length of the time period corresponding to each subtask.

15. The electronic device according to claim 10, wherein generating a plurality of candidate backup strategies for the data backup task comprises: The first network in the reinforcement learning model generates the multiple candidate backup strategies for the data backup task based on the system state.

16. An electronic device according to claim 15, wherein the target backup strategy is a candidate backup strategy with the largest reward value, and the reinforcement learning model also includes a second network, which is used to synchronize the parameters of the first network when generating the candidate backup strategy with the largest reward value, so as to output the target backup strategy to the first storage device based on the system state.

17. The electronic device of claim 10, wherein the system status further comprises a network status related to available network bandwidth and round-trip latency.

18. The electronic device according to claim 17, wherein The state of the client device includes at least one of the following: an IO object number of the IO task, a data volume of the IO task, a historical execution time of the IO task, or a historical execution result of the IO task; The state of the first storage device includes at least one of the following: a processor usage rate of the first storage device, a memory usage rate of the first storage device, an IO usage rate of the first storage device, a historical execution time of the data backup task, a historical execution success rate of the data backup task, an object number of the data backup task, a data volume of the data backup task, a historical redundancy deletion rate of the data backup task, or a historical execution result of the data backup task; The state of the second storage device includes at least one of the following: a processor usage rate of the second storage device, a memory usage rate of the second storage device, or an IO usage rate of the second storage device.

19. A computer-readable storage medium having computer-executable instructions stored thereon, wherein when the computer-executable instructions are executed, the computer: Acquire a system state of a storage system according to a data backup task for backing up data from a first storage device to a second storage device, wherein the system state includes a state of a client, a state of the first storage device, and a state of the second storage device; The reinforcement learning model generates a plurality of candidate backup strategies for the data backup task based on the system state; Determining multiple reward values ​​of the multiple candidate backup strategies based on the amount of resources occupied by the IO tasks in multiple time periods corresponding to the data backup tasks; selecting a target backup strategy from the plurality of candidate backup strategies based on the plurality of reward values; as well as The data backup task is executed using the target backup strategy.

20. A computer program product tangibly stored on a computer-readable storage medium and comprising computer-executable instructions which, when executed, cause a computer to: Acquire a system state of a storage system according to a data backup task for backing up data from a first storage device to a second storage device, wherein the system state includes a state of a client, a state of the first storage device, and a state of the second storage device; The reinforcement learning model generates a plurality of candidate backup strategies for the data backup task based on the system state; Determining multiple reward values ​​of the multiple candidate backup strategies based on the amount of resources occupied by the IO tasks in multiple time periods corresponding to the data backup tasks; selecting a target backup strategy from the plurality of candidate backup strategies based on the plurality of reward values; as well as The data backup task is executed using the target backup strategy.

Images