Backup task scheduling method, device and equipment

Abstract

The invention discloses a backup task scheduling method, device and equipment, and the method comprises the steps: adding a first backup task into a backup task queue when the scheduling device receives the first backup task; when the first backup task is extracted from the backup task queue for the first time, the first backup task is not processed, and the first backup task is added into the backup task queue again. And the scheduling device extracts the first backup task from the backup task queue again, determines a first target node from the multiple nodes in the backup cluster, and triggers the first target node to execute the first backup task. The scheduling device delays the processing time of the first backup task. The information of the nodes obtained when the scheduling device processes the first backup task is closer to the information of the nodes when the first backup task is extracted for the first time, so that the scheduling device can conveniently select the first target node suitable for executing the first backup task from all the nodes, and the execution efficiency of the backup task is guaranteed.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a backup task scheduling method, apparatus, and device. Background Technology

[0002] Data backup is an effective way to protect data in storage systems, as it can reduce the impact of data loss or damage.

[0003] Backup services in a storage system are typically considered ancillary services, and their impact on other services (such as storage operations or near-memory computing) should be minimized. This necessitates that backup operations do not consume excessive processor resources within the storage system. Therefore, allocating appropriate nodes from the storage system to execute the backup task is crucial; only by selecting suitable nodes can the execution efficiency of the backup task be guaranteed. Summary of the Invention

[0004] This application provides a backup task scheduling method, apparatus, and device to improve the execution efficiency of backup tasks.

[0005] Firstly, this application also provides a backup task scheduling method, which can be executed by a scheduling device. When the scheduling device receives a first backup task, it adds the first backup task to a backup task queue. The scheduling device sequentially retrieves each backup task according to the order of the backup tasks in the backup queue and decides whether to process the backup task. Taking the first backup task as an example, when the scheduling device retrieves the first backup task from the backup task queue for the first time, it does not process the first backup task but re-adds it to the backup task queue. After re-adding the first backup task to the backup task queue, if the scheduling device retrieves the first backup task from the backup task queue again, the scheduling device processes the backup task, determines a first target node from multiple nodes in the backup cluster, and triggers the first target node to execute the first backup task.

[0006] The backup service scenario described above has certain unique characteristics. Only after a backup task is completed can the computing power consumed and storage space occupied on a node be known. This results in a certain delay in node information (such as real-time status information representing the running status or node historical task information describing the tasks executed by the node) in the backup service scenario. In this application, the scheduling device does not process the first backup task when it is first retrieved, but processes it when it is retrieved again. The scheduling device postpones the processing time of the first backup task. The node information obtained by the scheduling device when processing the first backup task is closer to the node information when the first backup task is first retrieved. This facilitates the scheduling device in selecting the most suitable first target node from among the various nodes to execute the first backup task, thus ensuring the execution efficiency of the backup task.

[0007] In other words, when executing a backup task in the backup queue, it is necessary to further obtain node information. However, node information changes dynamically, and the process of obtaining node information is delayed. Therefore, if the backup task is executed directly, the node information obtained will not be the current node information, but rather past node information, and the two do not match.

[0008] For example: Suppose there is a 10-minute lag in obtaining node information (this is just an example and not actual time). If the first backup task is retrieved at 10:00, and the node information is retrieved directly at this time, due to the lag, the node information obtained will be from 9:50, resulting in a significant time discrepancy. If the first backup task is executed at this time, it means that the backup task, which should have been executed at 10:00, is being executed based on the node information obtained at 9:50.

[0009] However, if the first backup task is not executed upon initial acquisition, but instead re-entered into the backup task queue, and then acquired again at 10:11, the node information retrieved at this time will be from 10:01 (a 10-minute lag). If the first backup task is executed at 10:01, it means that the backup task originally scheduled for 10:00 is actually executed based on the node information acquired at 10:01. Compared to a backup task originally scheduled for 10:00 being executed based on node information acquired at 10:50, the matching degree between the backup task and node information is higher, and the task execution will be smoother. It should be noted that if the lag in the task node information is significant, the first backup task can be repeatedly placed into the backup task queue to further delay its execution time, thus ensuring a better match with the lag in the node information.

[0010] In one possible implementation, the first target node satisfies the following condition: when the first target node is performing the first backup task, multiple nodes are in a load-balanced state.

[0011] Using the above method, the scheduling device selects a first target node for the first backup task with load balancing as the goal. The load on each node in the backup cluster tends to be consistent. The load of multiple nodes in the backup cluster presents a "uniform" distribution or tends to a "uniform" distribution (here, uniformity only means uniformity that allows for some fluctuations, not strict uniformity). This ensures that after the first backup task arrives at the first target node, the first target node has sufficient computing power or storage space to support the execution of the target backup task, effectively guaranteeing the execution efficiency of the first backup task.

[0012] In one possible implementation, when the scheduling device determines the first target node from multiple nodes in the backup cluster, it assumes that each of the multiple nodes is executing the first backup task and determines the load status of each of the multiple nodes when executing the first backup task; the scheduling device determines the first target node based on the load status of each node when executing the first backup task.

[0013] Using the above method, the scheduling device considers the load status of each node when executing the first backup task when determining the first target node. By using the load status of each node executing the first backup task, it can more clearly determine whether each node is suitable to execute the first backup task, and thus determine the first target node.

[0014] In one possible implementation, if multiple storage pools are stored on the first target node, the scheduling device needs to further select a storage pool from the multiple storage pools as the target storage pool to carry the backup task. The target storage pool satisfies the following: when the first backup task is executed in the target storage pool, the multiple storage pools are in a load-balanced state, triggering the first target node to execute the first backup task in the target storage pool.

[0015] Using the above method, the scheduling device selects a target storage pool for the first backup task with load balancing as the goal. The load on each storage pool of the first target node tends to be consistent, and the load of multiple storage pools in the backup cluster presents a "uniform" distribution or tends to a "uniform" distribution. This ensures that when the first target node executes the first backup task in the target storage pool, the target storage pool has sufficient computing power or storage space to support the execution of the target backup task, effectively guaranteeing the execution efficiency of the first backup task.

[0016] In one possible implementation, when the scheduling device selects a storage pool as the target storage pool from multiple storage pools in the first target node, it assumes that the first target node has executed the first backup task in each storage pool, and determines the load status of each storage pool after executing the first backup task; the scheduling device determines the target storage pool based on the load status of each storage pool after executing the first backup task.

[0017] Using the above method, the scheduling device considers the load status of each storage pool after executing the first backup task when determining the target storage pool. By utilizing the load status of each storage pool after executing the first backup task, it can more clearly determine whether it is suitable to execute the first backup task in each storage pool, and thus determine the target storage pool.

[0018] In one possible implementation, to determine the load status of each of the multiple nodes while executing the first backup task, the scheduling device can calculate a first load parameter for each node. The specific process is as follows: The scheduling device calculates the backup data volume of the first backup task, which is the amount of data to be backed up when executing the backup task; for any node, the scheduling device obtains the node's first load parameter based on the backup data volume and the node's information. The node's first load parameter characterizes the node's load status while executing the first backup task. The node's information includes part or all of the node's historical task information and real-time status information. The node's historical task information is the task information of the backup tasks already executed by the node, and the node's real-time status information characterizes the current operating status of the node.

[0019] Using the above method, when determining the load status of each node executing the first backup task, the scheduling device combines the backup task volume of the first backup task with some information about the node to obtain the first load parameter of the node. The first load parameter of the node obtained in this way can more accurately characterize the load status of the node executing the first backup task.

[0020] In one possible implementation, to determine the load status of each storage pool after executing the first backup task, the scheduling device can calculate a first load parameter for each storage pool. The specific process is as follows: The scheduling device calculates the amount of backup data required for the first backup task, which is the amount of data that needs to be backed up when executing the first backup task; for any storage pool, the scheduling device obtains the first load parameter of the storage pool based on the storage pool's information and the amount of backup data. The first load parameter of the storage pool characterizes the capacity status of the storage pool when the first backup task is executed within it; the storage pool's information includes some or all of the storage pool's historical task information and its real-time status information. The historical task information of the storage pool is the task information of backup tasks already executed within the storage pool, and the real-time status information characterizes the current capacity status of the storage pool.

[0021] Using the above method, when determining the load status of the first backup task in each storage pool, the scheduling device combines the backup task volume of the first backup task with some information about the storage pool to obtain the first load parameter of the storage pool. The first load parameter of the storage pool obtained in this way can more accurately characterize the load status of the first backup task in the storage pool.

[0022] In one possible implementation, the scheduling device retrieves a second backup task from the backup task queue, wherein the time of receiving the second backup task and the time of receiving the first backup task belong to the same time period; here, the reception time of the second backup task and the first backup task refers to the reception time of the second backup task and the first backup task before they were first added to the backup task queue. Of course, in some scenarios, after receiving the backup task, it is immediately added to the backup task queue. In this scenario, the reception time of the second backup task is close to the time when the second backup task and the second backup task were first added to the backup task queue, and the reception time of the first backup task is close to the time when the first backup task and the second backup task were first added to the backup task queue.

[0023] When determining the second target node to execute the second backup task, the scheduling device assumes that the first backup task has already been assigned to the first target node, and that the second target node satisfies the following conditions: multiple nodes are in a load-balanced state while the first target node is executing the first backup task and the second target node is executing the second backup task. After determining the second target node, the scheduling device triggers the second target node to execute the second backup task.

[0024] Using the above method, the receiving time of the second backup task and the receiving time of the first backup task are in the same time period, indicating that the first backup task and the second backup task may be backup tasks that need to be completed in parallel, or backup tasks that are related to each other. When the scheduling device determines the second target node for the second backup task, it will assume that the first target node is executing the first backup task to avoid conflicts or mutual influence between the execution of the second backup task and the first backup task, and to ensure the execution efficiency of the second backup task.

[0025] In one possible implementation, when the scheduling device determines the second target node from multiple nodes in the backup cluster, it can determine the load status of the first target node when executing the first backup task and the second backup task, as well as the load status of the nodes other than the first target node when executing the second backup task. The scheduling device determines the second target node based on the load status of the first target node when executing the first backup task and the second backup task, as well as the load status of the nodes other than the first target node when executing the second backup task.

[0026] By using the above method, when determining the second backup node, the scheduling device takes into account the load status of each of the multiple nodes executing the second backup task when the first target node executes the first backup task. This allows for a clearer understanding of the impact of the execution of the second backup task on the load of each node, thereby selecting a second target node that is more suitable for executing the second backup task.

[0027] In one possible implementation, the scheduling device obtains the node's historical task information from the node. The node's historical task information includes some or all of the following: the number of backup tasks assigned to the node and the number of backup tasks that the node has successfully executed.

[0028] Using the above method, the scheduling device interacts with the nodes to obtain the nodes' historical task information, which facilitates the subsequent estimation of the load status of the nodes when executing the first backup task.

[0029] In one possible implementation, the scheduling device obtains the real-time status information of the node from the node. The real-time status information of the node includes some or all of the following: the number of backup tasks currently being executed by the node, the maximum number of backup tasks that the node is allowed to execute in parallel, the occupied storage space within the node, the free storage space within the node, and the total size of the storage space within the node.

[0030] Using the above method, the scheduling device interacts with the nodes to obtain the real-time status information of the nodes, which facilitates the subsequent estimation of the load status of the nodes when performing the first backup task.

[0031] In one possible implementation, the scheduling device obtains historical task information of the storage pool from the first target node. The historical task information of the storage pool includes some or all of the following: the number of backup tasks that have been executed in the storage pool and the number of backup tasks that have been successfully executed in the storage pool.

[0032] Using the above method, the scheduling device interacts with the first target node to obtain historical task information of each storage pool, which facilitates subsequent estimation of the load status of the storage pool executing the first backup task.

[0033] In one possible implementation, the scheduling device obtains real-time status information of the storage pool from the first target node. The real-time status information of the storage pool includes some or all of the following: the number of backup tasks currently being executed in the storage pool, the maximum number of backup tasks that can be executed in parallel in the storage pool, the occupied storage space in the storage pool, the free storage space in the storage pool, and the total storage space of the storage pool.

[0034] Using the above method, the scheduling device interacts with the first target node to obtain real-time status information of each storage pool, which facilitates subsequent estimation of the load status of the storage pool executing the first backup task.

[0035] Secondly, this application also provides a scheduling device that has the function of implementing the behavior in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the scheduling device includes an adding module, an extracting module, and a determining module. These modules can perform the corresponding functions in the first aspect described above, as detailed in the method example, and will not be repeated here.

[0036] Thirdly, this application also provides a computing device including at least one processor and a memory, and may further include a communication interface. The at least one processor executes program instructions stored in the memory to perform the method provided in the first aspect or any possible implementation thereof. The memory is coupled to the processor and stores computer program instructions and data necessary for the backup task allocation process. The communication interface is used for communicating with other devices.

[0037] Fourthly, this application provides a computing device system including at least one computing device. Each computing device includes a memory and a processor. The processor of at least one computing device is used to access code in the memory to execute the methods provided in the first aspect or any possible implementation thereof.

[0038] Fifthly, this application provides a computer-readable storage medium that, when executed by a computing device, allows the computing device to execute the method provided in the first aspect or any possible implementation thereof, or to execute the method provided in the second aspect. The storage medium stores computer program instructions. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid-state drive (SSD).

[0039] Sixthly, this application provides a computing device program product, which includes computer program instructions. When executed by a computing device, the computing device performs the methods provided in the first aspect or any possible implementation thereof. The computer program product can be a software installation package. When the methods provided in the first aspect or any possible implementation thereof are required, the computer program product can be downloaded and executed on the computing device.

[0040] In a seventh aspect, this application also provides a computer chip connected to a memory, the chip being used to read and execute computer program instructions stored in the memory, and to execute the methods described in the first aspect and various possible implementations of the first aspect.

[0041] For the technical effects that can be achieved in aspects two through eight above, please refer to the description of the technical effects that can be achieved in the corresponding design schemes in aspect one above. This application will not repeat them here. Attached Figure Description

[0042] Figure 1 This application provides a schematic diagram of the structure of a data system according to an embodiment of the present application.

[0043] Figure 2 This application provides a schematic diagram of the structure of a backup cluster.

[0044] Figure 3 A schematic diagram illustrating the initial extraction and subsequent extraction of a backup task as provided in an embodiment of this application;

[0045] Figure 4 This is a schematic diagram of a backup task scheduling method provided in an embodiment of this application;

[0046] Figure 5 This is a schematic diagram of a machine learning model training method provided in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of a dispensing device method provided in an embodiment of this application;

[0048] Figures 7-8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0049] This application provides a backup task scheduling method, apparatus, and device. Some concepts involved in this application are described below.

[0050] (1) Types of backup tasks.

[0051] A backup task instructs the backup of specific data. The data targeted by this backup task (i.e., the backup object) is related to the user who triggered the backup task and / or the data organization within the backup system. Assuming the data in the backup system is organized as a file system, the data targeted by the backup task can be one or more files within that file system, or subdirectories and files within a directory within that file system. For the user, they can decide which file or subdirectories and files within a directory to back up based on their needs. Users can also pre-set backup policies, which describe the files or directories that need to be backed up, the backup frequency, or the backup method. For example, a backup policy could be: if file A in the file system is modified, perform a full backup of file A.

[0052] When a backup task is triggered, the required backup method is determined. Different backup methods result in different backup task types. Below are some common backup task types (which can also be understood as backup methods).

[0053] A full backup involves making an exact copy of the entire dataset. This copy can then be used as a baseline for subsequent backups, whether incremental or differential.

[0054] Incremental backup refers to backing up only the changes that have occurred since the last backup, rather than the entire data. An incremental backup must be based on the previous full or incremental backup. Multiple incremental backups can form a backup chain, where each incremental backup depends on the previous backup. If any backup in the backup chain is lost or corrupted, the latest data cannot be recovered.

[0055] Differential backup: This is a special type of incremental backup that backs up changes made since the last full backup. Unlike incremental backups, differential backups only need to be based on the previous full backup and do not depend on earlier backups. Differential backups offer faster recovery speeds compared to incremental backups, but require more disk space.

[0056] Permanent incremental backup is similar to incremental backup in its basic approach. Permanent incremental backup is performed only as an incremental backup after a full backup of the data.

[0057] A snapshot is an image of data at a specific point in time. The first snapshot is an exact copy of the data, and subsequent snapshots, based on the first snapshot, record the data blocks that have been changed or added during that period.

[0058] Log backup: refers to backing up logs, which record information such as data creation time and data modification operations.

[0059] (2) Nodes and storage pools.

[0060] In this embodiment, the node has the capability to execute backup tasks; in other words, the node has data processing capabilities and can parse backup tasks. For example, the node can determine which data needs to be backed up and which does not (e.g., differential backup or incremental backup tasks) from the data targeted by the backup task. Optionally, the node can perform data deduplication or data compression to reduce the amount of data that needs to be backed up. The node also has storage capabilities to store the backup data; that is, the node can provide storage space that can be used to store the backup data.

[0061] Logically, any given node has one or more storage pools. The storage space within these pools collectively constitutes the storage space available to that node. Each storage pool is built upon the memory accessible to the node; a storage pool can be understood as an abstract logical storage space. Setting up storage pools helps manage the node's storage space. The memory accessible to the node can be internal to the node or externally deployed memory that the node can access via a network.

[0062] When a node has multiple storage pools, the storage pools are isolated from each other. Each storage pool can independently carry out data storage tasks (such as the backup tasks involved in the embodiments of this application), that is, any storage pool can support the storage of data required in the data storage task (such as, in the embodiments of this application, any storage pool can be used to store backup data). In the embodiments of this application, carrying out backup tasks within a storage pool can be described as the node executing the backup task within that storage pool. The embodiments of this application do not particularly distinguish between these two descriptions.

[0063] The term "mutual isolation" means that there is no mutual influence between the various storage pools. When actually executing a data storage task, only one storage pool can be selected from the multiple storage pools to store the data that needs to be stored in the data storage task, without having to distribute the data across various storage pools.

[0064] (3) The load status of the nodes and the load status of the storage pool.

[0065] A node's load status describes the workload or pressure it bears. This application's embodiments involve two types of node load status: one is the estimated load status of the node when performing a backup task; this type of load status assumes the node's load status while performing the backup task, in other words, it's the node's load status after completing the backup task. Estimating this load status requires considering the node's load status before performing the backup task (i.e., the current node load status) and the impact of the backup task on the node's load status. In this application's embodiments, a first load parameter of the node is used to characterize this type of node load status.

[0066] Another type of node load status is the current load status of the node. This type of node load status is the actual load status of the node and can be determined through some information of the node (such as the node's historical task information and the node's real-time status information). In the embodiments of this application, the second load parameter of the node is used to characterize this type of node load status.

[0067] Similarly, the load status of a storage pool describes the workload or pressure it experiences. This application embodiment involves two types of storage pool load statuses. One type is the estimated load status of the storage pool when a backup task is being performed. This type of load status assumes the load status during the execution of the backup task, and its estimation requires considering the load status when the backup task is not being performed (i.e., the current load status of the storage pool) and the impact of the backup task on the storage pool's load status. In this application embodiment, a first load parameter of the storage pool is used to characterize this type of storage pool load status.

[0068] Another type of storage pool load status is the current load status of the storage pool. This type of storage pool load status is the actual load status of the storage pool, which can be determined through some information of the storage pool (such as historical task information and real-time status information). In the embodiments of this application, the second load parameter of the storage pool is used to characterize this type of storage pool load status.

[0069] (4) Load balancing status.

[0070] Taking node load balancing as an example, load balancing refers to distributing load across multiple nodes to ensure the overall performance of these nodes (such as overall throughput, data processing capacity, and service availability). Load balancing is applied in different scenarios, and the required overall performance across the nodes varies. For example, in traffic distribution scenarios, load balancing distributes traffic across different nodes according to a set rule or strategy to improve the overall throughput and network data processing capacity of the system comprised of these nodes. As another example, in cloud service scenarios, load balancing distributes cloud service-related tasks to one or more normally functioning nodes to ensure the high availability of the cloud service.

[0071] In this embodiment, load balancing can be used in backup scenarios. In these scenarios, load balancing refers to distributing backup tasks to a suitable node among multiple nodes to perform the task. The "suitable" aspect of a node suitable for performing the backup task is primarily reflected in some or all of the following:

[0072] Aspect 1: The node has the capability to execute the backup task and can guarantee the execution efficiency of the backup task.

[0073] In other words, the node has the computing power and storage space to support the execution of the backup task. After the backup task is assigned to the node, the node can start or complete the backup task in a short period of time.

[0074] Aspect 2: The node is a node in a backup cluster that provides backup services. If the node is assigned the backup task, the load of each node in the backup cluster is close (that is, the difference between the loads of each node is less than the preset value) or the load ratio of each node in the backup cluster is close (the load ratio of any node is equal to the ratio of the load of the node to the maximum load that the node can bear, that is, the load ratio of each node is close means that the difference between the load ratios of each node is less than the preset value).

[0075] In this embodiment, after a node (such as the target mentioned in this embodiment) is allocated to a backup task (such as the first backup task or the second backup task mentioned in this embodiment), the multiple nodes in the backup cluster are in a load-balanced state. Corresponding to the definition of load balancing in the backup business scenario, the multiple nodes being in a load-balanced state is reflected in some or all of the following aspects:

[0076] Firstly, the node assigned to the backup task can both execute the backup task and ensure its execution efficiency.

[0077] Second, the load or load ratio of multiple nodes in the backup cluster is similar.

[0078] Load balancing aims to distribute the load across different nodes more evenly, reducing the occurrence of some nodes being extremely busy while others are very idle. It's important to note that load balancing is a relative, not an absolute, balance. Statistically, compared to randomly selecting nodes to perform backup tasks, the solution provided in this invention results in a more balanced load across nodes.

[0079] For example, if there are node A (currently with 5 backup tasks), node B (currently with 30 backup tasks), and node C (currently with 100 backup tasks), then when a new backup task is assigned, it will be distributed to node A, thus achieving a balanced state among the three nodes.

[0080] For another example: If there are nodes A (currently with 20 backup tasks), B (currently with 21 backup tasks), and C (currently with 100 backup tasks), then when a new backup task is added, assigning the new task to either node A or node B is balanced, but assigning it to node C is unbalanced.

[0081] For ease of understanding, in the two examples above, we assume that the amount of data to be backed up by different backup tasks is the same.

[0082] When the performance of nodes is consistent or similar, one way to load balance is to distribute backup tasks evenly (or close to evenly) among the nodes.

[0083] When nodes have different performance levels, one way to load balance is to distribute backup tasks among the nodes according to their performance, allocating more backup tasks to high-performance nodes and fewer tasks to low-performance nodes.

[0084] For load balancing of multiple storage pools in a node, the definition of load balancing of storage pools is similar to that of load balancing of nodes. The only difference is that the load balancing targets different objects. Load balancing of storage pools targets the storage pools, meaning that the load of the storage pools tends to be consistent. Load balancing of nodes targets the node pools, meaning that the load of the nodes tends to be consistent. For details, please refer to the above explanation, which will not be repeated here.

[0085] (5) Historical task information of the node.

[0086] A node's historical tasks represent the backup tasks it has executed. These include two types of backup tasks: First, those that the node successfully executed; meaning the node successfully backed up the data specified in the task after acquiring it. For simplicity, this type of backup task will be referred to as a successfully executed backup task. Second, those that the node executed but failed; meaning the node did not complete the backup of the data specified in the task after acquiring it. There are many reasons why a node might fail to back up the data, such as node failure, corrupted data, or insufficient computing power. For simplicity, this type of backup task will be referred to as a failed backup task.

[0087] The historical task information of a node refers to the task information of the backup tasks that the node has executed. There are many types of historical task information for a node. The historical task information of a node can be the number of backup tasks that the node has executed, the type of backup tasks that the node has executed, the duration of the backup tasks that the node has executed, or the amount of backup data. Any information that can describe the backup tasks that the node has executed can be used as the node's historical task information.

[0088] The node's historical task information includes, but is not limited to: the number of backup tasks assigned to the node (i.e., the total number of backup tasks successfully executed by the node and the number of backup tasks that the node failed to execute), the number of backup tasks that the node has successfully executed, and the number of backup tasks that the node has failed to execute.

[0089] In this embodiment, the scheduling device can collect historical task information of nodes. This embodiment does not limit the method by which the scheduling device collects the historical task information of a node. For example, the scheduling device sends a first request message to a node, which requests the node's historical task information. After receiving the first request message, the node sends its historical task information back to the scheduling device. Alternatively, the node can proactively and periodically send its historical task information to the scheduling device.

[0090] (6) Real-time status information of nodes.

[0091] The real-time status information of a node represents its current operational status. Any information that can represent the current operational status of a node can be used as its real-time status information.

[0092] The real-time status information of a node includes, but is not limited to: the number of backup tasks currently being executed by the node, the maximum number of backup tasks that the node is allowed to execute in parallel, the occupied storage space within the node, the free storage space within the node, and the total size of the storage space within the node.

[0093] In this embodiment, the scheduling device can collect real-time status information of nodes. This embodiment does not limit the method by which the scheduling device collects the real-time status information of the nodes. For example, the scheduling device may send a second request message to a node, requesting to obtain the node's real-time status information. After receiving the second request message, the node may send its real-time status information back to the scheduling device. Alternatively, the node may proactively and periodically send its real-time status information to the scheduling device.

[0094] (7) Historical task information of the storage pool.

[0095] The historical tasks of a storage pool refer to backup tasks that have been executed within that storage pool. These backup tasks fall into two categories: First, those that were successfully executed within the storage pool; meaning the node successfully completed the backup task and stored the backed-up data in the storage pool. For simplicity, this type of backup task will be referred to as a successfully executed backup task within the storage pool. Second, those that were executed but failed; meaning the node executed the backup task but did not successfully back up the data it was intended to back up, and the data was not successfully stored in the storage pool. There are many reasons why backup data might not be successfully stored in the storage pool, such as: the memory on which the storage pool is based is damaged; the data targeted by the backup task is corrupted; or the storage space within the storage pool is insufficient to support storing the backup data. For simplicity, this type of backup task will be referred to as a failed backup task within the storage pool.

[0096] Historical task information of a storage pool refers to the task information of backup tasks that have been executed within that storage pool. There are many types of historical task information for a storage pool. It can be the number of backup tasks executed within the storage pool, the type of backup tasks executed within the storage pool, the duration of backup tasks executed within the storage pool, or the amount of backup data. Any information that can describe the backup tasks executed within the storage pool can be used as historical task information for the storage pool.

[0097] Historical task information for a storage pool includes, but is not limited to: the number of backup tasks executed within the storage pool (i.e., the total number of backup tasks successfully executed and those that failed within the storage pool), the number of backup tasks successfully executed within the storage pool, and the number of backup tasks that failed within the storage pool.

[0098] In this embodiment, the scheduling device can collect historical task information of the storage pool. This embodiment does not limit the method by which the scheduling device collects the historical task information of the storage pool. For example, the scheduling device may send a third request message to the node to which the storage pool belongs, requesting access to the historical task information of the storage pool. After receiving the third request message, the node may send the historical task information of the storage pool back to the scheduling device. Alternatively, the node to which the storage pool belongs may proactively and periodically send the historical task information of the storage pool to the scheduling device.

[0099] (8) Real-time status information of the storage pool.

[0100] Real-time status information of a storage pool represents its current capacity status. Any information that can represent the current capacity status of a storage pool can be used as its real-time status information.

[0101] The real-time status information of the storage pool includes, but is not limited to: the number of backup tasks currently being executed in the storage pool, the maximum number of backup tasks that can be executed in parallel in the storage pool, the occupied storage space in the storage pool, the free storage space in the storage pool, and the total size of the storage space in the storage pool.

[0102] In this embodiment, the scheduling device can collect real-time status information of the storage pool. This embodiment does not limit the method by which the scheduling device collects the real-time status information of the storage pool. For example, the scheduling device may send a fourth request message to the node to which the storage pool belongs, requesting the acquisition of the storage pool's real-time status information. After receiving the fourth request message, the node to which the storage pool belongs will feed back the real-time status information of the storage pool to the scheduling device. Alternatively, the node to which the storage pool belongs may proactively and periodically send the real-time status information of the storage pool to the scheduling device.

[0103] like Figure 1 The diagram shown is a schematic representation of a backup system architecture provided in an embodiment of this application. The backup system includes a scheduling device 110 and a backup cluster 120. The backup cluster 120 includes multiple nodes 121, each node 121 having data processing and storage capabilities. Each node 121 includes one or more storage pools.

[0104] For users, this backup system provides backup services, capable of backing up user data according to user needs. For example, the backup system can create backup tasks for data according to user requirements; these backup tasks are the tasks that back up the data. The backup of the target data can be of various types, such as full backup, incremental backup, differential backup, or snapshot. This application embodiment does not limit the specific type of backup task; the specific type of backup task can be set based on user needs or based on the current storage space status within the backup system or the operating status of node 121.

[0105] Within this backup system, there are multiple nodes 121 with data processing and storage capabilities. For any backup task to be executed, there are always one or more nodes 121 capable of performing the backup task. However, even if these one or more nodes 121 can execute the backup task, their efficiency in performing the backup task may differ. Therefore, it is necessary to determine which node 121 will execute the backup task, thus completing the backup task scheduling. In this backup system, the scheduling device 110 is used to implement the scheduling of backup tasks.

[0106] In this backup system, all backup tasks transmitted to the backup system will reach the scheduling device 110, and on the scheduling device 110 side, the backup tasks will be arranged in sequence to form a backup task queue.

[0107] Typically, the scheduling device 110 implements backup task scheduling in the following manner: extracting backup tasks from the backup task queue, determining the node 121 to execute the backup task, and assigning the backup task to the node 121.

[0108] In this embodiment of the application, the scheduling device 110 proposes a new method for scheduling backup tasks. For ease of explanation, this new method for scheduling backup tasks is referred to as delayed scheduling. "Delayed scheduling" means that the scheduling device 110 can postpone the scheduling time of the backup task.

[0109] The delayed scheduling process is as follows: A backup task is retrieved from the backup task queue. If this is the first time the backup task has been retrieved from the queue, it is not scheduled, and is instead added back to the queue. If the backup task is retrieved again (e.g., the second or third time), a node 121 is assigned to execute it. If node 121 includes multiple storage pools, the specific storage pool within node 121 that will host the backup task can be further identified. After determining node 121 and its storage pool, the backup task is assigned to node 121, triggering node 121 (within its storage pool) to execute the backup task.

[0110] It should be noted that the initial retrieval and subsequent retrieval of backup tasks (such as the first or second backup task mentioned in the embodiments of this application) are both triggered in response to the processing progress of tasks in the backup task queue, when it is the turn of the backup task to be processed and retrieved. This differs from the conventional backup task scheduling method, where the backup task is only retrieved from the backup task queue once, and then scheduled, node 121 is assigned to it, and node 121 is triggered to execute the backup task. However, in the delayed scheduling method of this embodiment, when the backup task is retrieved for the first time, it is not actually processed (i.e., node 121 is not assigned to it, nor is node 121 triggered to execute the backup task), but rather it is added back to the backup task queue. Only when the backup task is retrieved again according to the processing progress of the backup task queue is it actually processed, node 121 is assigned to it, and node 121 is triggered to execute the backup task. In other words, the timing of retrieving backup tasks in delayed scheduling is the same as the timing of retrieving a backup task when it is its turn to be processed in regular backup task scheduling. However, the timing of backup processing differs. In both delayed scheduling and regular backup task scheduling, a backup task is retrieved from the backup task queue and then processed again. In the latter, it is processed the first time it is retrieved from the backup task queue.

[0111] Regarding the timing of task retrieval, we will further explain using a First-In-First-Out (FIFO) queue as an example. In a FIFO queue, new tasks are added to the queue from the front, and tasks at the back of the queue are retrieved for execution. In this embodiment, the first and second retrievals of the backup task refer to the situations where the backup task is retrieved because it has been queued to the end of the backup task queue.

[0112] From the perspective of the object that carries (or executes) the backup task, there are two levels of scheduling in this delayed scheduling. The first level of scheduling is to determine the node 121 that will execute the backup task from multiple nodes 121. The second level of scheduling is to determine one storage pool that will carry the backup task from multiple storage pools of the node 121 determined in the first level of scheduling.

[0113] The scheduling device 110 can complete the first-level scheduling in any of the following ways:

[0114] The first method involves estimating the load status of each node 121 in executing the backup task, and determining the node 121 to execute the backup task based on the estimated load status of each node 121.

[0115] The scheduling device 110 calculates the first load parameters of each node 121, which characterizes the load status of the node 121 in performing the backup task. The scheduling device 110 determines one node 121 from the nodes 121 to perform the backup task based on the first load parameters of each node 121.

[0116] In this approach, for the backup task, the scheduling device 110 needs to calculate the amount of backup data required for the backup task, that is, the amount of data that needs to be backed up when executing the backup task. In this embodiment, the scheduling device 110 can use one or more machine learning models to calculate the amount of backup data required for the backup task, wherein any machine learning model is used to predict the amount of backup data required when executing the backup task, that is, to predict the amount of backup data required for the backup task.

[0117] For any node 121, the first load parameter of the node 121 characterizes the load status of the node 121 in performing the backup task. When the scheduling device 110 calculates the first load parameter of the node 121, it needs to combine three different pieces of information: the amount of backup data of the backup task and the information of the node 121. The information of the node 121 includes part or all of the historical task information of the node 121 and the real-time status information of the node 121.

[0118] The second method involves determining the current load status of each node 121 and then determining the node 121 to perform the backup task based on the current load status of each node 121.

[0119] The scheduling device 110 calculates the second load parameters of each node 121, and the second load parameters of any node 121 characterize the current load status of that node 121. The scheduling device 110 determines one node 121 from among the nodes 121 to perform the backup task based on the second load parameters of each node 121.

[0120] In this method, the scheduling device 110 does not need to calculate the amount of backup data for the backup task. For any node 121, when the scheduling device 110 calculates the second load parameter of the node 121, it needs to combine some or all of the information in the historical task information and real-time status information of the node 121.

[0121] If the node 121 determined by the scheduling device 110 to perform the backup task includes multiple storage pools, the scheduling device 110 can further perform a second-level scheduling, that is, determine the storage pool to carry the backup task from the multiple storage pools. In this embodiment, the scheduling device 110 can implement the second-level scheduling in any of the following ways:

[0122] The first approach is to estimate the load status of each storage pool on the backup task, and then determine the storage pool to carry the backup task based on the estimated load status of each storage pool.

[0123] The scheduling device 110 calculates a first load parameter for each storage pool, which characterizes the load status of the backup task performed within that storage pool. Based on the first load parameter of each storage pool, the scheduling device 110 determines one storage pool from among the storage pools to undertake the backup task.

[0124] In this approach, for the backup task, the scheduling device 110 needs to calculate the amount of backup data required for the backup task, that is, the amount of data that needs to be backed up when executing the backup task. In this embodiment, the scheduling device 110 can use one or more machine learning models to calculate the amount of backup data required for the backup task, wherein any machine learning model is used to predict the amount of data that needs to be backed up when executing the backup task.

[0125] For any storage pool, the first load parameter of the storage pool characterizes the load status of the storage pool that performs the backup task. When the scheduling device 110 calculates the first load parameter of the storage pool, it needs to combine the amount of backup data of the backup task and the information of the storage pool. The information of the storage pool includes part or all of the historical task information of the storage pool and the real-time status information of the storage pool.

[0126] The second method involves determining the current load status of each storage pool and then selecting the storage pool to undertake the backup task based on that load status.

[0127] The scheduling device 110 calculates the second load parameters of each storage pool, which characterizes the current load status of that storage pool. Based on the second load parameters of each storage pool, the scheduling device 110 determines one storage pool from among the storage pools to undertake the backup task.

[0128] In this method, the scheduling device 110 does not need to calculate the amount of backup data for the backup task. For any storage pool, when calculating the second load parameter of the storage pool, the scheduling device 110 needs to combine some or all of the historical task information and real-time status information of the storage pool.

[0129] If the node 121 determined by the scheduling device 110 to perform the backup task includes only one storage pool, the scheduling device 110 may not perform the second-level scheduling, and the storage pool in the node 121 is the storage pool that carries the backup task.

[0130] This application does not limit the specific form or deployment location of the scheduling device 110. The scheduling device 110 can be hardware, such as a computing device or a component (such as a processor) within that computing device. The scheduling device 110 can also be a software module, such as an application running on a processor, an application running in a virtual machine or container, etc. The scheduling device 110 can be deployed on node 121 within the backup cluster 120, or it can be deployed outside the backup cluster 120.

[0131] The backup cluster 120 includes multiple nodes 121, and each node 121 includes multiple storage pools. In this embodiment, node 121 has the ability to perform backup tasks and also has storage space to store backup data, which is supported by the storage pool in node 121.

[0132] Node 121 can be a computing device, including but not limited to personal computers, servers, mobile phones, tablets, or smart cars. This application embodiment does not limit the specific type of the backup cluster 120. The backup cluster 120 can be a distributed storage system including multiple nodes 121, or a computing cluster including multiple computing devices, such as a high-performance computing (HPC) cluster. The following describes a possible structure of the backup cluster 120.

[0133] like Figure 3 The diagram shown is a system architecture diagram of a distributed system provided in an embodiment of this application. The distributed system includes a server cluster. The server cluster includes one or more servers 130 ( Figure 2 The diagram shows three servers 130 (but is not limited to two servers 130), and these servers 130 can communicate with each other. A server 130 is a device that has both computing and storage capabilities, such as a server or desktop computer.

[0134] Server 130 can function as node 121 mentioned in this embodiment, performing backup tasks and transmitting historical task information and real-time status information of node 121 to scheduling device 110. In terms of hardware, such as... Figure 3 As shown, server 130 includes at least processor 132, memory 133, network card 134, and hard disk 105. Processor 132, memory 133, network card 134, and hard disk 105 are connected via a bus.

[0135] Processor 132 can be a central processing unit (CPU) or other specific integrated circuits. Processor 201 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 132 is the main processing core inside the storage system, capable of maintaining the storage system, such as updating data metadata, performing persistent data storage, and backing up data.

[0136] Memory 133 refers to the internal memory that directly exchanges data with the processor. Memory 133 can be dynamic random access memory (DRAM). Besides DRAM, memory 202 can also be other types of random access memory, such as static random access memory (SRAM). Additionally, memory 202 can also be read-only memory (ROM). For example, read-only memory can be programmable read-only memory (PROM) or erasable programmable read-only memory (EPROM). Memory 202 can also be flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0137] The network card 134 can be used to communicate with the scheduling device 110 and obtain backup tasks from the scheduling device 110. The network card 134 can also be used to communicate with servers 130 other than the server 130 where it is located.

[0138] The hard disk 105 is used to provide storage resources, such as storing data. It can be a disk or other types of storage media, such as a solid-state drive or a shingled magnetic recording hard disk. Multiple hard disks 105 can be used to form multiple storage pools.

[0139] Unlike other business scenarios, backup scenarios have certain unique characteristics. Because backup tasks typically involve operations such as deduplication and compression, the storage space and computing power consumed by such tasks are difficult to estimate. Only after the backup task is successfully executed can the storage space usage be definitively known. Therefore, the real-time status information of node 121 obtained during the execution of this backup task essentially represents its running state before or just before the task's execution. In other words, the real-time status information representing the running state of node 121 in the backup scenario has a certain time delay and poor real-time performance. If a conventional method is used to schedule backup tasks, upon first retrieving a backup task from the backup task queue, the node 121 to execute the backup task is immediately determined based on its real-time status information. However, since the real-time status information of node 121 represents its running state from a point prior to the current time, which is a considerable time interval, the current running state of node 121 may not reflect its true running state. Therefore, the node 121 ultimately determined to execute the backup task may not be the most suitable node for the task. Similarly, the real-time status information of the storage pool within node 121 also suffers from a certain delay, resulting in poor real-time performance.

[0140] This application proposes the concept of "delayed scheduling," which postpones the determination of the node 121 to execute the backup task; similarly, it also postpones the determination of the storage pool to carry the backup task. Because the determination of the node 121 and the storage pool to carry the backup task is delayed, the real-time status information of node 121 obtained when determining the node 121 to execute the backup task more closely reflects its operating state when the backup task is first extracted from the backup task queue. Therefore, the node 121 determined using delayed scheduling is more suitable for executing the backup task. Similarly, when determining the storage pool to carry the backup task, the real-time status information of the storage pool obtained more closely reflects its capacity state when the backup task is first extracted from the backup task queue. Therefore, the node 121 and storage pool determined using delayed scheduling are more suitable for executing the backup task. The accuracy of backup task scheduling is higher.

[0141] To more clearly illustrate the difference between the conventional method and delayed scheduling, see [link to relevant documentation]. Figure 2 Taking the real-time status information of node 121 as an example, assuming backup task A is extracted from the backup task queue at time T2, the real-time status information of node 121 obtained at time T2 has a certain time delay, representing the running state of node 121 at time T0 before time T2. In the conventional method of implementing backup task scheduling, the node 121 to execute backup task A is determined at this time using the real-time status information of node 121 obtained at time T2. However, delayed scheduling will not schedule backup task A at time T2, but will instead add the backup task back to the backup task queue. Assuming that backup task A is extracted from the backup task queue at time T3 after time T2, this is the second time backup task A has been extracted from the backup task queue. The real-time status information of node 121 obtained at time T3 has a certain time delay, representing the running state of node 121 at time T1 before time T3. Compared to the real-time status information of node 121 obtained at time T2, the real-time status information of node 121 obtained at time T3 is closer to the running state of node 121 when backup task A was first extracted, which is the running state of node 121 at time T2. It is evident that delayed scheduling allows the obtained real-time status information of node 121 to more closely approximate the running state of node 121 when backup task A was first extracted, thus more accurately determining the node 121 executing backup task A.

[0142] like Figure 4 As shown, a backup task scheduling method provided in this application embodiment will be described. This method describes the scheduling process of two backup tasks. For ease of distinction, these two backup tasks are referred to as the first backup task and the second backup task, respectively. The first backup task instructs the backup of a first target data. The node 121 determined for executing the first backup task is referred to as the first target node. The node 121 determined for executing the second backup task is referred to as the second target node.

[0143] Step 400: The scheduling device 110 adds the received first backup task and second backup task to the backup task queue.

[0144] In a backup system, user-triggered backup tasks are added to a backup task queue in a certain order. This application embodiment does not limit the arrangement of backup tasks in the backup task queue. For example, the backup task queue can be arranged according to the order in which they are received. Alternatively, the backup task queue can be arranged according to the importance of the backup tasks (e.g., the importance of the backup task can be determined based on the user's level or the importance of the data targeted by the backup task).

[0145] When the backup system receives the first and second backup tasks triggered by the user, the first and second backup tasks will be added to the backup task queue.

[0146] In this application embodiment, the concept of a "batch task" is introduced, which describes one or more backup tasks received within the same time period. In other words, if one or more backup tasks are received within the same time period, these backup tasks belong to the same batch task. When a backup task is first received, the reception time of that backup task can be recorded to determine whether different backup tasks belong to the same batch task.

[0147] In practical applications, the receiving time can be divided according to the time granularity to form multiple time periods. One or more backup tasks received in each time period constitute a batch task.

[0148] Step 401: The scheduling device 110 first extracts the first backup task from the backup task queue and then extracts the second backup task from the backup task queue.

[0149] Step 402: The scheduling device 110 temporarily suspends processing the first backup task and the second backup task, and adds the first backup task and the second backup task back to the backup task queue.

[0150] Step 403: The scheduling device 110 retrieves the first backup task from the backup task queue again, and then retrieves the second backup task from the backup task queue again.

[0151] Both steps 402 and 403 involve retrieving the first backup task and the second backup task from the backup task queue. The difference is that in step 401, the first backup task and the second backup task are retrieved for the first time, while in step 403, the first backup task and the second backup task are retrieved for the second time.

[0152] To determine whether a backup task is being retrieved from the backup task queue for the first time, a retrieval identifier can be set for that task. This identifier indicates whether the backup task is being added to the queue for the first time. When a backup task is received, its retrieval identifier is set to a first value before it is added to the queue. If the backup task needs to be added to the queue again later, if the retrieval identifier is currently set to a first value, it is changed to a second value before the task is added to the queue. In other words, a first value indicates that the backup task is being added to the queue for the first time, while a second value indicates that the task is not being added to the queue for the first time.

[0153] When executing step 401, the scheduling device 110 determines that the first backup task is being extracted for the first time by determining that the extraction identifier of the first backup task is a first value. Similarly, when the scheduling device 110 determines that the second backup task is being extracted for the first time by determining that the extraction identifier of the second backup task is a first value, the second backup task is being extracted for the first time. When executing step 402, before adding the first and second backup tasks back to the backup task queue, the extraction identifier of the first backup task is changed from the first value to the second value, and the extraction identifier of the second backup task is also changed from the first value to the second value. Then, the first and second backup tasks are added to the backup task queue.

[0154] To determine whether a backup task is being retrieved from the backup task queue for the first time, a count identifier can be set for that task. This count identifier represents the number of times the backup task has been added to the queue. When a backup task is received, its count identifier is set to an initial value before it is added to the queue for the first time. Then, the backup task with the count identifier set is added to the queue. Each subsequent addition of the backup task to the queue increments the count identifier by one, and then the backup task with the changed count identifier value is added to the queue. In other words, the count identifier represents the number of times the backup task has been added to the queue. When the count identifier is at its initial value, it indicates that the backup task is being added to the queue for the first time. When the count identifier is not at its initial value, it indicates that the backup task is not being added to the queue for the first time. The total number of times the backup task has been added to the queue is equal to the difference between the count identifier value and the initial value, plus one.

[0155] When executing step 401, the scheduling device 110 determines that the first backup task is being retrieved for the first time by setting the count identifier of the first backup task to an initial value. Similarly, when the scheduling device 110 determines that the second backup task is being retrieved for the first time by setting the count identifier of the second backup task to an initial value, the second backup task is retrieved for the first time. When executing step 402, before adding the first and second backup tasks back to the backup task queue, the count identifier of the first backup task is incremented by one, and the count identifier of the second backup task is incremented by one. Then, the first and second backup tasks are added to the backup task queue.

[0156] In this embodiment of the application, the scheduling device 110 only needs to determine that the first backup task and the second backup task are not added to the backup task queue for the first time (e.g., the extraction identifier is the second value or the number identifier is not the initial value) to execute step 402.

[0157] It should be noted that, taking the first backup task as an example, the first target data involved in the first backup task refers to the object targeted by the first backup task, not the data that needs to be stored when executing the first backup task. For example, when the type of the first backup task is differential backup or incremental backup, the data that needs to be stored when executing the first backup task is a portion of the target data.

[0158] The following section will take the scheduling device 110's scheduling of the first backup task as an example, illustrating the process by which the scheduling device 110 determines the first target node and the storage pool within that first target node for the first backup task. The scheduling process for the second backup task will also be explained below.

[0159] Step 404: The scheduling device 110 selects the method for determining the first target node.

[0160] In this embodiment of the application, the scheduling device 110 provides two methods for determining the first target node, namely:

[0161] The first method involves estimating the load status of each node 121 in executing the backup task, and determining the first target node based on the estimated load status of each node 121.

[0162] The second method involves determining the current load status of each node 121 and then determining the first target node based on the current load status of each node 121.

[0163] The scheduling device 110 can select one of the two methods according to the user's needs. For example, the scheduling device 110 provides an interface for the user to select the method for determining the first target node. This interface can be understood as a function provided to the user. The embodiments of this application do not limit the specific form of the interface. The interface can be a visual interface provided to the user, which provides options for the two methods. The user can select one of the options according to their own needs. The interface can also be a message format for configuring the method for determining the first target node. The user can send a message conforming to the message format to the scheduling device 110 through a computing device deployed on the user side. This message is used to indicate the method for determining the first target node selected by the user.

[0164] The scheduling device 110 can also select one of the two methods based on a default selection strategy. This selection strategy is a pre-configured strategy set for selecting the method of determining the first target node. For example, in the first method, the scheduling device 110 needs to use a machine learning model. The selection strategy can be described as follows: if the machine learning model has been established, the first method is selected; if the machine information model has not been established, the second method is executed.

[0165] In this embodiment, the machine learning model can be trained based on the task information of the backup tasks already completed by each node 121. Therefore, the establishment of the machine learning model has a certain delay; that is, the machine learning model can only be trained when the task information of the backup tasks already completed by each node 121 reaches a certain amount. After training is completed, the machine learning model is established. The training process of the machine learning model can be found in the description below, and will not be detailed here.

[0166] Regardless of the method used to determine the first target node, the first target node determined by the allocation device from the backup cluster 120 must satisfy the following condition: when the first target node performs the first backup task, the multiple nodes 121 in the backup cluster 120 achieve a load-balanced state. For an explanation of load balancing and load-balanced states in backup service scenarios, please refer to the foregoing content, which will not be repeated here. This application embodiment does not limit the method by which the allocation device determines the first target node from the backup cluster 120; the aforementioned first and second methods are merely examples.

[0167] After determining the method for identifying the first target node, the scheduling device 110 performs first-level scheduling, the process of which can be seen in steps 405-406, or steps 409-410. Steps 405-406 involve determining the first target node using the first method, while steps 409-410 involve determining the first target node using the second method.

[0168] Step 405: If the scheduling device 110 selects to use the first method to determine the first target node, the scheduling device 110 calculates the first load parameters of each node 121.

[0169] Taking one of the nodes 121 as an example, the process of the scheduling device 110 calculating the first load parameter of node 121 is explained as follows:

[0170] Step 1: Calculate the amount of backup data for the first backup task.

[0171] The scheduling device 110 is equipped with a machine learning model, which is used to predict the amount of data to be backed up when executing the first backup task. The machine learning model can be a random forest-based machine learning model, an adaptive boosting machine learning model, a gradient boosting machine learning model, a deep neural network (DNN) model based on multi-head attention, etc.

[0172] For the machine learning model, the input is the information of the first backup task. The specific content of this information is related to the specific training process of the machine learning model. For example, the information of the first backup task includes: the type of the first backup task, the amount of backup data from historical backup tasks for the first target data, and the execution time of historical backup tasks for the first target data. The output of the machine learning model is the predicted amount of data that needs to be backed up when executing the first backup task.

[0173] The historical backup task for the first target data refers to the executed backup task that instructs the backup of the first target data. The amount of backup data for the historical backup task for the target data is the amount of data backed up when the historical backup task is executed. The execution time of the historical backup task for the first target data is the time from when the historical backup task is assigned to node 121 to when it is completed.

[0174] This application embodiment does not limit the number of historical backup tasks for the first target data involved in the information of the first backup task. The number of historical backup tasks for the first target data involved in the information of the backup task is related to the training process of the machine learning model. In other words, the scale of the input data of the machine learning model is determined during the training process of the machine learning model.

[0175] The types of first backup tasks include, but are not limited to: full backup, incremental backup, differential backup, permanent incremental backup, snapshot, log backup, etc.

[0176] The execution process of step 1 is as follows:

[0177] Step 1: The scheduling device 110 inputs the information of the first backup task into the machine learning model and obtains the output of each machine learning model. The output of the machine learning model is the amount of data that needs to be backed up when the first backup task is executed, which is the predicted amount of backup data for the first backup task.

[0178] After the scheduling device 110 obtains the output of the machine learning model, it can directly execute step ②, or it can first adjust the output of the machine learning model and then use the processed output of the machine learning model to execute step ②.

[0179] If the amount of backup data predicted by the machine learning model for a backup task differs significantly from the actual amount of backup data for that task (i.e., the difference is less than the preset value), then the historical prediction accuracy of the machine learning model is low, and the output of the machine learning model needs to be adjusted.

[0180] The scheduling device 110 adjusts the output of the machine learning model based on the historical prediction accuracy of the machine learning model.

[0181] If the machine learning model predicts that the total amount of backup data for each backup task is greater than the actual amount of backup data for each backup task, then the output of the machine learning model can be reduced, that is, the value of the backup data amount in the output of the machine learning model can be reduced. The reduced output of the machine learning model is the final output of the machine learning model.

[0182] If the backup data volume predicted by the machine learning model for each backup task is always less than the actual backup data volume for each backup task, then the output of the machine learning model can be improved, that is, the value of the backup data volume in the output of the machine learning model can be increased, and the increased output of the machine learning model can be used as the final output of the machine learning model.

[0183] Here's a way to adjust the output of a machine learning model:

[0184] If the number of backup tasks predicted by the machine learning model in the past is greater than N, based on the backup data volume of the N backup tasks predicted by the machine learning model and the actual backup data volume of the N backup tasks, if there are K backup tasks among the N backup tasks predicted by the machine learning model, and the predicted backup data volume of each of the K backup tasks is greater than the actual backup data volume of each of the K backup tasks, and the difference between the predicted backup data volume of each of the K backup tasks and the actual backup data volume of each of the K backup tasks is greater than a set value, then the backup data volume output by the machine learning model is reduced by the set value. If there are K backup tasks among the N backup tasks predicted by the machine learning model, and the predicted backup data volume of each of the K backup tasks is less than the actual backup data volume of each of the K backup tasks, and the difference between the predicted backup data volume of each of the K backup tasks and the actual backup data volume of each of the K backup tasks is less than a set value, then the backup data volume output by the machine learning model is increased by the set value.

[0185] Step 2: The scheduling device 110 determines the amount of backup data for the first backup task based on the output of the machine learning model. For example, the scheduling device 110 uses the average of the outputs of multiple machine learning models as the amount of backup data for the first backup task.

[0186] The scheduling device 110 can directly use the output of a machine learning model as the backup data volume of the first backup task; assuming that the output of the machine learning model was not adjusted in step ①, the scheduling device 110 can also adjust the output of the machine learning model and use the adjusted output of the machine learning model as the backup data volume of the first backup task. The adjustment method can be referred to the foregoing description, and will not be repeated here.

[0187] Unlike other business scenarios, backup scenarios have certain unique characteristics. Because backup tasks typically involve operations such as deduplication and compression, the storage space and computing power consumed by these tasks are difficult to estimate. Only after successful execution can the storage space required for the backup task be clearly determined. In this embodiment, a machine learning model is used to estimate the backup data volume of the first backup task. This ensures that the final estimated backup data volume is more accurate and closer to reality. The first load parameter of node 121 calculated using the estimated backup data volume from the machine learning model will also be more accurate. This further ensures that the backup task can be allocated to an appropriate node.

[0188] Step 2: Obtain the first load parameters of node 121 based on the information of node 121 and the amount of backup data of the first backup task. The information of node 121 includes part or all of the historical task information of node 121 and the real-time status information of node 121.

[0189] This application embodiment does not limit the way the scheduling device 110 performs step 2. The following describes a method for obtaining the first load parameter of node 121 based on the historical task information of node 121, the real-time status information of node 121, and the backup data volume of the first backup task.

[0190] First, define three parameters for node 121: the health parameter NHealthy, the capacity parameter NUsedCapPer`, and the parallelism parameter NPar.

[0191] The health parameter NHealthy of node 121 describes the probability that node 121 will successfully execute the backup task. The health parameter NHealthy of node 121 satisfies:

[0192] The health parameter NHealthy for node 121 is calculated as: (Number of backup tasks successfully executed on node 121, NoSJ) / (Number of backup tasks executed on node 121, NoHJ).

[0193] The parallelism parameter NPar of node 121 describes the state of parallel execution of backup tasks on node 121. The parallelism parameter NPar of node 121 satisfies:

[0194] The parallelism parameter NPar for node 121 is equal to the number of backup tasks currently being executed by node 121, NCorun, and the maximum number of backup tasks that node 121 is allowed to execute in parallel, NSpec.

[0195] The capacity parameter NUsedCapPer of node 121 describes the capacity status of node 121 after performing a backup task. The capacity parameter NUsedCapPer of node 121 satisfies:

[0196] The capacity parameter NUsedCapPer` of node 121 = (UsedCap of the storage space already occupied in node 121 + the backup amount of the first backup task) / the total size of the storage space in node 121, TolCap.

[0197] The scheduling device 110 can calculate the first load parameter of node 121 based on the health parameter NHealthy, the capacity parameter NUsedCapPer`, and the parallelism parameter NPar of node 121.

[0198] For example, the first load parameter W1 of node 121 satisfies:

[0199] W1 = (α - NPar) * (α - NUsedCapPer`) * NHealthy, where α is a positive integer greater than 1.

[0200] The first load parameter of node 121 that satisfies the above formula is such that the larger the first load parameter of node 121, the more suitable node 121 is as the first target node. The value range of W1 is 0≤W1≤α. In this calculation method, the larger the W1 of node 121, the more suitable node 121 is to perform the first backup task.

[0201] In practical scenarios, the importance of the health parameter NHealthy of node 121 in calculating the first load parameter of node 121 can be appropriately increased, such as by increasing the value of NHealthy in the formula satisfied by the first load parameter W1 of node 121. The higher the health parameter of node 121, the more historical backup tasks that node 121 has successfully executed, and the greater the probability that node 121 will successfully execute the first backup task.

[0202] For example, the formula satisfied by the first load parameter W1 of the aforementioned node 121 can be transformed into:

[0203] W1 = (α - NPar) * (α - NUsedCapPer`) * (1 + NHealthy), where α is a positive integer greater than 1.

[0204] Since the execution status of backup tasks on each node 121 or the size of unused storage space on each node 121 may differ, for some "special" nodes 121, the formula that the first load parameter of that node 121 satisfies can be appropriately adjusted.

[0205] Special Case 1: The unused storage space on node 121 is insufficient to support the backup task, that is, the unused storage space on node 121 is not greater than the amount of backup data required for the backup task.

[0206] For this node 121, the first load parameter W1 of node 121 satisfies:

[0207] W 1=(α-NPar)*(α-epsilon / (-ResCap / TolCap)*NHealthy

[0208] Where α is a positive integer greater than 1, and ResCap is the remaining storage space of node 121, which is the remaining storage space of node 121 after executing the first backup task. Since the unused storage space of node 121 is insufficient to support the first backup task, the remaining storage space ResCap of node 121 is a negative value.

[0209] The remaining storage space ResCap of node 121 satisfies:

[0210] Remaining storage space ResCap of node 121 = Total storage space in node 121 (TolCap - Used storage space in node 121 + Backup amount of backup tasks)

[0211] In a real-world scenario, if it is necessary to increase the importance of the health parameter NHealthy of node 121 in the first load parameter of computing node 121, the value of the health parameter of node 121 can also be increased.

[0212] For example, the first load parameter W1 of node 121 satisfies:

[0213] W 1=(α-NPar)*(α-epsilon / (-ResCap / TolCap)*(1+NHealthy)

[0214] Special Case 2: Node 121 frequently fails to perform backup tasks, such as when the health parameters of node 121 are less than the minimum threshold.

[0215] For this node 121, the first load parameter W1 of node 121 satisfies:

[0216] W 1=(α-NPar)*(1+epsilon / NoHJ)*NHealthy

[0217] Where α is a positive integer greater than 1, and epsilon is a constant.

[0218] Considering that node 121 frequently fails when performing backup tasks, it is preferable to select node 121 with a smaller number of backup tasks already performed. Therefore, when calculating the first load parameter W1 of node 121, the number of backup tasks already performed by node 121 needs to be taken into account.

[0219] In a real-world scenario, if it is necessary to increase the importance of the health parameter NHealthy of node 121 in the first load parameter of computing node 121, the value of the health parameter of node 121 can also be increased.

[0220] For example, the first load parameter W1 of node 121 satisfies:

[0221] W 1=(α-NPar)*(1+epsilon / NoHJ)*(1+NHealthy)

[0222] Special Case 3: Node 121 has not yet executed any backup tasks.

[0223] For this node 121, the first load parameter W1 of node 121 satisfies:

[0224] W 1=(α-NPar)*(epsilon / TolCap)*NHealthy

[0225] Where α is a positive integer greater than 1, and epsilon is a constant.

[0226] Considering that node 121 is not performing a backup task, in this case, node 121 with larger storage space is preferred. Therefore, when calculating the first load parameter W1 of node 121, the total size of the storage space of node 121 needs to be taken into account.

[0227] In a real-world scenario, if it is necessary to increase the importance of the health parameter NHealthy of node 121 in the first load parameter of computing node 121, the value of the health parameter of node 121 can also be increased.

[0228] For example, the first load parameter W1 of node 121 satisfies:

[0229] W 1=(α-NPar)*(epsilon / TolCap)*(1+NHealthy)

[0230] It should be noted that when using the above calculation method, there is a possibility that the calculated first load parameter may be abnormal. An abnormality refers to the calculated first load parameter exceeding the value range of W1. In this case, the first load parameter of node 121 can be set to the boundary value of the value range of W1. Furthermore, the formulas satisfied by the health parameter NHealthy, capacity parameter NUsedCapPer`, and parallelism parameter NPar of node 121 are merely examples. This application does not limit the specific calculation method or formulas satisfied by the health parameter NHealthy, capacity parameter NUsedCapPer`, and parallelism parameter NPar of node 121.

[0231] Step 406: The scheduling device 110 determines the first target node from each node 121 based on the first load parameter of each node 121. Combining the calculation method of the first load parameter of each node 121, the first target node selected by the scheduling device 110 can be the node 121 with the largest or smallest first load parameter among the nodes 121.

[0232] When determining the target node 121, the load status of each node 121 after executing the backup task is taken into account (which can be characterized by the first load parameter of node 121). By comparing the first load parameters of each node 121, the load status of each node 121 after executing the backup task can be clearly understood. Then, the node 121 with the better load status is determined from among the nodes 121 to execute the backup task. That is, the determined node 121 is more suitable to execute the backup task, thereby improving the execution efficiency of the backup task.

[0233] After determining the first target node, if the first target node includes multiple storage pools, it is necessary to further determine the storage pool that will carry out the first backup task from among these multiple storage pools, that is, to perform the second-level scheduling. For ease of explanation, the storage pool that will carry out the first backup task will be referred to as the target storage pool.

[0234] In this embodiment of the application, the scheduling device 110 provides two methods for determining the target storage pool, namely:

[0235] Method 1: Estimate the load status of each storage pool on the backup task, and determine the target storage pool based on the estimated load status of each storage pool.

[0236] Method 2: Determine the current load status of each storage pool, and then determine the target storage pool based on the current load status of each storage pool.

[0237] Since the first method of determining the first target node was used, the scheduling device 110 can use the first method of determining the target storage pool. For details, please refer to steps 405 to 407.

[0238] Step 407: The scheduling device 110 calculates the first load parameters for each storage pool.

[0239] Taking one of the storage pools as an example, the process of the scheduling device 110 calculating the first load parameter of the storage pool is explained as follows:

[0240] Step 1) Obtain the backup data volume of the first backup task. The backup data volume of the first backup task only needs to be the same as the backup data volume of the first backup task calculated when the first target node was determined.

[0241] Step 2) Obtain the first load parameters of the storage pool based on the information of the storage pool and the amount of backup data of the first backup task. The information of the storage pool includes some or all of the historical task information and real-time status information of the storage pool.

[0242] This application does not limit the way the scheduling device 110 executes step 2). The following describes a method for obtaining the first load parameter of the storage pool based on the historical task information of the storage pool, the real-time status information of the storage pool, and the backup data volume of the first backup task.

[0243] First, define three parameters for the storage pool: the health parameter SPHealthy, the capacity parameter SPUsedCapPer`, and the parallelism parameter SPPar.

[0244] The storage pool health parameter SPHealthy describes the probability of successfully executing a backup task within that storage pool. The storage pool health parameter SPHealthy satisfies the following conditions:

[0245] The storage pool health parameter SPHealthy = (Number of backup tasks successfully hosted on the storage pool, SPSJ) / (Number of backup tasks hosted on the storage pool, SPHJ).

[0246] The parallelism parameter SPPar of a storage pool describes the state of backup tasks executed in parallel within that storage pool. The parallelism parameter SPPar of a storage pool satisfies the following:

[0247] The parallelism parameter SPPar of the storage pool = the number of backup tasks currently being carried by the storage pool NCorun / the maximum number of backup tasks that the storage pool is allowed to carry in parallel NSpec.

[0248] The storage pool capacity parameter SPUsedCapPer` describes the capacity status after the first backup task is performed within the storage pool, and satisfies the following:

[0249] The storage pool capacity parameter SPUsedCapPer` = (Occupied storage space in the storage pool SPUsedCap + Backup amount of the first backup task) / Total storage space in the storage pool SPTolCap.

[0250] The scheduling device 110 can calculate the first load parameter of the storage pool based on the storage pool's health parameter SPHealthy, storage pool capacity parameter SPUsedCapPer`, and capacity parallelism parameter SPPar.

[0251] For example, the first load parameter w1 of the storage pool satisfies:

[0252] w1 = (α - SPPar) * (α - SPUsedCapPer`) * SPHealthy, where α is a positive integer greater than 1.

[0253] The first load parameter of the storage pool that satisfies the above formula is considered to be more suitable as the target storage pool if the first load parameter is larger. The value range of w1 is 0≤w1≤α. In this calculation method, the larger the w1 of the storage pool, the more suitable the storage pool is to carry out the first backup task.

[0254] In practical scenarios, the importance of the storage pool's health parameter SPHealthy in calculating the storage pool's second load parameter can be appropriately increased, such as by increasing the value of SPHealthy in the formula satisfied by the storage pool's second load parameter w1. A higher storage pool health parameter indicates more successfully executed historical backup tasks within that storage pool, and a greater probability of successfully executing the first backup task within that storage pool.

[0255] For example, the formula satisfied by the first load parameter w1 of the aforementioned storage pool can be transformed into:

[0256] w1=(α-SPPar)*(α-SPUsedCapPer`)*(1+SPHealthy)

[0257] Since the execution status of backup tasks in each storage pool or the size of unused storage space in each storage pool may differ, for some "special" storage pools, the formula that the first load parameter of the storage pool satisfies can be appropriately adjusted.

[0258] Special Case 1: The unused storage space in the current storage pool is insufficient to support the backup task, that is, the unused storage space in the storage pool is not greater than the amount of backup data required for the backup task.

[0259] For this type of storage pool, the first load parameter w1 of the storage pool satisfies:

[0260] w1=(α-SPPar)*(α-epsilon / (-SPResCap / SPTolCap)*SPHealthy

[0261] Where α is a positive integer greater than 1, SPResCap is the remaining storage space of the storage pool, which is the remaining storage space of the storage pool after the first backup task is carried out. Since the unoccupied storage space of the current storage pool is insufficient to support the first backup task, the remaining storage space SPResCap of the storage pool is a negative value.

[0262] The remaining storage space SPResCap in the storage pool satisfies:

[0263] The remaining storage space of the storage pool SPResCap = the total size of the storage space in the storage pool SPTolCap - the storage space already occupied in the storage pool SPPUsedCap + the backup amount of the backup task.

[0264] In real-world scenarios, if it is necessary to increase the importance of the storage pool's health parameter SPHealthy in calculating the first load parameter of the storage pool, the value of the storage pool's health parameter can also be increased.

[0265] For example, the first load parameter w1 of the storage pool satisfies:

[0266] w1=(α-SPPar)*(α-epsilon / (-SPResCap / TolCap)*(1+SPHealthy)

[0267] Special Case 2: The storage pool frequently fails to perform backup tasks, such as when the health parameters of the storage pool are below the minimum threshold.

[0268] For this type of storage pool, the first load parameter w1 of the storage pool satisfies:

[0269] w1=(α-SPPar)*(1+epsilon / SPHJ)*SPHealthy

[0270] Where α is a positive integer greater than 1, and epsilon is a constant.

[0271] Considering that storage pools often fail when performing backup tasks, it is preferable to choose a storage pool with a smaller number of backup tasks already hosted in that situation. Therefore, when calculating the first load parameter w1 of a storage pool, the number of backup tasks already hosted by that storage pool needs to be taken into account.

[0272] In real-world scenarios, if it is necessary to increase the importance of the storage pool's health parameter SPHealthy in calculating the first load parameter of the storage pool, the value of the storage pool's health parameter can also be increased.

[0273] For example, the first load parameter w1 of the storage pool satisfies:

[0274] w1=(α-SPPar)*(1+epsilon / SPHJ)*(1+SPHealthy)

[0275] Special Case 3: The storage pool has not yet executed any backup tasks.

[0276] For this type of storage pool, the first load parameter w1 of the storage pool satisfies:

[0277] w1=(α-SPPar)*(epsilon / SPTolCap)*SPHealthy

[0278] Where α is a positive integer greater than 1, and epsilon is a constant.

[0279] Considering the scenario where the storage pool is not handling backup tasks, a storage pool with a larger storage capacity is preferred. Therefore, when calculating the first load parameter w1 of the storage pool, the total size of the storage pool's storage capacity needs to be taken into account.

[0280] In real-world scenarios, if it is necessary to increase the importance of the storage pool's health parameter SPHealthy in calculating the first load parameter of the storage pool, the value of the storage pool's health parameter can also be increased.

[0281] For example, the first load parameter w1 of the storage pool satisfies:

[0282] w1=(α-SPPar)*(epsilon / SPTolCap)*(1+SPHealthy)

[0283] It should be noted that when using the above calculation method, there is a possibility that the calculated first load parameter may be abnormal. An abnormality refers to the calculated first load parameter exceeding the range of w1. In this case, the first load parameter of the storage pool can be set to the boundary value of the range of w1. Furthermore, the formulas satisfied by the aforementioned health parameter SPHealthy, capacity parameter SPUsedCapPer`, and parallelism parameter SPPar of the storage pool are merely examples. This application does not limit the specific calculation method or formulas satisfied by the health parameter SPHealthy, capacity parameter SPUsedCapPer`, and parallelism parameter SPPar of the storage pool.

[0284] Step 408: The scheduling device 110 selects a storage pool as the target storage pool from each storage pool according to the first load parameter of each storage pool. Based on the calculation method of the first load parameter of each storage pool, the target storage pool selected by the scheduling device 110 can be the storage pool with the largest or smallest first load parameter among the storage pools.

[0285] Regarding the scheduling method of the second backup task, if the second backup task and the first backup task are not batch tasks, the scheduling method of the second backup task is the same as that of the first backup task. For details, please refer to steps 404 to 408 above, which will not be repeated here. If the second backup task and the first backup task are batch tasks, that is, the receiving time of the second backup task and the receiving time of the first backup task belong to the same time period, the scheduling method of the second backup task is similar to that of the first backup task. The difference is that when scheduling the second backup task, when calculating the first load parameters of each node 121, it is necessary to assume that the first backup task has been allocated to the first target node. Therefore, when calculating the first load parameters of the first target node, the capacity parameters of the first target node involved satisfy:

[0286] The capacity parameter of the first target node = (the storage space already occupied in the first target node + the backup amount of the first backup task + the backup amount of the second backup task) / the total size of the storage space in node 121.

[0287] The capacity parameters of any other node 121 satisfy:

[0288] The capacity parameter of node 121 = (the storage space already occupied in node 121 + the backup amount of the second backup task) / the total size of the storage space in node 121.

[0289] Subsequently, based on the first load parameters of each node 121, the second target node for executing the second backup task is determined. If the first target node and the second target node are the same node 121, then the scheduling device 110 will further perform a second-level scheduling to determine the storage pool that will carry the second backup task from the first target node. Similarly, when calculating the first load parameters of each storage pool, it is assumed that the first backup task has been allocated to the target storage pool. Therefore, when calculating the first load parameters of the target storage pool, the capacity parameters of the target storage pool involved must satisfy:

[0290] The capacity parameter of the target storage pool = (the storage space already occupied in the target storage pool + the backup amount of the first backup task + the backup amount of the second backup task) / the total size of the storage space in the target storage pool.

[0291] The capacity parameters of any other storage pool satisfy:

[0292] The capacity parameter of the storage pool = (the storage space already occupied in the storage pool + the backup amount of the second backup task) / the total size of the storage space in the storage pool.

[0293] Step 409: If the scheduling device 110 selects the second method to determine the first target node, the scheduling device 110 calculates the second load parameters of each node 121. The load parameters of node 121 are used to characterize the current load status of node 121.

[0294] If the second method is selected, the scheduling device 110 does not need to consider the load status of each node 121 executing the first backup task, but only needs to select the first target node based on the current load status of each node 121.

[0295] Using this second method, the scheduling device 110 needs to first calculate the second load parameters of each node 121. For any node 121, the scheduling device 110 obtains the second load parameters of the node 121 based on part or all of the historical task information and real-time status information of the node 121.

[0296] This application embodiment does not limit the method by which the scheduling device 110 obtains the second load parameter of node 121 based on the historical task information and real-time status information of node 121. Several calculation methods are described below.

[0297] First, the capacity parameter NUsedCapPer` of node 121 when calculating the second load parameter of each node 121 is distinguished from the capacity parameter NUsedCapPer` of node 121. The capacity parameter of node 121 when calculating the second load parameter of node 121 is redefined. To facilitate the distinction between different capacity parameters, the capacity parameter of node 121 involved in the calculation of the second load parameter is represented by NUsedCapPer.

[0298] The capacity parameter NUsedCapPer of node 121 satisfies:

[0299] The capacity parameter NUsedCapPer of node 121 = UsedCap of the occupied storage space in node 121 / Total size of storage space in node 121, TolCap.

[0300] The definitions of the health parameter NHealthy and the parallelism parameter NPar of node 121 are the same as those mentioned above and will not be repeated here.

[0301] The scheduling device 110 can calculate the second load parameter of node 121 based on some or all of the health parameter NHealthy, the capacity parameter NUsedCapPer, and the parallelism parameter NPar of the capacity of node 121.

[0302] For example, the second load parameter W2 of node 121 satisfies:

[0303] W2 = (α - NPar) * (α - NUsedCapPer) * NHealthy, where α is a positive integer greater than 1.

[0304] The second load parameter of node 121 that satisfies the above formula is such that the larger the second load parameter of node 121, the more suitable node 121 is as the first target node. The value range of W2 is 0≤W2≤α. In this calculation method, the larger the W2 of node 121, the more suitable node 121 is to perform the first backup task.

[0305] In practical scenarios, the importance of the health parameter NHealthy of node 121 in calculating the second load parameter of node 121 can be appropriately increased, such as by increasing the value of NHealthy in the formula satisfied by the second load parameter W2 of node 121. The higher the health parameter of node 121, the more historical backup tasks that node 121 has successfully executed, and the greater the probability that node 121 has successfully executed the first backup task.

[0306] For example, the formula satisfied by the second load parameter W2 of the aforementioned node 121 can be transformed into:

[0307] W2 = (α - NPar) * (α - NUsedCapPer) * (1 + NHealthy), where α is a positive integer greater than 1.

[0308] Since the execution status of backup tasks on each node 121 or the size of unused storage space on each node 121 may differ, the formula for the second load parameter of some "special" nodes 121 can be appropriately adjusted. Regarding the calculation method for the second load parameter of node 121 in special cases, please refer to the aforementioned calculation method for the first load parameter of node 121 in special cases. The only difference is that the special case involving the second load parameter of node 121 does not include characteristic case one. For details, please refer to the aforementioned explanation; it will not be repeated here.

[0309] It should be noted that when using the above calculation method, there is a possibility that the calculated second load parameter may be abnormal. An abnormality means that the calculated second load parameter exceeds the value range of W2. In this case, the second load parameter of node 121 can be set to the boundary value of the value range of W2.

[0310] Step 410: The scheduling device 110 determines the first target node from each node 121 according to the second load parameters of each node 121. Based on the calculation method of the second load parameters of each node 121, the first target node selected by the scheduling device 110 can be the node 121 with the largest or smallest second load parameters among the nodes 121.

[0311] Since the second method for determining the first target node was used, the scheduling device 110 can use the second method for determining the target storage pool. For details, please refer to steps 409 to 410.

[0312] Step 411: The scheduling device 110 calculates the second load parameters for each storage pool.

[0313] Taking one of the storage pools as an example, the process of the scheduling device 110 calculating the second load parameter of the storage pool is explained as follows:

[0314] The scheduling device 110 can obtain the second load parameters of the storage pool from some or all of the historical task information and real-time status information of the storage pool.

[0315] This application embodiment does not limit the scheduling device 110 to obtain the second load parameters of the storage pool based on some or all of the historical task information and real-time status information of the storage pool. Several methods for obtaining the second load parameters of the storage pool based on the historical task information and real-time status information of the storage pool are described below.

[0316] First, unlike the storage pool capacity parameter defined in step 405, the storage pool capacity parameter is redefined here, and the redefined storage pool capacity parameter is identified as SPUsedCapPer.

[0317] The storage pool capacity parameter SPUsedCapPer satisfies:

[0318] The storage pool capacity parameter SPUsedCapPer = (Occupied storage space in the storage pool - SPUsedCap) / (Total storage space in the storage pool - SPTolCap).

[0319] The definitions of the storage pool's health parameter SPHealthy and the storage pool's parallelism parameter SPPar can be found in the aforementioned content and will not be repeated here.

[0320] The scheduling device 110 can calculate the second load parameter of the storage pool based on some or all of the health parameter SPHealthy, the capacity parameter SPUsedCapPer, and the parallelism parameter SPPar.

[0321] For example, the second load parameter w2 of the storage pool satisfies:

[0322] w2 = (α - SPPar) * (α - SPUsedCapPer) * SPHealthy, where α is a positive integer greater than 1.

[0323] The second load parameter of the storage pool that satisfies the above formula is such that the larger the second load parameter, the more suitable the storage pool is as the target storage pool. The value range of w2 is 0≤w2≤α. In this calculation method, the larger the w2 of the storage pool, the more suitable the storage pool is to carry out the first backup task.

[0324] In practical scenarios, the importance of the storage pool's health parameter SPHealthy in calculating the storage pool's second load parameter can be appropriately increased, such as by increasing the value of SPHealthy in the formula satisfied by the storage pool's second load parameter w2. A higher storage pool health parameter indicates more successfully executed historical backup tasks, and a greater probability of successfully executing the first backup task.

[0325] For example, the formula satisfied by the second load parameter w2 of the aforementioned storage pool can be transformed into:

[0326] w2 = (α - SPPar) * (α - SPUsedCapPer) * (1 + SPHealthy), where α is a positive integer greater than 1.

[0327] Because the execution status of backup tasks on different storage pools or the size of unused storage space on different storage pools may vary, the formula for the second load parameter of some "special" storage pools can be adjusted appropriately. Regarding the method for calculating the second load parameter of a storage pool in special cases, please refer to the aforementioned method for calculating the first load parameter of a storage pool in special cases. The only difference is that the special cases involved in the second load parameter of the storage pool do not include characteristic case one. For details, please refer to the aforementioned explanation; it will not be repeated here.

[0328] Step 412: The scheduling device 110 selects a storage pool as the target storage pool from each storage pool according to the second load parameter of each storage pool. Combined with the calculation method of the first load parameter of each storage pool, the target storage pool selected by the scheduling device 110 can be the storage pool with the largest or smallest second load parameter among the storage pools.

[0329] Regarding the scheduling method of the second backup task, if the second backup task and the first backup task are not batch tasks, the scheduling method of the second backup task is the same as that of the first backup task. For details, please refer to steps 402 to 406 above, which will not be repeated here. If the second backup task and the first backup task are batch tasks, that is, the receiving time of the second backup task and the receiving time of the first backup task belong to the same time period, the scheduling method of the second backup task is similar to that of the first backup task. The difference is that when scheduling the second backup task, when calculating the second load parameters of each node 121, it is necessary to assume that the first backup task has been allocated to the first target node. Therefore, when calculating the first load parameters of the first target node, the capacity parameters of the first target node involved satisfy:

[0330] The capacity parameter of the first target node = (the storage space already occupied in the first target node + the backup amount of the first backup task) / the total size of the storage space in node 121.

[0331] The capacity parameters of any other node 121 satisfy

[0332] The capacity parameter of node 121 = (the storage space already occupied in node 121) / the total size of the storage space in node 121.

[0333] Subsequently, based on the second load parameters of each node 121, the second target node for executing the second backup task is determined. If the first target node and the second target node are the same node 121, then the scheduling device 110 will further perform a second-level scheduling to determine the storage pool that will carry the second backup task from the first target node. Similarly, when calculating the second load parameters of each storage pool, it is assumed that the first backup task has been allocated to the target storage pool. Therefore, when calculating the second load parameters of the target storage pool, the capacity parameters of the target storage pool must satisfy the following:

[0334] The capacity parameter of the target storage pool = (the storage space already occupied in the target storage pool + the backup amount of the first backup task) / the total size of the storage space in the target storage pool.

[0335] The capacity parameters of any other storage pool satisfy

[0336] The capacity parameter of the storage pool = (the storage space already occupied in the storage pool) / the total size of the storage space in the storage pool.

[0337] Step 413: The scheduling device 110 assigns the first backup task to the first target node, triggering the first target node to execute the first backup task.

[0338] If the first target node includes multiple storage pools, the scheduling device 110 determines the target storage pool that will carry the first backup task from among the multiple storage pools, and the scheduling device 110 assigns the first backup task to the first target node, triggering the first target node to execute the first backup task in the target storage pool.

[0339] After the first target node is determined, the scheduling device 110 can transmit the first backup task to the first target node and notify the first target node to execute the first backup task in the target storage pool. After receiving the first backup task, the first target node executes the first backup task in the target storage pool.

[0340] Backup service scenarios have certain unique characteristics. When the first backup task is the first backup task targeting the target data, the scheduling device 110 can adopt... Figure 4 The method shown is to assign node 121 to the first backup task. If the first backup task is not the first backup task targeting the target data, such as an incremental or differential backup after a full backup of the target data, it can usually be assigned to node 121, which is performing the first backup task targeting the target data. If node 121 cannot execute the first backup task, such as if the first backup task fails, the scheduling device 110 can use... Figure 4 The method shown is to reassign node 121 for the first backup task.

[0341] The training process of the machine learning model is explained below. (See also...) Figure 5 The training method for the machine learning model provided in the embodiments of this application.

[0342] Step 501: Generate the training set for the machine learning model.

[0343] Since the information of the backup tasks executed by each node 121 (such as the type of backup task, the execution time of the backup task, and the actual amount of backup data) may vary greatly, it is necessary to obtain information on multiple different backup tasks for processing when generating this training set. The specific process is as follows:

[0344] Step 5011: Obtain information on the backup tasks executed by each node 121. The information on the executed backup tasks includes: the type of backup task, the execution time of the backup task, the actual amount of backup data, and the data targeted by the backup task.

[0345] The actual backup data volume of the backup task refers to the actual amount of data backed up when the backup task is executed, and the execution time of the backup task refers to the time spent executing the backup task.

[0346] Backup task types include, but are not limited to: full backup, incremental backup, differential backup, permanent incremental backup, snapshot, log backup, and log backup.

[0347] Regarding the configuration of backup task types, you can consider only the most frequently occurring backup task types and categorize the less frequently occurring backup task types as "Other". For example, only consider full backups, incremental backups, differential backups, and snapshots, and set the type of the remaining backup tasks to "Other".

[0348] The data targeted by the backup task refers to the data that needs to be backed up. The data targeted by the backup task does not refer to the actual data backed up when the backup task is executed, but rather to the data that the user requests to be backed up when the backup task is triggered.

[0349] Step 5012: Perform data cleaning on the information of the backup tasks executed by each node 121 that has been acquired. Delete abnormal and useless data from the information of the backup tasks executed by each node 121 that has been acquired, such as information on backup tasks whose type is not among the possible types, information on backup tasks with a small actual backup data volume (e.g., less than 10 kilobits), and information on backup tasks with a negative execution time.

[0350] Step 5013: Preprocess the information of the backup tasks executed on each node 121 after data cleaning. This embodiment of the application does not limit the method of preprocessing the information of the backup tasks executed on each node 121.

[0351] For example, the type of backup task can be converted into a single value using one-hot encoding.

[0352] For example, the unit of measurement should be standardized for the actual backup data volume of backup tasks.

[0353] For example, the actual backup data volume of a backup task can be decomposed into a two-dimensional vector, <normalized data volume, dimension>: where the normalized data volume is limited to a preset range, such as [1, 10]. The dimension refers to the unit of measurement used to limit the actual backup data volume to the preset range, and the dimension is a simplified representation of that unit, which can be designed according to actual needs. For example, the dimension of a unit of 1000 is 3, and the dimension of a unit of 10000 is 4. Specifically, for a backup task with a backup data volume of 3187MB, the backup data volume will be represented as <3.187, 3>.

[0354] For example, data can be filled into the information of backup tasks executed on each node 121 to fill in the missing data. If any backup task lacks necessary information, it can be filled in based on the information of historical backup tasks for the data it targets. For example, if a backup task is backing up data A, and the information obtained for this backup task is missing the actual backup amount, then the actual backup amount of this backup task can be determined based on the actual backup amount of historical backup tasks for data A (e.g., the mean or variance of the actual backup amount of historical backup tasks for data A can be used as the actual backup amount of this backup task).

[0355] Step 502: Train the machine learning model based on the training set. This application embodiment does not limit the method of training multiple machine learning models based on the training set. The training method includes, but is not limited to: supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and self-supervised learning.

[0356] Step 1: Define variables.

[0357] If backup tasks for the same data are aggregated, and there are M backup tasks for the same data, then the information of one of these M backup tasks is one record of that data.

[0358] definition The amount of backup data in the t-th record of backup task job j. To back up the execution duration of job j in the t-th record, The data type is defined in the t-th record of backup task job j, and the time window size is Tw (when Tw is 10, it means that information from 10 historical backup tasks will be used as input).

[0359] The backup task type Job_Type can be divided into seven types: full backup, incremental backup, differential backup, log backup, permanent incremental backup, snapshot, and others. The backup task type is converted into a positive integer with a value greater than or equal to 1 or less than or equal to 7 through one-hot encoding.

[0360] The backup data volume Data_Amount of the backup task is converted into a two-dimensional vector <normalized data volume, dimension>, which can be represented as follows:<Normed_Data_Amount,Data_Amount_Dimen> .

[0361] The backup task's execution duration, Used_Time, is converted into a two-dimensional vector <normalized duration, dimension>, which can be represented as:<Normed_Used_Time,Used_Time_Dimen> .

[0362] Step 2: For different machine learning models, set the input format applicable to the machine learning mode.

[0363] For example, the input format of machine learning models based on RandomForest, AdaBoost, and GradientBoosting is as follows: The training sets for machine learning models based on RandomForest, AdaBoost, and GradientBoosting are the sets of inputs mentioned above.

[0364] The input and output of the machine learning models based on RandomForest, AdaBoost, and GradientBoosting are: the (t+1)th record of backup task job j, which represents the amount of backup data for the (t+1)th backup task targeting the same data. This means that the output of the machine learning model is a prediction of the amount of backup data for the next backup task based on information from the previous historical backup task.

[0365] For example, the input features of a DNN model based on Multi-head Attention are: the t-th record of backup task j.

[0366] Extract Tw historical backup records for each backup task j according to the time window Tw as the time sequence input:

[0367]

[0368] In the DNN model based on Multi-head Attention, the time series is processed through an FC Embedding layer consisting of two fully-connected layers (FC) and one non-linear layer for feature extraction. Then, it is processed through the Multi-head Attention network, and finally through an FCEmbedding layer consisting of two fully-connected layers and one non-linear layer to extract information and obtain the final output.

[0369] The output of the DNN model based on Multi-head Attention is: the predicted value of the backup data volume for the (t+1)th record.

[0370] Step 3: Parameter Tuning: Tune the parameters of the machine learning model to complete model training. The model training process involves iteratively training the machine learning model in multiple rounds, adjusting the parameters of the machine learning model based on the results of the previous round of training, and then proceeding to the next round of training until the loss function of the machine learning model converges.

[0371] For machine learning models based on RandomForest, AdaBoost, and GradientBoosting, the parameters are adjusted according to the common parameter tuning range.

[0372] For DNN models based on Multi-head Attention, during the training of each set of hyperparameters, the parameters of the DNN that minimize the mean-squared error (MSE) (i.e., the loss function of the DNN) on the training set after each training epoch are retained. For example, if no better model parameters are found for evaluating MSE after more than 10 training epochs and the number of training epochs has exceeded 30, training of the current hyperparameter combination is stopped, and training of the next set of hyperparameters continues, continuously updating the model parameters that evaluate better MSE, until all hyperparameter combinations have been traversed. These hyperparameter combinations are then fixed as initial parameters, and the model is built using these parameters without manual adjustment.

[0373] At this point, the training of multiple machine learning models is complete. In this embodiment, the scheduling device 110 can perform update operations on the established machine learning models according to a preset model update frequency. These update operations include model training and model replacement. The model training operation retrains the established machine learning modules; the training process can be found in steps 501-503, and will not be repeated here. It should be noted that the training set used for model training differs in each update operation. The training set used in each update operation may include information from the backup tasks executed by each node 121 after the last update operation. The model replacement operation replaces the established machine learning model with the retrained machine learning model.

[0374] Based on the same inventive concept as the method embodiments, this application also provides a scheduling device for executing the method executed by the scheduling device 110 in the above method embodiments. For example... Figure 6As shown, the scheduling device 600 includes an adding module 601, an extracting module 602, and a determining module 603. Specifically, in the scheduling device 600, the modules are connected through a communication path.

[0375] Add module 601 to receive the first backup task and add it to the backup task queue.

[0376] The extraction module 602 is used to extract the first backup task from the backup task queue for the first time, without processing the first backup task, and to add the first backup task back to the backup task queue; and to extract the first backup task from the backup task queue again.

[0377] The determination module 603 is used to determine the first target node from multiple nodes in the backup cluster and trigger the first target node to execute the first backup task.

[0378] As one possible implementation, the first target node satisfies the following condition: when the first target node is performing the first backup task, multiple nodes are in a load-balanced state.

[0379] As one possible implementation, when determining the first target node, the determining module 603 determines the load status of each of the multiple nodes when performing the first backup task; the determining module 603 determines the first target node based on the load status of each node when performing the first backup task.

[0380] As one possible implementation, after determining the first target node, the determining module 603 selects a storage pool from multiple storage pools among the first target node as the target storage pool. The target storage pool satisfies the following: when the first backup task is executed in the target storage pool, the multiple storage pools are in a load-balanced state, triggering the first target node to execute the first backup task in the target storage pool.

[0381] As one possible implementation, when determining the target storage pool, the determining module 603 determines the load status of each of the multiple storage pools after executing the first backup task; the determining module 603 determines the target storage pool based on the load status of each storage pool after executing the first backup task.

[0382] In one possible implementation, when determining the load state of each of the multiple nodes while executing the first backup task, the determining module 603 calculates the backup data volume of the first backup task, which is the amount of data that needs to be backed up when executing the backup task. For any node, the determining module 603 obtains the first load parameter of the node based on the backup data volume and the node's information. The first load parameter of the node characterizes the load state of the node while executing the first backup task. The node's information includes part or all of the node's historical task information and the node's real-time status information. The node's historical task information is the task information of the backup tasks that the node has executed, and the node's real-time status information characterizes the current running status of the node.

[0383] As one possible implementation, when determining the load status of each storage pool after executing the first backup task in multiple storage pools, the determining module 603 calculates the backup data volume of the first backup task, which is the amount of data that needs to be backed up when executing the first backup task. For any storage pool, the determining module 603 obtains the first load parameter of the storage pool based on the information of the storage pool and the backup data volume. The first load parameter of the storage pool characterizes the capacity status of the storage pool when the first backup task is executed in the storage pool. The information of the storage pool includes part or all of the historical task information and the real-time status information of the storage pool. The historical task information of the storage pool is the task information of the backup tasks that have been executed in the storage pool, and the real-time status information of the storage pool characterizes the current capacity status of the storage pool.

[0384] In one possible implementation, the extraction module 602 extracts a second backup task from the backup task queue, and the time of receiving the second backup task is within the same time period as the time of receiving the first backup task.

[0385] The determination module 603 determines the second target node from multiple nodes in the backup cluster and triggers the second target node to execute the second backup task. The second target node satisfies the following condition: when the first target node is executing the first backup task and the second target node is executing the second backup task, the multiple nodes are in a load-balanced state.

[0386] As one possible implementation, when determining the second target node from multiple nodes in the backup cluster, the determining module 603 determines the load status of the first target node when performing the first backup task and the second backup task, and the load status of the nodes other than the first target node when performing the second backup task.

[0387] The determination module 603 determines the second target node based on the load status of the first target node when executing the first backup task and the second backup task, and the load status of the nodes other than the first target node among the multiple nodes when executing the second backup task.

[0388] As one possible implementation, the determining module 603 obtains the node's historical task information from the node. The node's historical task information includes some or all of the following: the number of backup tasks assigned to the node and the number of backup tasks that the node has successfully executed.

[0389] As one possible implementation, the determining module 603 obtains the real-time status information of the node from the node. The real-time status information of the node includes some or all of the following: the number of backup tasks currently being executed by the node, the maximum number of backup tasks that the node is allowed to execute in parallel, the occupied storage space within the node, the free storage space within the node, and the total size of the storage space within the node.

[0390] As one possible implementation, the determining module 603 obtains historical task information of the storage pool from the first target node. The historical task information of the storage pool includes some or all of the following: the number of backup tasks that have been executed in the storage pool and the number of backup tasks that have been successfully executed in the storage pool.

[0391] As one possible implementation, the determining module 603 obtains the real-time status information of the storage pool from the first target node. The real-time status information of the storage pool includes some or all of the following: the number of backup tasks currently being executed in the storage pool, the maximum number of backup tasks that can be executed in parallel in the storage pool, the storage space that has been occupied in the storage pool, the free storage space in the storage pool, and the total storage space of the storage pool.

[0392] The module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0393] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a personal computer, mobile phone, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0394] This application also provides, for example Figure 7 The computing device 700 shown includes a bus 701, a processor 702, a communication interface 703, and a memory 704. The processor 702, the memory 704, and the communication interface 703 communicate with each other via the bus 701.

[0395] The processor 702 can be a CPU, or other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0396] The memory 704 can be DRAM. Besides DRAM, the memory 704 can also be other random access memory (RAM), such as SRAM. Additionally, the memory 704 can also be ROM. For read-only memory, for example, it could be PROM, EPROM, etc. The memory 704 can also be flash memory, HDD, or SSD, etc.

[0397] The memory 704 stores computer program instructions, and the processor 702 executes these computer program instructions to perform the aforementioned tasks. Figure 3 The steps performed by scheduling device 110 in the described method. The memory 704 may also include other software modules required for running processes, such as an operating system (e.g., multiple modules in scheduling device 600). The operating system may be LINUX. TM UNIX TM WINDOWS TM wait.

[0398] This application also provides a computing device system, the computing device system including at least one such as Figure 8The computing device 800 shown includes a bus 801, a processor 802, a communication interface 803, and a memory 804. The processor 802, memory 804, and communication interface 803 communicate with each other via the bus 801. At least one computing device 800 in the computing device system communicates with each other via a communication path.

[0399] The specific types of processor 802 and memory 804 can be found in the relevant descriptions of processor 702 and memory 704, and will not be repeated here. Processor 802 executes the computer program instructions stored in memory 804 to perform the aforementioned tasks. Figure 3 The described method may include some or all of the steps executed by the scheduling device 110. The memory may also include other software modules required for running processes, such as an operating system. The operating system may be Linux. TM UNIX TM WINDOWS TM wait.

[0400] At least one computing device 800 in the computing device system establishes communication with each other through a communication network, and each computing device 800 runs any one or any multiple modules of the scheduling device 600.

[0401] The descriptions of the processes corresponding to the above-mentioned figures each have their own emphasis. For parts of a process that are not described in detail, please refer to the relevant descriptions of other processes.

[0402] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes computer program instructions, which, when loaded and executed on a computer, generate, in whole or in part, the product according to the embodiments of the present invention. Figure 2 The process or function described.

[0403] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD).

[0404] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A backup task scheduling method, characterized in that, include: Receive the first backup task and add it to the backup task queue; When the first backup task is retrieved from the backup task queue for the first time, the first backup task is not processed, and the first backup task is re-added to the backup task queue; The first backup task is retrieved again from the backup task queue, and the first target node is determined from multiple nodes in the backup cluster, triggering the first target node to execute the first backup task.

2. The method as described in claim 1, characterized in that, The first target node satisfies the following condition: when the first target node is executing the first backup task, the multiple nodes are in a load-balanced state.

3. The method as described in claim 1 or 2, characterized in that, The step of determining the first target node from multiple nodes in the backup cluster includes: Determine the load status of each of the plurality of nodes while executing the first backup task; The first target node is determined based on the load status of each node when performing the first backup task.

4. The method according to any one of claims 1 to 3, characterized in that, The step of triggering the first target node to execute the first backup task includes: A storage pool is selected from multiple storage pools in the first target node as the target storage pool. The target storage pool satisfies the following condition: when the first backup task is executed in the target storage pool, the multiple storage pools are in a load-balanced state, triggering the first target node to execute the first backup task in the target storage pool.

5. The method as described in claim 4, characterized in that, The step of selecting a storage pool as the target storage pool from multiple storage pools in the first target node includes: Determine the load status of each of the plurality of storage pools after executing the first backup task; The target storage pool is determined based on the load status after the first backup task is executed in each storage pool.

6. The method as described in claim 3, characterized in that, Determining the load status of each of the plurality of nodes while executing the first backup task includes: Calculate the backup data volume of the first backup task, where the backup data volume is the amount of data that needs to be backed up when the backup task is executed; For any of the nodes, a first load parameter of the node is obtained based on the amount of backup data and the node's information. The first load parameter of the node characterizes the load status of the node when executing the first backup task. The node's information includes part or all of the node's historical task information and the node's real-time status information. The node's historical task information is the task information of the backup tasks that the node has executed. The node's real-time status information characterizes the current running status of the node.

7. The method as described in claim 6, characterized in that, Determining the load status of each of the plurality of storage pools after executing the first backup task includes: Calculate the backup data volume of the first backup task, where the backup data volume is the amount of data that needs to be backed up when the first backup task is executed; For any of the storage pools, a first load parameter of the storage pool is obtained based on the information of the storage pool and the amount of backup data. The first load parameter of the storage pool characterizes the capacity status of the storage pool when the first backup task is executed in the storage pool. The information of the storage pool includes part or all of the historical task information of the storage pool and the real-time status information of the storage pool. The historical task information of the storage pool is the task information of the backup tasks that have been executed in the storage pool. The real-time status information of the storage pool characterizes the current capacity status of the storage pool.

8. The method as described in claim 1, characterized in that, The method further includes: The second backup task is retrieved again from the backup task queue, and the time of receiving the second backup task and the time of receiving the first backup task belong to the same time period; A second target node is determined from multiple nodes in the backup cluster, and the second target node is triggered to execute the second backup task. The second target node satisfies the following condition: when the first target node is executing the first backup task and the second target node is executing the second backup task, the multiple nodes are in a load-balanced state.

9. The method as described in claim 8, characterized in that, The step of determining the second target node from multiple nodes in the backup cluster includes: Determine the load status of the first target node when executing the first backup task and the second backup task, and the load status of the nodes other than the first target node among the plurality of nodes when executing the second backup task; The second target node is determined based on the load status of the first target node when executing the first backup task and the second backup task, and the load status of the nodes other than the first target node among the plurality of nodes when executing the second backup task.

10. A scheduling device, characterized in that, The scheduling device includes: An add module is used to receive the first backup task and add the first backup task to the backup task queue; The extraction module is configured to, when extracting the first backup task from the backup task queue for the first time, not process the first backup task, but add the first backup task back to the backup task queue; and extract the first backup task from the backup task queue again. The determination module is used to determine the first target node from multiple nodes in the backup cluster and trigger the first target node to execute the first backup task.

11. The apparatus as claimed in claim 10, characterized in that, The first target node satisfies the following condition: when the first target node is executing the first backup task, the multiple nodes are in a load-balanced state.

12. The apparatus as claimed in claim 10 or 11, characterized in that, The determining module is used for: Determine the load status of each of the plurality of nodes while executing the first backup task; The first target node is determined based on the load status of each node when performing the first backup task.

13. The apparatus according to any one of claims 10 to 12, characterized in that, The determining module is used for: A storage pool is selected from multiple storage pools in the first target node as the target storage pool. The target storage pool satisfies the following condition: when the first backup task is executed in the target storage pool, the multiple storage pools are in a load-balanced state, triggering the first target node to execute the first backup task in the target storage pool.

14. A computing device, characterized in that, The computing device includes a processor and memory; The memory is used to store computer program instructions; The processor executes computer program instructions in the memory to perform the method as described in any one of claims 1 to 9.

15. A computer-readable storage medium, characterized in that, When the computer-readable storage medium is executed by a computing device, the computing device performs the method according to any one of claims 1 to 9.

Images