Resource control method, device, equipment and system for backup recovery

Abstract

A resource control method, apparatus, device, and system for backup and recovery are disclosed, relating to the field of computers. The method includes allocating first resources to a first backup and recovery task based on initial system resources and system-used resources, and executing the first backup and recovery task based on the first resources. The first resources include at least one of computing resources or storage resources, and the system-used resources include resources used by applications. The method provided in this application can dynamically allocate resources from system resources to the backup and recovery task based on real-time system resource occupancy, improving resource utilization and backup and recovery efficiency while ensuring backup and recovery reliability.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a resource control method, apparatus, device, and system for backup and recovery. Background Technology

[0002] Currently, in the event of application system failure, data recovery from backups is used to improve system high availability and disaster recovery. Typically, to ensure the normal operation of application systems, more resources are reserved for the application system, while fewer resources are allocated to the backup system, resulting in a relatively conservative backup and recovery strategy. Furthermore, using the same backup and recovery strategy for multiple application systems leads to lower reliability, efficiency, and resource utilization during backup and recovery. Summary of the Invention

[0003] This application provides a resource control method, apparatus, device, and system for backup and recovery, which improves the reliability, efficiency, and resource utilization of backup and recovery.

[0004] Firstly, a resource control method for backup and recovery is provided. The method includes allocating first resources to a first backup and recovery task based on initial system resources and system-used resources, and executing the first backup and recovery task based on the first resources. The first resources include at least one of computing resources or storage resources, and the system-used resources include resources used by applications.

[0005] The method provided in this application can dynamically allocate resources from system resources for backup and recovery tasks based on the real-time system resource usage, thereby improving resource utilization and backup and recovery efficiency while ensuring the reliability of backup and recovery.

[0006] In one possible implementation, the system-used resources also include the second resources used by the second backup and recovery task; allocating the first resources to the first backup and recovery task based on the system initial resources and the system-used resources includes: when the second backup and recovery task is performed based on the second resources, allocating the first resources to the first backup and recovery task based on the system initial resources and the system-used resources.

[0007] In scenarios where resources have already been allocated to backup and recovery tasks, resources can also be allocated to backup and recovery tasks based on the real-time resource usage of the system, thereby improving resource utilization and backup and recovery efficiency while ensuring the reliability of backup and recovery.

[0008] In another possible implementation, the first resource is allocated to the first backup and recovery task based on the initial system resources and the resources already used by the system, including: allocating the first resource to the first backup and recovery task based on the initial system resources, the resources already used by the system, and the historical resource allocation of the first backup and recovery task.

[0009] When allocating resources for the first backup and recovery task, referring to the historical resource allocation of that backup and recovery task can predict the resource consumption trend of that backup and recovery task, thereby allocating resources more accurately for this backup and recovery task.

[0010] For example, a neural network model can be trained based on historical resource allocation and real-time system resources to automatically output backup and recovery strategies suitable for the current system resource usage.

[0011] In another possible implementation, allocating first resources to the first backup and recovery task based on the initial system resources and the resources already used by the system includes: allocating first resources to the first backup and recovery task based on the initial system resources and the resources already used by the system resources, wherein the system resource threshold is used to indicate the occupancy status of the initial system resources.

[0012] After allocating resources to backup and recovery tasks, the relationship between resource utilization and threshold is compared, and the resources allocated to backup and recovery tasks are dynamically adjusted to maximize resource utilization and backup efficiency while ensuring system reliability.

[0013] In another possible implementation, the first backup and recovery task includes at least one backup and recovery subtask, and the first resource includes the resources of at least one backup and recovery subtask.

[0014] The method provided in this application can also allocate resources to backup and recovery subtasks with smaller granularity, enabling more precise control over backup and recovery tasks and improving system resource utilization and backup and recovery efficiency.

[0015] In another possible implementation, the system includes an application system and a backup system, with the application system deployed on the first node and the backup system deployed on the second node; the first resource also includes network resources.

[0016] When transferring backup data between different computing nodes, the resources allocated to the backup and recovery task also include network resources.

[0017] For example, when the network is idle, increase network resource allocation for backup and recovery tasks to improve resource utilization and backup and recovery efficiency. When the network is busy, reduce network resource allocation for backup and recovery tasks to avoid consuming network bandwidth of the application system and improve the stability of the application system.

[0018] In another possible implementation, the method also includes: displaying the resource information used by the backup and restore task.

[0019] By fully leveraging the advantages of digitalization, we can, on the one hand, display the current system resource usage and the execution status of backup and recovery tasks in real time, thereby increasing usability; on the other hand, we can provide data support for establishing a backup and recovery resource control model.

[0020] In another possible implementation, allocating first resources to the first backup and recovery task based on the initial system resources and the resources already used by the system includes: receiving an adaptive backup and recovery start instruction, and allocating first resources to the first backup and recovery task based on the initial system resources and the resources already used by the system.

[0021] After enabling adaptive backup and recovery, the system will automatically output a resource allocation strategy, issue instructions to each backup and recovery task, and dynamically adjust the backup and recovery tasks in real time.

[0022] In another possible implementation, the method also includes: displaying an adaptive backup and recovery start control interface.

[0023] The adaptive backup and recovery function can be selectively enabled by the user. In scenarios where the efficiency, reliability, and resource utilization of backup and recovery tasks are not critical, it can be disabled to reduce costs.

[0024] In another possible implementation, the method further includes: unbinding the relationship between the first backup and recovery task and the first resource after the first backup and recovery task has been completed.

[0025] After the backup and restore task is completed, release the occupied resources in a timely manner. The released resources will be added to the available resources of the system to improve resource utilization.

[0026] In a second aspect, a resource control device is provided, comprising modules for performing the methods of the first aspect or any possible design of the first aspect. For example, the resource control device includes a communication module and a processing module.

[0027] The communication module is used to acquire the initial system resources and the system resources already used; the processing module is used to allocate first resources to the first backup and recovery task based on the initial system resources and the system resources already used, wherein the first resources include at least one of computing resources or storage resources, and the system resources already used include resources used by the application; the processing module is also used to execute the first backup and recovery task based on the first resources.

[0028] In one possible implementation, the system-used resources also include the second resources used by the second backup and recovery task: when the processing module allocates the first resources to the first backup and recovery task based on the system initial resources and the system-used resources, it is specifically used to: allocate the first resources to the first backup and recovery task based on the system initial resources and the system-used resources when the second backup and recovery task is performed based on the second resources.

[0029] In another possible implementation, the processing module allocates the first resource to the first backup and recovery task based on the initial system resources and the resources already used by the system. Specifically, it allocates the first resource to the first backup and recovery task based on the initial system resources, the resources already used by the system, and the historical resource allocation of the first backup and recovery task.

[0030] In another possible implementation, when the processing module allocates the first resource to the first backup and recovery task based on the initial system resources and the system resources already used, it is specifically used to: allocate the first resource to the first backup and recovery task based on the system resource threshold, the initial system resources and the system resources already used, whereby the system resource threshold is used to indicate the occupancy status of the initial system resources.

[0031] In another possible implementation, the first backup and recovery task includes at least one backup and recovery subtask, and the first resource includes the resources of at least one backup and recovery subtask.

[0032] In another possible implementation, the system includes an application system and a backup system, with the application system deployed on the first node and the backup system deployed on the second node; the first resource also includes network resources.

[0033] In another possible implementation, the device also includes a display module; the display module is used to display the resource status used by the backup and recovery task.

[0034] In another possible implementation, when the processing module allocates the first resource to the first backup and recovery task based on the initial system resources and the system resources already used, it is specifically used to: allocate the first resource to the first backup and recovery task based on the adaptive backup and recovery start instruction, the initial system resources, and the system resources already used.

[0035] In another possible implementation, the display module is also used to display the adaptive backup and recovery start control interface.

[0036] In another possible implementation, the processing module is also used to unbind the first backup and recovery task from the first resource after the first backup and recovery task has been completed.

[0037] Thirdly, a data processing system is provided, which includes an application system and a backup and recovery system; the backup and recovery system is used to perform the operation steps of the method in the first aspect or any possible implementation of the first aspect to back up and restore the data processed by the application system.

[0038] Fourthly, a computer device is provided, the computer device including a memory and a plurality of processors, the memory being used to store a set of computer instructions; when the processors execute the set of computer instructions, the plurality of processors jointly execute the operation steps of the method as described in the first aspect or any possible implementation of the first aspect.

[0039] Fifthly, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a processor, causing the processor to perform operational steps of the method as described in the first aspect or any possible implementation thereof.

[0040] In a sixth aspect, a computer program product is provided that, when run on a computer, causes the computer to perform the operational steps of the method as described in the first aspect or any possible implementation thereof.

[0041] The technical effects of any of the design methods in aspects two through six can be found in aspect one or in different design methods in aspect one, and will not be repeated here.

[0042] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0043] Figure 1 A schematic diagram of a backup and recovery system provided in this application;

[0044] Figure 2 A flowchart illustrating a backup and recovery resource control method provided in this application;

[0045] Figure 3 A schematic diagram of a resource preprocessing method provided in this application;

[0046] Figure 4 A schematic diagram illustrating an adjustment of the concurrent number of backup and recovery tasks provided in this application;

[0047] Figure 5 A schematic diagram of a backup and recovery visualization interface provided in this application;

[0048] Figure 6 A schematic diagram of the structure of a resource control device provided in this application;

[0049] Figure 7 This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0050] To better understand the embodiments of this application, some terms or technologies involved in the embodiments of this application will be explained below.

[0051] Backup and recovery primarily refer to data backup and data recovery. Data backup is the process of copying data from its original storage location to another storage location. The purpose of data backup is to ensure data integrity and availability, preventing losses due to data loss. Data recovery is the process of restoring data or rebuilding the system using backup data when data or system problems occur. The goal of data recovery is to restore business operations as quickly as possible and minimize losses caused by data loss.

[0052] In some embodiments, to ensure the normal operation of the application system, a larger amount of resources are typically pre-configured for the application system, while a smaller amount of resources are pre-configured for the backup system. When the application system uses fewer resources to process data, while the backup system requires more resources to back up and restore data, the system's resources cannot be fully utilized, leading to low reliability and efficiency in backup and recovery.

[0053] For example, different application systems using the same backup and recovery strategy may lead to backup and recovery failures or low resource utilization. For instance, both Region A and Region B are configured with the first backup and recovery strategy. However, the data volume of application systems in Region A is significantly larger than that in Region B. If an application system in Region A fails, the amount of faulty data may be substantial. Due to limited resources for backup and recovery, this can lead to backup and recovery failures and low reliability. Conversely, if an application system in Region B fails, the amount of faulty data may be smaller, allowing for the allocation of more resources for backup and recovery, resulting in low resource utilization.

[0054] To address the issues of low reliability, efficiency, and resource utilization in backup and recovery, this application provides a resource control method for backup and recovery. The method includes allocating resources to the backup and recovery task based on initial system resources and system-used resources, and executing the backup and recovery task according to the available resources. Resources include at least one of computing resources or storage resources. System-used resources include resources used by applications.

[0055] Using a single backup and recovery strategy across multiple application systems often results in lower reliability, efficiency, and resource utilization during backup and recovery. The method provided in this application dynamically allocates resources to backup and recovery tasks based on the system's real-time resource usage. This improves resource utilization and efficiency while ensuring backup and recovery reliability, enabling the backup and recovery system to intelligently and automatically adjust its strategy.

[0056] The following describes in detail, with reference to the accompanying drawings, an implementation method for a backup and recovery resource control system provided in this application.

[0057] Figure 1 This is a schematic diagram of the architecture of a backup and recovery system provided in this application. Figure 1 As shown, the backup and recovery system 100 includes an application system 110, a backup system 120, and a monitoring system 130.

[0058] Application system 110 refers to a system that processes business or application-related data in an application scenario. Examples include intelligent transportation systems and logistics management systems.

[0059] Application system 110 includes an application layer 111, a database layer 112, and an operating system layer 113. The application layer 111 runs application software within the system and generates data such as files stored on the disk. The database layer 112 stores data for application system 110, such as databases like MySQL and Oracle. The operating system layer 113 supports the application software running in the application layer and generates data such as driver information and configuration files. Examples include raw devices / volumes, device I / O, and file systems.

[0060] Backup system 120 is used to perform backup and recovery operations on the data generated by application system 110.

[0061] The backup system 120 includes a backup management module 121, a data source acquisition module 122, a data processing module 123, a data storage management module 124, and an agent module 125.

[0062] The proxy module 125 is used to extract data generated by the application system and send the data to the data source acquisition module 122. For example, the proxy module 125 is deployed in the application layer 111, database layer 112, and operating system layer 113 of the application system 110. The proxy module 125 deployed in the application layer 111 is used to acquire data such as files stored on the disk. The proxy module 125 deployed in the database layer 112 is used to acquire data stored in the database. The proxy module 125 deployed in the operating system layer 113 is used to acquire data such as drivers and configuration files.

[0063] The data source acquisition module 122 is used to receive data generated by the application system. For example, it receives data acquired and sent by the proxy module 125 from the application layer 111, the database layer 112, and the operating system layer 113.

[0064] The data processing module 123 processes the data from the data source acquisition module 122 and sends the processed data to the data storage management module 124. For example, data processing includes operations such as compression and deduplication.

[0065] The data storage management module 124 is used to receive data sent by the data processing module 123 and save the received data to a storage medium. For example, the storage medium includes, but is not limited to, servers, tape drives, hard drives, public clouds, private clouds, etc.

[0066] Application system 110 and backup system 120 report resource metrics to monitoring system 130. Resources include computing resources, storage resources, and network resources. For example, computing resources include at least one of a central processing unit (CPU), input / output (I / O), handles, or number of threads. Storage resources include at least one of memory or disk. Network resources include at least one of network transmission rate, packet loss rate, or latency.

[0067] The backup management module 121 is used to obtain indicator information from the monitoring system 130 and issue instructions to the data source acquisition module 122, data processing module 123, data storage management module 124 and agent module 125 based on the resource indicator information. The instructions include information such as resource allocation.

[0068] The monitoring system 130 is used to receive resource indicators reported by the application system 110 and the backup system 120. Optionally, the monitoring system 130 provides a page for visualizing the above resource indicators and provides the function of enabling intelligent calculation of dynamic backup strategies.

[0069] In some embodiments, the backup system 120, the application system 110, and the monitoring system 130 are deployed on a single node.

[0070] In other embodiments, the backup system 120, the application system 110, and the monitoring system 130 are deployed separately on different nodes.

[0071] The node can be in an environment such as a disaster recovery appliance, a pure software deployment, a public cloud, or a private cloud. In the case of a pure software deployment, the agent module 125 in the backup system 120 needs to be deployed on the same node as the application system 120.

[0072] Next, the resource control method for backup and recovery will be explained in detail with reference to the accompanying diagram.

[0073] Figure 2 This is a flowchart illustrating a resource control method for backup and recovery provided in this application. Here, we mainly focus on... Figure 1 The schematic diagram of the backup and recovery system shown is used for illustration.

[0074] Step 210: Allocate the first resource to the first backup and recovery task based on the initial system resources and the system resources already used.

[0075] Initial system resources refer to the resources provided by the system. In other words, initial system resources refer to the inherent resources of the system. For example, a system refers to a computer device, and its initial resources are the resources provided by that computer device. Other examples of systems include, but are not limited to, disaster recovery appliances, public clouds, and private clouds.

[0076] System resources used refer to the resources occupied by the system deployed on the computing node during operation.

[0077] In some embodiments, system-used resources include resources consumed by the application system. For example, system-used resources include... Figure 1 The resources consumed by at least one of the application layer 111, database layer 112, or operating system layer 113 in the application system 110 shown.

[0078] In other embodiments, the resources used by the system also include resources used by the backup system. For example, the resources used by the system include... Figure 1 The backup system 120 shown in the diagram consumes the following resources: For example, the backup management module 121 consumes resources to acquire metrics, formulate resource allocation strategies, and issue instructions. The data source acquisition module 121 transmits backup data to the application system 110, and the resource consumed during data transmission between the application system 110 and the backup system 120. The data processing module 123 performs operations such as compression and deduplication on the data, consuming resources. The data storage management module 124 saves the processed data to the storage medium, consuming resources.

[0079] Resources include computing resources and storage resources. For example, computing resources include at least one of CPU, I / O, handles, or number of threads. Storage resources include at least one of memory or disk.

[0080] Optionally, resources also include network resources. For example, network resources include at least one of network transmission rate, packet loss rate, or latency. For instance, when the backup system and application system are deployed on different nodes, the backup system and application system transmit data over a network. For example, the backup system and application system transmit data via fiber optic cables or similar means.

[0081] Backup and recovery tasks include at least one of the following: backup management tasks, data storage tasks, data processing tasks, data source acquisition tasks, or proxy tasks. For example, Figure 2 The backup management module 121 performs backup management tasks, the data storage management module 124 performs data storage tasks, the data processing module 123 performs data processing tasks, the data source acquisition module 122 performs data source acquisition tasks, and the proxy module 125 performs proxy tasks.

[0082] In some embodiments, available system resources are determined based on initial system resources and system-used resources; available system resources are the remaining usable resources in the system. First resources are allocated to the first backup and recovery task from these available system resources.

[0083] In one scenario, system resources used include resources consumed by running application systems. Available system resources include the remaining system resources beyond those consumed by application systems from the initial system resources.

[0084] In another scenario, the system's used resources include the sum of resources consumed by the application system and resources consumed by backup and recovery tasks. The system's available resources include the remaining resources after deducting the resources consumed by the application system and backup and recovery tasks from the initial system resources.

[0085] For example, when performing a second backup and recovery task based on a second resource, the first resource is allocated to the first backup and recovery task according to the initial system resources and the system resources already used.

[0086] The difference between the second backup and recovery task and the first backup and recovery task lies in their execution order. For example, the system executes the second backup and recovery task first. During the execution of the second backup and recovery task, resources are allocated to the first backup and recovery task, and the first backup and recovery task is executed.

[0087] In one scenario, the first backup recovery task and the second backup recovery task are the same task. The second backup recovery task is executed before the first backup recovery task. When resources are allocated for the first backup recovery task, the system's used resources include the second resource allocated for the second backup recovery task. For example, both the first and second backup recovery tasks are data source acquisition tasks. If the data source acquisition task is executed based on the second resource, and the first resource is also allocated to the data source acquisition task, the data source acquisition task is executed based on both the second and first resources.

[0088] In another scenario, the first backup and recovery task and the second backup and recovery task are different tasks. When allocating resources for the first backup and recovery task, the system's used resources include the second resource allocated for the second backup and recovery task. For example, the first backup and recovery task is a data source acquisition task, and the second backup and recovery task is a data processing task. If the data processing task is executed based on the second resource, the first resource is also allocated to the data source acquisition task. The data source acquisition task is then executed based on both the second and first resources.

[0089] In some embodiments, a first resource is allocated to the first backup and recovery task based on a system resource threshold, initial system resources, and system-used resources. The system resource threshold is used to indicate the occupancy of initial system resources. For example, the sum of the resources allocated to the first backup and recovery task and the system-used resources is determined. If the ratio of the sum of resources to the initial system resources is greater than or less than the threshold, the resources allocated to the first backup and recovery task are adjusted.

[0090] In other embodiments, first resources are allocated to the first backup and recovery task based on the initial system resources, the system's used resources, and the historical resources allocated to the first backup and recovery task. For example, the trend of the first backup and recovery task is predicted based on the historical resource allocation, and the first resources are allocated to the first backup and recovery task according to the trend of the first backup and recovery task.

[0091] Historical resource allocation refers to the resources allocated by the system for the first backup and recovery task that has already been executed. Trends include the increasing or decreasing trend of resources used by the first backup and recovery task over time. For example, if the application system is busy in the first time period, the predicted trend of historical resource allocation indicates a decreasing trend, reducing resource allocation for the current backup and recovery task. If the application system is idle in the second time period, the predicted trend of historical resource allocation indicates an increasing trend, increasing resource allocation for the current backup and recovery task.

[0092] In other embodiments, a first resource is allocated to the first backup and recovery task based on the system's initial resources, the system's used resources, historically allocated resources, and a threshold. First, the first resource is predicted based on historically allocated resources and trends. Then, the sum of the predicted first resource and the system's used resources is calculated. If the ratio of the sum of resources to the system's initial resources is greater than the threshold, the first resource is reduced; conversely, if the ratio of the sum of resources to the system's initial resources is less than the threshold, the first resource is increased.

[0093] In other embodiments, an artificial intelligence model is trained to allocate resources from available system resources to backup and recovery tasks. For example, the AI ​​model is trained to obtain a resource control model for backup and recovery. The model's output includes the resources allocated to backup and recovery tasks. The model's input includes real-time resource usage.

[0094] Optionally, the model's input may also include at least one of historical resource allocation or a threshold.

[0095] For example, real-time resources include node-level resources, process-level resources, and inter-node-level resources.

[0096] Node-level resources refer to the initial system resources and used system resources of the node where the backup system or application system is deployed.

[0097] Process-level resources refer to the resources used by a process. For example, a process includes resources that support... Figure 3 The processes running in each module of the backup system 120 shown. For example, processes include those that support the operation of the backup management module 121. That is, the backup management module is used to handle backup management tasks, and the processes refer to the processes that execute backup management tasks.

[0098] Inter-node resources refer to at least one of the following: network transmission rate, packet loss rate, or latency between the node where the backup system is deployed and the node where the application system is deployed.

[0099] In some embodiments, real-time resources are preprocessed, and resources are allocated to backup and recovery tasks based on the preprocessed resources. Preprocessing includes determining node-level resource availability, process-level resource utilization, and inter-node-level resource availability.

[0100] Node-level resource availability includes node-level CPU resource availability, node-level memory resource availability, and node-level I / O resource availability.

[0101] The availability of CPU resources at the node level satisfies formula (1).

[0102] X=(C1-C2) / C1 Formula (1)

[0103] Where X represents the CPU resource availability of the node where the application system or backup system is deployed, C1 represents the initial CPU resources of the system on the node where the application system or backup system is deployed, and C2 represents the CPU resources already used by the system on the node where the application system or backup system is deployed.

[0104] The availability of node-level memory resources satisfies formula (2).

[0105] Y = (M1 - M2) / M1 Formula (2)

[0106] Where Y represents the memory resource availability of the node where the application system or backup system is deployed, M1 represents the initial memory resources of the system on the node where the application system or backup system is deployed, and M2 represents the memory resources already used by the system on the node where the application system or backup system is deployed.

[0107] The availability of node-level I / O resources satisfies formula (3).

[0108] Z=(I1-I2) / I1 Formula (3)

[0109] Where Z represents the I / O resource availability of the node where the application system or backup system is deployed, I1 represents the initial I / O resources of the system on the node where the application system or backup system is deployed, and I2 represents the I / O resources already used by the system on the node where the application system or backup system is deployed.

[0110] Process-level resource utilization includes process-level CPU resource utilization, process-level memory resource utilization, and process-level I / O resource utilization.

[0111] The process-level CPU resource utilization rate satisfies formula (4).

[0112] C4 = C3 / C1 Formula (4)

[0113] Where C4 represents the ratio of the CPU resources occupied by the process to the initial CPU resources of the system on the node where the process resides, C3 represents the CPU resources occupied by the process, and C1 represents the initial CPU resources of the system on the node where the process resides.

[0114] The process-level memory resource utilization rate satisfies formula (5).

[0115] M4=M3 / M1 Formula (5)

[0116] Where M4 represents the ratio of the memory resources occupied by the process to the initial memory resources of the system on the node where the process resides, M3 represents the memory resources occupied by the process, and M1 represents the initial memory resources of the system on the node where the process resides.

[0117] The process-level I / O resource utilization rate satisfies formula (6).

[0118] I4=I3 / I1 Formula (6)

[0119] Where I4 represents the ratio of the I / O resources occupied by the process to the initial I / O resources of the system on the node where the process resides, I3 represents the I / O resources occupied by the process, and I1 represents the initial I / O resources of the system on the node where the process resides.

[0120] The inter-node resource availability satisfies formula (7):

[0121] NB3=(NB1-NB2) / NB1 Formula (7)

[0122] Among them, NB3 refers to the availability of network transmission resources between nodes, NB1 refers to the initial network bandwidth resources, and NB2 refers to the resources occupied by the transmission of backup data.

[0123] Optionally, neural networks can be used to optimize the preprocessed data, historical resource allocation, and trends. Neural networks include, but are not limited to, Long Short-Term Memory (LSTM) algorithms.

[0124] Machine learning techniques are used to model the optimized data. Machine learning techniques include, but are not limited to, linear programming (LP) algorithms.

[0125] For example, the backup management module 121 takes resource usage and constraints as input and uses the LP algorithm to establish a resource control model for backup and recovery.

[0126] Resource usage includes preprocessed node-level resource availability rates X, Y, and Z, process-level resource availability rates C4, M4, and I4, and inter-node-level resource availability rate NB3.

[0127] The constraint conditions satisfy formula (8):

[0128]

[0129] Where m represents the number of nodes, Pi represents node i, T∈T(Pi) represents the task set on node Pi, CPU(T) represents the CPU resources occupied by task set T, and CPU_Pi represents the initial CPU resources of node Pi. For example, the threshold is 0.9. Mem(T) represents the memory resources occupied by task set T, and Mem_Pi represents the initial memory resources of node Pi. IO(T) represents the I / O resources occupied by task set T, and IO_Pi represents the initial I / O resources of node Pi.

[0130] The model must satisfy the objective formula (9) while satisfying the constraints.

[0131]

[0132] in, Let represent the set of tasks on node Pi, Constraint represent the constraint condition in formula (8), and obj represent the goal that the set of tasks on node Pi needs to achieve under the constraint condition. The function of formula (9) is to enable the task scheduling of node Pi to utilize system resources to the maximum extent while keeping the resource utilization rate below the threshold.

[0133] After the resource control model for backup and recovery is established, a backup and recovery strategy is output, which satisfies formula (10):

[0134]

[0135] Where Pi represents node i. Represents the set of tasks for node Pi.

[0136] This application does not limit the granularity of backup and recovery tasks. A backup and recovery task may be, for example, at least one of a backup management task, a data storage task, a data processing task, a data source acquisition task, or a proxy task. Alternatively, a backup and recovery task may be a sub-task of at least one of a backup management task, a data storage task, a data processing task, a data source acquisition task, or a proxy task.

[0137] The backup and recovery resource control method provided in this application can allocate resources to backup and recovery tasks of any granularity.

[0138] For example, resources are allocated to at least one of the backup and recovery tasks, such as backup management tasks, data storage tasks, data processing tasks, data source acquisition tasks, or proxy tasks, based on the system's initial resources and the resources already used by the system.

[0139] For example, resources can be allocated to multiple backup and recovery tasks based on the system's initial resources and the resources already used. These multiple backup and recovery tasks include node-level, process-level, and inter-node-level tasks. Node-level tasks include at least one of the following: backup management tasks, data storage tasks, data processing tasks, data source acquisition tasks, or proxy tasks. Process-level tasks include subtasks of at least one of these. Inter-node-level tasks include, but are not limited to, tasks that transfer backup data between proxy tasks and data source acquisition tasks. For example, when the backup system and application system are deployed on different nodes, a proxy task in the application system transfers backup data with a data source acquisition task in the backup system.

[0140] The backup and recovery tasks described in this application include backup tasks and recovery tasks. A recovery task is the reverse operation of a backup task, referring to the process of data recovery or system reconstruction using backup data. Resources can be allocated to backup and recovery tasks according to the resource control method provided in this application.

[0141] like Figure 1 As shown, the data processing module 123 obtains backup data from the data storage management module 124, performs decompression and other processing on the backup data, and sends it to the data source acquisition module 122. The data source acquisition module 122 sends the processed data to the agent module 125 in the application system 110, and the agent module 125 copies the data to the original storage location.

[0142] Step 220: Execute the first backup and recovery task based on the first resource.

[0143] In some embodiments, the first backup and recovery task includes at least one backup and recovery subtask, and the first resource includes the resources of at least one backup and recovery subtask.

[0144] refer to Figure 4 Explain the backup and restore subtasks. For example... Figure 4 As shown, the application system 110 includes database 1, database 2 and database 3. Database 1 contains table 1 and table 2, database 2 contains table 3, and database 3 contains table 4 and table 5.

[0145] For example, if the first backup and recovery task is an agent task executed by agent module 125, the first backup and recovery task includes subtask 1, subtask 2, and subtask 3. Subtask 1 retrieves data from database 1, subtask 2 retrieves data from database 2, and subtask 3 retrieves data from database 3. The resources allocated to the first backup and recovery task include the resources allocated to subtask 1, subtask 2, and subtask 3.

[0146] For example, if the first backup and recovery task is to retrieve data from database 1, it includes subtasks 4 and 5. Subtask 4 retrieves data from table 1, and subtask 5 retrieves data from table 2. The resources allocated to the first backup and recovery task include the resources allocated to subtasks 4 and 5.

[0147] In one scenario, after the backup management module establishes a resource control model for backup and recovery, it issues instructions to the first backup and recovery task. The first backup and recovery task dynamically adjusts the number of concurrent tasks of the first backup and recovery task and its subtasks according to the number of threads in the instructions.

[0148] In another scenario, after the backup management module establishes a resource control model for backup and recovery, it issues instructions to the first backup and recovery subtask. The first backup and recovery subtask then dynamically adjusts the number of concurrent tasks of the first backup and recovery subtask and its subtasks according to the number of threads in the instructions.

[0149] The instruction is represented as follows: Pi{ <process name ,task num >, <process name ,task num >, <process name ,task num >,…}

[0150] Here, Pi represents node i, process_name represents all processes in node Pi, and task_num represents the number of threads allocated to each process. After receiving instructions from the backup management module, each process creates threads according to the number of threads, and the created threads concurrently execute the backup and restore tasks.

[0151] In other embodiments, the relationship between the first backup and recovery task and the first resource is unbound after the first backup and recovery task has been completed.

[0152] In one scenario, the first backup and recovery task is completed while the second backup and recovery task is still in progress. At this point, the first resource occupied by the first backup and recovery task is released and added to the system's currently available resources.

[0153] In another scenario, the second backup recovery task is completed while the first backup recovery task is still in progress. At this time, the second resource occupied by the second backup recovery task is released and added to the system's currently available resources.

[0154] Before allocating the first resource to the first backup and recovery task based on the initial system resources and the resources already used by the system, i.e., before step 210, the system monitors the real-time resources. For example... Figure 3 As shown, the backup management module 121, data source acquisition module 122, data processing module 123, and data storage management module 124 in the backup system 120 report indicators to the monitoring system 130. The agent module 125 in the application system 110 reports indicators to the monitoring system 130. Indicators include node-level resources, process-level resources, and inter-node-level resources. After obtaining real-time resources, the monitoring system 130 can visualize and display the real-time resources.

[0155] In some embodiments, when the backup system 120 obtains the initial system resources and the resources already used by the system, the backup management module 121 sends a request to the monitoring system 130 to obtain the real-time resources returned by the monitoring system 130. The real-time resources include the initial system resources and the resources already used by the system. This allows the backup system 120 to allocate resources to the backup and recovery task based on the initial system resources and the resources already used by the system.

[0156] In some embodiments, a real-time resource usage display page is provided. For example, Figure 5 The resource usage displayed in (a) includes, but is not limited to, the overall utilization rate and historical resource utilization rate changes of node-level resources such as CPU, I / O, and memory. The process-level resource utilization rate is shown as the percentage of total node resources such as CPU, I / O, and memory, along with its historical resource utilization rate changes.

[0157] Optionally, when the backup system and application system are deployed on different nodes, the displayed content also includes the overall utilization rate of resources such as network transmission rate, packet loss rate, and latency at the node level, as well as historical resource change trend charts.

[0158] In other embodiments, a real-time display page for dynamically adjusting the number of concurrent tasks is provided, for example, Figure 5 As shown in (b) above, the content displayed on the page includes, but is not limited to, displaying the current number of concurrent tasks of processes in the backup system, application system, and monitoring system.

[0159] In other embodiments, different configuration methods are provided to enable the adaptive backup and restore function. These different configuration methods include, but are not limited to, page-based, license-based, and backend configuration items. For example, Figure 5 The method for enabling the backup and restore function is shown in (c) above.

[0160] It is understood that, in order to achieve the functions in the above embodiments, the computer device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0161] The above text combines Figures 1 to 5 This application details the resource control method for backup and recovery provided in this application. The following section will combine... Figure 6 This application describes the apparatus provided according to the present application. These apparatuses are used to implement the functions of the backup system in the above-described method embodiments, and therefore also achieve the beneficial effects of the above-described method embodiments. In this embodiment, the apparatus is as follows: Figure 1 The backup system shown is still a module (such as a chip) used in computer equipment.

[0162] like Figure 6 As shown, the resource control device 600 includes a communication module 610, a processing module 620, and a storage module 630.

[0163] Resource control device 600 is used to achieve the above. Figure 1 The method embodiment shown illustrates the function of the backup management module 121.

[0164] Communication module 610 is used to obtain initial system resources and resources already used by the system. For example, communication module 610 is used to execute... Figure 1 The task of transmitting backup data and processed data.

[0165] Processing module 620 is configured to allocate first resources to the first backup and recovery task based on the initial system resources and the system's already used resources. The first resources include at least one of computing resources or storage resources, and the system's already used resources include resources used by applications. For example, processing module 620 is configured to execute... Figure 2 Steps 210 and 220.

[0166] Storage module 630 is used to store initial system resources, system resources already used, backup data, etc.

[0167] It should be understood that the resource control device 600 in this application embodiment is implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented using software. Figure 2 The method shown, and its various modules, are also software modules, as are the resource control device 600 and its various modules.

[0168] The resource control device 600 according to the embodiments of this application can correspond to the execution of the methods described in the embodiments of this application, and the above and other operations and / or functions of each unit in the resource control device 600 are respectively for implementing Figure 2 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0169] Figure 7 This is a structural schematic diagram of a computer device 700 provided in this application. Figure 7 As shown, the computer device 700 includes a processor 710, a bus 720, a memory 730, a communication interface 740, a main memory unit 750 (also called a main memory unit), and a processor 760. The processor 710, processor 760, memory 730, main memory unit 750, and communication interface 740 are connected via the bus 720.

[0170] It should be understood that in this embodiment, the processor 710 is a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor is a microprocessor or any conventional processor.

[0171] The computer device 700 also includes a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of programs according to the present application. For example, the processor 760 is a GPU or an NPU.

[0172] The communication interface 740 is used to enable communication between the computer device 700 and external devices or components.

[0173] In this application, computer device 700 is used to implement Figure 1 When performing the backup system function as shown, the communication interface 740 is used to obtain the initial system resources and the system's used resources, so that the processor 710 can execute the first backup and recovery task based on the first resources. The processor 760 is used to allocate first resources to the first backup and recovery task based on the initial system resources and the system's used resources.

[0174] Bus 720 includes a pathway for transferring information between the aforementioned components (such as processor 710, memory 750, and storage 730). In addition to the data bus, bus 720 also includes a power bus, control bus, and status signal bus. However, for clarity, all buses are labeled as bus 720 in the diagram. Bus 720 is a Peripheral Component Interconnect Express (PCIe) bus, or an Extended Industry Standard Architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cachecoherent interconnect for accelerators (CCIX), etc. Bus 720 is divided into address bus, data bus, and control bus.

[0175] As an example, computer device 700 includes multiple processors. The processor is a multi-core (multi-CPU) processor. Here, a processor refers to one or more devices, circuits, and / or computing units used to process data (e.g., computer program instructions).

[0176] It is worth noting that, Figure 7 Taking a computer device 700 comprising one processor 710 and one memory 730 as an example, the processor 710 and memory 730 are used to indicate a type of device or equipment. In a specific embodiment, the number of each type of device or equipment is determined according to business requirements. For example, the computer device 700 includes multiple GPUs or NPUs.

[0177] Memory 750 can be non-transitory memory, which can be a pool of volatile memory or a pool of non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory is random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). Memory 750 is used to store initial system resources, system-used resources, backup data, etc.

[0178] The memory 730 corresponds to the storage medium used in the above method embodiments for storing information such as initial system resources, system used resources, and backup data, for example, a disk, such as a mechanical hard disk or a solid-state hard disk.

[0179] The aforementioned computer device 700 may be a general-purpose device or a special-purpose device. For example, computer device 700 may also be a server or other device with computing capabilities.

[0180] It should be understood that the computer device 700 according to this embodiment may correspond to the resource control device 600 in this embodiment, and correspond to the execution according to Figure 2 The corresponding subject in any of the methods, and the above and other operations and / or functions of each module in the resource control device 700 are respectively for the purpose of implementing Figure 2 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0181] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a computing device. Of course, the processor and storage medium can also exist as discrete components in the computing device.

[0182] Some embodiments of this application provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform one or more steps in the backup and recovery resource control method as described in any of the above embodiments.

[0183] For example, the aforementioned computer-readable storage media include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., CDs (Compact Disks), DVDs (Digital Versatile Disks), etc.), smart cards, and flash memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memory), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the embodiments of this application may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0184] Some embodiments of this application also provide a computer program product. This computer program product includes computer program instructions carried on a non-transitory computer-readable storage medium, which, when executed on a computer, cause the computer to perform one or more steps of the data storage method as described in the above embodiments.

[0185] The beneficial effects of the computer-readable storage medium and computer program product described above are the same as those of the data storage method described in some of the above embodiments, and will not be repeated here.

[0186] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD). The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A resource control method for backup and recovery, characterized in that, include: The first resource is allocated to the first backup and recovery task based on the initial system resources and the system resources already used. The first resource includes at least one of computing resources or storage resources. The system resources already used include resources used by the application. Perform the first backup and recovery task based on the first resource.

2. The method according to claim 1, characterized in that, The system's used resources also include the second resources used by the second backup and recovery task; Allocate primary resources to the first backup and recovery task based on the system's initial resources and the resources already used by the system, including: When performing the second backup and recovery task based on the second resource, the first resource is allocated to the first backup and recovery task according to the initial system resources and the resources already used by the system.

3. The method according to claim 1 or 2, characterized in that, Allocate primary resources to the first backup and recovery task based on the system's initial resources and the resources already used by the system, including: The first resource is allocated to the first backup and recovery task based on the initial resources of the system, the resources already used by the system, and the historical resources allocated to the first backup and recovery task.

4. The method according to any one of claims 1-3, characterized in that, Allocate primary resources to the first backup and recovery task based on the system's initial resources and the resources already used by the system, including: Based on the system resource threshold, the initial system resources and the system resources already used are allocated to the first backup and recovery task as first resources, and the system resource threshold is used to indicate the occupancy status of the initial system resources.

5. The method according to any one of claims 1-4, characterized in that, The first backup and recovery task includes at least one backup and recovery subtask, and the first resource includes the resources of the at least one backup and recovery subtask.

6. The method according to any one of claims 1-5, characterized in that, The system includes an application system and a backup system, with the application system deployed on the first node and the backup system deployed on the second node; The first resource also includes network resources.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Displays the resources used by the backup and restore task.

8. The method according to any one of claims 1-7, characterized in that, Allocate primary resources to the first backup and recovery task based on the system's initial resources and the resources already used by the system, including: Upon receiving an adaptive backup and recovery enable instruction, the system allocates the first resources to the first backup and recovery task based on the initial system resources and the system resources already in use.

9. The method according to claim 8, characterized in that, The method further includes: The adaptive backup and restore control interface is displayed.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: Once the first backup and recovery task is completed, the relationship between the first backup and recovery task and the first resource is unbound.

11. A resource control device, characterized in that, The device includes: The communication module is used to obtain the initial system resources and the resources already used by the system. The processing module is configured to allocate first resources to the first backup and recovery task based on the initial system resources and the system's used resources, wherein the first resources include at least one of computing resources or storage resources, and the system's used resources include resources used by applications. The processing module is further configured to perform a first backup and recovery task based on the first resource.

12. The apparatus according to claim 11, characterized in that, The system's used resources also include the second resources used by the second backup and recovery task; When the processing module allocates the first resources to the first backup and recovery task based on the initial system resources and the system's used resources, it is specifically used for: When performing the second backup and recovery task based on the second resource, the first resource is allocated to the first backup and recovery task according to the initial system resources and the resources already used by the system.

13. The apparatus according to claim 11 or 12, characterized in that, When the processing module allocates the first resources to the first backup and recovery task based on the initial system resources and the system's used resources, it is specifically used for: The first resource is allocated to the first backup and recovery task based on the initial resources of the system, the resources already used by the system, and the historical resources allocated to the first backup and recovery task.

14. The apparatus according to any one of claims 11-13, characterized in that, When the processing module allocates the first resources to the first backup and recovery task based on the initial system resources and the system's used resources, it is specifically used for: Based on the system resource threshold, the initial system resources and the system resources already used are allocated to the first backup and recovery task as first resources, and the system resource threshold is used to indicate the occupancy status of the initial system resources.

15. The apparatus according to any one of claims 11-14, characterized in that, The first backup and recovery task includes at least one backup and recovery subtask, and the first resource includes the resources of the at least one backup and recovery subtask.

16. A computer device, characterized in that, The computer device includes a memory and a plurality of processors, the memory being used to store a set of computer instructions; when the processors execute the set of computer instructions, the plurality of processors perform the operation steps of the method according to any one of claims 1-10.

17. A data processing system, characterized in that, It includes an application system and a backup and recovery system; the backup and recovery system is used to perform the method as described in any one of claims 1-10 to back up and restore the data processed by the application system.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program or instructions, which, when executed by a resource control device, implement the method as described in any one of claims 1-10.

19. A computer program product, characterized in that, The computer program product includes a computer program or instructions, which, when executed by a communication device, implement the method as described in any one of claims 1-10.

Images