Data backup method, device, equipment, system and storage medium

Abstract

The present application discloses a data backup method, which is applied to a scheduling node running a backup scheduling container. The method includes: if it is detected that the current moment is the moment for performing a full backup of a target database, determining at least one first reference node through the backup scheduling container; using the backup scheduling container to respectively calculate the target predicted resource consumption of the first reference node when performing a full backup of the target database, thereby obtaining at least one target predicted resource consumption; using the backup scheduling container to determine a target node from the at least one first reference node based on the at least one target predicted resource consumption; and sending an execution instruction to the target node through the backup scheduling container, thereby achieving centralized scheduling and management of backups, improving backup efficiency, and ensuring database service performance. The present application also discloses a data backup device, equipment, system, and storage medium.

Description

Technical Field

[0001] The present application relates to the field of cloud computing service technology, and in particular to a data backup method, apparatus, device, system, and storage medium. Background Art

[0002] With the rapid development of internet technology and the advent of the cloud computing era, various cloud computing solutions have emerged one after another. Platform as a Service (Paas), as a new service provision model, has been rapidly deployed and applied. Currently, in the Paas field, there are two main approaches to cloud data: one is to deploy database services through virtualization based on cloud computing management platforms (OpenStack), and the other is to deploy containerized database services based on platforms for managing container clusters (Kubernetes, Kubernetes). On Kubernetes-based containerized database service platforms, due to the isolation of containers, each machine runs multiple database instances. As a result, each database instance is allocated relatively few operating resources, resulting in poor operational stability of the database instances. Therefore, data backup is required for Kubernetes-based containerized database service platforms. Currently, there are two main solutions for backing up data on Kubernetes-based containerized database service platforms: one is for users to set a backup policy and bind a database instance in the cloudified database cluster to perform regular backups according to the backup policy; the other is for the cloudified database cluster to control backup execution and select a database instance from the database cluster for backup when the backup time arrives.

[0003] Currently, in the above two backup solutions, when selecting a database instance for backup operation, the load of the backup node corresponding to the database instance is not considered. If the load of the physical backup device corresponding to the database instance to be backed up is high, the backup efficiency of the database instance will be low, which will seriously reduce the service performance of the corresponding database.

[0004] Application Contents

[0005] In order to solve the above technical problems, the present application hopes to provide a data backup method, device, equipment, system and storage medium, which solves the problem that the load of the backup node corresponding to the database instance is not fully considered when performing data backup, resulting in low backup efficiency. A method for determining the database instance used for backing up data is proposed, which realizes centralized scheduling and management of backup, improves backup efficiency, and ensures the service performance of the database.

[0006] The technical solution of this application is achieved as follows:

[0007] In a first aspect, a data backup method is provided, the method being applied to a scheduling node running a backup scheduling container, the method comprising:

[0008] If it is detected that the current time is a time to perform a full backup of the target database, at least one first reference node is determined through the backup scheduling container; wherein the at least one first reference node is used to run a backup remote procedure call (RPC) container bound to a target instance corresponding to the target database;

[0009] Obtaining at least one target predicted resource consumption by counting target predicted resource consumption when the first reference node fully backs up the target database through a backup scheduling container;

[0010] Determining a target node from the at least one first reference node by using a backup scheduling container to consume resources based on the at least one target prediction;

[0011] An execution instruction is sent to the target node through a backup scheduling container; wherein the execution instruction is used to instruct the target RPC container running on the target node to perform a full backup operation on the target database.

[0012] Optionally, the obtaining of at least one target predicted resource consumption by respectively counting target predicted resource consumption when the first reference node fully backs up the target database through the backup scheduling container includes:

[0013] receiving, through a backup scheduling container, the target predicted consumed resource sent by each of the first reference nodes, to obtain the at least one target predicted consumed resource;

[0014] Alternatively, a first consumable resource and a second consumable resource sent by each first reference node are received through a backup scheduling container; wherein the first consumable resource includes the consumable resource when the corresponding first reference node runs the target instance, and the second consumable resource includes the consumable resource other than the first consumable resource when the corresponding first reference node runs the target instance;

[0015] The sum of the first consumed resource and the second consumed resource of each first reference node is determined through a backup scheduling container to obtain at least one of the target predicted consumed resources.

[0016] Optionally, the second consumable resources include at least the consumable resources of the reference instance running when the corresponding first reference node runs the target instance and the running consumable resources of the target running statement other than the target instance and the reference instance when the corresponding first reference node runs the target instance; wherein, the reference instance is an instance other than the target instance running when the target instance is run in the corresponding first reference node.

[0017] Optionally, if it is detected that the current time is a time for performing a full backup of the target database, determining at least one first reference node through the backup scheduling container includes:

[0018] If it is detected that the current time is a time for performing a full backup of the target database, the target information of the target database sent by each second reference node is received through the backup scheduling container to obtain at least one target information;

[0019] At least one first reference node is determined from at least one second reference node based on at least one of the target information.

[0020] Optionally, the determining, based on at least one of the target information, at least one of the first reference nodes from at least one of the second reference nodes includes:

[0021] Determining at least one target information including a target identifier from at least one of the target information; wherein the target identifier is used to identify that the corresponding second reference node is not down;

[0022] From at least one of the second reference nodes, a node matching at least one target information including a target identifier is determined to obtain at least one of the first reference nodes.

[0023] Optionally, if it is detected that the current time is a time for performing a full backup of the target database, before determining at least one first reference node through the backup scheduling container, the method further includes:

[0024] Obtaining a preset backup strategy for the target database through the backup scheduling container;

[0025] The preset backup policy is sent to at least one second reference node through the backup scheduling container; wherein, at least one second reference node is a node running a backup RPC container bound to a target instance corresponding to the target database, and the preset backup policy is used to instruct each second reference node to send the target predicted resource consumption to the scheduling node when detecting that the size of the target database exceeds a preset database threshold through the set backup RPC container, and at least one second reference node includes at least one first reference node.

[0026] Optionally, the execution instruction is further used to indicate that all incremental backup operations between the current time and the next full backup operation time of the target database are executed by the target node.

[0027] Optionally, the predicting consumption of resources based on the at least one target by the backup scheduling container and determining the target node from the at least one first reference node includes:

[0028] determining a maximum resource of at least one of the first reference nodes through the backup scheduling container;

[0029] Determine, by the backup scheduling container, a difference between each first reference node and a corresponding target predicted resource consumption, to obtain at least one reference difference;

[0030] Determining, by the backup scheduling container, a target difference value having a maximum value among the at least one reference difference value;

[0031] The backup scheduling container determines, from the at least one first reference node, a first reference node corresponding to the target difference as the target node.

[0032] In a second aspect, a data backup device is provided, comprising: a first determining unit, a counting unit, a second determining unit, and a sending unit; wherein:

[0033] The first determining unit is configured to, if it is detected that the current moment is a moment for performing a full backup of the target database, determine at least one first reference node through the backup scheduling container; wherein the at least one first reference node is configured to run a backup remote procedure call (RPC) container bound to a target instance corresponding to the target database;

[0034] The statistical unit is configured to respectively count target predicted resource consumption when the first reference node fully backs up the target database through the backup scheduling container to obtain at least one target;

[0035] The second determining unit is configured to determine a target node from the at least one first reference node by using a backup scheduling container to consume resources based on the at least one target prediction;

[0036] The sending unit is used to send an execution instruction to the target node through a backup scheduling container; wherein the execution instruction is used to instruct the target RPC container running on the target node to perform a full backup operation on the target database.

[0037] In a third aspect, a data backup device is provided, comprising: a memory, a processor, and a communication bus; wherein:

[0038] The memory is used to store executable instructions;

[0039] The communication bus is used to realize the communication connection between the processor and the memory;

[0040] The processor is configured to run the backup scheduling container to execute the data backup method stored in the memory, thereby implementing the steps of any of the aforementioned data backup methods.

[0041] In a fourth aspect, a data backup system is provided, comprising: a scheduling node for running a backup scheduling container, and at least one second reference node for running a backup RPC container bound to a target instance corresponding to a target database; wherein:

[0042] The scheduling node is used to determine at least one first reference node through a backup scheduling container if it is detected that the current moment is the moment for performing a full backup of the target database; wherein, the at least one first reference node is used to run a backup remote procedure call RPC container bound to a target instance corresponding to the target database; wherein, at least one second reference node includes at least one first reference node, and the target predicted resource consumption when the first reference node performs a full backup of the target database is respectively counted through the backup scheduling container to obtain at least one target predicted resource consumption; the target node is determined from the at least one first reference node based on the at least one target predicted resource consumption through the backup scheduling container; and an execution instruction is sent to the target node through the backup scheduling container; wherein, the execution instruction is used to instruct the target RPC container run by the target node to perform a full backup operation on the target database;

[0043] The target node is configured to receive an execution instruction sent by the scheduling node through the target RPC container; and in response to the execution instruction, perform a full backup operation on the target database at a full backup time corresponding to the target database.

[0044] In a fifth aspect, a storage medium is provided, on which a data backup method is stored, and when the data backup method is executed by a processor, the steps of the data backup method as described in any one of the above items are implemented.

[0045] The embodiment of the present application provides a data backup method, apparatus, device, system and storage medium. If it is detected that the current moment is the time to perform a full backup of the target database, the scheduling node determines at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database through the backup scheduling container, and then uses the backup scheduling container to respectively count the target predicted resource consumption of the first reference node when performing a full backup of the target database, obtain at least one target predicted resource consumption, and based on the at least one target predicted resource consumption, determine the target node from the at least one first reference node, and finally send an execution instruction to the target node. In this way, the scheduling node centrally analyzes at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database, determines the target node based on the target predicted resource consumption of each first reference node, and instructs the target node to perform a full backup operation for the target database, thereby solving the problem that the load of the backup node corresponding to the database instance is not fully considered when performing data backup, resulting in low backup efficiency. A method for determining the database instance used to perform data backup is proposed, which realizes centralized scheduling and management of backup, improves backup efficiency, and ensures the service performance of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the data backup method provided in this embodiment Figure 1 ;

[0047] Figure 2 Schematic diagram of the data backup method provided in this embodiment Figure 2 ;

[0048] Figure 3 An application scenario of the data backup method provided in the embodiment of the present application;

[0049] Figure 4 Schematic diagram of the data backup method provided in this embodiment Figure 3 ;

[0050] Figure 5 A schematic diagram of the structure of a data backup device provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of a data backup device provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of the structure of a data backup system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0054] The embodiment of the present application provides a data backup method, referring to Figure 1 As shown, the scheduling node for running the backup scheduling container includes the following steps:

[0055] Step 101: If it is detected that the current time is a time for performing a full backup of a target database, at least one first reference node is determined through a backup scheduling container.

[0056] The at least one first reference node is used to run a backup remote procedure call RPC container bound to a target instance corresponding to the target database.

[0057] In an embodiment of the present application, a scheduling node is a node device running a backup scheduling container, such as a server or other device. At least one first reference node is a node device running a backup remote procedure call (RPC) container bound to a target instance corresponding to a target database, such as a computer device or other device with a storage function. The current moment for performing a full backup of the target database can be determined based on the full backup cycle of the target database. When the backup scheduling container detects that the current moment is the backup moment for performing a full backup of the target database, it determines at least one first reference node corresponding to the target database at the current moment.

[0058] It should be noted that the at least one first reference node may be a node that meets preset requirements among all nodes running a backup RPC container bound to a target instance corresponding to the target database. The preset requirement may be, for example, that the corresponding backup RPC container running on the node is alive.

[0059] Step 102: Count target predicted resource consumption when the first reference node fully backs up the target database through the backup scheduling container to obtain at least one target predicted resource consumption.

[0060] In an embodiment of the present application, the backup scheduling container calculates the target predicted resource consumption corresponding to each of the at least one determined first reference node when performing a full backup of the target database to obtain the target predicted resource consumption of each first reference node, thereby obtaining the target predicted resource consumption corresponding to the at least one first reference node. The target predicted resource consumption of each first reference node is primarily disk resource input / output (IO) consumption.

[0061] Step 103: Determine a target node from at least one first reference node by using a backup scheduling container to consume resources based on at least one target prediction.

[0062] In an embodiment of the present application, after obtaining at least one target predicted consumption resource, the backup scheduling container determines a target predicted consumption resource with the minimum predicted consumption resource from the at least one target predicted consumption resource, and determines the node corresponding to the target predicted consumption resource with the minimum predicted consumption resource from at least one first reference node as the target node. Alternatively, the backup scheduling container determines the remaining resources of each first reference node based on the at least one target predicted consumption resource.

[0063] Step 104: Send the execution instruction to the target node through the backup scheduling container.

[0064] The execution instruction is used to instruct the target RPC container running on the target node to perform a full backup operation on the target database.

[0065] In an embodiment of the present application, after the backup scheduling container determines the target node, it generates instructions for instructing the target node to perform a full backup operation on the target database. Correspondingly, after the target node receives the execution instruction, the target RPC container in the target node controls its bound target instance to perform a full backup operation on the target database. In this way, by analyzing the resource consumption of nodes corresponding to multiple database instances, a database instance that can perform the backup operation is determined, reducing the network bottleneck of disk resource IO on some nodes during the backup peak period and improving the service performance of the database.

[0066] The embodiment of the present application provides a data backup method. If it is detected that the current moment is the time to perform a full backup of the target database, the scheduling node determines at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database through the backup scheduling container, and then uses the backup scheduling container to respectively count the target predicted resource consumption of the first reference node when performing a full backup of the target database, obtains at least one target predicted resource consumption, and based on the at least one target predicted resource consumption, determines the target node from the at least one first reference node, and finally sends an execution instruction to the target node. In this way, the scheduling node centrally analyzes at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database, determines the target node based on the target predicted resource consumption of each first reference node, and instructs the target node to perform a full backup operation for the target database, thereby solving the problem that the load of the backup node corresponding to the database instance is not fully considered when performing data backup, resulting in low backup efficiency. A method for determining the database instance used to perform data backup is proposed, which realizes centralized scheduling and management of backup, improves backup efficiency, and ensures the service performance of the database.

[0067] Based on the foregoing embodiments, embodiments of the present application provide a data backup method, which is applied to a scheduling node running a backup scheduling container. The method includes the following steps:

[0068] Step 201: If it is detected that the current time is a time for performing a full backup of a target database, at least one first reference node is determined through a backup scheduling container.

[0069] The at least one first reference node is used to run a backup remote procedure call RPC container bound to a target instance corresponding to the target database.

[0070] In an embodiment of the present application, a cloud database cluster in a platform of a containerized database service of k8s is used as an example for explanation. In addition to running a backup scheduling container, the scheduling node may also run containers that implement other functions, such as a backup RPC container bound to an instance corresponding to another database. Similarly, in addition to running a backup RPC container bound to a target instance corresponding to a target database, at least one first reference node may also run a backup RPC container bound to an instance corresponding to another database. The backup RPC container bound to the target instance corresponding to the target database may be set in only some nodes in the cloud database cluster in the platform of the containerized database service of k8s, or may be set in all nodes in the cloud database cluster in the platform of the containerized database service of k8s, depending on the actual application scenario.

[0071] Taking database A as the target database for explanation, when the backup scheduling container determines the time when a full backup of database A is required at the current moment based on the full backup cycle of database A, the backup scheduling container determines that at least one first reference node B1, B2, B3, and B4 corresponding to the backup RPC container bound to the database instance of database A is running.

[0072] Step 202: Count target predicted resource consumption when the first reference node fully backs up the target database through the backup scheduling container to obtain at least one target predicted resource consumption.

[0073] In an embodiment of the present application, the backup mobilization container respectively counts the target predicted consumption resources corresponding to the first reference nodes B1, B2, B3, and B4 when performing a full backup of the target database, and obtains the target predicted consumption resources X1, X2, X3, and X4 corresponding to the first reference nodes B1, B2, B3, and B4 respectively.

[0074] Step 203: Determine a target node from at least one first reference node by using a backup scheduling container to consume resources based on at least one target prediction.

[0075] In an embodiment of the present application, the backup scheduling container determines the target predicted resource consumption with the smallest value from X1, X2, X3, and X4. Assuming it is X3, the first reference node B3 corresponding to X3 can be determined as the target node from the first reference nodes B1, B2, B3, and B4. That is, the target predicted resource consumption corresponding to the target node B3 is the smallest when performing a full backup of the target database.

[0076] Step 204: Send the execution instruction to the target node through the backup scheduling container.

[0077] The execution instruction is used to instruct the target RPC container running on the target node to perform a full backup operation on the target database.

[0078] In the embodiment of the present application, the backup scheduling container sends an execution instruction to the target node B3, so that the target node B3 performs a full backup operation on the target database.

[0079] Based on the foregoing embodiment, in other embodiments of the present application, step 202 may be implemented by step 202a, that is, the first reference node calculates its target predicted resource consumption when performing a full backup of the target database and directly sends it to the backup scheduling container, or step 202 may be implemented by steps 202b to 202c, that is, the first reference node sends the first consumption resource and the second consumption resource used to predict its target predicted resource consumption when performing a full backup of the target database to the backup scheduling container, and the backup scheduling container determines the target predicted resource consumption corresponding to the first reference node based on the received first consumption resource and the second consumption resource:

[0080] Step 202a: Receive the target predicted resource consumption sent by each first reference node through the backup scheduling container to obtain at least one target predicted resource consumption.

[0081] In the embodiment of the present application, after each first reference node has determined the target predicted resource consumption when performing a full backup operation on its corresponding target database, the corresponding target predicted resource consumption is directly sent to the backup scheduling container.

[0082] Step 202b: Receive the first consumed resource and the second consumed resource sent by each first reference node through the backup scheduling container.

[0083] The first consumable resources include the consumable resources when the corresponding first reference node runs the target instance, and the second consumable resources include the consumable resources when the corresponding first reference node runs the target instance except the consumable resources of running the target instance.

[0084] In an embodiment of the present application, the first consumable resource is the consumable resource required by the first reference node when the first reference node runs the target instance to perform a full backup of the target database. Since other processes other than the full backup operation of the target database may be running in the first reference node, when considering the consumable resource when the first reference node performs a full backup of the target database, the consumable resource of other processes, i.e., the second consumable resource, must also be considered.

[0085] Step 202c: Determine the sum of the first consumed resource and the second consumed resource of each first reference node through the backup scheduling container to obtain at least one target predicted consumed resource.

[0086] In an embodiment of the present application, the backup scheduling container calculates the sum of the first consumption resource and the second consumption resource received for each first reference node to obtain the target predicted consumption resource of each first reference node, thereby obtaining at least one target predicted consumption resource. For example, assuming that the first consumption resource received by the backup scheduling container is X11 for the first reference node B1, the second consumption resource of the first reference node B1 is X12, the first consumption resource of the first reference node B2 is X21, the second consumption resource of the first reference node B2 is X22, the first consumption resource of the first reference node B3 is X31, the second consumption resource of the first reference node B3 is X32, the first consumption resource of the first reference node B4 is X41, and the second consumption resource of the first reference node B4 is X42, accordingly, the target predicted consumption resource of the first reference node B1 determined by the backup scheduling container is X1=X11+X12, the target predicted consumption resource of the first reference node B2 is X2=X21+X22, the target predicted consumption resource of the first reference node B3 is X3=X31+X32, and the target predicted consumption resource of the first reference node B4 is X4=X41+X42.

[0087] It should be noted that in step 202a, each first reference node determines its own corresponding target predicted consumption resources by referring to the implementation process in step 202b and step 203c, that is, after each first reference node determines the first consumption resources and second consumption resources corresponding to the full backup of the target database, it calculates the first consumption resources and the second consumption resources to obtain its own corresponding target predicted consumption resources, and sends the target predicted consumption resources to the backup scheduling container.

[0088] Based on the foregoing embodiments, in other embodiments of the present application, the second consumable resources include at least the consumable resources of the reference instance running when the corresponding first reference node runs the target instance and the running consumable resources of the target running statement other than the running target instance and the reference instance when the corresponding first reference node runs the target instance; wherein, the reference instance is an instance other than the target instance running when the target instance is running in the corresponding first reference node.

[0089] In an embodiment of the present application, when determining the second resource consumption, if other reference instances besides the target database are running in the first reference node, it is necessary to count the resource consumption when the target instance is running to fully back up the target database, when the other reference instances are also running, and the corresponding resource consumption when some other target execution statements are running. The target execution statement may be, for example, Structured Query Language (SQL).

[0090] Based on the above embodiment, in other embodiments of the present application, step 201 can be implemented by steps 201a to 201b:

[0091] Step 201a: If it is detected that the current time is the time for performing a full backup of the target database, the target information of the target database sent by each second reference node is received through the backup scheduling container to obtain at least one target information.

[0092] In an embodiment of the present application, the target information is information used to indicate whether the target database in the first reference node is down.

[0093] Step 201b: Based on at least one target information, determine at least one first reference node from at least one second reference node.

[0094] In an embodiment of the present application, at least one target information corresponding to at least one second reference node is analyzed to determine at least one first reference node from the at least one second reference node.

[0095] Based on the above embodiment, in other embodiments of the present application, step 201b can be implemented by steps a11 to:

[0096] Step a11: Determine at least one target information including a target identifier from at least one target information.

[0097] The target identifier is used to identify that the corresponding second reference node is not down.

[0098] Step a12: Determine, from the at least one second reference node, a node that matches at least one target information including a target identifier, and obtain at least one first reference node.

[0099] In an embodiment of the present application, nodes in a downtime state are removed from at least one second reference node to obtain at least one first reference node in a normal working state.

[0100] Based on the above embodiments, in other embodiments of the present application, refer to Figure 2As shown, before the scheduling node executes step 201, it is also used to execute steps 205 to 206:

[0101] Step 205: Obtain a preset backup policy for the target database through the backup scheduling container.

[0102] In the embodiment of the present application, the preset backup strategy can be set according to the actual backup requirements of the target database.

[0103] Step 206: Send the preset backup policy to at least one second reference node through the backup scheduling container.

[0104] Among them, at least one second reference node is a node running a backup RPC container bound to a target instance corresponding to the target database. The preset backup strategy is used to instruct each second reference node to send target predicted consumption resources to the scheduling node when it detects through the set backup RPC container that the size of the target database exceeds the preset database threshold. The at least one second reference node includes at least one first reference node.

[0105] In an embodiment of the present application, the backup scheduling container sends the preset backup policy to at least one second reference node, and the at least one second reference node can execute the backup policy for the target database according to the preset backup policy. The preset database threshold is an empirical value obtained based on a large number of experiments.

[0106] Based on the foregoing embodiment, in other embodiments of the present application, the execution instruction is also used to indicate that the incremental backup operations between the current time and the next full backup operation time of the target database are all executed by the target node.

[0107] Based on the above embodiment, in other embodiments of the present application, step 203 can be implemented by steps 203a to 203d:

[0108] Step 203a: Determine the maximum resource of at least one first reference node through the backup scheduling container.

[0109] In the embodiment of the present application, the maximum resource of each first reference node is the maximum resource that each first reference node can provide.

[0110] Step 203b: Determine the difference between each first reference node and the corresponding target predicted resource consumption through the backup scheduling container to obtain at least one reference difference.

[0111] Step 203c: Determine a target difference value having the largest value among at least one reference difference value through the backup scheduling container.

[0112] Step 203d: Determine, through the backup scheduling container, from at least one first reference node, a first reference node corresponding to the target difference as the target node.

[0113] In this way, when the maximum resources of at least one first reference node are different, the first reference node with the most remaining resources is determined as the target node by judging the remaining resources of each first reference node, ensuring that the node with the most remaining resources can be determined to perform the backup operation, thereby improving the backup operation for the target database.

[0114] Based on the above embodiments, the present application provides an application scenario of a data backup method. Figure 3 As shown, it includes a scheduling node 31, a database cluster 32 and a cloud storage distributed system (Ceph) 33; wherein, a backup scheduling container and a backup policy are running in the scheduling node 31, and the database cluster 32 includes node 1 running a backup RPC container 1 bound to the target instance of the target database, node 2 running a backup RPC container 2 bound to the target instance of the target database, node 3 running a backup RPC container 3 bound to the target instance of the target database, ..., node n running a backup RPC container n bound to the target instance of the target database.

[0115] based on Figure 3 The application scenario shown in Figure 4 The implementation process of the data backup method can be as follows:

[0116] Step 401, start.

[0117] Step 402: The backup scheduling container sends the user backup policy.

[0118] Among them, the scheduling node collects the user backup policy set by the user on the web front end through the backup scheduling container, and sends the user backup policy to nodes 1, 2, 3, ..., and n corresponding to the backup RPC container bound to the corresponding database cluster instance.

[0119] Step 403 : The backup RPC container determines whether the database size exceeds the database threshold. If the database size exceeds the database threshold, execute step 404 ; otherwise, execute step 407 .

[0120] First, the backup scheduling container checks the heartbeat signals of the nodes, specifically detecting whether the target database in nodes 1, 2, 3, ..., and n has crashed. If a crash is detected in any node, the crashed node is removed so that it is not considered in subsequent analysis. For example, if node 3 is detected to be down, node 3 is removed.

[0121] Second, after receiving the user backup policy, the backup RPC containers of node 1, node 2, node 4, ..., node n periodically collect the corresponding database instance size according to the collection period set in the user backup policy.

[0122] Step 404: Back up the RPC container and upload the consumed resource information.

[0123] When the database size exceeds the threshold set in the user's backup policy, resource consumption information for the target instance corresponding to the target database is sent to the backup scheduling container of the scheduling node. The uploaded resource consumption information includes at least the preset backup duration required for each node to fully back up the target database, the SQL statement execution start time, and the SQL statement execution duration. The preset backup duration can be calculated as follows: the preset backup duration is equal to the database instance size divided by the disk IO rate. The SQL execution duration can be calculated as: the sum of the scanned record sizes in the SQL execution cost divided by the disk IO rate.

[0124] Step 405: The backup scheduling container determines the target node.

[0125] The backup scheduling container of the scheduling node aggregates resource consumption information uploaded by nodes 1, 2, 4, ..., and n for the target database. The backup start time of the target database is determined according to the user backup policy, which can be determined by the full backup cycle for the target database set in the user backup policy.

[0126] When the target database is deployed in a multi-instance database cluster, the instances in the cluster are typically distributed across different nodes. The total resource consumption for each node (i.e., the target predicted resource consumption) must take into account the resource consumption caused by the backup period of other database instances on the node that are not part of the cluster, as well as the execution period of all SQL statements on the node during the target database backup phase. Therefore, the total resource consumption for the target database on each node can be calculated using the following formula: ((Execution duration of each SQL statement on the node + Start time of each corresponding SQL statement execution - Start time of the cluster backup) + (Backup duration of other instances on the node + Start time of other instances backup execution - Start time of the cluster backup) + Target database's) * Disk I / O rate. For example, assuming there are three SQL statements, SQL1 and SQL2, and other instances include Instance 1 and Instance 2, the total resource consumption of the corresponding nodes can be calculated as: ((SQL1 execution duration + SQL1 execution start time - target database backup start time) + (SQL2 execution duration + SQL2 execution start time - target database backup start time) + (Instance 1 backup execution duration + Instance 2 backup execution start time - target database backup start time) + (Instance 2 backup execution duration + Instance 2 backup execution start time - target database backup start time)) * disk I / O rate. The backup scheduling container determines the node with the lowest total resource consumption as the target node and sends the backup execution task to the target node.

[0127] Step 406: The target node executes the backup task for the target database.

[0128] After receiving the backup task, the target node executes the backup task for the target database. It should be noted that all incremental backups before the next full backup task of the target database are executed by the target node determined in step 405. For example, Figure 3 Node 2 is identified as the target node. Node 2 can back up the target database in full to Ceph cloud storage using the Simple Storage Service (S3) object storage interface.

[0129] Step 407: The backup RPC container does not report resource consumption information.

[0130] Step 408, end.

[0131] It should be noted that, for the description of the same steps and contents in this embodiment as those in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0132] The embodiment of the present application provides a data backup method. If it is detected that the current moment is the time to perform a full backup of the target database, the scheduling node determines at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database through the backup scheduling container, and then uses the backup scheduling container to respectively count the target predicted resource consumption of the first reference node when performing a full backup of the target database, obtains at least one target predicted resource consumption, and based on the at least one target predicted resource consumption, determines the target node from the at least one first reference node, and finally sends an execution instruction to the target node. In this way, the scheduling node centrally analyzes at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database, determines the target node based on the target predicted resource consumption of each first reference node, and instructs the target node to perform a full backup operation for the target database, thereby solving the problem that the load of the backup node corresponding to the database instance is not fully considered when performing data backup, resulting in low backup efficiency. A method for determining the database instance used to perform data backup is proposed, which realizes centralized scheduling and management of backup, improves backup efficiency, and ensures the service performance of the database.

[0133] Based on the above embodiments, the embodiments of the present application provide a data backup device, which can be applied to Figures 1-2 In the data backup method provided in the corresponding embodiment, refer to Figure 5 As shown, the data backup device 5 may include: a first determination unit 51, a statistics unit 52, a second determination unit 53 and a sending unit 54; wherein:

[0134] A first determining unit 51 is configured to, upon detecting that the current time is a time for performing a full backup of the target database, determine at least one first reference node through a backup scheduling container; wherein the at least one first reference node is configured to run a backup remote procedure call (RPC) container bound to a target instance corresponding to the target database;

[0135] A statistics unit 52 is configured to count target predicted resource consumption when the first reference node fully backs up the target database using the backup scheduling container to obtain at least one target predicted resource consumption;

[0136] A second determining unit 53 is configured to determine a target node from at least one first reference node by using a backup scheduling container to consume resources based on at least one target prediction;

[0137] The sending unit 54 is configured to send an execution instruction to the target node through the backup scheduling container; wherein the execution instruction is configured to instruct the target RPC container running on the target node to perform a full backup operation on the target database.

[0138] In other embodiments of the present application, the statistical unit includes: a first receiving module and a first determining module; wherein:

[0139] A first receiving module is configured to receive, through a backup scheduling container, a target predicted resource consumption sent by each first reference node, and obtain at least one target predicted resource consumption;

[0140] Alternatively, the first receiving module is configured to receive, through the backup scheduling container, first consumed resources and second consumed resources sent by each first reference node; wherein the first consumed resources include consumed resources when the corresponding first reference node runs the target instance, and the second consumed resources include consumed resources when the corresponding first reference node runs the target instance except for the consumed resources of running the target instance;

[0141] The first determining module is configured to determine the sum of the first consumed resource and the second consumed resource of each first reference node through a backup scheduling container to obtain at least one target predicted consumed resource.

[0142] In other embodiments of the present application, the second consumable resources include at least the consumable resources of the reference instance running when the corresponding first reference node runs the target instance and the running consumable resources of the target running statement other than the running target instance and the reference instance when the corresponding first reference node runs the target instance; wherein, the reference instance is an instance other than the target instance running when the target instance is running in the corresponding first reference node.

[0143] In other embodiments of the present application, the first determining unit includes: a second receiving module and a second determining module; wherein:

[0144] a second receiving module configured to, if detecting that the current moment is a moment for performing a full backup of the target database, receive target information of the target database sent by each second reference node through the backup scheduling container to obtain at least one target information;

[0145] The second determining module is configured to determine at least one first reference node from at least one second reference node based on at least one target information.

[0146] In other embodiments of the present application, the second determining module is specifically configured to implement the following steps:

[0147] Determining at least one target information including a target identifier from the at least one target information; wherein the target identifier is used to identify that the corresponding second reference node is not down;

[0148] From the at least one second reference node, a node matching at least one target information including a target identifier is determined to obtain at least one first reference node.

[0149] In other embodiments of the present application, before the first determining unit, the data backup device further includes an acquiring unit; wherein:

[0150] An acquisition unit, configured to acquire a preset backup policy for a target database through a backup scheduling container;

[0151] A sending unit is used to send a preset backup policy to at least one second reference node through a backup scheduling container; wherein, the at least one second reference node is a node running a backup RPC container bound to a target instance corresponding to a target database, and the preset backup policy is used to instruct each second reference node to send a target predicted resource consumption to the scheduling node when detecting that the size of the target database exceeds a preset database threshold through the set backup RPC container, and the at least one second reference node includes at least one first reference node.

[0152] In other embodiments of the present application, the execution instruction is also used to indicate that all incremental backup operations between the current time and the next full backup operation time of the target database are executed by the target node.

[0153] In other embodiments of the present application, the second determining unit includes a third determining module; wherein:

[0154] A third determining module, configured to determine a maximum resource of at least one first reference node through a backup scheduling container;

[0155] A third determining module is configured to determine, through the backup scheduling container, a difference between each first reference node and the corresponding target predicted resource consumption to obtain at least one reference difference;

[0156] A third determining module is configured to determine, through the backup scheduling container, a target difference value having the largest determined value among at least one reference difference value;

[0157] The third determining module is configured to determine, from at least one first reference node using a backup scheduling container, a first reference node corresponding to the target difference as a target node.

[0158] It should be noted that the information interaction process between the units and modules of the data backup device in this embodiment can be referred to Figures 1-2 The implementation process of the data backup method provided in the corresponding embodiment will not be repeated here.

[0159] The embodiment of the present application provides a data backup device. If it is detected that the current moment is the time to perform a full backup of the target database, the scheduling node determines at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database through the backup scheduling container, and then uses the backup scheduling container to respectively count the target predicted resource consumption of the first reference node when performing a full backup of the target database, obtains at least one target predicted resource consumption, and based on the at least one target predicted resource consumption, determines the target node from the at least one first reference node, and finally sends an execution instruction to the target node. In this way, the scheduling node centrally analyzes at least one first reference node running a backup RPC container bound to the target instance corresponding to the target database, determines the target node based on the target predicted resource consumption of each first reference node, and instructs the target node to perform a full backup operation for the target database, thereby solving the problem that the load of the backup node corresponding to the database instance is not fully considered when performing data backup, resulting in low backup efficiency. A method for determining the database instance used to perform data backup is proposed, which realizes centralized scheduling and management of backup, improves backup efficiency, and ensures the service performance of the database.

[0160] Based on the above embodiments, the embodiments of the present application provide a data backup device, which can be applied to Figures 1-2 In the data backup method provided in the corresponding embodiment, refer to Figure 6 As shown, the data backup device 6 may include: a processor 61, a memory 62 and a communication bus 63, wherein:

[0161] Memory 62, for storing executable instructions;

[0162] A communication bus 63 is used to implement communication between the processor 61 and the memory 62;

[0163] Processor 61 is used to execute the data backup method stored in memory 62 to achieve Figures 1-2 The implementation process of the data backup method provided in the corresponding embodiment will not be repeated here.

[0164] Based on the above embodiments, the present application provides a data backup system. Figure 7 As shown, the data backup system 7 includes: a scheduling node 71 for running a backup scheduling container, and at least one second reference node 72 for running a backup RPC container bound to a target instance corresponding to a target database; wherein:

[0165] Scheduling node 71 is configured to, upon detecting that the current moment is a moment for performing a full backup of a target database, determine at least one first reference node through a backup scheduling container; wherein the at least one first reference node is configured to run a backup remote procedure call (RPC) container bound to a target instance corresponding to the target database, and the at least one second reference node includes the at least one first reference node; calculate, through the backup scheduling container, target predicted resource consumption when the first reference node performs a full backup of the target database, to obtain at least one target predicted resource consumption; determine, through the backup scheduling container, a target node from the at least one first reference node based on the at least one target predicted resource consumption; and send an execution instruction to the target node through the backup scheduling container; wherein the execution instruction is configured to instruct the target RPC container running on the target node to perform a full backup operation on the target database;

[0166] The target node 72 is configured to receive the execution instruction sent by the scheduling node through the target RPC container; in response to the execution instruction, perform a full backup operation on the target database at the full backup time corresponding to the target database.

[0167] Based on the above embodiment, in other embodiments of the present application, the specific implementation process of the scheduling node 71 for running the backup scheduling container can be referred to Figures 1-2 The implementation process of the embodiment shown will not be described in detail here. It should be noted that the scheduling node 71 is the aforementioned data backup device.

[0168] Based on the above embodiments, the embodiments of the present application provide a computer-readable storage medium, referred to as a storage medium, which stores one or more programs, which can be executed by one or more processors to implement the reference Figures 1-2 The implementation process of the data backup method provided in the corresponding embodiment will not be repeated here.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

Claims

1. A data backup method, characterized in that: The method is applied to a scheduling node running a backup scheduling container, and the method includes: If it is detected that the current time is a time to perform a full backup of the target database, at least one first reference node is determined through the backup scheduling container; wherein the at least one first reference node is used to run a backup remote procedure call (RPC) container bound to a target instance corresponding to the target database; Obtaining at least one target predicted resource consumption by respectively counting target predicted resource consumption when the first reference node fully backs up the target database through a backup scheduling container; Determining a target node from the at least one first reference node by using a backup scheduling container to consume resources based on at least one of the target predictions; Sending an execution instruction to the target node through the backup scheduling container; wherein the execution instruction is used to instruct the target RPC container running on the target node to perform a full backup operation on the target database; The step of respectively counting target predicted resource consumption when the first reference node fully backs up the target database through the backup scheduling container to obtain at least one target predicted resource consumption includes: receiving, through a backup scheduling container, the target predicted consumed resource sent by each of the first reference nodes, and obtaining at least one of the target predicted consumed resources; Alternatively, a first consumable resource and a second consumable resource sent by each first reference node are received through a backup scheduling container; wherein the first consumable resource includes the consumable resource when the corresponding first reference node runs the target instance, and the second consumable resource includes the consumable resource other than the first consumable resource when the corresponding first reference node runs the target instance; Determine the sum of the first consumed resource and the second consumed resource of each first reference node through a backup scheduling container to obtain at least one target predicted consumed resource; The step of consuming resources based on at least one target prediction by using a backup scheduling container and determining a target node from the at least one first reference node includes: determining a maximum resource of at least one of the first reference nodes through the backup scheduling container; Determine, by the backup scheduling container, a difference between each first reference node and a corresponding target predicted resource consumption, to obtain at least one reference difference; Determining, by the backup scheduling container, a target difference value having a maximum value among the at least one reference difference value; The backup scheduling container determines, from the at least one first reference node, a first reference node corresponding to the target difference as the target node.

2. The method according to claim 1, characterized in that The second consumable resources include at least the consumable resources of the reference instance running when the corresponding first reference node runs the target instance and the running consumable resources of the target running statement other than the target instance and the reference instance when the corresponding first reference node runs the target instance; wherein, the reference instance is an instance other than the target instance running when the target instance is run in the corresponding first reference node.

3. The method according to any one of claims 1 to 2, characterized in that If it is detected that the current time is the time for performing a full backup of the target database, determining at least one first reference node through the backup scheduling container includes: If it is detected that the current time is a time for performing a full backup of the target database, the target information of the target database sent by each second reference node is received through the backup scheduling container to obtain at least one target information; At least one first reference node is determined from at least one second reference node based on at least one of the target information.

4. The method according to claim 3, characterized in that The determining, based on at least one of the target information, at least one of the first reference nodes from at least one of the second reference nodes comprises: Determining at least one target information including a target identifier from at least one of the target information; wherein the target identifier is used to identify that the corresponding second reference node is not down; From at least one of the second reference nodes, a node matching at least one target information including a target identifier is determined to obtain at least one of the first reference nodes.

5. The method according to any one of claims 1 to 2 and 4, characterized in that: If it is detected that the current time is the time for performing a full backup of the target database, before determining at least one first reference node through the backup scheduling container, the method further includes: Obtaining a preset backup strategy for the target database through the backup scheduling container; The preset backup policy is sent to at least one second reference node through the backup scheduling container; wherein, at least one second reference node is a node running a backup RPC container bound to a target instance corresponding to the target database, and the preset backup policy is used to instruct each second reference node to send the target predicted resource consumption to the scheduling node when detecting that the size of the target database exceeds a preset database threshold through the set backup RPC container, and at least one second reference node includes at least one first reference node.

6. The method according to claim 1, characterized in that The execution instruction is further used to instruct the target node to execute all incremental backup operations between the current time and the next full backup operation time of the target database.

7. A data backup device, characterized in that: The device includes: a first determining unit, a counting unit, a second determining unit and a sending unit; wherein: The first determining unit is configured to, if it is detected that the current moment is a moment for performing a full backup of the target database, determine at least one first reference node through a backup scheduling container; wherein the at least one first reference node is configured to run a backup remote procedure call (RPC) container bound to a target instance corresponding to the target database; The statistical unit is configured to respectively count target predicted consumed resources when the first reference node fully backs up the target database through the backup scheduling container to obtain at least one target predicted consumed resource; The second determining unit is configured to determine a target node from the at least one first reference node by using a backup scheduling container to consume resources based on the at least one target prediction; The sending unit is configured to send an execution instruction to the target node through a backup scheduling container; wherein the execution instruction is configured to instruct the target RPC container running on the target node to perform a full backup operation on the target database; The statistical unit is specifically configured to implement the following steps: receiving, through a backup scheduling container, the target predicted consumed resource sent by each of the first reference nodes, and obtaining at least one of the target predicted consumed resources; Alternatively, a first consumable resource and a second consumable resource sent by each first reference node are received through a backup scheduling container; wherein the first consumable resource includes the consumable resource when the corresponding first reference node runs the target instance, and the second consumable resource includes the consumable resource other than the first consumable resource when the corresponding first reference node runs the target instance; Determine the sum of the first consumed resource and the second consumed resource of each first reference node through a backup scheduling container to obtain at least one target predicted consumed resource; The second determining unit is specifically configured to implement the following steps: determining a maximum resource of at least one of the first reference nodes through the backup scheduling container; Determine, by the backup scheduling container, a difference between each first reference node and a corresponding target predicted resource consumption, to obtain at least one reference difference; Determining, by the backup scheduling container, a target difference value having a maximum value among the at least one reference difference value; The backup scheduling container determines, from the at least one first reference node, a first reference node corresponding to the target difference as the target node.

8. A data backup device, characterized in that: The device comprises: a memory, a processor and a communication bus; wherein: The memory is used to store executable instructions; The communication bus is used to realize the communication connection between the processor and the memory; The processor is configured to run the backup scheduling container to execute the data backup method stored in the memory, thereby implementing the steps of the data backup method according to any one of claims 1 to 6.

9. A data backup system, characterized in that: The system includes: a scheduling node for running a backup scheduling container, and at least one second reference node for running a backup RPC container bound to a target instance corresponding to a target database; wherein: The scheduling node is used to determine at least one first reference node through a backup scheduling container if it is detected that the current moment is the moment for performing a full backup of the target database; wherein, at least one first reference node is used to run a backup remote procedure call RPC container bound to a target instance corresponding to the target database, and at least one second reference node includes at least one first reference node; through the backup scheduling container, respectively count the target predicted consumption resources when the first reference node performs a full backup of the target database to obtain at least one target predicted consumption resource; through the backup scheduling container, determine the target node from the at least one first reference node based on the at least one target predicted consumption resource; send an execution instruction to the target node through the backup scheduling container; wherein, the execution instruction is used to instruct the target RPC container run by the target node to perform a full backup operation on the target database; The target node is configured to receive an execution instruction sent by the scheduling node through the target RPC container; in response to the execution instruction, perform a full backup operation on the target database at the full backup time corresponding to the target database; The step of respectively counting target predicted resource consumption when the first reference node fully backs up the target database through the backup scheduling container to obtain at least one target predicted resource consumption includes: receiving, through a backup scheduling container, the target predicted consumed resource sent by each of the first reference nodes, and obtaining at least one of the target predicted consumed resources; Alternatively, a first consumable resource and a second consumable resource sent by each first reference node are received through a backup scheduling container; wherein the first consumable resource includes the consumable resource when the corresponding first reference node runs the target instance, and the second consumable resource includes the consumable resource other than the first consumable resource when the corresponding first reference node runs the target instance; Determine the sum of the first consumed resource and the second consumed resource of each first reference node through a backup scheduling container to obtain at least one target predicted consumed resource; The step of consuming resources based on at least one target prediction by using a backup scheduling container and determining a target node from the at least one first reference node includes: determining a maximum resource of at least one of the first reference nodes through the backup scheduling container; Determine, by the backup scheduling container, a difference between each first reference node and a corresponding target predicted resource consumption, to obtain at least one reference difference; Determining, by the backup scheduling container, a target difference value having a maximum value among the at least one reference difference value; The backup scheduling container determines, from the at least one first reference node, a first reference node corresponding to the target difference as the target node.

10. A storage medium, characterized in that: The storage medium stores a data backup method, which, when executed by a processor, implements the steps of the data backup method according to any one of claims 1 to 6.

Images