Methods, apparatus and equipment for backup and recovery of non-peer resources of distributed databases

Abstract

The present invention provides a distributed database non-peer resource backup and recovery method, device and equipment, which can be applied to the field of database technology. The method includes: in response to detecting a backup and recovery instruction, reading a first parameter and a second parameter from a recovery configuration file, and generating a reading result, wherein the first parameter includes a first storage address and a first backup address; the second parameter includes a deployment parameter characterizing a target database cluster, and a first data sharding parameter characterizing a resource difference between an original database cluster and a target database cluster; in response to determining that the reading result characterizes that the recovery configuration file includes the second parameter, obtaining backup data from the first storage address; based on the deployment parameter, restoring the backup data to the first backup address; and based on the first data sharding parameter, performing a redistribution operation on the backup data to obtain final backup data.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and more specifically to a distributed database non-peer resource backup and recovery method, device and equipment. Background Art

[0002] With the popularization of the Internet, traditional centralized databases are difficult to meet the needs of large data volumes and high concurrency. Therefore, distributed databases that can be horizontally expanded are usually used to meet the needs of large data volumes and high concurrency.

[0003] When backing up and restoring data in a distributed database, it is usually necessary to back up the data in all servers in the original database cluster, and then restore the backed-up data to the target database with the same number of servers. However, if the number of servers in the original database cluster is different from the number of servers in the target database cluster, it is difficult to back up and restore between the original database cluster and the target database cluster. Summary of the invention

[0004] In view of the above problems, the present invention provides a distributed database non-peer resource backup and recovery method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present invention, a distributed database non-peer resource backup and recovery method is provided, comprising: in response to detecting a backup and recovery instruction, reading a first parameter and a second parameter from a recovery configuration file, and generating a reading result for the second parameter, wherein the first parameter comprises a first storage address and a first backup address, the first storage address represents a temporary storage address of backup data pre-acquired from an original database cluster, and the first backup address is an address of a target database cluster; the second parameter comprises a deployment parameter representing a hardware deployment method of the target database cluster, and a first data sharding parameter representing a hardware resource difference between the original database cluster and the target database cluster; in response to determining that the reading result represents that the recovery configuration file includes the second parameter, acquiring the backup data from the first storage address; based on the deployment parameter, restoring the backup data to the first backup address; and based on the first data sharding parameter, performing a redistribution operation on the backup data to obtain final backup data.

[0006] According to an embodiment of the present invention, the above-mentioned redistribution operation is performed on the above-mentioned backup data based on the above-mentioned first data sharding parameter to obtain the final backup data, including: on the master server of the above-mentioned target database cluster, redistributing the backup data on the target data shard indicated by the above-mentioned first data sharding parameter to the preset data shard in the above-mentioned master server to obtain the above-mentioned final backup data; and synchronizing the above-mentioned final backup data on the above-mentioned master server to at least one backup server of the above-mentioned target database cluster.

[0007] According to an embodiment of the present invention, the redistribution operation includes a capacity expansion operation or a capacity reduction operation.

[0008] According to an embodiment of the present invention, the method further includes: before performing a redistribution operation on the backup data based on the first data sharding parameter, adding a preset database lock to the target database cluster to stop the service provided by the target database cluster based on the final backup data; after performing a redistribution operation on the backup data based on the first data sharding parameter, releasing the preset database lock to restore the service provided by the target database cluster based on the final backup data.

[0009] According to an embodiment of the present invention, the method further includes: in response to determining that the reading result indicates that the recovery configuration file does not include the second parameter, obtaining the backup data from the first storage address, and restoring the backup data to the first backup address.

[0010] According to an embodiment of the present invention, the above-mentioned recovery configuration file may further include a second data sharding parameter, and the above-mentioned obtaining the above-mentioned backup data from the above-mentioned first storage address includes: obtaining the backup data corresponding to the above-mentioned second data sharding parameter from the above-mentioned first storage address; wherein, the backup data corresponding to the above-mentioned second data sharding parameter is the backup data stored in the above-mentioned first storage address, or, the backup data corresponding to the above-mentioned second data sharding parameter is a part of the backup data stored in the above-mentioned first storage address.

[0011] According to an embodiment of the present invention, obtaining backup data from the original database cluster in advance includes: in response to detecting a backup instruction, reading a second storage address and a second backup address from a backup configuration file, wherein the second backup address is the same as the first storage address; the second storage address is the address of the original database cluster; obtaining the backup data from the second storage address, and storing the backup data to the second backup address.

[0012] According to an embodiment of the present invention, the above-mentioned backup configuration file also includes a third data sharding parameter, and the above-mentioned obtaining of the above-mentioned backup data from the above-mentioned second storage address includes: obtaining the backup data corresponding to the above-mentioned third data sharding parameter from the above-mentioned second storage address; wherein, the backup data corresponding to the above-mentioned third data sharding parameter is the backup data stored in the above-mentioned second storage address, or, the backup data corresponding to the above-mentioned third data sharding parameter is a part of the backup data stored in the above-mentioned second storage address.

[0013] The second aspect of the present invention provides a distributed database non-peer resource backup and recovery device, including: a reading module, which is used to read a first parameter and a second parameter from a recovery configuration file in response to detecting a backup and recovery instruction, and generate a reading result for the above-mentioned second parameter, wherein the above-mentioned first parameter includes a first storage address and a first backup address, the above-mentioned first storage address represents a temporary storage address of backup data pre-acquired from an original database cluster, and the above-mentioned first backup address is the address of a target database cluster; the above-mentioned second parameter includes a deployment parameter representing a hardware deployment method of the above-mentioned target database cluster, and a first data sharding parameter representing a hardware resource difference between the above-mentioned original database cluster and the above-mentioned target database cluster; a data acquisition module, which is used to obtain the above-mentioned backup data from the above-mentioned first storage address in response to determining that the above-mentioned reading result represents that the above-mentioned recovery configuration file includes the second parameter; a first recovery module, which is used to restore the above-mentioned backup data to the above-mentioned first backup address based on the above-mentioned deployment parameter; and a redistribution module, which is used to redistribute the above-mentioned backup data based on the above-mentioned first data sharding parameter to obtain the final backup data.

[0014] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0016] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0017] According to an embodiment of the present invention, when there is a difference in hardware resources between the original data cluster and the target data cluster, the backup data is restored to the target database cluster according to the hardware deployment mode of the target database cluster represented by the deployment parameters, thereby enhancing the durability of the backup data. In addition, the backup data in the target database cluster is redistributed according to the first data sharding parameter, so that the final backup data matches the hardware resources of the target database cluster, thereby improving the reliability of the target database cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0019] Figure 1 An application scenario diagram of a distributed database non-peer resource backup and recovery method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0020] Figure 2 The flowchart of the distributed database non-peer resource backup and recovery method according to an embodiment of the present invention is shown.

[0021] Figure 3 A schematic diagram of a backup and recovery method according to a specific embodiment of the present invention is shown.

[0022] Figure 4 A schematic diagram of a backup and recovery method according to another specific embodiment of the present invention is shown.

[0023] Figure 5 A schematic diagram showing partial recovery of backup data according to a specific embodiment of the present invention is shown.

[0024] Figure 6 A schematic diagram showing part of the backup data according to a specific embodiment of the present invention is shown.

[0025] Figure 7 The structure block diagram of the distributed database non-peer resource backup and recovery device according to an embodiment of the present invention is shown.

[0026] Figure 8 A block diagram of an electronic device suitable for implementing a distributed database non-peer resource backup and recovery method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] Normally, when backing up and restoring data in the original database cluster to the target database cluster, it is necessary to ensure that the number of servers in the target database cluster is the same as the number of servers in the target database cluster. Therefore, if the number of servers in the original database cluster is different from the number of servers in the target database cluster, data backup and restoration cannot be performed. In addition, during the backup and restoration process, it is not possible to back up only specific servers in the original database cluster, nor is it possible to restore specific servers in the target database cluster.

[0032] An embodiment of the present invention provides a distributed database non-peer resource backup and recovery method, characterized in that the method includes: in response to detecting a backup and recovery instruction, reading a first parameter and a second parameter from a recovery configuration file, and generating a reading result for the second parameter, wherein the first parameter includes a first storage address and a first backup address, the first storage address represents a temporary storage address of backup data pre-acquired from an original database cluster, and the first backup address is the address of a target database cluster; the second parameter includes a deployment parameter representing a hardware deployment method of the target database cluster, and a first data sharding parameter representing a hardware resource difference between the original database cluster and the target database cluster; in response to determining that the reading result represents that the recovery configuration file includes the second parameter, acquiring backup data from the first storage address; based on the deployment parameter, restoring the backup data to the first backup address; and based on the first data sharding parameter, redistributing the backup data to obtain final backup data.

[0033] Figure 1 An application scenario diagram of a distributed database non-peer resource backup and recovery method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0034] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a terminal device 110, a network 120, a backup and recovery server 130, an original database cluster 140, and a target database cluster 150. The network 120 is used as a medium for providing a communication link between the terminal device 110 and the backup and recovery server 130, between the original database cluster 140 and the backup and recovery server 130, and between the target database cluster 150 and the backup and recovery server 130. The network 120 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0035] The user can use the terminal device 110 to interact with the backup and recovery server 130 through the network 120 to receive or send messages, etc. The terminal device 110 can be installed with various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0036] The terminal device 110 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0037] The backup and recovery server 130 may be a server that provides backup and recovery services. The backup and recovery server 130 may back up the data in the original database cluster 140 according to the backup instruction issued by the user, or may restore the backup data to the target database cluster 150 according to the backup and recovery instruction issued by the user, and feed back the processing result to the terminal device 110.

[0038] Both the original database cluster 140 and the target database cluster 150 are distributed databases. The number of servers included in the original database cluster 140 and the number of servers included in the target database cluster may be the same or different.

[0039] It should be noted that the distributed database non-peer resource backup and recovery method provided in the embodiment of the present invention can generally be executed by the backup and recovery server 130. Correspondingly, the distributed database non-peer resource backup and recovery device provided in the embodiment of the present invention can generally be set in the backup and recovery server 130. The distributed database non-peer resource backup and recovery method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the backup and recovery server 130 and can communicate with the terminal device 110, the original database cluster 140, the target database cluster 150 and / or the backup and recovery server 130. Correspondingly, the distributed database non-peer resource backup and recovery device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the backup and recovery server 130 and can communicate with the terminal device 110, the original database cluster 140, the target database cluster 150 and / or the backup and recovery server 130.

[0040] It should be understood that Figure 1 The number of terminal devices, networks, backup recovery servers, and database cluster servers in the embodiment is only for illustration. Any number of terminal devices, networks, backup recovery servers, and data nodes may be provided according to implementation requirements.

[0041] The following will be based on Figure 1 The scene described by Figure 2~Figure 6 The distributed database non-peer resource backup and recovery method of the disclosed embodiment is described in detail.

[0042] Figure 2 The flowchart of the distributed database non-peer resource backup and recovery method according to an embodiment of the present invention is shown.

[0043] like Figure 2 As shown, the distributed database non-peer resource backup and recovery of this embodiment includes operations S210 to S240.

[0044] In operation S210, in response to detecting a backup recovery instruction, a first parameter and a second parameter are read from a recovery configuration file, and a read result for the second parameter is generated.

[0045] According to an embodiment of the present invention, the backup and recovery instructions may be issued by a user through a terminal, or may be generated according to a scheduled backup and recovery task. The recovery configuration file may be pre-written and stored in a backup and recovery database of a backup and recovery server, or may be stored in other locations of the backup and recovery server.

[0046] According to an embodiment of the present invention, the backup and recovery server detects the backup and recovery instruction, searches the backup and recovery database for a recovery configuration file that matches the backup and recovery instruction, and reads the first parameter and the second parameter from the recovery configuration file. Alternatively, after searching the backup and recovery database for the recovery configuration file, the first parameter and the second parameter that match the backup and recovery instruction are searched from the recovery configuration file.

[0047] According to an embodiment of the present invention, the restoration configuration file may store a first parameter and a second parameter that match the backup restoration instruction, or may store only the first parameter that matches the backup restoration instruction.

[0048] According to an embodiment of the present invention, the reading result is used to indicate whether the second parameter is stored in the recovery configuration file. When determining whether the second parameter is stored in the recovery configuration file, the recovery configuration file may be traversed according to the second parameter identifier corresponding to the recovery configuration file.

[0049] According to an embodiment of the present invention, the first parameter includes a first storage address and a first backup address, the first storage address represents a temporary storage address of backup data pre-acquired from the original database cluster, and the first backup address is the address of the target database cluster.

[0050] According to an embodiment of the present invention, the original database cluster may be a data source for backup data. Specifically, when the data in the original database cluster needs to be backed up, the data in the original database cluster may be backed up through a backup instruction.

[0051] According to an embodiment of the present invention, the first storage address can be used to temporarily store backup data obtained from the original database cluster. Specifically, the first storage address can be in the backup recovery database of the backup recovery server. After the backup recovery is completed, the backup data in the first storage address can also be deleted to save storage resources.

[0052] According to an embodiment of the present invention, the target database cluster may be used to receive backup data. The first backup address may be a primary server in the target database cluster.

[0053] According to an embodiment of the present invention, the second parameter includes a deployment parameter characterizing a hardware deployment mode of the target database cluster and a first data sharding parameter characterizing a hardware resource difference between the original database cluster and the target database cluster.

[0054] According to an embodiment of the present invention, the restoration configuration file may store the first parameter and the second parameter, or may store only the first parameter.

[0055] According to an embodiment of the present invention, before performing backup and recovery, it can be determined whether there is a hardware resource difference between the original database cluster and the target database cluster. When it is determined that there is a hardware resource difference between the original database cluster and the target database cluster, the first parameter and the second parameter are configured in the recovery configuration file. When it is determined that there is no hardware resource difference between the original database cluster and the target database cluster, only the first parameter can be configured in the recovery configuration file.

[0056] According to an embodiment of the present invention, the deployment parameter can be used to characterize the hardware deployment mode of the target database cluster, and the first data sharding parameter can be used to characterize the hardware resource difference between the original database cluster and the target database cluster. The hardware can be a server in the database cluster.

[0057] In operation S220 , in response to determining that the read result indicates that the recovery configuration file includes the second parameter, the backup data is acquired from the first storage address.

[0058] According to an embodiment of the present invention, the reading result indicates that the recovery configuration file includes the second parameter, that is, there is a hardware resource difference between the original database cluster and the target database cluster.

[0059] According to the embodiment of the present invention, since the first storage address represents a temporary storage address of the backup data, the backup and recovery server can obtain the backup data from the first storage address.

[0060] In operation S230, the backup data is restored to the first backup address based on the deployment parameters.

[0061] According to an embodiment of the present invention, when restoring the backup data to the first backup address, since there are hardware resource differences between the original database cluster and the target database cluster, the backup data needs to be restored to the first backup address according to the deployment parameters of the target database cluster.

[0062] According to an embodiment of the present invention, when the number of hardware resources in the original database cluster is different from the number of hardware resources in the target database cluster, since the hardware resources in the database cluster correspond to the data shards, the correspondence between multiple first hardware resources in the original database cluster and multiple second hardware resources in the target database cluster can be specified through deployment parameters, for example, restoring the data in the first hardware resource 1 to the second hardware resource 1.

[0063] In operation S240, a redistribution operation is performed on the backup data based on the first data sharding parameter to obtain final backup data.

[0064] According to an embodiment of the present invention, since there are hardware resource differences between the original database cluster and the target database cluster, after the backup data is restored to the first backup address, it may cause an unbalanced allocation of hardware resources for storing the backup data in the target database cluster, and therefore it is necessary to redistribute the backup data in the target database cluster.

[0065] According to an embodiment of the present invention, although the hardware resources of the target database cluster and the original database cluster are not equal (such as the number of servers is not equal), since the data shard size of the backup data in the target database cluster still matches that of the original database cluster, the backup data can be redistributed based on the first data shard parameter so that the data shard size of the final backup data after redistribution matches that of the target database cluster.

[0066] According to an embodiment of the present invention, when there is a difference in hardware resources between the original data cluster and the target data cluster, the backup data is restored to the target database cluster according to the hardware deployment mode of the target database cluster represented by the deployment parameters, thereby enhancing the durability of the backup data. In addition, the backup data in the target database cluster is redistributed according to the first data sharding parameter, so that the final backup data matches the hardware resources of the target database cluster, thereby improving the reliability of the target database cluster.

[0067] According to an embodiment of the present invention, based on the first data sharding parameter, a redistribution operation is performed on the backup data to obtain final backup data, including: on the master server of the target database cluster, redistributing the backup data on the target data shard indicated by the first data sharding parameter to the preset data shard in the master server to obtain the final backup data; and synchronizing the final backup data on the master server to at least one standby server of the target database cluster.

[0068] According to an embodiment of the present invention, the first data shard parameter can be used to indicate that the backup data in the target data shard is redistributed to the preset data shard. For example, the data on the first hardware resource 1 in the original database cluster belongs to the first data shard 1, and still belongs to the first data shard 1 before redistribution. When the first data shard parameter indicates that the data on the first data shard 1 is redistributed to the second hardware resource 1, the first data shard 1 is the target data shard, and the second data shard 1 corresponding to the second hardware resource 1 is the preset data shard.

[0069] According to an embodiment of the present invention, the number of target data shards and preset data shards may be the same or different. The backup data in one target data shard may be redistributed to multiple preset data shards, or the backup data in multiple target data shards may be redistributed to one preset data shard, or the backup data in multiple target data shards may be redistributed to multiple preset data shards.

[0070] According to an embodiment of the present invention, in order to improve the durability and reliability of the target database cluster, a primary server and a backup server can be set in the target database cluster, wherein the backup server is used to back up the final backup data in the primary server, and there is no limit on the number of primary servers and the number of backup servers.

[0071] According to an embodiment of the present invention, the backup data in the main server is redistributed to obtain the final backup data, thereby improving the reliability of the main server. In addition, the main server and the backup server are set in the target server cluster to improve the reliability and durability of the target server cluster.

[0072] According to an embodiment of the present invention, the redistribution operation includes a capacity expansion operation and a capacity reduction operation.

[0073] According to an embodiment of the present invention, a redistribution operation can be determined based on the first data sharding parameter. When the second hardware resource quantity of the target database cluster is greater than the first hardware resource quantity of the original database cluster, the redistribution operation can be determined to be a capacity expansion operation. When the second hardware resource quantity of the target database cluster is less than the first hardware resource quantity of the original database cluster, the redistribution operation can be determined to be a capacity reduction operation.

[0074] According to an embodiment of the present invention, when the redistribution operation is a capacity expansion operation, the first data shard parameter can be used to indicate that the backup data on a target data shard is redistributed to multiple preset data shards, or the first data shard parameter can be used to indicate that the backup data on multiple target data shards is redistributed to multiple preset data shards, wherein the number of target data shards is less than the number of preset data shards.

[0075] According to an embodiment of the present invention, when the redistribution operation is a shrinking operation, the first data shard parameter can be used to indicate that the backup data on multiple target data shards are redistributed into one preset data shard, or the first data shard parameter can be used to indicate that the backup data on multiple target data shards are redistributed into multiple preset data shards, wherein the number of target data shards is greater than the number of preset data shards.

[0076] According to the embodiment of the present invention, by performing expansion and reduction operations, the reliability of the target database cluster is improved, so that the storage method of storing the final backup data in the target database cluster matches the hardware resources.

[0077] Figure 3 A schematic diagram of a backup and recovery method according to a specific embodiment of the present invention is shown.

[0078] like Figure 3 As shown, the original database cluster includes six first servers, such as first server 1, ..., first server 6, each first server corresponds to a first shard, such as the first server corresponds to first shard 1, ..., first server 6 corresponds to first shard 6, and the backup data in the original database cluster are respectively stored on first shard 1, first shard 2, first shard 3, first shard 4, first shard 5 and first shard 6.

[0079] like Figure 3 As shown, the number of shards in the backup recovery database is equal to that in the original database cluster, and the backup data in the backup recovery database is stored on the first shard 1, the first shard 2, the first shard 3, the first shard 4, the first shard 5 and the first shard 6 respectively.

[0080] According to an embodiment of the present invention, the target database cluster includes four second servers, such as second server 1, ..., second server 4. According to the deployment parameters, the first shard 1 and the first shard 2 are restored to the second server 1, the first shard 3 is stored in the second server 2, the first shard 4 is stored in the second server 3, and the first shard 5 and the first shard 6 are stored in the second server 4.

[0081] According to an embodiment of the present invention, since the number of second servers in the target database cluster is less than the number of first servers in the original database cluster, the redistribution operation for the target database cluster is a capacity reduction operation. Specifically, the backup data on the first shard 2 and the first shard 6 can be evenly distributed to the four second servers using the first data shard parameter to obtain the second shard 1, the second shard 2, the second shard 3, and the second shard 4.

[0082] Figure 4 A schematic diagram of a backup and recovery method according to another specific embodiment of the present invention is shown.

[0083] like Figure 4 As shown, the original database cluster includes four first servers, such as first server 1, ..., first server 4, each first server corresponds to a first shard, such as first server 1 corresponds to first shard 1, ..., first server 4 corresponds to first shard 4, and the backup data in the original database cluster is stored on first shard 1, first shard 2, first shard 3 and first shard 4, respectively.

[0084] like Figure 4 As shown, the number of data shards in the backup recovery database is equal to that in the original database cluster, and the backup data in the backup recovery database is stored in the first shard 1, the first shard 2, the first shard 3 and the first shard 4 respectively.

[0085] According to an embodiment of the present invention, the target database cluster includes six second servers, such as second server 1, ..., second server 6. According to the deployment parameters, the first shard 1 is restored to the second server 1, the first shard 2 is restored to the second server 2, the first shard 3 is restored to the second server 3, and the first shard 4 is restored to the second server 4.

[0086] According to an embodiment of the present invention, since the number of second servers in the target database cluster is less than the number of first servers in the original database cluster, the redistribution operation for the target database cluster is a capacity expansion operation. Specifically, the backup data on the first shard 1, the first shard 2, the first shard 3, and the first shard 4 can be evenly distributed to the six second servers using the first data shard parameter to obtain the second shard 1, the second shard 2, the second shard 3, the second shard 4, the second shard 5, and the second shard 6.

[0087] According to an embodiment of the present invention, the above-mentioned distributed database non-peer resource backup and recovery method also includes: based on the first data sharding parameter, before performing a redistribution operation on the backup data, adding a preset database lock to the target database cluster to stop the service provided by the target database cluster based on the final backup data; based on the first data sharding parameter, after performing a redistribution operation on the backup data, releasing the preset database lock to restore the service provided by the target database cluster based on the final backup data.

[0088] According to an embodiment of the present invention, after the backup data is restored to the target database cluster, the target database cluster is already able to provide services. However, since the backup data in the target database cluster has not yet been redistributed, in order to prevent the user from using the services of the target database cluster before the redistribution operation is performed, which may affect the backup data that needs to be redistributed, it is necessary to add a preset database lock to the target database cluster.

[0089] According to the embodiment of the present invention, after adding the preset database lock, the user cannot operate the backup data in the target database cluster, thereby ensuring the normal operation of the redistribution operation and improving the stability of the target database cluster.

[0090] According to an embodiment of the present invention, before the preset database lock is released, the data stored in the original database can be used to provide services to the user until the preset database lock is released.

[0091] According to an embodiment of the present invention, after the redistribution operation is completed, the target database cluster at this time has the ability to provide services to users, so the preset database lock can be released to provide users with services based on the final backup data.

[0092] According to an embodiment of the present invention, by adding a preset database lock to the target database cluster during the redistribution operation, the integrity and accuracy of the backup data are ensured.

[0093] According to an embodiment of the present invention, the distributed database non-peer resource backup and recovery method also includes: in response to determining that the read result indicates that the recovery configuration file does not include the second parameter, obtaining backup data from the first storage address, and restoring the backup data to the first backup address.

[0094] According to an embodiment of the present invention, the reading result indicates that the restoration configuration file does not include the second parameter, that is, there is no hardware resource difference between the original database cluster and the target database cluster, and the backup data can be directly restored to the first backup address.

[0095] According to an embodiment of the present invention, in the case of equal hardware resources, there is no need to use deployment parameters to restore backup data, nor is there a need to perform redistribution operations. Specifically, during the recovery process, the one-to-one correspondence between the first hardware resources of the original database cluster and the second hardware resources of the target database cluster can be used to store the data on each first hardware resource in the original database cluster to the second hardware resources of the target database cluster that match it.

[0096] According to the embodiment of the present invention, when the resources of the original database cluster and the target database cluster are equal, the backup data is directly restored, thereby improving the processing efficiency of the backup and restoration.

[0097] According to an embodiment of the present invention, obtaining backup data from a first storage address includes: obtaining backup data corresponding to a second data sharding parameter from the first storage address.

[0098] According to an embodiment of the present invention, the backup data corresponding to the second data sharding parameter is the backup data stored in the first storage address, or the backup data corresponding to the second data sharding parameter is a part of the backup data stored in the first storage address.

[0099] According to an embodiment of the present invention, the recovery configuration file may further include a second data sharding parameter. Since the user may not need all the backup data, the second data sharding parameter may be used to indicate the recovery of all or part of the backup data when the backup data is restored to the target database cluster. Specifically, the second data sharding parameter may be used to indicate the data shards that need to be restored.

[0100] According to the embodiment of the present invention, the second data sharding parameter is used to implement partial recovery of backup data, thereby improving the user experience.

[0101] Figure 5 A schematic diagram showing partial recovery of backup data according to a specific embodiment of the present invention is shown.

[0102] like Figure 5 As shown, the backup recovery database includes six first servers, such as first server 1, ..., first server 6, each first server corresponds to a first shard, such as first server 1 corresponds to first shard 1, ..., first server 6 corresponds to first shard 6, and the backup data in the backup recovery database are stored on first shard 1, first shard 2, first shard 3, first shard 4, first shard 5 and first shard 6, respectively.

[0103] According to an embodiment of the present invention, it is determined according to the second data sharding parameters that only the first shard 1, the first shard 3, the first shard 4 and the first shard 5 are restored. The target database cluster includes four second servers, such as the second server 1, ..., the second server 4, and according to the deployment parameters, the first shard 1 can be restored to the second server 1, the first shard 3 can be restored to the second server 2, the first shard 4 can be restored to the second server 3, and the first shard 5 can be restored to the second server 4.

[0104] According to an embodiment of the present invention, pre-acquiring backup data from the original database cluster includes: in response to detecting a backup instruction, reading a second storage address and a second backup address from a backup configuration file; acquiring backup data from the second storage address, and storing the backup data to the second backup address.

[0105] According to an embodiment of the present invention, the second backup address is the same as the first storage address, and the second storage address is the address of the original database cluster.

[0106] According to an embodiment of the present invention, the backup instruction may be issued by a user through a terminal, or may be generated according to a scheduled backup task. The backup configuration file may be pre-written and stored in a backup recovery database of a backup recovery server, or may be stored in other locations of the backup recovery server.

[0107] According to an embodiment of the present invention, the backup and recovery server detects the backup instruction, searches for a backup configuration file matching the backup instruction from the backup and recovery database, and reads the second storage address and the second backup address from the backup configuration file. Alternatively, after searching for the backup configuration file from the backup and recovery database, the second storage address and the second backup address matching the backup instruction may be searched from the backup configuration file.

[0108] According to an embodiment of the present invention, the backup instruction is used to back up the data in the original database cluster to a second backup address. The second backup address may be an address of a backup recovery database.

[0109] According to an embodiment of the present invention, the accuracy of the backup data is improved by backing up the data in the original database cluster in the second backup address.

[0110] According to an embodiment of the present invention, obtaining backup data from the second storage address includes: obtaining backup data corresponding to the third data sharding parameter from the second storage address.

[0111] According to an embodiment of the present invention, the backup data corresponding to the third data sharding parameter is the backup data stored in the second storage address, or the backup data corresponding to the third data sharding parameter is a part of the backup data stored in the second storage address.

[0112] According to an embodiment of the present invention, the backup configuration file further includes a third data sharding parameter. Since the user may not need to back up all the data, in order to improve the backup efficiency, when acquiring the backup data in the second storage address, the third data sharding parameter can be used to indicate backing up all or part of the backup data. Specifically, the third data sharding parameter can be used to indicate the data shards that need to be backed up.

[0113] According to the embodiment of the present invention, the third data sharding parameter is used to implement partial backup of backup data, thereby improving the user experience.

[0114] Figure 6 A schematic diagram showing part of the backup data according to a specific embodiment of the present invention is shown.

[0115] like Figure 6As shown, the original database cluster includes six first servers, such as first server 1, ..., first server 6, each first server corresponds to a first shard, such as first server 1 corresponds to first shard 1, ..., first server 6 corresponds to first shard 6, and the backup data in the original database cluster are stored on first shard 1, first shard 2, first shard 3, first shard 4, first shard 5 and first shard 6, respectively.

[0116] According to an embodiment of the present invention, it is determined that only the data on the first shard 1, the first shard 3, the first shard 4 and the first shard 5 are backed up according to the third data shard parameter. The backup recovery database includes four second servers, such as the second server 1, ..., the second server 4. According to the third data shard parameter, the first shard 1 can be backed up to the second server 1, the first shard 3 can be backed up to the second server 2, the first shard 4 can be backed up to the second server 3, and the first shard 5 can be backed up to the second server 4. The backup recovery database stores the backup data of the first shard 1, the first shard 3, the first shard 4 and the first shard 5.

[0117] Based on the above-mentioned distributed database non-peer resource backup and recovery method, the present invention also provides a distributed database non-peer resource backup and recovery device. Figure 7 The device is described in detail.

[0118] Figure 7 The structure block diagram of the distributed database non-peer resource backup and recovery device according to an embodiment of the present invention is shown.

[0119] like Figure 7 As shown, the distributed database non-peer resource backup and recovery device 700 of this embodiment includes a reading module 710 , a data acquisition module 720 , a first recovery module 730 and a redistribution module 740 .

[0120] The reading module 710 is used to respond to the detection of the backup recovery instruction, read the first parameter and the second parameter from the recovery configuration file, and generate a reading result for the second parameter, wherein the first parameter includes a first storage address and a first backup address, the first storage address represents a temporary storage address of the backup data pre-acquired from the original database cluster, and the first backup address is the address of the target database cluster; the second parameter includes a deployment parameter representing the hardware deployment mode of the target database cluster, and a first data sharding parameter representing the hardware resource difference between the original database cluster and the target database cluster. In one embodiment, the reading module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0121] The data acquisition module 720 is used to acquire the backup data from the first storage address in response to determining that the read result indicates that the recovery configuration file includes the second parameter. In one embodiment, the data acquisition module 720 can be used to perform the operation S220 described above, which will not be described in detail here.

[0122] The first restoration module 730 is used to restore the backup data to the first backup address based on the deployment parameters. In one embodiment, the first restoration module 730 can be used to perform the operation S230 described above, which will not be described in detail here.

[0123] The redistribution module 740 is used to perform a redistribution operation on the backup data based on the first data sharding parameter to obtain final backup data. In one embodiment, the redistribution module 740 can be used to perform the operation S240 described above, which will not be described in detail here.

[0124] According to an embodiment of the present invention, the redistribution module 740 includes a redistribution submodule and a synchronization submodule.

[0125] The redistribution submodule is used to redistribute the backup data on the target data shard indicated by the first data shard parameter to the preset data shard in the main server on the main server of the target database cluster to obtain the final backup data.

[0126] The synchronization submodule is used to synchronize the final backup data on the primary server to at least one standby server of the target database cluster.

[0127] According to an embodiment of the present invention, the redistribution operation includes a capacity expansion operation or a capacity reduction operation.

[0128] According to an embodiment of the present invention, the distributed database non-peer resource backup and recovery device 700 further includes a locking module and an unlocking module.

[0129] The locking module is used to add a preset database lock to the target database cluster before performing a redistribution operation on the backup data based on the first data sharding parameter, so as to stop the service provided by the target database cluster based on the final backup data.

[0130] The unlocking module is used to release the preset database lock after redistributing the backup data based on the first data sharding parameter, so as to restore the service provided by the target database cluster based on the final backup data.

[0131] According to an embodiment of the present invention, the distributed database non-peer resource backup and recovery device 700 further includes a second recovery module.

[0132] The second restoration module is used for, in response to determining that the reading result indicates that the restoration configuration file does not include the second parameter, obtaining the backup data from the first storage address and restoring the backup data to the first backup address.

[0133] According to an embodiment of the present invention, the recovery configuration file may further include a second data sharding parameter, and the distributed database non-peer resource backup and recovery device 700 may further include a partial recovery module.

[0134] The partial recovery module is used to obtain backup data corresponding to the second data sharding parameter from the first storage address; wherein the backup data corresponding to the second data sharding parameter is the backup data stored in the first storage address, or the backup data corresponding to the second data sharding parameter is a part of the backup data stored in the first storage address.

[0135] According to the embodiment of the present invention, the distributed database non-peer resource backup and recovery device 700 further includes a backup acquisition module and a backup module.

[0136] The backup acquisition module is used to respond to detecting a backup instruction and read a second storage address and a second backup address from a backup configuration file, wherein the second backup address is the same as the first storage address; and the second storage address is the address of the original database cluster.

[0137] The backup module is used to obtain backup data from the second storage address and store the backup data to the second backup address.

[0138] According to an embodiment of the present invention, the backup configuration file further includes a third data sharding parameter. The backup module further includes a partial backup submodule.

[0139] The partial backup submodule is used to obtain backup data corresponding to the third data sharding parameter from the second storage address; wherein the backup data corresponding to the third data sharding parameter is the backup data stored in the second storage address, or the backup data corresponding to the third data sharding parameter is a part of the backup data stored in the second storage address.

[0140] According to an embodiment of the present invention, any multiple modules among the reading module 710, the data acquisition module 720, the first recovery module 730 and the redistribution module 740 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the reading module 710, the data acquisition module 720, the first recovery module 730 and the redistribution module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the reading module 710, the data acquisition module 720, the first recovery module 730 and the redistribution module 740 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0141] Figure 8 A block diagram of an electronic device suitable for implementing a distributed database non-peer resource backup and recovery method according to an embodiment of the present invention is shown.

[0142] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0143] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0144] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 808 including a network interface card such as a LAN card, a modem, etc. The communication portion 808 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.

[0145] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0146] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0147] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the distributed database non-peer resource backup and recovery method provided by the embodiment of the present invention.

[0148] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the processor 801 executes the computer program. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0149] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 808, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0150] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 808, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0151] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0152] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0154] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A distributed database non-peer resource backup and recovery method, characterized in that: The method comprises: In response to detecting a backup and recovery instruction, a first parameter and a second parameter are read from a recovery configuration file, and a reading result for the second parameter is generated, wherein the first parameter includes a first storage address and a first backup address, the first storage address represents a temporary storage address of backup data pre-acquired from an original database cluster, and the first backup address is an address of a target database cluster; the second parameter includes a deployment parameter representing a hardware deployment mode of the target database cluster, and a first data sharding parameter representing a hardware resource difference between the original database cluster and the target database cluster; In response to determining that the read result indicates that the recovery configuration file includes a second parameter, acquiring the backup data from the first storage address; Based on the deployment parameters, restoring the backup data to the first backup address; Based on the first data sharding parameter, redistribute the backup data to obtain final backup data; The redistributing operation on the backup data based on the first data sharding parameter to obtain final backup data includes: On the master server of the target database cluster, redistribute the backup data on the target data shard indicated by the first data shard parameter to the preset data shards in the master server to obtain the final backup data; and The final backup data on the primary server is synchronized to at least one standby server of the target database cluster.

2. The method according to claim 1, characterized in that The redistribution operation includes a capacity expansion operation or a capacity reduction operation.

3. The method according to claim 1, characterized in that The method further comprises: Before performing a redistribution operation on the backup data based on the first data sharding parameter, adding a preset database lock to the target database cluster to stop a service provided by the target database cluster based on the final backup data; After performing a redistribution operation on the backup data based on the first data sharding parameter, the preset database lock is released to restore the service provided by the target database cluster based on the final backup data.

4. The method according to claim 1, characterized in that: The method further comprises: In response to determining that the read result indicates that the recovery configuration file does not include the second parameter, the backup data is obtained from the first storage address, and the backup data is restored to the first backup address.

5. The method according to claim 1, characterized in that The restoration configuration file further includes a second data shard parameter, and the acquiring the backup data from the first storage address includes: Acquire backup data corresponding to the second data sharding parameter from the first storage address; The backup data corresponding to the second data sharding parameter is the backup data stored in the first storage address, or the backup data corresponding to the second data sharding parameter is a part of the backup data stored in the first storage address.

6. The method according to claim 1, characterized in that Obtain backup data from the original database cluster in advance, including: In response to detecting the backup instruction, reading a second storage address and a second backup address from the backup configuration file, wherein the second backup address is the same as the first storage address; and the second storage address is the address of the original database cluster; The backup data is acquired from the second storage address, and the backup data is stored in the second backup address.

7. The method according to claim 6, characterized in that The backup configuration file further includes a third data sharding parameter, and the acquiring the backup data from the second storage address includes: Acquire backup data corresponding to the third data sharding parameter from the second storage address; The backup data corresponding to the third data sharding parameter is the backup data stored in the second storage address, or the backup data corresponding to the third data sharding parameter is a part of the backup data stored in the second storage address.

8. A distributed database non-peer resource backup and recovery device, characterized in that: The device comprises: A reading module, configured to read a first parameter and a second parameter from a recovery configuration file in response to detecting a backup recovery instruction, and generate a reading result for the second parameter, wherein the first parameter includes a first storage address and a first backup address, the first storage address represents a temporary storage address of backup data pre-acquired from an original database cluster, and the first backup address is an address of a target database cluster; the second parameter includes a deployment parameter representing a hardware deployment mode of the target database cluster, and a first data sharding parameter representing a difference in hardware resources between the original database cluster and the target database cluster; A data acquisition module, configured to acquire the backup data from the first storage address in response to determining that the read result indicates that the recovery configuration file includes a second parameter; A first recovery module, configured to restore the backup data to the first backup address based on the deployment parameters; A redistribution module, configured to perform a redistribution operation on the backup data based on the first data sharding parameter to obtain final backup data; The redistribution module includes a redistribution submodule and a synchronization submodule: The redistribution submodule is used to redistribute the backup data on the target data shard indicated by the first data shard parameter to the preset data shard in the primary server on the target database cluster to obtain the final backup data; and The synchronization submodule is used to synchronize the final backup data on the primary server to at least one backup server of the target database cluster.

9. An electronic device, comprising: one or more processors; a memory for storing one or more programs, It is characterized in that when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

Images