Database cluster physical backup method and related product

Abstract

The application provides a database cluster physical backup method and related products. The database cluster physical backup method comprises the following steps: an initiating node reads a configuration file stored by itself, initiates a physical backup process, and acquires a starting log sequence number; a backup set label of this backup is constructed; the starting log sequence number and the backup set label are synchronized to other nodes, and the other nodes are guided to verify whether they participate in this backup; the initiating node acquires meta information of all files in a database directory, and distributes a required copy file list to each node participating in this backup according to the configuration file of the initiating node; each node is guided to copy files according to the distributed file list, so that each node records the meta information of the copied files in the copying process; and the initiating node receives the meta information of the copied files returned by each node. The application has the advantage that all available nodes can participate in backup, and resource consumption and occupation caused by single node backup are avoided.

Description

Technical Field

[0001] This invention relates to the field of databases, and in particular to a physical backup method for database clusters and related products. Background Technology

[0002] In today's digital age, data has become one of the core assets of enterprises and organizations. As a key tool for data storage and management, the stability and security of databases are paramount. Database failures, data loss, or malicious attacks can lead to significant economic losses and reputational damage for businesses. Therefore, database backup has become an important means of ensuring data security.

[0003] For read-write split database clusters, physical backup is a common data backup strategy. When performing a physical backup of a read-write split database cluster, the typical approach is to select a database node and then copy the physical files of the database directory within that node. The target storage location for this copy is primarily one of three types: 1) dedicated storage media directly connected to the local node; 2) storage media mounted over a network; and 3) a remote dedicated storage node.

[0004] However, this traditional physical backup method has some significant drawbacks. Firstly, regardless of the storage method used, it relies on dedicated storage media, which undoubtedly increases the cost of physical backups. Secondly, if a network is used during the backup process, the physical backup speed will be affected by network factors. When copying large files, not only may network failures cause backup failures, but network speed limitations may also significantly increase backup time, thus affecting backup efficiency and timely data recovery. Summary of the Invention

[0005] One objective of this invention is to enable all available nodes to participate in backup, thereby avoiding resource consumption and occupation caused by a single node performing backup.

[0006] Another objective of this invention is to quickly identify nodes that can participate in backup.

[0007] Another objective of this invention is to improve the efficiency and accuracy of data restoration.

[0008] Specifically, according to a first aspect of the present invention, the present invention provides a physical backup method for a database cluster, comprising:

[0009] The initiating node reads its own stored configuration file, initiates the physical backup process, and obtains the starting log sequence number for the physical backup.

[0010] The initiating node constructs a backup set tag for this backup, which is used to back up the parent directory of all files. Each backup set corresponds to a separate subdirectory.

[0011] The initiating node synchronizes the starting log sequence number and the backup set tag to other nodes in the database cluster, guides the other nodes to verify whether they are participating in this backup, and receives feedback information from other nodes on whether they are participating in this backup.

[0012] The initiating node obtains the metadata of all files in the database directory and allocates the list of files to be copied to each node participating in this backup according to its own configuration file.

[0013] The initiating node guides each node participating in this backup to copy files according to the allocated file list, prompting each node to record the metadata of the copied files during the copying process;

[0014] The initiating node receives the metadata of the copied files returned by each node participating in the backup.

[0015] Optionally, the step of guiding the other nodes to verify whether they are participating in this backup includes:

[0016] After receiving the starting log sequence number, each node is guided to verify whether its own database directory has reached the starting log sequence number;

[0017] If the starting log sequence number has been reached, then the node is determined to be eligible to participate in this backup.

[0018] If the starting log sequence number is not reached, then the node is determined not to participate in this backup.

[0019] Each node will send information about whether it is participating in this backup back to the initiating node.

[0020] Optionally, the configuration file of each node records the basic information of all database nodes, the directory for storing physical backup set metadata for each node, and the directory for storing physical backup files for each node, and sets the storage limit, priority, and available storage capacity of each directory.

[0021] The steps involved in allocating the list of files to be copied to each node participating in this backup based on its own configuration file include:

[0022] The initiating node allocates the physical files to be copied to each node participating in this backup according to the storage limit, available storage capacity and priority of each node, and forms a list of files to be copied.

[0023] Optionally, the step of the initiating node guiding each node participating in this backup to copy files according to the allocated file list, and prompting each node to record the metadata of the copied files during the copying process, includes:

[0024] The initiating node prompts each node participating in this backup to copy the files corresponding to the file list in its own database directory to the preset physical file storage directory in the configuration file. This directory is named with the backup set tag to form a separate subdirectory.

[0025] Each node participating in this backup records the metadata of the copied files and writes the recorded metadata into a backup list file named with the backup set tag in the backup set metadata directory.

[0026] Optionally, after the step of the initiating node receiving the metadata of the copied files returned by each node participating in the backup, the method further includes:

[0027] The initiating node summarizes the metadata of all files copied by all nodes into a backup list file named with the backup set tag in its own backup set metadata directory;

[0028] The initiating node synchronizes its own backup manifest file with each node participating in this backup, prompting each node to update its own backup manifest file according to the backup manifest file.

[0029] Optionally, data synchronization between nodes in the database cluster is achieved through a network shell protocol, a custom network protocol, or a temporary database table.

[0030] Optionally, when data needs to be restored, the required backup set is determined from the node that initiated the restoration based on the restoration parameters;

[0031] The node that initiates the restoration reads the backup manifest file of the backup set to obtain the storage information of all files;

[0032] The node that initiates the restoration copies the file that needs to be restored from the corresponding node that stores the file.

[0033] Optionally, during the process of the node initiating the restore reading the backup manifest file of the backup set,

[0034] If its database directory exists, perform incremental restore;

[0035] If the database directory does not exist, create the database directory and perform a full restore.

[0036] According to a second aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the physical backup method for a database cluster as described above.

[0037] According to a third aspect of the present invention, the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the physical backup method for a database cluster as described above.

[0038] The physical backup method for database clusters in this invention involves an initiating node coordinating the process, with each node participating according to its configuration and status. This avoids excessive resource consumption by a single node, ensuring the overall stability of the cluster performance and fully utilizing the computing, storage, and network resources of each node. Multi-node parallel operation shortens backup time. During backup, each node copies files according to its assigned file list, accelerating data copying. Each node records and transmits file metadata, which is then summarized by the initiating node, ensuring data integrity and accuracy, providing a reliable basis for data recovery, and effectively guaranteeing data consistency.

[0039] Furthermore, the physical backup method for database clusters of the present invention uses the starting log sequence number as a judgment criterion to ensure that the data of the nodes participating in the backup is in the latest state. This allows for the rapid screening of nodes with lagging data, promptly removing these nodes from the backup operation and avoiding invalid data processing flows. The initiating node can then quickly advance subsequent tasks, greatly saving backup time, significantly improving backup efficiency, and meeting the needs of efficient database operation and maintenance.

[0040] Furthermore, in the physical backup method for the database cluster of the present invention, the node initiating the restoration determines the backup set based on the restoration parameters and obtains the file storage information by reading the backup list file. This enables the node initiating the restoration to accurately locate the corresponding node, avoiding blind searching and improving file retrieval efficiency. In the file copying stage, files are copied from the corresponding node to ensure complete and accurate file transmission, ultimately completing data restoration and meeting the business requirements for timeliness and accuracy in data recovery.

[0041] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0042] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0043] Figure 1 This is a flowchart of a physical backup method for a database cluster according to an embodiment of the present invention;

[0044] Figure 2 This is a flowchart illustrating the aggregation and synchronization of metadata according to an embodiment of the present invention;

[0045] Figure 3 Based on Figure 1 The flowchart for data restoration using physical backup methods;

[0046] Figure 4 This is a schematic diagram of a computer program product according to an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0049] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0050] This invention provides a physical backup method for a database cluster, aiming to fully leverage the advantages of clustering and involve all nodes in the physical backup process. Each node only needs to copy a portion of the files to its local storage medium; the total number of files copied by all nodes constitutes a complete physical backup set. After the backup is complete, the metadata of the backup set is transmitted to each node via the network, ensuring that each node retains complete backup set metadata, facilitating subsequent data restoration.

[0051] This method not only effectively reduces storage costs, but also allows all available nodes to participate in the backup or restore process, changing the situation where a single node bears the backup and restore task. It avoids excessive consumption and occupation of resources caused by a single node operating under high load for a long time, and achieves efficient utilization of cluster resources.

[0052] The physical backup method for database clusters of the present invention is mainly applied to read-write separation database clusters. Before the implementation of this method, there is an implicit default condition, namely, the read-write separation cluster is in a synchronized state, and the contents of the data directory files in each node are consistent.

[0053] During the initial deployment configuration phase, the specific steps are as follows:

[0054] All nodes need to have a physical backup and recovery binary tool called sys_rman installed; if it cannot be installed, the tool's capabilities can be integrated into the database kernel, enabling each node to perform backup, restore, and communication with other nodes.

[0055] A sys_rman.conf configuration file is generated in the data directory of each node to record the basic information of all database nodes, such as IP, port, database name, username, etc. It also records the directory used by each node to store physical backup set metadata and the directory used by each node to store physical backup files. There can be multiple directories, or they can be the same directory as the directory storing metadata.

[0056] Set storage limits and priorities for each directory to store more physical files in high-priority node directories, thus achieving load balancing.

[0057] The priority setting principle is mainly based on the type of disk where the directory is located. For example, solid-state drives (SSDs) are usually set to high priority because of their high read and write speeds. At the same time, the hard drive's activity level can also be taken into account. If no other programs are accessing or using the hard drive, it can also be set to high priority.

[0058] Figure 1 This is a flowchart of a physical backup method for a database cluster according to an embodiment of the present invention, such as... Figure 1 As shown, the physical backup method includes at least the following steps S102 to S112.

[0059] In step S102, the initiating node reads its own stored configuration file, initiates the physical backup process, and obtains the starting log sequence number of the physical backup.

[0060] Step S104: The initiating node constructs a backup set label for this backup, which is used to back up the parent directory of all files. Each backup set corresponds to a separate subdirectory.

[0061] In step S106, the initiating node synchronizes the starting log sequence number and backup set label to other nodes in the database cluster, guides other nodes to verify whether they are participating in this backup, and receives feedback information from other nodes regarding whether they are participating in this backup.

[0062] In step S108, the initiating node obtains the metadata of all files in the database directory and allocates the list of files to be copied to each node participating in this backup according to its own configuration file.

[0063] Step S110: The initiating node guides each node participating in this backup to copy files according to the allocated file list, prompting each node to record the metadata of the copied files during the copying process.

[0064] Step S112: The initiating node receives the metadata of the copied files returned by each node participating in the backup.

[0065] The physical backup method for database clusters in this invention involves an initiating node coordinating the process, with each node participating according to its configuration and status. This avoids excessive resource consumption by a single node, ensuring the overall stability of the cluster performance and fully utilizing the computing, storage, and network resources of each node. Multi-node parallel operation shortens backup time. During backup, each node copies files according to its assigned file list, accelerating data copying. Each node records and transmits file metadata, which is then summarized by the initiating node, ensuring data integrity and accuracy, providing a reliable basis for data recovery, and effectively guaranteeing data consistency.

[0066] In practical applications, the selection of the initiating node can be based on various strategies. For example, a round-robin approach can be used, selecting a node sequentially according to a pre-defined node order. This ensures that each node has the opportunity to undertake the initiation task, thus allocating resources relatively fairly.

[0067] Alternatively, the decision can be made based on the node load. The system monitors the current load of each node in real time, such as CPU utilization, memory usage, and network bandwidth usage, and selects the node with the lowest load as the initiating node. This ensures that the backup operation is started on a node with relatively sufficient resources, reducing the impact on the normal operation of the database cluster.

[0068] You can also select the most powerful node as the initiating node based on the node's performance parameters, such as the node's hardware configuration (e.g., number of CPU cores, memory capacity, disk read / write speed, etc.), because a high-performance node may be more efficient in handling backup startup-related operations and can complete the backup initialization work faster.

[0069] In physical backups of a database cluster, the starting log sequence number of the physical backup represents a status identifier of the database at the start of the backup, and is an important basis for determining whether node data is suitable to participate in this backup.

[0070] The initiating node will synchronize the starting log sequence number and backup set tag to other nodes in the database cluster. Regarding data synchronization, the nodes in the database cluster can securely and efficiently transmit data using SSH (Secure Shell); they can also use custom network protocols to meet the personalized needs of specific business scenarios; or they can utilize temporary database tables to complete data synchronization operations within the database, ensuring accurate data sharing among all nodes.

[0071] In an optional embodiment, the step of the initiating node guiding other nodes to verify whether it is participating in this backup may be as follows: after each node receives the starting log sequence number, it guides each node to verify whether its own database directory has reached the starting log sequence number. If it has reached the starting log sequence number, it is determined that the node can participate in this backup. If it has not reached the starting log sequence number, it is determined that the node cannot participate in this backup. Each node feeds back the information on whether it is participating in this backup to the initiating node.

[0072] Upon receiving the starting log sequence number, each node verifies the status of its own database directory to see if it has reached or exceeded that sequence number. Log sequence numbers typically record database transaction operations in ascending order, reflecting the progress of database data updates. If a node's database directory reaches the starting log sequence number, it means that the node's data has been updated to a state consistent with or later than the initial backup state, and the data is complete and up-to-date.

[0073] When a node verifies that its own database directory has reached the starting log sequence number, it indicates that the node's data matches or is better than the data status of the backup starting point, and can provide complete and valid data for this backup. Therefore, it is determined that the node can participate in this backup.

[0074] If a node's database directory has not reached the starting log sequence number, it indicates that the node's data updates are lagging, and some transaction operations may not have been recorded. If it is allowed to participate in the backup, it will lead to incomplete or inconsistent backup data. Therefore, it is determined that this node cannot participate in this backup.

[0075] After determining whether it will participate in the backup, each node will send this information back to the initiating node. Based on this feedback, the initiating node can clearly identify which nodes can participate in the backup, and thus prepare to assign file copy tasks to these eligible nodes.

[0076] Using the starting log sequence number as the criterion ensures that the data on nodes participating in the backup is up-to-date, quickly identifying lagging nodes and promptly removing them from the backup operation, thus avoiding ineffective data processing. The initiating node can then rapidly advance subsequent tasks, significantly saving backup time, greatly improving backup efficiency, and meeting the needs of efficient database operation and maintenance.

[0077] During the physical backup of a database cluster, determining whether a node has reached the starting log sequence number includes, but is not limited to, the following two methods:

[0078] Method 1: Determine the relevant fields in your own control file.

[0079] In a database, the control file records key information about the database, such as the database name, the location of data files and log files, the database creation time, and also includes information about log sequence numbers.

[0080] A node can read the log sequence number-related fields from its own control file and compare their values ​​with the received starting log sequence number. If the log sequence number in the control file is greater than or equal to the starting log sequence number, it indicates that the node's data has reached the required state for backup and can participate in this backup; otherwise, it cannot participate.

[0081] Method 2: Perform an SQL query on the current node to retrieve the log sequence numbers that have already been flushed to disk.

[0082] In a database, data is typically first written to a buffer in memory, and then written to disk according to a preset strategy. This process is called disk flushing. A flushed log sequence number means that the corresponding data has been persisted to disk and is stable and reliable.

[0083] Nodes can retrieve flushed log sequence numbers from the database system by executing SQL queries. The database system typically maintains system tables or views to record this information. The node compares the retrieved flushed log sequence number with the starting log sequence number. If the flushed log sequence number matches the starting log sequence number, the node's data status meets the backup requirements and it can participate in the backup; otherwise, it does not meet the conditions.

[0084] The node configuration file can also set the available storage space capacity of the node. The available storage space capacity of each node is the actual usable storage space capacity, which is a configuration item in sys_rman.conf. After each backup, it is necessary to synchronize and update this configuration item in the sys_rman.conf configuration file of all nodes.

[0085] In an optional embodiment, the step of allocating the list of files to be copied to each node participating in this backup according to its own configuration file may be: the initiating node allocates the physical files to be copied to each node participating in this backup according to the storage limit, available storage capacity and priority of each node, and forms the list of files to be copied.

[0086] In a database cluster, different nodes have varying storage capabilities and performance. To efficiently and effectively complete backup tasks, avoid excessive storage pressure on some nodes, and ensure even distribution of backup data, the initiating node needs to allocate files based on the specific conditions of each node. Information such as storage limits, available storage capacity, and priorities for each node is recorded in the initiating node's own configuration file, and this information serves as a crucial reference for file allocation.

[0087] Taking "the initiating node comprehensively considers the available storage space capacity and priority of each node to allocate physical files" as an example, the available storage space capacity determines the upper limit of the file size that a node can accommodate, while the priority reflects the relative importance of a node when storing backup files. By combining these two factors, backup tasks can be allocated more rationally, and resources can be used more effectively.

[0088] The specific allocation method is as follows:

[0089] First, calculate the total size of the physical files to be backed up. To ensure a smooth backup process and to reserve sufficient buffer space, the total available storage space of the nodes participating in the backup must be greater than 1.1 times this total size. If this condition is not met, it indicates that the overall storage capacity of the nodes currently participating in the backup is insufficient. In this case, the system will report an error and exit the backup allocation process to avoid backup failure due to insufficient storage space.

[0090] Secondly, based on the priority of the storage directories on each node, the expected storage size for each node is calculated proportionally. By allocating the total backup size to each node according to priority, higher-priority nodes can undertake relatively more backup tasks, fully leveraging their advantages. For example, when the total size is 1000G and the priority ratio of each node is 3:4:5:7:1, the expected backup file sizes for each node are calculated to be 150G, 200G, 250G, 350G, and 50G, respectively.

[0091] Secondly, after initially calculating the expected backup size for each node, the actual available storage capacity of each node also needs to be considered. Some nodes may, for various reasons, have insufficient available storage capacity to store the expected backup file size. In this case, the difference in capacity will be redistributed to other nodes according to the priority strategy described above. This process will be repeated until each node can store the corresponding expected file size, ensuring the feasibility of the allocation scheme.

[0092] Finally, after determining the final expected storage capacity for each node, the physical files to be backed up are sorted by file length. Then, the file combinations closest to the expected storage capacity of each node are selected from the sorted file list. This ensures that the file size stored on each node is as close as possible to the expected capacity, further optimizing the allocation of backup files and improving the utilization of storage resources.

[0093] In an optional embodiment, the step of the initiating node guiding each node participating in this backup to copy files according to the allocated file list and prompting each node to record the metadata of the copied files during the copying process can be as follows: the initiating node prompts each node participating in this backup to copy the files corresponding to the file list in its own database directory to the physical file storage directory preset in the configuration file. This directory is named with a backup set tag to form a separate subdirectory; each node participating in this backup records the metadata of the copied files and writes the recorded metadata into a backup list file named with a backup set tag in the backup set metadata directory.

[0094] The initiating node acts as a coordinator and guide, directing the participating nodes to perform file copying operations. Specifically, each node needs to locate the files corresponding to the allocated file list in its own database directory and then copy these files to the physical file storage directory pre-defined in the configuration file. To facilitate management and differentiation between different backup sets, this storage directory is named using the backup set tag for this backup, forming a separate subdirectory. This ensures that the files for each backup set are clearly organized and stored, preventing files from different backup sets from being mixed up.

[0095] During the file copying process, each node involved in the backup records the metadata of the copied files. Metadata is descriptive data about the files, typically including the filename, file path, file length, last modified time, and checksum value.

[0096] After each node records its metadata, it writes this information to a backup manifest file named after the backup set tag in the backup set metadata directory. The backup manifest file acts as a detailed index, recording the metadata of all files in this backup set. Using this manifest file, during data restoration, the required files can be quickly and accurately located and retrieved. It also allows for the verification and checking of backup data, ensuring data consistency and reliability.

[0097] Figure 2 This is a flowchart of the aggregation and synchronization of metadata according to an embodiment of the present invention, such as... Figure 2 As shown, after the initiating node receives the metadata of the copied files returned by each node participating in the backup, it is also necessary to summarize and synchronize the metadata. The summary and synchronization of metadata includes at least the following steps S202 to S204.

[0098] In step S202, the initiating node summarizes the metadata of all node copied files into a backup list file named with the backup set tag in its own backup set metadata directory.

[0099] In step S204, the initiating node synchronizes its own backup manifest file with each node participating in this backup, prompting each node to update its own backup manifest file according to the backup manifest file.

[0100] This ensures that the backup list information on all nodes is up-to-date and consistent, preventing errors or omissions during data recovery due to inconsistent metadata. During subsequent data restoration, each node can accurately locate and restore the corresponding files based on this unified backup list, ensuring the database can be correctly restored to its state at the time of backup.

[0101] Figure 3 Based on Figure 1 The flowchart for data restoration using physical backup methods, such as Figure 3 As shown, when data needs to be restored, the following steps S302 to S306 can be performed.

[0102] Step S302: Determine the required backup set from the node that initiated the restore, based on the restore parameters.

[0103] Restore parameters may include specific timestamps, version numbers, or other identifying information. These parameters help nodes accurately locate the backup set that meets the restore requirements. For example, if a user wants to restore the database to a state on a specific date, the node initiating the restore will find the corresponding backup set based on this time parameter.

[0104] In step S304, the node initiating the restoration reads the backup list file of the backup set to obtain the storage information of all files.

[0105] The backup manifest file records detailed storage information for all files in the backup set, such as the specific node where the file is located, the file path, and the file name. By reading this file, the node initiating the restore can clearly know the storage location of each file that needs to be restored, providing necessary guidance for subsequent file copy operations.

[0106] In step S306, the node that initiates the restoration copies the file to be restored from the corresponding node that stores the file.

[0107] For example, when copying via SSH, the node initiating the restore uses the SSH protocol to establish a secure connection with the corresponding node storing the file, and then uses SSH-based tools such as scp (Secure Copy) to copy the file to be restored from the storage node to the specified location on the node initiating the restore.

[0108] Alternatively, the storage node first reads the file to be restored, and then transmits the file data over the network to the node initiating the restoration. Upon receiving the data, the node initiating the restoration writes it to a pre-defined recovery directory.

[0109] Understandably, the node initiating the restore determines the backup set based on the restore parameters and obtains the file storage information by reading the backup manifest file. This allows the node initiating the restore to accurately locate the corresponding node, avoiding blind searches and improving file retrieval efficiency. During the file copying phase, files are copied from the corresponding node, ensuring complete and accurate file transmission, ultimately completing the data restoration and meeting the business's requirements for timely and accurate data recovery.

[0110] In one optional embodiment, during the process of the node initiating the restore reading the backup manifest file of the backup set, if its database directory exists, an incremental restore is performed to improve restore efficiency and avoid repeatedly restoring unchanged data. If its database directory does not exist, a new database directory is created, and a full restore is performed to restore the data of the entire database.

[0111] Specifically, the initiating node first reads the metadata of all files in its own database directory. It then compares the collected local file metadata with the file metadata recorded in the backup list file. Based on the comparison results, if the local file's file length, last modification time, and checksum value are completely consistent with the corresponding file in the backup list, it means that the file has not changed since the last backup and no restoration operation is needed. Conversely, if any of these pieces of information are inconsistent, it indicates that the file content may have changed, and the file needs to be restored from the backup to ensure that the database's data state is consistent with that at the time of the backup.

[0112] If the data directory of the initiating node is unavailable, it may be due to deletion, corruption, or other reasons preventing normal access. In this case, a full restore is required to recover the entire database. The initiating node first creates a new database directory to provide storage for subsequent file restore operations. This new directory will serve as the storage space for the restored data. Since the previous data state cannot be determined, to ensure that the database can be completely restored to the state at the time of backup, all files recorded in the backup list need to be restored to the newly created data directory. A full restore ensures that the database contains all the data from the backup, preventing data loss.

[0113] After the file copy is complete, the data directory containing the restored data on the initiating node needs to be started. Once the data directory is started, the database kernel will automatically execute a series of recovery operations to restore the database to the specified state. Since this part of the operation is a routine function of the database system and is not the focus of this invention, it will not be described in detail here.

[0114] In a read / write separation cluster that is in a synchronized state and whose data directory files are identical across all nodes, this invention proposes a novel physical backup scheme. All available nodes participate in the physical backup, with each node copying only a portion of the files to its local storage medium. The set of files copied by all nodes constitutes a physical backup set. The metadata of the backup set is transmitted to each node via the network, ensuring that each node possesses complete metadata for the backup set, facilitating restoration.

[0115] Compared to traditional methods, this backup solution is expected to reduce reliance on dedicated storage media and lower backup costs. At the same time, because each node operates in parallel, it can effectively improve backup efficiency and avoid problems caused by network factors affecting the backup of large files on a single node.

[0116] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0117] It should be understood that in some embodiments, the components may be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods may be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0118] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 4 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 5 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 6 This is a schematic diagram of a computer device 30 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which, when executed by the processor 32, implements the steps of any of the above-described physical backup methods for a database cluster. A computer-readable storage medium 20 stores the computer program 11 thereon, which, when executed by the processor 32, implements the steps of the physical backup method for a database cluster according to any of the above-described embodiments. The computer device 30 may include a memory 31, a processor 32, and the computer program 11 stored on the memory 31 and running on the processor 32.

[0119] The computer program 11 used to perform the operations of this invention may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, Field-Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), may execute computer-readable program instructions using status information from computer-readable program instructions to personalize the electronic circuits.

[0120] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0121] For the purposes of this embodiment, the computer-readable storage medium 20 is a tangible device capable of holding and storing a computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting the program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0122] Computer device 30 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 30 can be a cloud computing node. Computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 30 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.

[0123] Computer device 30 may include a processor 32 adapted to execute stored instructions and a memory 31 that provides temporary storage space for the operation of said instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0124] Computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows external devices that can be connected to the computer device to input and output data. The network adapter / interface provides communication between the computer device and a network, typically represented as a communication network.

[0125] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A physical backup method for a database cluster, comprising: The initiating node reads its own stored configuration file, initiates the physical backup process, and obtains the starting log sequence number for the physical backup. The initiating node constructs a backup set tag for this backup, which is used to back up the parent directory of all files. Each backup set corresponds to a separate subdirectory. The initiating node synchronizes the starting log sequence number and the backup set tag to other nodes in the database cluster, guides the other nodes to verify whether they are participating in this backup, and receives feedback information from other nodes on whether they are participating in this backup. The initiating node obtains the metadata of all files in the database directory and allocates the list of files to be copied to each node participating in this backup according to its own configuration file. The initiating node guides each node participating in this backup to copy files according to the allocated file list, prompting each node to record the metadata of the copied files during the copying process; The initiating node receives the metadata of the copied files returned by each node participating in the backup.

2. The physical backup method for a database cluster according to claim 1, wherein, The steps to guide the other nodes to verify whether they are participating in this backup include: After receiving the starting log sequence number, each node is guided to verify whether its own database directory has reached the starting log sequence number; If the starting log sequence number has been reached, then the node is determined to be eligible to participate in this backup. If the starting log sequence number is not reached, then the node is determined not to participate in this backup. Each node will send information about whether it is participating in this backup back to the initiating node.

3. The physical backup method for a database cluster according to claim 1, wherein, Each node's configuration file records the basic information of all database nodes, the directory for storing physical backup set metadata for each node, and the directory for storing physical backup files for each node. It also sets the storage quota, priority, and available storage capacity for each directory. The steps involved in allocating the list of files to be copied to each node participating in this backup based on its own configuration file include: The initiating node allocates the physical files to be copied to each node participating in this backup according to the storage limit, available storage capacity and priority of each node, and forms a list of files to be copied.

4. The physical backup method for a database cluster according to claim 3, wherein, The steps of the initiating node guiding each participating node to copy files according to the allocated file list, and prompting each node to record the metadata of the copied files during the copying process, include: The initiating node prompts each node participating in this backup to copy the files corresponding to the file list in its own database directory to the preset physical file storage directory in the configuration file. This directory is named with the backup set tag to form a separate subdirectory. Each node participating in this backup records the metadata of the copied files and writes the recorded metadata into a backup list file named with the backup set tag in the backup set metadata directory.

5. The physical backup method for a database cluster according to claim 4, wherein, After the step of the initiating node receiving the metadata of the copied files returned by each node participating in the backup, the method further includes: The initiating node summarizes the metadata of all files copied by all nodes into a backup list file named with the backup set tag in its own backup set metadata directory; The initiating node synchronizes its own backup manifest file with each node participating in this backup, prompting each node to update its own backup manifest file according to the backup manifest file.

6. The physical backup method for a database cluster according to claim 1, wherein, The nodes of the database cluster synchronize data through a network shell protocol, a custom network protocol, or a temporary database table.

7. The physical backup method for a database cluster according to claim 1, wherein, When data needs to be restored, the required backup set is determined from the node that initiated the restoration based on the restoration parameters. The node that initiates the restoration reads the backup manifest file of the backup set to obtain the storage information of all files; The node that initiates the restoration copies the file that needs to be restored from the corresponding node that stores the file.

8. The physical backup method for a database cluster according to claim 7, wherein, During the process of the node initiating the restore reading the backup manifest file of the backup set, If its database directory exists, perform incremental restore; If the database directory does not exist, create the database directory and perform a full restore.

9. A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the physical backup method for a database cluster according to any one of claims 1-8.

10. A computer program product comprising a computer program that, when executed by a processor, implements the physical backup method for a database cluster according to any one of claims 1-8.

Images