Rapid cloning method for distributed postgresql database

Abstract

The present application belongs to the technical field of IT software development, and relates to a rapid cloning method for a distributed PostgreSQL database. The method comprises the following steps: creating a distributed cluster consistent restore-point; performing snapshot-based cloning on underlying database instances; restoring an underlying database by means of the restore point; and performing metadata restoration. In the present application, on the basis of the study on the principle of a Citus plug-in, a metadata repair script is customized, which is used as a post step following the restoration of an underlying database, and realizes a cluster-dimension cloning operation, and thus metadata of a cluster obtained after physical cloning replication of a distributed database which is obtained on the basis of extension performed by Citus is directly repairable, and it is not necessary to sequentially execute, according to a conventional process, the operations of newly building an empty cluster, calling a Citus interface by the new cluster to regenerate metadata and performing logic importing of service data, thereby significantly shortening a cloning time; moreover, in view of the general capability of a cloud computing infrastructure layer, a Citus open interface and a PostgreSQL self-restoration mechanism, the cloning operation for an underlying database is realized.

Description

A fast cloning method for distributed PostgreSQL database

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on December 13, 2023, with application number 202311706756.3 and invention name “A method for rapid cloning of a distributed PostgreSQL database”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application belongs to the field of IT software development technology, and in particular relates to a method for quickly cloning a distributed PostgreSQL database. Background Art

[0004] Citus is distributed middleware for the PostgreSQL database. It does not intrude or modify the database's kernel code. Instead, it writes metadata to the underlying database and extends PostgreSQL capabilities as a plug-in, enabling horizontal scaling of the database. Citus primarily consists of CN and DN nodes. Each node is a single or a group of PostgreSQL instances in a master-slave relationship.

[0005] For distributed databases extended by Citus, there is no standard cloud cloning solution in the existing technology. One possible solution is to clone the underlying database through physical replication. Although this solution can generate a new cluster, the IP address of the server where the new cluster is located often has changed, while the metadata information of the new cluster is exactly the same as that of the original cluster. In fact, this will cause the CN nodes of the new cluster to be reorganized into a cluster with the DN nodes of the old cluster, which does not meet the expected goal of the cloning operation. Another possible solution is to call the Citus interface in the empty new cluster to regenerate metadata and then import business data. Although this solution can achieve the expected goal of the cloning operation, it is time-consuming and will drag down the service recovery time indicator in disaster recovery scenarios. Summary of the Invention

[0006] In view of the above-mentioned deficiencies in the existing technology, the purpose of the application is to provide a method for quickly cloning a distributed PostgreSQL database, which mainly includes two steps: cloning the underlying database instance and metadata recovery. The cloning step of the underlying database instance is to use snapshots, volume cloning and other operations of the cloud computing infrastructure layer to achieve physical cloning of the underlying database instance, and clone the business data and metadata as is to the new cloud server. The metadata recovery step is based on the study of the Citus plug-in principle, and executes a customized metadata repair script to replace and repair the metadata information of Citus, so that the distributed database can restore the correct forwarding mechanism and correctly respond to the SQL instructions issued by the client again, thereby realizing cloning operations in the cluster dimension.

[0007] This application proposes a method for quickly cloning a distributed PostgreSQL database, comprising the following steps:

[0008] S1. Create a distributed cluster consistent recovery point;

[0009] S2. Snapshot clones of the underlying database instance;

[0010] S3. Restore the underlying database from the recovery point.

[0011] S4. Perform metadata recovery.

[0012] Optionally, the step of creating a distributed cluster consistent recovery point specifically includes:

[0013] S11. Call the citus_create_restore_point interface provided by Citus;

[0014] S12. Use the citus_create_restore_point interface to seize the exclusive lock of the entire distributed cluster at the bottom layer, prohibiting write operations to the cluster within a preset time.

[0015] S13. After the lock is successfully seized, a restore point is automatically created on all underlying database instances.

[0016] Optionally, in S13, the restore point has cluster-wide consistency.

[0017] Optionally, the step of cloning the underlying database instance from the snapshot specifically includes:

[0018] S21. Based on the cloud computing infrastructure layer, treat the cloud disks where all underlying database instances reside as a whole and create a snapshot consistency group to ensure the time consistency of data written to the cloud disks.

[0019] S22. Perform a volume clone operation based on the snapshot consistency group to create a new set of cloud servers to mount the cloned volumes.

[0020] Optionally, the step of restoring the underlying database through the recovery point specifically includes:

[0021] S31. Start all underlying PostgreSQL instances on the new cloud server.

[0022] S32.After the PostgreSQL instance is started, it will automatically use the pre-written log file to perform the crash recovery process.

[0023] Optionally, the metadata recovery step specifically includes:

[0024] S41. Execute the custom metadata repair script on the server corresponding to the underlying PostgreSQL instance cloned from the original Citus CN node.

[0025] Optionally, the execution logic of the script is to scan metadata information in the cluster.

[0026] Optionally, the step of executing the custom metadata repair script on the server corresponding to the underlying PostgreSQL instance cloned from the original Citus CN node includes:

[0027] S42. Call the citus_update_node interface provided by Citus based on the IP port correspondence before and after cloning to update the metadata information and restore the distributed cluster function.

[0028] Optionally, in S41, the input parameters of the script are connection information of each underlying database of the original cluster and connection information of each underlying database of the new cluster.

[0029] Optionally, the underlying database has one master architecture and two slave architectures.

[0030] The beneficial effects of this application are as follows:

[0031] 1. Based on research on the principles of the Citus plug-in, this application customizes a metadata repair script as a post-step to the recovery of the underlying database, implementing a cluster-level cloning operation. This allows the metadata of a cluster to be repaired after the physical cloning of a distributed database based on Citus extensions. This eliminates the need to follow the traditional process of creating a new empty cluster, calling the Citus interface to regenerate metadata, and then logically importing business data, significantly reducing cloning time.

[0032] 2. This application combines the general capabilities of the cloud computing infrastructure layer, the Citus open interface, and the PostgreSQL self-recovery mechanism to implement the cloning operation of the underlying database, making the physical cloning of the distributed database based on the Citus extension operational. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the present application. Throughout the drawings, the same reference numerals represent the same components. Obviously, the drawings described below are only some of the embodiments described in the present application. Those skilled in the art can also obtain other drawings based on these drawings.

[0034] FIG1 is a flow chart of an embodiment of the present application;

[0035] FIG2 is a schematic diagram of distributed database cloning based on Citus extension according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work should fall within the scope of protection of this application.

[0037] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in this application.

[0038] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of methods and systems consistent with certain aspects of the present application, as detailed in the appended claims.

[0039] This application proposes a fast cloning method for a distributed PostgreSQL database to solve the problem that cloning a distributed database expanded from Citus in a disaster recovery scenario takes too long.

[0040] Method Example

[0041] In order to assist people in understanding this application, the terms appearing in this article are explained.

[0042] Citus: Distributed middleware for the PostgreSQL database, extending PostgreSQL capabilities in the form of plug-ins.

[0043] CN (Coordinator Node): A role of Citus node, mainly responsible for storing metadata and routing forwarding.

[0044] DN (Worker Node): A role of Citus node that mainly stores actual data.

[0045] RTO (Recovery Time Objective): An indicator that reflects the timeliness of business recovery, indicating the time required for business to return to normal after interruption. The smaller the RTO value, the stronger the data recovery capability of the disaster recovery system.

[0046] VIP (Virtual IP Address): Users can assign virtual IP addresses to multiple servers, achieving load balancing and high availability. Virtual IPs can also quickly switch servers, improving service reliability.

[0047] In addition, in this application, as shown in Figures 1 and 2, a method for quickly cloning a distributed PostgreSQL database is provided, and the following implementation methods are provided:

[0048] Implementation Method 1

[0049] S1. Create a distributed cluster consistent recovery point;

[0050] S2. Snapshot clones of the underlying database instance;

[0051] S3. Restore the underlying database from the recovery point.

[0052] S4. Perform metadata recovery.

[0053] Implementation Method 2

[0054] S1. Call the citus_create_restore_point interface provided by Citus;

[0055] S2. Use the citus_create_restore_point API to seize the exclusive lock of the entire distributed cluster at the bottom layer, prohibiting write operations to the cluster within a preset time.

[0056] S3. After the lock is successfully preempted, a restore point is automatically created on all underlying database instances. This restore point is consistent across the cluster. This step ensures that business data is flushed to disk and that the database can automatically perform crash recovery to restore services after physical replication.

[0057] S4. Snapshot and clone the underlying database instance;

[0058] S5. Restore the underlying database using the recovery point.

[0059] S6. Perform metadata recovery.

[0060] Implementation 3

[0061] S1. Create a distributed cluster consistent recovery point;

[0062] S2. Based on the cloud computing infrastructure layer, the cloud disks containing all underlying database instances are treated as a whole and a snapshot consistency group is created to ensure the time consistency of data written to the cloud disks.

[0063] S3. Then, perform a volume clone operation based on the snapshot consistency group, creating a new set of cloud servers to mount the cloned volumes. This step achieves a complete clone of the underlying database architecture, business data, and metadata.

[0064] S4. Restore the underlying database using the recovery point.

[0065] S5. Perform metadata recovery.

[0066] Implementation 4

[0067] S1. Create a distributed cluster consistent recovery point;

[0068] S2. Snapshot clones of the underlying database instance;

[0069] S3. Start all underlying PostgreSQL instances on the new cloud server.

[0070] S4. After the PostgreSQL instance starts, it automatically uses the write-ahead log file to perform crash recovery. This step utilizes PostgreSQL's self-recovery mechanism after an abnormal shutdown, allowing the underlying database to independently provide services.

[0071] S5. Perform metadata recovery.

[0072] Implementation 5

[0073] S1. Create a distributed cluster consistent recovery point;

[0074] S2. Snapshot clones of the underlying database instance;

[0075] S3. Restore the underlying database from the recovery point.

[0076] S4. Execute a custom metadata repair script on the server corresponding to the underlying PostgreSQL instance cloned from the original Citus CN node. The script's input parameters include the connection information for each underlying database in the original cluster and the connection information for each underlying database in the new cluster.

[0077] S5. The execution logic is to scan the metadata information in the cluster, call the citus_update_node interface provided by Citus according to the IP port correspondence before and after cloning, update the metadata information, and restore the function of the distributed cluster. By studying the principle of the Citus plug-in, it can be known that operations on the Citus shard table will eventually be forwarded to one or more DN nodes. During forwarding, the metadata information is directly read to obtain specific connection information. In the cloning scenario, since the architecture and actual data are exactly the same as the original cluster, it is actually only necessary to correct the connection information related to the DN node in the metadata, and the new cluster can restore the correct forwarding mechanism.

[0078] Implementation Method 6

[0079] S1. Call the citus_create_restore_point interface provided by Citus;

[0080] S2. Use the citus_create_restore_point API to seize the exclusive lock of the entire distributed cluster at the bottom layer, prohibiting write operations to the cluster within a preset time.

[0081] S3. After the lock is successfully preempted, a restore point is automatically created on all underlying database instances. This restore point is consistent across the cluster. This step ensures that business data is flushed to disk and that the database can automatically perform crash recovery to restore services after physical replication.

[0082] S4. Based on the cloud computing infrastructure layer, the cloud disks containing all underlying database instances are treated as a whole and a snapshot consistency group is created to ensure the time consistency of data written to the cloud disks.

[0083] S5. Then, perform a volume clone operation based on the snapshot consistency group, creating a new set of cloud servers to mount the cloned volumes. This step achieves a complete clone of the underlying database architecture, business data, and metadata.

[0084] S6. Start all underlying PostgreSQL instances on the new cloud server.

[0085] S7. After the PostgreSQL instance starts, it automatically uses the write-ahead log file to perform crash recovery. This step utilizes PostgreSQL's self-recovery mechanism after an abnormal shutdown, allowing the underlying database to independently provide services.

[0086] S8. Execute the custom metadata repair script on the server corresponding to the underlying PostgreSQL instance cloned from the original Citus CN node.

[0087] S9. Call the citus_update_node interface provided by Citus based on the IP-port correspondence before and after cloning to update the metadata information and restore the distributed cluster function. By studying the principle of the Citus plug-in, it can be known that operations on the Citus shard table will eventually be forwarded to one or more DN nodes. During forwarding, the metadata information is directly read to obtain specific connection information. In the cloning scenario, since the architecture and actual data are exactly the same as the original cluster, it is actually only necessary to correct the connection information related to the DN node in the metadata, and the new cluster can restore the correct forwarding mechanism.

[0088] In addition, in this application, embodiment 7 is provided:

[0089] Regarding the above underlying databases, the underlying databases all have a 1-master-2-slave architecture. The connection information for the underlying database 1 is:

[0090] 192.168.18.20:5432

[0091] 192.168.18.30:5432

[0092] 192.168.18.40:5432

[0093] External connections use VIP:192.168.18.50,

[0094] The connection information of the underlying database 2 is:

[0095] 192.168.18.60:5432

[0096] 192.168.18.70:5432

[0097] 192.168.18.80:5432

[0098] External connections use VIP:192.168.18.90,

[0099] The connection information of the underlying database 3 is:

[0100] 192.168.18.100:5432

[0101] 192.168.18.110:5432

[0102] 192.168.18.120:5432

[0103] External connections use VIP:192.168.18.130,

[0104] The distributed database is extended by Citus on three underlying PostgreSQL databases. The CN node is 192.168.18.90:5432, the DN1 node is 192.168.18.50:5432, and the DN2 node is 192.168.18.130:5432.

[0105] Stop writing business data and prepare to clone the distributed database.

[0106] Connect to the CN node and execute select citus_create_restore_point('restore_p1'); create a cluster-consistent restore point named restore_p1. This operation prohibits write operations to the cluster for a preset time. Citus automatically creates restore points on the three underlying databases.

[0107] Utilize the common capabilities of the cloud computing infrastructure layer to treat the cloud disks corresponding to the database instance installation directories on servers 192.168.18.20, 192.168.18.30, 192.168.18.40, 192.168.18.60, 192.168.18.70, 192.168.18.80, 192.168.18.100, 192.168.18.110, and 192.168.18.120 as a whole. Create a snapshot consistency group, then clone volumes based on the snapshot consistency group to create a new set of cloud servers and mount the cloned volumes.

[0108] Start all underlying database instances on the new cloud server. After PostgreSQL starts, it will automatically use the write-ahead log file for crash recovery. The cloned underlying database connection information is as follows:

[0109] The connection information for cloning the underlying database 1 is:

[0110] 10.10.5.20:5432;

[0111] 10.10.5.30:5432;

[0112] 10.10.5.40:5432;

[0113] External connections use VIP:10.10.5.50,

[0114] The connection information for cloning the underlying database 2 is:

[0115] 10.10.5.60:5432;

[0116] 10.10.5.70:5432;

[0117] 10.10.5.80:5432;

[0118] External connections use VIP:10.10.5.90,

[0119] The connection information for cloning the underlying database 3 is:

[0120] 10.10.5.100:5432;

[0121] 10.10.5.110:5432;

[0122] 10.10.5.120:5432;

[0123] External connections use VIP:10.10.5.130.

[0124] Access the actual server corresponding to VIP 10.10.5.90 and execute the custom metadata repair script on the server. Run the command: sh citus_fix.sh 5432postgres business_db 192.168.18.505432 10.10.5.50 5432 192.168.18.130 5432 10.10.5.130 5432. The custom script content is as follows:

[0125] Implementation 8

[0126] 1. Create a consistent recovery point for a distributed cluster

[0127] S1. Call the citus_create_restore_point interface provided by Citus. This interface allows us to create a consistent recovery point in the underlying distributed cluster. By using this interface, we can ensure that all underlying database instances in the distributed cluster are in a consistent state.

[0128] S2. Seize an exclusive lock on the entire distributed cluster at the underlying layer using the citus_create_restore_point API. This step temporarily disables write operations to the cluster to prevent data inconsistencies during the restore point creation process.

[0129] S3. After successfully preempting the lock, a restore point is automatically created on all underlying database instances. This restore point ensures that the underlying database is consistent when cloning, and thus the cloned database instances are also consistent.

[0130] 2. Snapshot cloning of underlying database instances

[0131] S4. Based on the cloud computing infrastructure layer, all cloud disks containing the underlying database instances are treated as a whole and a snapshot consistency group is created. This step ensures that the timing of data writes is consistent during data write operations, thus ensuring data consistency in the cloned database instances.

[0132] S5. Clone the volume based on the snapshot consistency group and create a new set of cloud servers to mount the cloned volume. This step quickly replicates the underlying database instance data and creates a new set of cloud servers to mount the cloned volume.

[0133] 3. Restore the underlying database through the recovery point

[0134] S6. Start all underlying PostgreSQL instances on the new cloud server. This step will restore the underlying database to normal operation.

[0135] After the S7.PostgreSQL instance is started, it automatically uses the write-ahead log files for crash recovery. This step ensures the data consistency and integrity of the underlying database.

[0136] 4. Perform metadata recovery

[0137] S8. Execute the custom metadata repair script on the server corresponding to the underlying PostgreSQL instance cloned from the original Citus CN node. This step allows you to scan and repair the metadata information in the cluster by executing the custom metadata repair script.

[0138] During the metadata repair script execution process, we can use the script's execution logic to scan the metadata information in the cluster. The specific execution logic can be defined according to actual needs. For example, we can repair metadata by scanning all tables in the cluster or by scanning all indexes in the cluster.

[0139] 5. Update metadata information and restore distributed cluster functionality

[0140] S9. Call the Citus_update_node interface provided by Citus based on the IP-port mapping before and after cloning to update the metadata information and restore the distributed cluster's functionality. This step can be done after the metadata repair is completed, based on the IP-port mapping before and after cloning, to update the metadata information and restore the distributed cluster's functionality.

[0141] To update metadata, we can use the Citus_update_node interface provided by Citus. This interface allows us to update metadata in a distributed cluster, thereby restoring the functionality of the distributed cluster.

[0142] 6. The input parameters of the script are the connection information of each underlying database of the original cluster and the connection information of each underlying database of the new cluster

[0143] When executing the metadata repair script, you need to pass in the connection information for each underlying database in the original cluster and the connection information for each underlying database in the new cluster as input parameters. These input parameters help us connect to each database instance in the original and new clusters to repair and update metadata information.

[0144] In summary, after Citus expands multiple underlying databases into a distributed database, the difficulty in cloning an entire cluster lies in how to handle metadata due to the read-only and self-maintaining nature of its metadata. One possible solution is to physically clone the underlying database. While this approach can generate a new cluster, the IP addresses of the servers hosting the new cluster often change, while the metadata of the new cluster is identical to the original cluster. This effectively results in the new cluster's CN nodes being reorganized with the old cluster's DN nodes, defeating the intended purpose of the cloning operation. Another possible solution is to call Citus interfaces to regenerate metadata in the empty new cluster before importing business data. This approach can achieve the intended purpose of the cloning operation, but is time-consuming and can hinder service recovery time in disaster recovery scenarios. Research into the principles of the Citus plugin reveals that operations on Citus sharded tables are ultimately forwarded to one or more DN nodes. This forwarding process directly reads metadata to obtain specific connection information. In cloning scenarios, since the architecture and actual data are identical to the original cluster, simply modifying the DN node-related connection information in the metadata is sufficient to restore the correct forwarding mechanism in the new cluster. This patented method analyzes the principles of the Citus plug-in and combines the general capabilities of the cloud computing infrastructure layer, the Citus open interface, and the PostgreSQL self-recovery mechanism. On the basis of physically cloning the underlying database, it realizes cloning operations in the cluster dimension by correcting metadata. In disaster recovery scenarios, it can significantly reduce the recovery time of disaster recovery services and has the characteristics of simplicity, strong operability, and good results.

[0145] Based on the same inventive concept, another embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus.

[0146] Memory for storing computer programs;

[0147] The processor is used to implement the fast cloning method based on the distributed PostgreSQL database of the present application when executing the program stored in the memory.

[0148] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above terminal and other devices. The memory can include a random access memory (RAM) or a non-volatile memory, such as at least one disk storage. Optionally, the memory can also be at least one storage system located away from the aforementioned processor.

[0149] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0150] In addition, to achieve the above-mentioned purpose, an embodiment of the present application further proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the fast cloning method based on a distributed PostgreSQL database of an embodiment of the present application.

[0151] It will be understood by those skilled in the art that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of the present application may take the form of a computer program product implemented on one or more computer-usable devices (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The present application embodiment is described with reference to the flow chart and / or block diagram according to the present application embodiment method, terminal equipment (system) and computer program product.It should be understood that each flow process and / or box in the flow chart and / or block diagram and the flow chart and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of general-purpose computer, special-purpose computer, embedded processing machine or other programmable data processing terminal equipment to produce a machine, so that the instruction executed by the processor of computer or other programmable data processing terminal equipment produces the system for realizing the function specified in flow chart one flow chart or multiple flow charts and / or block diagram one block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction system that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0155] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. "And / or" means that either one of the two can be selected, or both can be selected. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or terminal device comprising the elements.

[0156] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of this application, and are not intended to limit them. Although this application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of this application. Any changes or replacements that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered by the scope of protection of this application.

Claims

1. A method for quickly cloning a distributed PostgreSQL database, characterized in that: The steps include: S1. Create a distributed cluster consistent recovery point; S2. Snapshot clones the underlying database instance; S3. Restore the underlying database through the recovery point; S4. Perform metadata recovery.

2. A method for rapid cloning of a distributed PostgreSQL database according to claim 1, characterized in that: The steps of creating a distributed cluster consistent recovery point specifically include: S11. Call the interface citus_create_restore_point provided by Citus; S12. Seize the exclusive lock of the entire distributed cluster at the bottom layer through the citus_create_restore_point interface, and prohibit writing operations to the cluster within the preset time; S13. After the lock is successfully seized, a restore point is automatically created on all underlying database instances.

3. The fast cloning method of a distributed PostgreSQL database according to claim 2, characterized in that: In S13, the restore point has cluster-wide consistency.

4. The fast cloning method of a distributed PostgreSQL database according to claim 1, characterized in that: The steps of cloning the underlying database instance by snapshot specifically include: S21. Based on the cloud computing infrastructure layer, the cloud disks where all underlying database instances are located are treated as a whole, and a snapshot consistency group is created to ensure the time consistency of data written to the cloud disks; S22. Perform volume cloning based on the snapshot consistency group and create a new set of cloud servers to mount the cloned volumes.

5. The fast cloning method of a distributed PostgreSQL database according to claim 1, characterized in that: The steps of restoring the underlying database through the recovery point specifically include: S31. Start all underlying PostgreSQL instances on the new cloud server respectively; S32.After the PostgreSQL instance is started, it will automatically use the pre-written log file to perform the crash recovery process.

6. The method for rapid cloning of a distributed PostgreSQL database according to claim 1, characterized in that: The steps of recovering metadata specifically include: S41. Execute the custom metadata repair script on the server corresponding to the underlying PostgreSQL instance cloned from the original Citus CN node.

7. The method for rapid cloning of a distributed PostgreSQL database according to claim 6, characterized in that: The execution logic of the script is to scan the metadata information in the cluster.

8. The method for rapid cloning of a distributed PostgreSQL database according to claim 6, characterized in that: The step of executing the custom metadata repair script on the server corresponding to the underlying PostgreSQL instance cloned from the original Citus CN node includes: S42. Call the interface citus_update_node provided by Citus according to the corresponding relationship between the IP ports before and after cloning, update the metadata information, and restore the function of the distributed cluster.

9. The method for rapid cloning of a distributed PostgreSQL database according to claim 6, characterized in that: In S41, the input parameters of the script are the connection information of each underlying database of the original cluster and the connection information of each underlying database of the new cluster.

10. The method for rapid cloning of a distributed PostgreSQL database according to claim 1, characterized in that: The underlying database has one master architecture and two slave architectures.

Images