Postgresql high-availability instance cross-cloud backup method and system based on cloud

Abstract

The invention discloses a cloud-based postgresql high-availability instance cross-cloud backup method and system, and belongs to the technical field of database backup and recovery. A Postgresql high-availability instance is created on a cloud platform, a manual / automatic backup tool pg-tools is installed for the postgresql high-availability instance, and manual / automatic backup of the Postgresql high-availability instance is achieved; the method comprises the following steps: binding an elastic public network IP, configuring a tool backup configuration file, modifying user object storage information, automatically backing up the information, creating a crontab timed task, and after manual / automatic backup of a PostgreSQL high-availability instance is completed, transmitting a backup file to a target object for storage and verifying the integrity of the configuration file. According to the method and the system, efficient backup and recovery of the PostgreSQL database can be realized, and the reliability and the security of the data are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of database backup and recovery, and in particular to a method and system for cross-cloud backup of a cloud-based PostgreSQL high-availability instance. Background Art

[0002] With the rapid development of cloud computing technology, more and more enterprises choose to migrate their business data and applications to cloud platforms for storage and management. As a powerful and widely used relational database management system, PostgreSQL has also been widely used on cloud platforms. However, with the increasing diversity and complexity of cloud computing environments, backing up PostgreSQL databases across cloud environments based on traditional backup systems faces a series of challenges.

[0003] Traditional backup systems are usually designed for specific cloud platforms. Cross-cloud backup requires the deployment of backup proxy nodes and data transmission on different cloud platforms, which makes it difficult to achieve unified management and operation. In addition, there are risks such as data inconsistency and data loss during data transmission, and the security and integrity of data are difficult to guarantee. At the same time, traditional backup systems have low backup efficiency and a single backup method, which cannot meet diverse needs such as disaster recovery and scheduled backup. Summary of the invention

[0004] The technical task of the present invention is to address the above shortcomings and provide a cloud-based postgresql high-availability instance cross-cloud backup method and system, which can realize efficient backup and recovery of PostgreSQL database, improve data reliability and security, and ensure that business operations can be quickly restored in the event of data loss or other system failures; provide powerful data management functions, reduce costs and improve data operation efficiency.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A cloud-based cross-cloud backup method for a PostgreSQL high-availability instance, which creates a PostgreSQL high-availability instance on a cloud platform and installs a manual / automatic backup service pg-tools for the PostgreSQL high-availability instance to implement manual / automatic backup of the PostgreSQL high-availability instance;

[0007] The manual backup process of a PostgreSQL high-availability instance includes:

[0008] Bind an elastic public IP, configure the tools backup configuration file, modify user object storage information and automatic backup information, verify configuration information, and perform manual cross-cloud backup tasks;

[0009] The process of implementing automatic backup of a PostgreSQL high-availability instance includes:

[0010] Bind an elastic public IP, configure the tools backup configuration file, modify the user object storage information and automatic backup information, verify the configuration information, modify the scheduled task configuration, create a crontab scheduled task, and execute the scheduled task to start automatic backup;

[0011] After the manual / automatic backup of the PostgreSQL high-availability instance is completed, transfer the backup file to the target object storage and verify the integrity of the configuration file.

[0012] The user will not be aware of the disaster recovery process, and the PostgreSQL high availability service will not be affected. When performing disaster recovery tasks, PostgreSQL high availability will be backed up across clouds to ensure data security and integrity in abnormal environments.

[0013] Furthermore, the specific implementation of this method includes:

[0014] Create a PostgreSQL high-availability instance on the cloud platform. After the PostgreSQL high-availability instance virtual machine is created, install the backup tool and S3 tool into the virtual machine through orchestration.

[0015] Pull and install the pg_monitor service on the virtual machine (this service is a PostgreSQL instance status monitoring service);

[0016] Through the visual console page, a backup command is issued to the PostgreSQL high-availability instance, and a related backup configuration file is generated on the virtual machine side, including information such as OSS address, OSS bucket name, accessKey, and secretKey. The backup command is issued to the virtual machine, and the backup tool is called to back up the PostgreSQL instance and compress the PostgreSQL service backup data file. After completion, the file is transferred to the target address. The data file can be backed up to other cloud OSS storage or self-built OSS storage service to ensure user data security and prevent data loss caused by unavailability of PostgreSQL high-availability instance service in the region. It also facilitates users to manage and use backup files.

[0017] Or you can set a scheduled backup for the PostgreSQL high-availability instance through the visual console. After the virtual machine receives the user's instruction, add a crontab scheduled task in the virtual machine. The execution time can be set through the visual page; call the backup tool at the specified time to back up the PostgreSQL high-availability instance to the specified cloud. This realizes the requirement of cross-cloud backup of the PostgreSQL high-availability instance on the cloud.

[0018] Furthermore, the disaster recovery target may include cloud OSS storage, or own OSS storage;

[0019] After the backup is completed, the file is transferred to the target address, and the data file is backed up to other cloud OSS storage or self-built OSS storage service.

[0020] Furthermore, the overall steps for manually backing up a PostgreSQL high-availability instance include:

[0021] 1) The public cloud environment executes resource orchestration scripts through openstack to create a PostgreSQL high-availability database instance and ensure that the database runs normally, the communication between nodes is normal, and the data synchronization between nodes is normal;

[0022] 2) In the public cloud environment, use the PostgreSQL high-availability database instance created in step 1), download the pg-tools image from the image repository, and use docker run to start the service; here you need to specify the virtual machine network used by the docker network;

[0023] 3) Use the openstack command to bind the enabled elastic public network IP to the PostgreSQL high-availability virtual machine master node; and add the user object storage IP to the network policy to prevent access being prohibited; in this way, the pg-tools tool can access the public network through the virtual machine network;

[0024] 4) You can use the pg-tools visualization window to configure the user object storage related parameters: object storage address ossAddress, object storage common name bucketName, object storage accessKey, object storage secretKey; or log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration;

[0025] 5) Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the directory specified by the object storage; or log in to the virtual machine and call the pg-tools script check_config.sh script for testing;

[0026] 6) Use the pg-tools visualization window or manually call the backup script to back up the PostgreSQL high-availability instance. The backup file can be compressed in a specified format according to the configuration.

[0027] 7) Use the backup file generated in step 6) to read the user object storage configuration file, and use the pg-tools tool to transfer the file to the specified bucket path of the user object storage to achieve the purpose of off-site disaster recovery.

[0028] 8) After the backup operation is completed, a scheduled task will be created to check whether the disaster recovery operation is completed at a fixed time interval; if it is successfully completed, the check script will be automatically executed to verify the integrity of the backup file in the target storage bucket; and the backup file size will be returned for the user to display.

[0029] Furthermore, the overall steps for automatically backing up a PostgreSQL high-availability instance include:

[0030] 1) The public cloud environment executes the resource orchestration script through open stack to create a PostgreSQL high-availability database instance;

[0031] 2) In the public cloud environment, use the PostgreSQL high-availability database instance created in step 1), download the pg-tools image from the image repository, and use docker run to start the service; here you need to specify the virtual machine network used by the docker network;

[0032] 3) Use the open stack command to bind the enabled elastic public network IP to the RPostgreSQL high-availability virtual machine master node; and add the user object storage IP to the network policy to prevent access being prohibited, so that the pg-tools tool can access the public network through the virtual machine network;

[0033] 4) You can use the pg-tools visualization window to configure user object storage related parameters: object storage address, object storage pass name, object storage accessKey, object storage secretKey; or log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration;

[0034] 5) Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the object storage; or log in to the virtual machine and call the tools script check_config.sh script for testing;

[0035] 6) Use the pg-tools visualization window or manually configure the automatic backup configuration file to modify the automatic backup switch and automatic backup time;

[0036] 7) Create a crontab scheduled task to perform the automatic backup operation at regular intervals. The compression format can be specified according to parameters. After the automatic backup is executed, a specified backup file will be generated.

[0037] 8) Use the backup file generated in step 7) to read the user object storage configuration file and obtain the configuration information, including the oss address, oss bucket name, accessKey, and secretKey. Use the tools command to transfer it to the specified bucket path of the user object storage.

[0038] 9) After the backup transfer is completed, the check script will be automatically executed to verify the integrity of the backup file in the target storage bucket. The size of the backup file will be returned for the user to display.

[0039] Furthermore, if the backup file is too large, it will be uploaded in slices by configuring the slice size and the number of slices to ensure sufficient virtual machine resources.

[0040] Furthermore, the overall steps for deleting the backup file of the PostgreSQL high-availability instance include:

[0041] 1) In the public cloud environment, the PostgreSQL high-availability instance has been created and a successful off-site disaster recovery has been completed.

[0042] 2) Trigger the operation of deleting the disaster recovery file through the pg-tools visualization window or by calling the script delete.sh (this script can be directly called through the server-side code).

[0043] 3) Obtain the disaster recovery configuration information, the target object storage address, the specific storage bucket and path, and the accessKey and secretKey for accessing the user storage by reading the configuration file.

[0044] 4) Call the check_config.sh script to perform a configuration test to check whether the storage address information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the permission to access the target file to be deleted in the object storage.

[0045] 5) Verify whether the target file to be deleted exists. If it does not exist, return directly. If the file exists, call the asynchronous method to delete the file.

[0046] 6) Create a scheduled task through the pg-tools tool. The scheduled task can be used to detect whether the disaster recovery deletion task has ended, whether it has been executed successfully or failed at regular intervals. If the execution fails, record the error log, which can be viewed through the visualization page. Or execute the deletion operation again to delete it.

[0047] The present invention also claims a cloud-based postgresql high-availability instance cross-cloud backup system, including a postgresql high-availability instance manual backup module and a postgresql high-availability instance automatic backup module;

[0048] Create a PostgreSQL high-availability instance on the cloud platform and install the manual / automatic backup service pg-tools for the PostgreSQL high-availability instance to implement manual / automatic backup of the PostgreSQL high-availability instance.

[0049] The system specifically implements cross-cloud backup of postgresql high-availability instances through the above method.

[0050] The present invention also claims a device for implementing cross-cloud backup of a cloud-based PostgreSQL high-availability instance, comprising: at least one memory and at least one processor;

[0051] The at least one memory is used to store a machine-readable program;

[0052] The at least one processor is used to call the machine-readable program to implement the above method.

[0053] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which, when executed by a processor, causes the processor to perform the above method.

[0054] Compared with the prior art, the cloud-based PostgreSQL high-availability instance cross-cloud backup method and system of the present invention have the following beneficial effects:

[0055] When initializing a virtual machine, the present invention can directly run the pg-tools service in all nodes of the cluster, and achieve the purpose of cross-cloud backup of PostgreSQL high-availability instances by configuring user object storage information, thereby ensuring the security and integrity of user data.

[0056] The present invention solves the challenges faced by database backup in the current cloud environment, such as data security, data recovery efficiency, and data management costs. By implementing this technical solution, efficient backup and recovery of the PostgreSQL database can be achieved, data reliability and security can be improved, and business operations can be quickly restored in the event of data loss or other system failures. At the same time, the flexibility of this technical solution can also adapt to the backup needs of different cloud service providers, provide powerful data management functions, reduce costs and improve data operation efficiency. It has the following advantages:

[0057] Efficiency: By combining incremental backup and differential backup, unnecessary data transmission and storage costs are reduced, and backup efficiency is improved. At the same time, data compression technology is used to further reduce the size of backup data and increase data transmission speed.

[0058] Reliability: Distributed storage technology is used to store backup data in cloud storage services of multiple cloud service providers, ensuring data reliability and availability. Even if data loss occurs on a single cloud service provider, data can be restored from other cloud service providers.

[0059] Security: Data encryption technology is used to protect the security of backup data and prevent unauthorized access and data leakage. At the same time, the technical solution also provides access control and permission management functions to ensure that only authorized users can access and manage backup data.

[0060] Flexibility: This technical solution can adapt to different cloud service providers and cloud storage service types, support multiple backup strategies and data recovery methods. At the same time, it can also be integrated with other data management systems and services to provide richer data management functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flowchart of a manual backup process of a cloud-based PostgreSQL high-availability instance provided by an embodiment of the present invention;

[0062] Figure 2 It is a diagram of a cloud-based automatic backup process of a PostgreSQL high-availability instance provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0064] The embodiment of the present invention provides a method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance. A PostgreSQL high-availability instance is created on a cloud platform, and a manual / automatic backup service pg-tools is installed on the PostgreSQL high-availability instance to implement manual / automatic backup of the PostgreSQL high-availability instance.

[0065] The manual backup process of a PostgreSQL high-availability instance includes:

[0066] Bind an elastic public IP, configure the tools backup configuration file, modify user object storage information and automatic backup information, verify configuration information, and perform manual cross-cloud backup tasks;

[0067] The process of implementing automatic backup of a PostgreSQL high-availability instance includes:

[0068] Bind an elastic public IP, configure the tools backup configuration file, modify the user object storage information and automatic backup information, verify the configuration information, modify the scheduled task configuration, create a crontab scheduled task, and execute the scheduled task to start automatic backup;

[0069] After the manual / automatic backup of the PostgreSQL high-availability instance is completed, transfer the backup file to the target object storage and verify the integrity of the configuration file.

[0070] The user will not be aware of the disaster recovery process, and the PostgreSQL high availability service will not be affected. When performing disaster recovery tasks, PostgreSQL high availability will be backed up across clouds to ensure data security and integrity in abnormal environments.

[0071] Backup and recovery are very important tasks in database management, involving operations such as regular and manual backup of data, as well as disaster recovery and data restoration. This method uses the pg-tools tool on the cloud platform to implement the backup and recovery of the PostgreSQL database, and uses pg_monitor to monitor the status of database backup and recovery, providing an efficient and reliable data protection solution. This method uses cloud computing technology, combined with the resource management, virtualization, elastic scaling and other functions of the cloud platform, to create a high-availability instance of PostgreSQL in the cloud database. Use elastic public networks to ensure that the PostgreSQL high-availability instance is connected to the cloud storage network, and add inbound and outbound policies to ensure network security.

[0072] The specific implementation of this method includes:

[0073] Create a PostgreSQL high-availability instance on the cloud platform. After the PostgreSQL high-availability instance virtual machine is created, install the backup tool and S3 tool into the virtual machine through orchestration.

[0074] Pull and install the pg_monitor service on the virtual machine (this service is the PostgreSQL instance status monitoring service).

[0075] The disaster recovery target can be cloud OSS storage or self-owned OSS storage.

[0076] Sending a backup command to the PostgreSQL high-availability instance through the visual console page will generate a related backup configuration file on the virtual machine side, including information about the OSS address, OSS bucket name, accessKey, and secretKey; the backup command is sent to the virtual machine, calling the backup tool to back up the PostgreSQL instance, and compressing the PostgreSQL service backup data file. After completion, the file is transferred to the target address, and the data file is backed up to other cloud OSS storage or self-built OSS storage services to ensure user data security and prevent data loss caused by the unavailability of the PostgreSQL high-availability instance service in the region. It also facilitates users to manage and use backup files.

[0077] We also set up scheduled backups for PostgreSQL high-availability instances through the visual console. After the virtual machine receives the user's instructions, we add a crontab scheduled task to the virtual machine. The execution time can be set through the visual page. The backup tool is called at the specified time to back up the PostgreSQL high-availability instance to the specified cloud. This meets the requirements for cross-cloud backup of PostgreSQL high-availability instances on the cloud.

[0078] Experiments have shown that this cross-cloud backup method has the advantages of high efficiency, reliability, and elastic scalability, and can effectively guarantee the backup and recovery capabilities of the database in a multi-cloud environment. This method has broad application prospects and commercial value in the field of cloud computing.

[0079] The implementation of this method is further described in detail below with reference to the accompanying drawings.

[0080] The implementation of this method requires the following prerequisites:

[0081] 1. There are sufficient resources in the public cloud environment to create a PostgreSQL high-availability instance.

[0082] 2. The environment where the PostgreSQL high-availability instance is located is interconnected with the object server network.

[0083] 3. Prepare the images to be used in the mirror repository (PostgreSQL high-availability service image, pg-tools tool image, PostgreSQL monitoring image).

[0084] 4. Make sure the virtual machine image is uploaded to openstack.

[0085] 5. The VPC where the created PostgreSQL high-availability instance is located maintains normal network.

[0086] 6. There is an available elastic public IP in the public cloud environment, and it is necessary to ensure that the elastic public IP is accessible through the external network; or an accessible elastic public IP can be created.

[0087] 7. Ensure that users have built their own object storage service and can access it through the public network; or have opened object storage service on the cloud platform and opened public network access rights.

[0088] 1. The specific process of manually backing up a PostgreSQL high-availability instance is as follows Figure 1 As shown, the overall steps include:

[0089] 1. The public cloud environment executes resource orchestration scripts through openstack to create a PostgreSQL high-availability database instance and ensure that the database runs normally, the communication between nodes is normal, and the data synchronization between nodes is normal.

[0090] 2. In the public cloud environment, use the PostgreSQL high-availability database instance created in step 1, download the pg-tools image from the image repository, and use docker run to start the service; note that the docker network must be specified to use the virtual machine network, otherwise other nodes and the public network will not be accessible.

[0091] 3. Use the openstack command to bind the enabled elastic public network IP to the master node of the PostgreSQL high-availability virtual machine; and add the user object storage IP to the network policy to prevent access being prohibited. In this way, the pg-tools tool can access the public network through the virtual machine network.

[0092] 4. You can use the pg-tools visualization window to configure user object storage related parameters: object storage address ossAddress, object storage common name bucketName, object storage accessKey, object storage secretKey; you can also log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration.

[0093] 5. Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the directory specified by the object storage. You can also log in to the virtual machine and call the pg-tools script check_config.sh script for testing.

[0094] 6. Use the pg-tools visualization window or manually call the backup script to back up the PostgreSQL high-availability instance. You can compress the backup file in a specified format according to the configuration. The compression process will take some time.

[0095] 7. Use the backup file generated in step 6 to read the user object storage configuration file, and use the pg-tools tool to transfer the file to the specified bucket path of the user object storage to achieve the purpose of off-site disaster recovery. If the backup file is too large, you can upload it in pieces by configuring the size and number of pieces to ensure sufficient virtual machine resources.

[0096] 8. After the backup operation is completed, a scheduled task will be created to check whether the disaster recovery operation is completed every 30 seconds; if it is successfully completed, the check script will be automatically executed to verify the integrity of the backup file in the target storage bucket; and the backup file size will be returned for users to display.

[0097] 2. Specific process of automatic backup of PostgreSQL high-availability instance Figure 2 As shown, the overall steps include:

[0098] 1. The public cloud environment executes the resource orchestration script through open stack to create a PostgreSQL high-availability database instance.

[0099] 2. In the public cloud environment, use the PostgreSQL high-availability database instance created in step 1, download the pg-tools image from the image repository, and use docker run to start the service; note that the docker network must be specified to use the virtual machine network, otherwise other nodes and the public network will not be accessible.

[0100] 3. Use the open stack command to bind the enabled elastic public network IP to the RPostgreSQL high-availability virtual machine master node; and add the user object storage IP to the network policy to prevent access being prohibited. In this way, the pg-tools tool can access the public network through the virtual machine network.

[0101] 4. You can use the pg-tools visualization window to configure user object storage related parameters: object storage address, object storage pass name, object storage accessKey, object storage secretKey; you can also log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration.

[0102] 5. Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have permission to access the object storage. You can also log in to the virtual machine and call the tools script check_config.sh script for testing.

[0103] 6. Use the pg-tools visualization window or manually configure the automatic backup configuration file to modify the automatic backup switch and automatic backup time.

[0104] 7. Create a crontab scheduled task to perform automatic backup operations at a fixed time. The compression format can be specified according to the parameters; after executing the automatic backup, the specified backup file will be generated.

[0105] 8. Use the backup file generated in step 7 to read the user object storage configuration file to obtain the configuration information OSS address, OSS bucket name, accessKey, and secretKey; use the tools command to transfer it to the specified bucket path of the user object storage.

[0106] 9. After the backup transfer is completed, the check script will be automatically executed to verify the integrity of the backup file in the target bucket. The backup file size will be returned for users to display.

[0107] 3. The overall steps for deleting backup files from a PostgreSQL high-availability instance include:

[0108] 1. The public cloud environment has completed the creation of a PostgreSQL high-availability instance, and has successfully completed an off-site disaster recovery.

[0109] 2. Trigger the operation of deleting the disaster recovery file through the pg-tools visualization window or by calling the script delete.sh (the script can be directly called through the server code).

[0110] 3. Obtain the disaster recovery configuration information, target object storage address, specific storage bucket and path, and access the accessKey and secretKey stored by the user by reading the configuration file.

[0111] 4. Perform a configuration test by calling the check_config.sh script to test whether the storage address information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have permission to access the target file to be deleted by the object storage.

[0112] 5. Check whether the target file to be deleted exists. If it does not exist, return directly. If the file exists, call the asynchronous method to delete the file.

[0113] 6. Use the pg-tools tool to create a scheduled task. The scheduled task can be used to regularly detect whether the disaster recovery deletion task has been completed, whether it has been executed successfully or failed. If the execution fails, an error log is recorded, which can be viewed through the visualization page. Or the deletion operation can be performed again to delete it.

[0114] Traditional database backup methods have problems with data redundancy and low backup efficiency, especially in a multi-cloud environment, where the synchronization and consistency of backup data are more difficult to guarantee. The present invention deploys backup proxy nodes on a cloud platform, establishes a secure communication channel, implements online backup and recovery of the PostgreSQL database, and provides flexible backup policy settings and a user-friendly management interface, thereby solving the problems faced by existing backup systems when backing up PostgreSQL databases in a cross-cloud environment, and improving backup efficiency and data protection reliability. This technology has good application prospects and market potential, and is expected to be widely used in the field of cloud computing.

[0115] An embodiment of the present invention also provides a cloud-based PostgreSQL high-availability instance cross-cloud backup system, including a PostgreSQL high-availability instance manual backup module and a PostgreSQL high-availability instance automatic backup module.

[0116] Create a PostgreSQL high-availability instance on the cloud platform and install the manual / automatic backup service pg-tools for the PostgreSQL high-availability instance to implement manual / automatic backup of the PostgreSQL high-availability instance.

[0117] The postgresql high-availability instance manual backup module includes:

[0118] Bind an elastic public IP, configure the tools backup configuration file, modify user object storage information and automatic backup information, verify configuration information, and perform manual cross-cloud backup tasks.

[0119] The postgresql high-availability instance automatic backup module includes:

[0120] Bind an elastic public IP, configure the tools backup configuration file, modify the user object storage information and automatic backup information, verify the configuration information, modify the scheduled task configuration, create a crontab scheduled task, and execute the scheduled task to start automatic backup.

[0121] After the manual / automatic backup of the PostgreSQL high-availability instance is completed, transfer the backup file to the target object storage and verify the integrity of the configuration file.

[0122] The system specifically implements cross-cloud backup of a high-availability postgresql instance through the cloud-based cross-cloud backup method of a high-availability postgresql instance described in the embodiment:

[0123] The overall steps for manually backing up a PostgreSQL high-availability instance include:

[0124] (1) The public cloud environment executes resource orchestration scripts through OpenStack to create a PostgreSQL high-availability database instance and ensure that the database runs normally, the communication between nodes is normal, and the data synchronization between nodes is normal.

[0125] (2) Use the PostgreSQL high-availability database instance created in step (1) in the public cloud environment, download the pg-tools image from the image repository, and use docker run to start the service. Note that you need to specify the virtual machine network of the docker network, otherwise you will not be able to access other nodes and the public network.

[0126] (3) Use the openstack command to bind the enabled elastic public network IP to the PostgreSQL high-availability virtual machine master node; and add the user object storage IP to the network policy to prevent access being prohibited. In this way, the pg-tools tool can access the public network through the virtual machine network.

[0127] (4) You can use the pg-tools visualization window to configure user object storage related parameters: object storage address ossAddress, object storage common name bucketName, object storage accessKey, object storage secretKey; you can also log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration.

[0128] (5) Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the directory specified by the object storage. You can also log in to the virtual machine and call the pg-tools script check_config.sh script for testing.

[0129] (6) Use the pg-tools visualization window or manually call the backup script to back up the PostgreSQL high-availability instance. You can compress the backup file in a specified format according to the configuration. The compression process will take some time.

[0130] (7) Use the backup file generated in step (6) to read the user object storage configuration file and use the pg-tools tool to transfer the file to the specified bucket path of the user object storage to achieve the purpose of off-site disaster recovery. If the backup file is too large, you can upload it in pieces by configuring the size and number of pieces to ensure sufficient virtual machine resources.

[0131] (8) After the backup operation is completed, a scheduled task will be created to check whether the disaster recovery operation is completed every 30 seconds. If it is successfully completed, the check script will be automatically executed to verify the integrity of the backup file in the target storage bucket. The backup file size will also be returned for the user to display.

[0132] The overall steps for automatically backing up a PostgreSQL high-availability instance include:

[0133] (1) The public cloud environment executes the resource orchestration script through Open Stack to create a PostgreSQL high-availability database instance.

[0134] (2) Use the PostgreSQL high-availability database instance created in step (1) in the public cloud environment, download the pg-tools image from the image repository, and use docker run to start the service. Note that you need to specify the virtual machine network of the docker network, otherwise you will not be able to access other nodes and the public network.

[0135] (3) Use the open stack command to bind the enabled elastic public network IP to the RPostgreSQL high-availability virtual machine master node; and add the user object storage IP to the network policy to prevent access being prohibited. In this way, the pg-tools tool can access the public network through the virtual machine network.

[0136] (4) You can use the pg-tools visualization window to configure user object storage related parameters: object storage address, object storage pass name, object storage accessKey, object storage secretKey; you can also log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration.

[0137] (5) Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have permission to access the object storage. You can also log in to the virtual machine and call the tools script check_config.sh script for testing.

[0138] (6) Use the pg-tools visualization window or manually configure the automatic backup configuration file to modify the automatic backup switch and automatic backup time.

[0139] (7) Create a crontab scheduled task to perform automatic backup operations at a fixed time. The compression format can be specified according to the parameters. After the automatic backup is executed, the specified backup file will be generated.

[0140] (8) Use the backup file generated in step (7) to read the user object storage configuration file and obtain the configuration information OSS address, OSS bucket name, accessKey, and secretKey; use the tools command to transfer it to the specified bucket path of the user object storage.

[0141] (9) After the backup is transferred, the check script will be automatically executed to verify the integrity of the backup file in the target bucket. The backup file size will be returned for the user to display.

[0142] The overall steps for deleting backup files in a PostgreSQL high-availability instance include:

[0143] (1) The public cloud environment has created a PostgreSQL high-availability instance and has successfully completed an off-site disaster recovery.

[0144] (2) Trigger the operation of deleting the disaster recovery file through the pg-tools visualization window or by calling the script delete.sh (the script can be directly called through the server code).

[0145] (3) Obtain the disaster recovery configuration information, target object storage address, specific storage bucket and path, and access the accessKey and secretKey stored by the user by reading the configuration file.

[0146] (4) Perform a configuration test by calling the check_config.sh script to test whether the storage address information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the target file to be deleted by the object storage.

[0147] (5) Check whether the target file to be deleted exists. If it does not exist, return directly. If the file exists, call the asynchronous method to delete the file.

[0148] (6) Use the pg-tools tool to create a scheduled task. The scheduled task can be used to regularly check whether the disaster recovery deletion task has been completed, whether it has been executed successfully or failed. If the execution fails, an error log is recorded, which can be viewed on the visualization page. Alternatively, the deletion operation can be executed again to delete it.

[0149] The embodiment of the present invention also provides a device for implementing cross-cloud backup of a cloud-based PostgreSQL high-availability instance, comprising: at least one memory and at least one processor;

[0150] The at least one memory is used to store a machine-readable program;

[0151] The at least one processor is used to call the machine-readable program to implement the cloud-based postgresql high-availability instance cross-cloud backup method described in the above embodiment.

[0152] The embodiment of the present invention also provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the processor executes the method for cross-cloud backup of a cloud-based postgresql high-availability instance described in the above embodiment. Specifically, a system or device equipped with a storage medium can be provided, on which software program codes implementing the functions of any of the above embodiments are stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0153] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0154] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0155] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0156] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0157] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance, characterized in that: Create a PostgreSQL high-availability instance on the cloud platform and install the manual / automatic backup tool pg-tools on the PostgreSQL high-availability instance to implement manual / automatic backup of the PostgreSQL high-availability instance; The manual backup process of a PostgreSQL high-availability instance includes: Bind an elastic public IP, configure the tools backup configuration file, modify user object storage information and automatic backup information, verify configuration information, and perform manual cross-cloud backup tasks; The process of implementing automatic backup of a PostgreSQL high-availability instance includes: Bind an elastic public IP, configure the tools backup configuration file, modify the user object storage information and automatic backup information, verify the configuration information, modify the scheduled task configuration, create a crontab scheduled task, and execute the scheduled task to start automatic backup; After the manual / automatic backup of the PostgreSQL high-availability instance is completed, transfer the backup file to the target object storage and verify the integrity of the configuration file.

2. A method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance according to claim 1, characterized in that: The specific implementation of this method includes: Create a PostgreSQL high-availability instance on the cloud platform. After the PostgreSQL high-availability instance virtual machine is created, install the backup tool and S3 tool into the virtual machine through orchestration. Pull and install the pg_monitor service on the virtual machine; Through the visual console page, send a backup command to the PostgreSQL high-availability instance, generate a related backup configuration file on the virtual machine side, including information OSS address, OSS bucket name, accessKey, secretKey; the backup command is sent to the virtual machine, call the backup tool to back up the PostgreSQL instance, and compress the PostgreSQL service backup data file. After completion, transfer the file to the target address. Or you can set up scheduled backup for the PostgreSQL high-availability instance through the visual console. After the virtual machine receives the user's instructions, add a crontab scheduled task in the virtual machine. The execution time can be set through the visual page; call the backup tool at the specified time to back up the PostgreSQL high-availability instance to the specified cloud.

3. A method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance according to claim 1 or 2, characterized in that: Disaster recovery targets can include cloud OSS storage or own OSS storage; After the backup is completed, the file is transferred to the target address, and the data file is backed up to other cloud OSS storage or self-built OSS storage service.

4. The method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance according to claim 1, characterized in that: The overall steps for manually backing up a PostgreSQL high-availability instance include: 1) The public cloud environment executes resource orchestration scripts through openstack to create a PostgreSQL high-availability database instance and ensure that the database runs normally, the communication between nodes is normal, and the data synchronization between nodes is normal; 2) In the public cloud environment, use the PostgreSQL high-availability database instance created in step 1), download the pg-tools image from the image repository, and use docker run to start the service; 3) Use the openstack command to bind the enabled elastic public network IP to the master node of the PostgreSQL high-availability virtual machine; and add the user object storage IP to the network policy, so that the pg-tools tool can access the public network through the virtual machine network; 4) You can use the pg-tools visualization window to configure the user object storage related parameters: object storage address ossAddress, object storage common name bucketName, object storage accessKey, object storage secretKey; or log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration; 5) Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the directory specified by the object storage; or log in to the virtual machine and call the pg-tools script check_config.sh script for testing; 6) Use the pg-tools visualization window or manually call the backup script to back up the PostgreSQL high-availability instance. The backup file can be compressed in a specified format according to the configuration. 7) Use the backup file generated in step 6) to read the user object storage configuration file, and use the pg-tools tool to transfer the file to the specified bucket path of the user object storage to achieve the purpose of off-site disaster recovery. 8) After the backup operation is completed, a scheduled task will be created to check whether the disaster recovery operation is completed at a fixed time interval; if it is successfully completed, the check script will be automatically executed to verify the integrity of the backup file in the target storage bucket; and the backup file size will be returned for the user to display.

5. The method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance according to claim 1, characterized in that: The overall steps for automatically backing up a PostgreSQL high-availability instance include: 1) The public cloud environment executes the resource orchestration script through open stack to create a PostgreSQL high-availability database instance; 2) In the public cloud environment, use the PostgreSQL high-availability database instance created in step 1), download the pg-tools image from the image repository, and use docker run to start the service; 3) Use the open stack command to bind the enabled elastic public network IP to the RPostgreSQL high-availability virtual machine master node; and add the user object storage IP to the network policy, so that the pg-tools tool can access the public network through the virtual machine network; 4) You can use the pg-tools visualization window to configure user object storage related parameters: object storage address, object storage pass name, object storage accessKey, object storage secretKey; or log in to the virtual machine and edit the above parameters in the pg-tools configuration file backup_config to modify the configuration; 5) Use the test function on the pg-tools visualization window to connect to the user object storage, test whether the configuration information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the object storage; or log in to the virtual machine and call the tools script check_config.sh script for testing; 6) Use the pg-tools visualization window or manually configure the automatic backup configuration file to modify the automatic backup switch and automatic backup time; 7) Create a crontab scheduled task to perform automatic backup operations at a fixed time. The compression format can be specified according to the parameters; after the automatic backup is executed, the specified backup file will be generated; 8) Use the backup file generated in step 7) to read the user object storage configuration file, obtain the configuration information OSS address, OSS bucket name, accessKey, secretKey; use the tools command to transfer it to the specified bucket path of the user object storage; 9) After the backup transfer is completed, the check script will be automatically executed to verify the integrity of the backup file in the target bucket; and the backup file size will be returned for the user to display.

6. A method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance according to claim 4 or 5, characterized in that: If the backup file is too large, upload it in pieces by configuring the piece size and number of pieces to ensure sufficient virtual machine resources.

7. The method for cross-cloud backup of a cloud-based PostgreSQL high-availability instance according to claim 1, characterized in that: The overall steps for deleting backup files in a PostgreSQL high-availability instance include: 1) The public cloud environment has completed the creation of a PostgreSQL high-availability instance and has successfully completed an off-site disaster recovery; 2) Use the pg-tools visualization window or call the script delete.sh to trigger the operation of deleting the disaster recovery file; 3) Obtain the disaster recovery configuration information, target object storage address, specific storage bucket and path, and access the user's stored accessKey and secretKey by reading the configuration file; 4) Perform configuration test by calling the check_config.sh script to test whether the storage address information is accurate, whether the network communication is normal, and whether the accessKey and secretKey have the authority to access the target file to be deleted by the object storage; 5) Check whether the target file to be deleted exists. If it does not exist, return directly. If the file exists, call the asynchronous method to delete the file. 6) Use the pg-tools tool to create a scheduled task. The scheduled task can be used to regularly detect whether the disaster recovery deletion task has been completed, whether it has been executed successfully or failed. If the execution fails, an error log is recorded, which can be viewed through the visualization page. Or the deletion operation can be performed again to delete it.

8. A cloud-based postgresql high-availability instance cross-cloud backup system, characterized in that: Including postgresql high availability instance manual backup module and postgresql high availability instance automatic backup module; Create a PostgreSQL high-availability instance on the cloud platform and install the manual / automatic backup service pg-tools for the PostgreSQL high-availability instance to implement manual / automatic backup of the PostgreSQL high-availability instance. The system specifically implements cross-cloud backup of PostgreSQL high-availability instances through any one of the methods described in claims 1 to 7.

9. A cloud-based postgresql high-availability instance cross-cloud backup implementation device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, which, when executed by a processor, cause the processor to execute the method according to any one of claims 1 to 7.

Images