OpenGauss database external table space compression backup and recovery method and system

Abstract

The invention relates to the technical field of database storage optimization, and provides an openGauss database external table space compression backup and recovery method and system.The method comprises the steps that an external table space catalog is recognized, and data files meeting compression conditions in the external table space catalog are screened and marked; performing full-amount or incremental compression backup on the data files meeting the compression conditions according to the compression attributes; and when the compressed data of the external table space needs to be recovered, recovering the file by adopting a corresponding sequence according to the category of the compressed backup data. According to the compression backup and recovery method and system for the external table space of the openGauss database, the full-amount and incremental compression backup function of the external table space data of the openGauss database can be achieved, and the compression backup efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of database storage optimization, and in particular to a method and system for compressing, backing up and restoring an external table space of an openGauss database. Background Art

[0002] As an advanced, enterprise-grade, open-source relational database, the openGauss database already includes a built-in data backup and recovery mechanism. To optimize storage space, openGauss's existing backup solution supports compressing data files in the database home directory ($PGDATA). However, in real-world enterprise applications, particularly those dealing with massive amounts of data, users often store large tables or partitions in external tablespaces. These external tablespaces typically point to independent disks or storage systems outside the database instance home directory.

[0003] Compression backup technology is one of the key solutions to the aforementioned storage space problem. Its core goal is to significantly reduce the disk space occupied by backup data while ensuring accurate identification and reliable decompression of compressed data during the recovery process, strictly guaranteeing restored data integrity and database availability. To meet diverse business needs (such as pursuing extreme compression ratios, high backup / restore speeds, or balanced modes), modern compression backup technologies typically offer a variety of compression algorithms (such as zlib, lz4, and zstd) and adjustable compression levels.

[0004] In practice, the openGauss database suffers from the following deficiencies when performing compressed backups: 1. Compressed backups of external tablespaces are unavailable: Currently, the compressed backup feature in openGauss only applies to data files in the $PGDATA directory. Data files stored in external tablespaces are not compressed during backup by the existing backup mechanism. These data files are copied to the backup media at their original size. 2. High storage costs: When business systems store large amounts of data (such as historical archives and large analytical tables) in external tablespaces, the data volume is often very large. Because this data cannot be compressed during backup, the overall disk space occupied by the backup set increases dramatically, significantly increasing storage costs. This is particularly disadvantageous for scenarios that require long-term retention of multiple backups or frequent backups. 3. Lack of targeted optimization: Existing incremental backup mechanisms (such as those using PITR or CBM technology to track modified pages) and backup verification are not adapted for data files in external tablespaces. In the context of compressed backups, this means that there is no native support for incremental identification and efficient compressed backup of external tablespace data, as well as integrity verification of compressed data. 3. The recovery process does not consider external compressed data: The existing recovery process is used to process compressed data files under $PGDATA. For external tablespaces, the recovery mechanism lacks the ability to automatically identify and decompress compressed data files from backup sets stored in the external tablespace path. Additional and complex manual intervention steps are required to complete the recovery of compressed backup sets containing external tablespaces.

[0005] Chinese patent publication number CN105677509A discloses a method and device for recovering data in a database. In actual applications, this solution is only applicable to main directory data and cannot process table files in distributed external storage.

[0006] Therefore, there is an urgent need for a technical solution that can seamlessly support efficient compressed backup and reliable recovery of external tablespace data to solve the backup space optimization problem in large-scale data storage scenarios and ensure the integrity and automation level of the entire backup and recovery process. Summary of the Invention

[0007] In view of this, in order to overcome the deficiencies of the prior art, the present invention aims to provide a method and system for compressing and backing up external tablespaces of an openGauss database.

[0008] According to a first aspect of the present invention, a method for compressing and backing up an external tablespace of an openGauss database is provided, the method comprising: Step S1: Identify the external tablespace directory, filter and mark the data files in the external tablespace directory that meet the compression conditions; Step S2: Perform full or incremental compression backup of data files that meet the compression conditions according to the compression attributes; Step S3: When the compressed data of the external table space needs to be restored, the files are restored in a corresponding order according to the category of the compressed backup data.

[0009] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S1, the external tablespace directory to be compressed and backed up is identified by comparing the database tablespace path with the external directory path input by the user, and the oid of the external tablespace directory is recorded.

[0010] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S1, by connecting to the database, obtaining a tablespace path list, traversing the external directory paths input by the user, and comparing each external directory path with the tablespace path list item by item. When the character string of the external directory path input by the user completely matches the path in the tablespace path list, the external directory path is marked as an external tablespace directory, and the parent-child inclusion relationship between directories is excluded during the comparison.

[0011] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S1, data files that comply with the database naming rules in the external tablespace directory are obtained based on the directory information and file name information of the data files, and the metadata of the identified data files are marked to mark the data files as meeting the compression conditions. The data files that meet the compression conditions in the external tablespace directory are compressed and backed up and saved at the page level using the compression attributes parsed from the user compression command.

[0012] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S2, a full compression backup is performed on the data files that meet the compression conditions in the following manner: Determine whether the data file has not been changed since the last backup. If not, skip the backup and write an invalid value flag. If the data file has changed since the last backup, obtain all page numbers of the data file and pull page data from the database source according to the page number; Check whether the pulled page has the corresponding compression attribute, and compress the pulled page data using the corresponding compression attribute; Save the page header information of the pulled page and the corresponding compressed page data in the corresponding folder.

[0013] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S2, the modified page information of the external tablespace directory file is mounted, the modified page is obtained from the source end and compressed and saved, thereby realizing incremental compression backup of the external tablespace data.

[0014] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S2, based on the OID of the external tablespace directory, the link path of the data file under the external directory relative to the database directory is obtained, the file path tracked by the CBM is matched, and the modification page information of the external tablespace data file is obtained. When it is identified that the data file has mounted modification page information, all modification pages are received in sequence from front to back and stored according to the page header information and page number format until all modification pages are obtained.

[0015] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S3, when it is necessary to restore the full compressed backup data of the external tablespace, the recovery files are taken out from the backup set in sequence and restored in the order of the backup set chain.

[0016] Optionally, in the openGauss database external table space compression backup and recovery method of the present invention, in step S3, when it is necessary to restore the incremental compressed backup data of the external table space, starting from the end of the backup set, the metadata corresponding to the data file is searched, and the page numbers to be restored are taken out in sequence according to the information in the metadata, and it is checked whether the pages have been restored by the previous backup set. If not, the file pages of the current backup set are restored, and the restored page numbers are recorded until all backup sets are traversed.

[0017] According to a second aspect of the present invention, a compression backup and recovery system for an openGauss database external tablespace is provided. The system includes a compression backup and recovery server, and the compression backup and recovery server includes: Identification module, used to identify external tablespace directories, filter and mark data files in the external tablespace directories that meet compression conditions; The compression backup module is used to perform full or incremental compression backup of data files that meet the compression conditions according to the compression attributes; The recovery module is used to restore files in a corresponding order according to the category of the compressed backup data when the compressed data of the external table space needs to be restored.

[0018] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect of the present invention when executing the program.

[0019] The openGauss database external table space compression backup and recovery method and system of the present invention have the following beneficial technical effects: 1. Implement full and incremental compressed backup of external tablespace data in the openGauss database, effectively resolving the limitation of existing technologies that only support compressed backup of the main directory ($PGDATA), and accurately tracking the modification status of external tablespace data pages; 2. Based on LSN and the enhanced CBM mechanism, a precise backup strategy is implemented, which only backs up data pages that have been modified since the last backup. Through innovative tablespace OID path mapping, the external file physical path is converted into a standard logical path recognizable by the database, ensuring accurate identification and matching of incrementally modified pages. Combined with the invalid value identification mechanism, unmodified files can be skipped, significantly improving the efficiency and accuracy of compressed backups. 3. Through a reverse chain recovery process, recovery operations are performed backwards from the latest incremental backup to the full backup. A global bitmap is maintained to record the restored page numbers, effectively avoiding duplicate data flushing to disk. The I / O operations during the recovery process have been reduced by over 60%. Compressed data is stored in a header+page format, supporting page-level atomic recovery. When the recovery process is interrupted, the unrestored page can be accurately located, reducing fault recovery time to seconds. 4. This system implements an end-to-end automated process, from automatic path identification and intelligent compression backup to efficient recovery. While maintaining full compatibility with native openGauss backup tools, it also supports hybrid backup modes for both home directories and external tablespaces. The system utilizes a modular architecture that supports both standalone deployment and seamless integration into cloud environments, meeting deployment requirements of varying scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is an example diagram of the architecture of an openGauss database external table space compression backup and recovery system according to an embodiment of the present invention; Figure 2 This is an example diagram of the architecture of a compression backup and recovery server of an openGauss database external table space compression backup and recovery system according to an embodiment of the present invention; Figure 3A flowchart of a method for compressing, backing up, and restoring an external tablespace of an openGauss database according to an embodiment of the present invention; Figure 4 This is an example diagram of metadata in the method according to an embodiment of the present invention; Figure 5 This is an example flow chart of performing data file compression backup according to the method of an embodiment of the present invention; Figure 6 This is an example diagram of a process for performing incremental page identification and compression backup of external tablespace data according to a method according to an embodiment of the present invention; Figure 7 This is an example diagram of the technical principle of recovering compressed data in an external table space according to a method according to an embodiment of the present invention; Figure 8 This is a schematic structural diagram of the device provided by the present invention. DETAILED DESCRIPTION

[0022] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments may be combined with each other; and, based on the embodiments in this disclosure, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of protection of this disclosure.

[0024] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement an apparatus and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this apparatus and / or practice this method.

[0025] Figure 1 FIG. 1 is an example diagram of an architecture of an openGauss database external table space compression backup and recovery system according to an embodiment of the present invention. Figure 1 As shown, the system may include a compression backup and recovery server 101, a communication network 102 and / or one or more compression backup and recovery clients 103. Figure 1 The example in FIG. 1 is a plurality of compression backup and recovery clients 103 .

[0026] The compression backup and recovery server 101 can be any appropriate server for storing information, data, programs and / or any other suitable type of content. In some embodiments, the compression backup and recovery server 101 can perform appropriate functions. For example, in some embodiments, the compression backup and recovery server 101 can be used for compression backup and recovery of external tablespaces of the openGauss database. As an optional example, in some embodiments, the compression backup and recovery server 101 can be used to: identify external tablespace directories, filter and mark data files that meet the compression conditions under the external tablespace directory; perform full or incremental compression backup of data files that meet the compression conditions according to the compression attributes; when it is necessary to restore the compressed data of the external tablespace, restore the files in the corresponding order according to the category of the compressed backup data.

[0027] Figure 2 FIG. 1 is an example diagram of the architecture of a compression backup and recovery server of an openGauss database external table space compression backup and recovery system according to an embodiment of the present invention. Figure 2 As shown, in this embodiment, the compression backup and recovery server includes: Identification module, used to identify external tablespace directories, filter and mark data files in the external tablespace directories that meet compression conditions; The compression backup module is used to perform full or incremental compression backup of data files that meet the compression conditions according to the compression attributes; The recovery module is used to restore files in a corresponding order according to the category of the compressed backup data when the compressed data of the external table space needs to be restored.

[0028] As another example, in some embodiments, the compression backup and recovery server 101 may send the openGauss database external tablespace compression backup and recovery method to the compression backup and recovery client 103 for user use based on a request from the compression backup and recovery client 103 .

[0029] As an optional example, in some embodiments, the compression backup and recovery client 103 is used to provide a visual compression backup and recovery interface, which is used to receive the user's selection input operation for compression backup and recovery of the external table space of the openGauss database, and, in response to the selection input operation, obtain the compression backup and recovery interface corresponding to the option selected by the selection input operation from the compression backup and recovery server 101 and display the compression backup and recovery interface, wherein the compression backup and recovery interface at least displays information on compression backup and recovery of the external table space of the openGauss database and operation options for compression backup and recovery of the information on the external table space of the openGauss database.

[0030] In some embodiments, the communication network 102 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 102 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The compression backup and restore client 103 can be connected to the communication network 102 via one or more communication links (e.g., communication link 104), and the communication network 102 can be linked to the compression backup and restore server 101 via one or more communication links (e.g., communication link 105). The communication link can be any communication link suitable for transmitting data between the compression backup and restore client 103 and the compression backup and restore server 101, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communication link, or any suitable combination of such links.

[0031] The compression backup and restore client 103 may include any one or more clients that present, in an appropriate form, an interface related to compression backup and restore of an openGauss database external tablespace for user use and operation. In some embodiments, the compression backup and restore client 103 may include any suitable type of device. For example, in some embodiments, the compression backup and restore client 103 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of client device.

[0032] Although the compression backup and restore server 101 is illustrated as a single device, in some embodiments, any suitable number of devices may be used to perform the functions performed by the compression backup and restore server 101. For example, in some embodiments, multiple devices may be used to implement the functions performed by the compression backup and restore server 101. Alternatively, a cloud service may be used to implement the functions of the compression backup and restore server 101.

[0033] Based on the above system, an embodiment of the present invention provides an openGauss database external table space compression backup and recovery method, which is described below through the following embodiments.

[0034] Figure 3 This is a flowchart of a method for compressing, backing up, and restoring an external tablespace of an openGauss database according to an embodiment of the present invention. Figure 3As shown, the openGauss database external table space compression backup and recovery method of this embodiment can be executed on the compression backup and recovery server. Specifically, the openGauss database external table space compression backup and recovery method of this embodiment is implemented according to the following steps: Step S1: Identify the external tablespace directory, and screen and mark the data files in the external tablespace directory that meet the compression conditions.

[0035] In the prior art, when traversing backup files and generating metadata, only files in the database directory are checked for data file identity. In this embodiment, the data file determination scope is expanded to include tablespace directories. First, a determination is made as to which external directories are tablespace directories and which are non-data (ordinary) directories. By comparing the database tablespace path with the external directory path entered by the user, the external tablespace directory to be compressed and backed up is identified.

[0036] As an optional example, this embodiment queries tablespace information through a database connection and compares the obtained tablespace information with the external directory path. This restricts the external directory and the tablespace directory from being in a parent-child relationship. The OID of the external tablespace directory is also recorded. The OID is then used to map the path when modifying the page mount. In the openGauss database and PostgreSQL ecosystem, OIDs (object identifiers) uniquely identify various objects within the database.

[0037] For example, in actual application, the identification of tablespace directory can be performed as follows: Connect to the database and execute SELECT spcpath FROM pg_tablespace to obtain a list of tablespace paths. Iterate through the external directory paths entered by the user and compare each one against the tablespace path list. When the string of the user-entered external directory path exactly matches a path in the tablespace path list, mark the external directory path as an external tablespace directory. Parent-child relationships between directories are excluded during the comparison. In openGauss / PostgreSQL, SELECT spcpath FROM pg_tablespace is a system table query statement used to obtain physical storage path information for all tablespaces in the database.

[0038] After identifying the tablespace directory, the data files under the tablespace directory are identified and marked. As an optional example, this embodiment obtains data files under the external tablespace directory that comply with database naming rules based on the directory information and file name information of the data files, and marks the identified data files as meeting compression conditions by marking metadata of the data files.

[0039] For example, in actual applications, judgment logic is added to the directory scanning function dir_list_file_internal(). If the directory to which the data file belongs is marked as an external tablespace directory and the file name matches the regular rule "base / [0-9]+ / [0-9]+" or "pg_tblspc / [0-9]+ / [0-9]+ / [0-9]+", "is_datafile=true" is stored in the file metadata. Figure 4 This is an example diagram of metadata in the method of the embodiment of the present invention, such as Figure 4 As shown, in this embodiment, the metadata information of the data file also includes the file path, file size, modification time, etc.

[0040] Step S2: Perform full or incremental compression backup on data files that meet the compression conditions according to the compression attributes.

[0041] Next, this embodiment uses the compression attribute parsed from the user compression command to perform compressed page-level backup and save the data files that meet the compression conditions in the external table space directory.

[0042] Figure 5 This is an example flow chart of performing data file compression backup according to an embodiment of the present invention. Figure 5 As shown, as an optional example, in this embodiment, it is first determined whether the data file has not been changed since the last round of backup. If not, the backup is skipped and an invalid value flag is written. If the data file has been changed since the last round of backup, all page numbers of the data file are obtained, and page data is pulled from the database source according to the page number. It is checked whether the pulled page has a corresponding compression attribute, and the pulled page data is compressed using the corresponding compression attribute, and the page header information of the pulled page and the corresponding compressed page data are saved in the corresponding folder.

[0043] It should be noted that, in this embodiment, it is still possible to obtain the modified page information of the external tablespace directory file from the source end, compress and save it, and realize incremental page identification and compressed backup of the external tablespace data.

[0044] In the existing technology, only the changed data pages under the database directory are tracked through CBM (Change Block Tracking) technology, and then before the incremental backup, the modified bitmap information of the modified page is mounted on the corresponding data file through file path matching.

[0045] This embodiment expands the scope of file matching. According to the OID of the external tablespace directory, the link path of the data file in the external directory relative to the database directory is obtained. Based on this, the file path tracked by CBM is matched to obtain the modified page information of the external tablespace data file.

[0046] This embodiment obtains the relative path of the data file in the external directory through the OID of the external tablespace directory, matches the file path tracked by CBM, and obtains the modified page information of the external tablespace data file.

[0047] For example, a query statement retrieves the relative path and modified page number of a file in the source repository relative to the database directory file. Matching is achieved by mapping the OID of the external tablespace directory to the CBM query item. In practice, the relative path obtained by the query consists of pg_tblspc + the OID of the external tablespace directory + the actual file location. When a data file is identified as having mounted modified pages, this embodiment sequentially receives all modified pages from the front to the back and stores them in a header+page format until all modified pages are retrieved.

[0048] Figure 6 FIG. 1 is an example flow chart of performing incremental page recognition and compression backup of external table space data according to an embodiment of the present invention. Figure 6 As shown, the modification bitmap information of all data pages is obtained from the CBM system through the last round of LSN. LSN is the unique serial number of the database transaction log, marking each transaction in the WAL (Write-Ahead Logging). In the backup scenario, the LSN recorded at the end of the previous backup is used as the reference point for incremental backups. It determines whether the pgFile has been read, that is, whether the file has been processed. If so, the process ends; otherwise, the next file is read. It identifies whether the read file is an external file. If not, openGauss uses the default backup logic. If the read file is an external file, it checks whether the current file belongs to an external tablespace directory. If so, it uses the tablespace OID to map the path. For example, based on the OID of the external tablespace directory (such as 12345), it generates its link path relative to the database directory (such as pg_tblspc / 12345 / 67890). It searches for the path in the modified file list and matches the mapped path with the modified page path tracked by CBM. If a match is successful, the file's modified page information (such as page number and compression attributes) is mounted. If a match fails, the file that does not need to be backed up is skipped. Finally, the modified page data is stored in the format of header (page header information) + page (page number) for subsequent compression processing.

[0049] Step S3: When the compressed data of the external table space needs to be restored, the files are restored in a corresponding order according to the category of the compressed backup data.

[0050] When the compressed data of the external table space needs to be restored, this embodiment sequentially retrieves the restoration files from the backup set and performs restoration in the order of the backup set chain.

[0051] It should be noted that in this embodiment, the incremental data recovery flow of a data file is opposite to the backup flow, and the recovery starts from the end of the backup set. As an optional example, in this embodiment, when restoring the incremental data of a data file, the corresponding metadata of the data file is searched from the end of the backup set. The page numbers to be restored are sequentially retrieved based on the information in the metadata. The page is checked to see if it has been restored by the previous backup set. If not, the file page of the current backup set is restored, and the restored page numbers are recorded until all backup sets are traversed.

[0052] For example, the write size of the metadata is used to determine whether the data file has content to be recovered. When the write size write_size in the metadata is greater than 0 and is not an invalid value, it means that there is content to be recovered; then the page numbers to be recovered are taken out in sequence according to the page header information, the modified page list and the corresponding compressed data are read, the compressed data is decompressed and written to the target file according to the page number.

[0053] Figure 7 This is an example diagram of the technical principle of recovering compressed data in an external table space according to an embodiment of the present invention. Figure 7 As shown, there is currently a backup chain of full backup, incremental backup 1, and incremental backup 2 waiting to be restored. From the backup perspective, the time flow is from full backup to incremental backup, and finally to incremental backup 2; during recovery, the data files are restored from incremental 2, incremental 1, and finally to the full backup.

[0054] Each backup saves the data of file file1, and incremental backup 2 saves the modified data in red. During recovery, the red pages are restored first and the page numbers are recorded; incremental backup 1 saves the modified green pages, and two of them have been processed in the recovery of incremental backup 2, so only the remaining modified pages are restored; 6 of the purple pages recorded in the full backup have been recovered and processed previously, so only the remaining pages need to be processed.

[0055] This recovery process only consumes bitmap-level resources. By recording the modified pages that have been recovered, it avoids multiple disk flushes of the same page, effectively reducing the amount of IO data.

[0056] In practical applications, the openGauss database external tablespace compression backup and recovery method and system of this embodiment have the following beneficial technical effects: 1. Implement full and incremental compressed backup of external tablespace data in the openGauss database, effectively resolving the limitation of existing technologies that only support compressed backup of the main directory ($PGDATA), and accurately tracking the modification status of external tablespace data pages; 2. Based on LSN and the enhanced CBM mechanism, a precise backup strategy is implemented, which only backs up data pages that have been modified since the last backup. Through innovative tablespace OID path mapping, the external file physical path is converted into a standard logical path recognizable by the database, ensuring accurate identification and matching of incrementally modified pages. Combined with the invalid value identification mechanism, unmodified files can be skipped, significantly improving the efficiency and accuracy of compressed backups. 3. Through a reverse chain recovery process, recovery operations are performed backwards from the latest incremental backup to the full backup. A global bitmap is maintained to record the restored page numbers, effectively avoiding duplicate data flushing to disk. The I / O operations during the recovery process have been reduced by over 60%. Compressed data is stored in a header+page format, supporting page-level atomic recovery. When the recovery process is interrupted, the unrestored page can be accurately located, reducing fault recovery time to seconds. 4. This system implements an end-to-end automated process, from automatic path identification and intelligent compression backup to efficient recovery. While maintaining full compatibility with native openGauss backup tools, it also supports hybrid backup modes for both home directories and external tablespaces. The system utilizes a modular architecture that supports both standalone deployment and seamless integration into cloud environments, meeting deployment requirements of varying scales.

[0057] like Figure 8 As shown, the present invention further provides a device comprising a processor 210, a communication interface 220, a memory 230 for storing a computer program executable by the processor, and a communication bus 240. The processor 210, communication interface 220, and memory 230 communicate with each other via the communication bus 240. The processor 210 executes the executable computer program to implement the aforementioned openGauss database external tablespace compression backup and recovery method.

[0058] Among them, the computer program in the memory 230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0059] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0060] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods of each embodiment or certain portions of the embodiments.

[0061] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for compressing and restoring an external tablespace of an openGauss database, characterized in that: The method comprises: Step S1: Identify the external tablespace directory, filter and mark the data files in the external tablespace directory that meet the compression conditions; Step S2: Perform full or incremental compression backup of data files that meet the compression conditions according to the compression attributes; Step S3: When the compressed data of the external table space needs to be restored, the files are restored in a corresponding order according to the category of the compressed backup data.

2. The openGauss database external table space compression backup and recovery method according to claim 1, characterized in that: In step S1, the external tablespace directory to be compressed and backed up is identified by comparing the database tablespace path with the external directory path input by the user, and the OID of the external tablespace directory is recorded.

3. The openGauss database external table space compression backup and recovery method according to claim 2, characterized in that: In step S1, by connecting to the database, obtaining the tablespace path list, traversing the external directory path entered by the user, and comparing each external directory path with the tablespace path list item by item, when the character string of the external directory path entered by the user completely matches the path in the tablespace path list, the external directory path is marked as an external tablespace directory, and the parent-child inclusion relationship between directories is excluded during the comparison.

4. The openGauss database external table space compression backup and recovery method according to claim 1, characterized in that: In step S1, based on the directory information and file name information of the data file, the data file that complies with the database naming rules in the external tablespace directory is obtained, and the metadata of the identified data file is marked to mark the data file as meeting the compression conditions. The compression attributes parsed from the user compression command are used to perform compressed page-level backup and preservation of the data file that meets the compression conditions in the external tablespace directory.

5. The openGauss database external table space compression backup and recovery method according to claim 1, characterized in that: In step S2, the data files that meet the compression conditions are fully compressed and backed up in the following manner: Determine whether the data file has not been changed since the last backup. If not, skip the backup and write an invalid value flag. If the data file has changed since the last backup, obtain all page numbers of the data file and pull page data from the database source according to the page number; Check whether the pulled page has the corresponding compression attribute, and compress the pulled page data using the corresponding compression attribute; Save the page header information of the pulled page and the corresponding compressed page data in the corresponding folder.

6. The openGauss database external table space compression backup and recovery method according to claim 1, characterized in that: In step S2, by mounting the modified page information of the external tablespace directory file, the modified page is obtained from the source end and compressed and saved, thereby realizing incremental compression backup of the external tablespace data.

7. The openGauss database external table space compression backup and recovery method according to claim 6, characterized in that: In step S2, based on the OID of the external tablespace directory, the link path of the data file under the external directory relative to the database directory is obtained, the file path tracked by CBM is matched, and the modified page information of the external tablespace data file is obtained. When it is identified that the data file has mounted modified page information, all modified pages are received in sequence from front to back and stored in the page header information and page number format until all modified pages are obtained.

8. The openGauss database external table space compression backup and recovery method according to claim 1, characterized in that: In step S3, when the full compressed backup data of the external tablespace needs to be restored, the restoration files are sequentially taken out from the backup set and restored in the order of the backup set chain.

9. The openGauss database external table space compression backup and recovery method according to claim 1, characterized in that: In step S3, when it is necessary to restore the incremental compressed backup data of the external tablespace, start from the end of the backup set, search for the metadata corresponding to the data file, take out the page numbers to be restored in sequence according to the information in the metadata, check whether the pages have been restored by the previous backup set, if not, restore the file pages of the current backup set, and record the restored page numbers until all backup sets are traversed.

10. An openGauss database external table space compression backup and recovery system, characterized in that: The system includes a compression backup and recovery server, which includes: Identification module, used to identify external tablespace directories, filter and mark data files in the external tablespace directories that meet compression conditions; The compression backup module is used to perform full or incremental compression backup of data files that meet the compression conditions according to the compression attributes; The recovery module is used to restore files in a corresponding order according to the category of the compressed backup data when the compressed data of the external table space needs to be restored.

Images