An openGauss database external table space compression backup and recovery method and system

Abstract

The application relates to the technical field of database storage optimization, and provides an openGauss database external table space compression backup and recovery method and system, which comprises the following steps: identifying an external table space directory, screening and marking data files under the external table space directory which meet compression conditions; performing full-amount or incremental compression backup on the data files which meet the compression conditions according to compression attributes; and when it is necessary to recover the compressed data of the external table space, recovering the files in a corresponding order according to the categories of the compressed backup data. According to the openGauss database external table space compression backup and recovery method and system, full-amount and incremental compression backup functions of the external table space data of the openGauss database can be realized, and the efficiency and accuracy of the compression backup are improved.

Description

Technical Field

[0001] This invention relates to the field of database storage optimization technology, and in particular to a method and system for compressing, backing up, and restoring external tablespaces in an OpenGauss database. Background Technology

[0002] As an advanced enterprise-grade open-source relational database, openGauss already includes built-in data backup and recovery mechanisms. To optimize storage space, openGauss's existing backup scheme supports compression of data files in the database's main directory ($PGDATA). However, in real-world enterprise applications, especially with massive amounts of data, users often store large tables or partitions in external tablespaces, which typically point to independent disks or storage systems outside the database instance's main directory.

[0003] Compression backup technology is one of the key means to solve the aforementioned storage space problems. Its core objective is to significantly reduce the disk space occupied by backup data, while ensuring accurate identification and reliable decompression of compressed data during the recovery process, strictly guaranteeing the integrity of the restored data and the availability of the database. To adapt to different business needs (such as pursuing extreme compression ratios, high backup / recovery speeds, or balanced modes), modern compression backup technologies typically offer a variety of compression algorithms (such as zlib, lz4, zstd, etc.) and adjustable compression levels for users to choose from.

[0004] In practical applications, openGauss database has the following shortcomings when performing compressed backups: 1. External tablespace compressed backups are unavailable: Currently, openGauss's compressed backup function only applies to data files in the $PGDATA main directory. For data files stored in external tablespaces, the existing backup mechanism does not perform any compression during backup; these data files in external tablespaces are copied to the backup media at their original size. 2. High storage costs: When business systems deploy large-scale data (such as historical archives, large analytical tables, etc.) in external tablespaces, the data volume is often enormous. Since this data cannot be compressed during backup, the overall disk space occupied by the backup set increases dramatically, significantly increasing storage costs. This is particularly unfavorable for scenarios that require long-term retention of multiple backups or frequent backups. 3. Lack of targeted optimization: Existing incremental backup mechanisms (such as tracking modified pages based on PITR or CBM technology) and backup verification functions are not adapted for data files in external tablespaces. In the context of compressed backups, this means that incremental identification and efficient compressed backup of external tablespace data, as well as integrity verification of compressed data, lack native support. 3. The recovery process does not consider external compressed data: The existing recovery process is used to process compressed data files under $PGDATA. For external tablespaces, the recovery mechanism lacks the ability to automatically identify and decompress compressed data files from the backup set stored in the external tablespace path. Additional and complex manual intervention steps are required to complete the recovery of compressed backup sets that include external tablespaces.

[0005] Chinese patent publication number CN105677509A discloses a method and apparatus for recovering data in a database. In practical applications, this solution is only applicable to main directory data and cannot handle table files in distributed external storage.

[0006] Therefore, there is an urgent need for a technical solution that can seamlessly support efficient compression backup and reliable recovery of external tablespace data to solve the backup space optimization problem in large-scale data storage scenarios and ensure the integrity and automation of the entire backup and recovery process. Summary of the Invention

[0007] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide a method and system for compressed backup and recovery of external tablespaces of openGauss database.

[0008] According to a first aspect of the present invention, a method for compressed backup and recovery of external tablespaces in an openGauss database is provided, the method comprising: Step S1: Identify the external tablespace directory, filter and mark the data files in the external tablespace directory that meet the compression conditions; Step S2: Perform full or incremental compression backup on data files that meet the compression conditions according to the compression attributes; Step S3: When it is necessary to restore compressed data from an external tablespace, restore the files in the corresponding order according to the category of the compressed backup data.

[0009] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S1, the external tablespace directory to be compressed and backed up is identified by comparing the database tablespace path with the external directory path input by the user, and the oid of the external tablespace directory is recorded.

[0010] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S1, the tablespace path list is obtained by connecting to the database, the external directory paths input by the user are traversed, and each external directory path is compared with the tablespace path list item by item. When the string of the external directory path input by the user completely matches the path in the tablespace path list, the external directory path is marked as an external tablespace directory, and the parent-child containment relationship between directories is excluded during the comparison.

[0011] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S1, data files that conform to the database naming rules in the external tablespace directory are obtained according to the directory information and file name information of the data files. The data files are marked as meeting the compression conditions by marking the metadata of the identified data files. The data files that meet the compression conditions in the external tablespace directory are compressed and backed up at the page level using the compression attributes parsed from the user compression command.

[0012] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, step S2 involves performing a full compression backup of data files that meet the compression conditions in the following manner: Determine if the data file has not been changed since the last backup. If not, skip the backup and write an invalid value flag. If the data file has been modified since the last backup, obtain all page numbers of the data file and pull the page data from the database source according to the page number; Check if the fetched page has the corresponding compression attribute, and compress the fetched page data using the corresponding compression attribute; Save the page header information and corresponding compressed page data of the retrieved page in the corresponding folder.

[0013] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S2, the modified pages are obtained from the source by mounting the modified page information of the external tablespace directory file, and then compressed and saved to achieve incremental compression backup of the external tablespace data.

[0014] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S2, the link path of the data file under the external directory relative to the database directory is obtained according to the OID of the external tablespace directory, the file path tracked by CBM is matched, and the modified page information of the external tablespace data file is obtained. When the modified page information of the data file is identified, all modified pages are received from front to back in sequence and stored in the format of page header information and page number until all modified pages are obtained.

[0015] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S3, when it is necessary to restore the full compressed backup data of the external tablespace, the recovery files are retrieved sequentially from the backup set and restored according to the order of the backup set chain.

[0016] Optionally, in the openGauss database external tablespace compression backup and recovery method of the present invention, in step S3, when it is necessary to restore the incremental compressed backup data of the external tablespace, starting from the end of the backup set, the corresponding metadata of the data file is searched, and the page number to be restored is retrieved sequentially according to the information in the metadata. It is checked whether the page has been restored by the previous backup set. If it has not been restored, the file page of the current backup set is restored, and the restored page number is recorded, until all backup sets are traversed.

[0017] According to a second aspect of the present invention, an openGauss database external tablespace compression backup and recovery system is provided. The system includes a compression backup and recovery server, which comprises: The identification module is used to identify external tablespace directories, filter and mark data files in external tablespace directories that meet compression conditions; The compression backup module is used to perform full or incremental compression backups on data files that meet the compression conditions based on compression attributes. The recovery module is used to restore files in the corresponding order according to the category of compressed backup data when it is necessary to restore compressed data from external tablespaces.

[0018] According to a third aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect of the present invention.

[0019] The method and system for compressed backup and recovery of external tablespaces in the OpenGauss database of this invention have the following beneficial technical effects: 1. Implement full and incremental compressed backup functions for external tablespace data in openGauss database, effectively solving the limitation of existing technologies that only support compressed backup of the main directory ($PGDATA), and can accurately track the modification status of external tablespace data pages; 2. Based on LSN and enhanced CBM mechanisms, a precise backup strategy is implemented that only backs up data pages that have changed since the last backup. Through innovative tablespace OID path mapping, the physical path of external files is converted into a standard logical path that the database can recognize, ensuring accurate identification and matching of incrementally modified pages. Combined with an invalid value identification mechanism, unmodified files can be skipped, significantly improving the efficiency and accuracy of compressed backups. 3. Through a reverse chain recovery process, the recovery operation is performed in reverse from the latest incremental backup to the full backup. By maintaining a global bitmap to record the number of recovered pages, duplicate data flushing is effectively avoided. Actual tests show that the amount of IO operations during the recovery process is reduced by more than 60%. Compressed data is stored in header+page format, and page-level atomic recovery is supported. When the recovery process is interrupted, the location of the unrecovered page can be accurately located, and the fault recovery time is controlled within seconds. 4. It achieves an end-to-end automated process from automatic path identification and intelligent compression backup to efficient recovery. While maintaining full compatibility with the openGauss native backup tool, it supports a hybrid backup mode for the main directory and external tablespaces. The system adopts a modular architecture design, supporting both independent deployment and seamless integration into cloud environments to meet deployment needs of different scales. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an example architecture diagram of an openGauss database external tablespace compression backup and recovery system according to an embodiment of the present invention; Figure 2 This is an example diagram of the architecture of a compressed backup and recovery server for an openGauss database external tablespace compressed backup and recovery system according to an embodiment of the present invention. Figure 3This is a flowchart illustrating the steps of an openGauss database external tablespace compression backup and recovery method according to an embodiment of the present invention. Figure 4 This is an example diagram of metadata in the method of an embodiment of the present invention; Figure 5 This is an example flowchart of a method for compressing and backing up data files according to an embodiment of the present invention; Figure 6 This is a flowchart illustrating the incremental page identification and compression backup process for external tablespace data according to an embodiment of the present invention. Figure 7 This is an example diagram illustrating the technical principle of recovering compressed data from an external tablespace according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of the device provided by the present invention. Detailed Implementation

[0022] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0023] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0024] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0025] Figure 1 This is an example architecture diagram of an openGauss database external tablespace compression backup and recovery system according to an embodiment of the present invention, as shown below. Figure 1 As shown, the system may include a compressed backup and recovery server 101, a communication network 102, and / or one or more compressed backup and recovery clients 103. Figure 1 The example in the text is a combination of multiple compressed backup and restore clients 103.

[0026] The compressed backup and recovery server 101 can be any suitable server used to store information, data, programs, and / or any other suitable type of content. In some embodiments, the compressed backup and recovery server 101 can perform appropriate functions. For example, in some embodiments, the compressed backup and recovery server 101 can be used for compressed backup and recovery of external tablespaces in the openGauss database. As an optional example, in some embodiments, the compressed backup and recovery server 101 can be used to: identify the external tablespace directory, filter and mark data files in the external tablespace directory that meet compression conditions; perform full or incremental compressed backups of data files that meet the compression conditions according to compression attributes; and when it is necessary to restore compressed data from the external tablespace, restore files in the corresponding order according to the category of compressed backup data.

[0027] Figure 2 This is an example architecture diagram of a compressed backup and recovery server for an openGauss database external tablespace compressed backup and recovery system according to an embodiment of the present invention. Figure 2 As shown, in this embodiment, the compressed backup and recovery server includes: The identification module is used to identify external tablespace directories, filter and mark data files in external tablespace directories that meet compression conditions; The compression backup module is used to perform full or incremental compression backups on data files that meet the compression conditions based on compression attributes. The recovery module is used to restore files in the corresponding order according to the category of compressed backup data when it is necessary to restore compressed data from external tablespaces.

[0028] As another example, in some embodiments, the compressed backup and recovery server 101 may send the compressed backup and recovery method for the external tablespace of the openGauss database to the compressed backup and recovery client 103 for user use, based on a request from the compressed backup and recovery client 103.

[0029] As an optional example, in some embodiments, the compressed backup and recovery client 103 is used to provide a visual compressed backup and recovery interface. This visual compressed backup and recovery interface is used to receive user selection input operations for compressed backup and recovery of openGauss database external tablespaces, and to obtain and display the compressed backup and recovery interface corresponding to the option selected by the selection input operation from the compressed backup and recovery server 101 in response to the selection input operation. The compressed backup and recovery interface displays at least information on compressed backup and recovery of openGauss database external tablespaces and operation options for compressed backup and recovery of openGauss database external tablespaces.

[0030] In some embodiments, communication network 102 can be any suitable combination of one or more wired and / or wireless networks. For example, communication network 102 can include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. Compressed backup and recovery client 103 can be connected to communication network 102 via one or more communication links (e.g., communication link 104), which can be linked to compressed backup and recovery server 101 via one or more communication links (e.g., communication link 105). Communication links can be any communication link suitable for transmitting data between compressed backup and recovery client 103 and compressed backup and recovery server 101, such as network links, dial-up links, wireless links, hardwired links, any other suitable communication links, or any suitable combination of such links.

[0031] The compressed backup and restore client 103 may include any one or more clients that present an interface related to compressed backup and restore of openGauss database external tablespaces in a suitable form for user use and operation. In some embodiments, the compressed backup and restore client 103 may include any suitable type of device. For example, in some embodiments, the compressed backup and restore client 103 may include a mobile device, tablet computer, laptop computer, desktop computer, and / or any other suitable type of client device.

[0032] Although the compressed backup and recovery server 101 is illustrated as a single device, in some embodiments, any suitable number of devices may be used to perform the functions performed by the compressed backup and recovery server 101. For example, in some embodiments, multiple devices may be used to implement the functions performed by the compressed backup and recovery server 101. Alternatively, cloud services may be used to implement the functions of the compressed backup and recovery server 101.

[0033] Based on the above system, this invention provides a method for compressed backup and recovery of external tablespaces in an openGauss database, which is illustrated in the following embodiments.

[0034] Figure 3 This is a flowchart illustrating the steps of a method for compressing, backing up, and restoring the external tablespace of an openGauss database according to an embodiment of the present invention. Figure 3As shown, the openGauss database external tablespace compression backup and recovery method of this embodiment can be executed on the compression backup and recovery server. Specifically, the openGauss database external tablespace compression backup and recovery method of this embodiment is implemented according to the following steps: Step S1: Identify the external tablespace directory, filter and mark the data files in the external tablespace directory that meet the compression conditions.

[0035] In existing technologies, the process of traversing backup files and generating metadata only checks whether files in the database directory are data files. In this embodiment, the scope of data file determination is extended to the tablespace directory. First, it determines which of the set external directories are tablespace directories and which are non-data (ordinary) directories. By comparing the database tablespace path with the user-input external directory path, the external tablespace directories that need to be compressed and backed up are identified.

[0036] As an optional example, in this embodiment, tablespace information is queried via a database connection. The obtained tablespace information is compared with the external directory path to restrict the external directory and tablespace directory from having a parent-child relationship. Simultaneously, the OID of the external tablespace directory is recorded. Subsequent path matching during page mounting modifications is mapped using the OID. In the openGauss database and PostgreSQL ecosystem, the OID (object identifier) ​​is used to uniquely identify various objects within the database.

[0037] For example, in practical applications, tablespace directory identification can be performed as follows: Connect to the database, execute `SELECT spcpath FROM pg_tablespace` to retrieve a list of tablespace paths, iterate through the user-inputted external directory paths, and compare each external directory path with the list of tablespace paths. When a user-input external directory path string exactly matches a path in the list, that external directory path is marked as an external tablespace directory, excluding parent-child relationships between directories during the comparison. In openGauss / PostgreSQL, `SELECT spcpath FROM pg_tablespace` is a system table query statement used to retrieve the physical storage path information of all tablespaces in the database.

[0038] After identifying the tablespace directory, the data files under the tablespace directory are identified and marked. As an optional example, this embodiment obtains data files that conform to the database naming rules under the external tablespace directory based on the directory information and file name information of the data files, and marks the data files as meeting the compression conditions by marking the metadata of the identified data files.

[0039] For example, in practical applications, a judgment logic can be added to the directory scanning function dir_list_file_internal(). If the directory to which the data file belongs is marked as an external tablespace directory, and the filename matches the regular expression rule "base / [0-9]+ / [0-9]+" or "pg_tblspc / [0-9]+ / [0-9]+ / [0-9]+", then "is_datafile=true" can be stored in the file's metadata. Figure 4 This is an example diagram of metadata in the method of this embodiment of the invention, such as... Figure 4 As shown in this embodiment, the metadata information of the data file also includes the file path, file size, modification time, etc.

[0040] Step S2: Perform full or incremental compression backup on data files that meet the compression conditions according to the compression attributes.

[0041] Next, this embodiment uses the compression attributes parsed from the user compression command to perform page-level backup and save of data files in the external tablespace directory that meet the compression conditions.

[0042] Figure 5 This is an example flowchart of a method for compressing and backing up data files according to an embodiment of the present invention, such as... Figure 5 As shown, as an optional example, in this embodiment, it is first determined whether the data file has not been changed since the last backup. If it has not been changed, the backup is skipped and an invalid value is written. If the data file has been changed since the last backup, all page numbers of the data file are obtained, page data is pulled from the database source according to the page number, it is checked whether the pulled page has a corresponding compression attribute, the pulled page data is compressed using the corresponding compression attribute, and the page header information of the pulled page and the corresponding compressed page data are saved in the corresponding folder.

[0043] It should be noted that in this embodiment, the modified pages can still be obtained from the source by mounting the modified page information of the external tablespace directory file, and then compressed and saved to achieve incremental page identification and compressed backup of external tablespace data.

[0044] In existing technologies, only the data pages in the database directory that have been changed are tracked using CBM (Change Block Tracking) technology. Then, before incremental backup, the modified bitmap information of the modified pages is mounted on the corresponding data file by matching file paths.

[0045] This embodiment expands the scope of file matching. Based on the OID of the external tablespace directory, the link path of the data file in the external directory relative to the database directory is obtained. Based on this, the file path tracked by CBM is matched to obtain the modified page information of the external tablespace data file.

[0046] In this embodiment, the relative path of the data file in the external directory is obtained by using the OID of the external tablespace directory, and the file path is matched with the file path tracked by CBM to obtain the modified page information of the external tablespace data file.

[0047] For example, a query retrieves the relative path and modified page number of a file in the source database relative to the database directory file. The matching is then completed by mapping the OID of the external tablespace directory to the CBM query item. In practical applications, the relative path obtained from the query includes pg_tblspc + the OID of the external tablespace directory + the actual file location. When modified page information of a mounted data file is detected, this embodiment receives all modified pages sequentially from front to back, storing them in header+page format until all modified pages have been retrieved.

[0048] Figure 6 This is a flowchart illustrating the incremental page identification and compression backup process for external tablespace data according to an embodiment of the present invention. Figure 6 As shown, the modification bitmap information of all data pages is obtained from the CBM system using the LSN from the previous round. The LSN is a unique sequence number in the database transaction log, marking each transaction in the WAL (Write-Ahead Log). In the Logging process, the LSN recorded at the end of the previous backup round serves as the baseline for incremental backups. It checks if pgFile has been completely read, i.e., if the file processing is complete. If all files are processed, the process ends; otherwise, it reads the next file. It identifies whether the read file is an external file. If the file is not external, openGauss uses its default backup logic. If the file is external, it checks if it belongs to an external tablespace directory. If it does, it maps the path using the tablespace OID. For example, based on the OID of the external tablespace directory (e.g., 12345), it generates a link path relative to the database directory (e.g., pg_tblspc / 12345 / 67890). It searches the modified file list for the path and matches the mapped path with the modified page path tracked by CBM. If a match is successful, it mounts the modified page information (e.g., page number, compression attributes). If a match fails, it skips files that do not need backup. Finally, it stores the modified page data in the format of header (page header information) + page (page number) for subsequent compression processing.

[0049] Step S3: When it is necessary to restore compressed data from an external tablespace, restore the files in the corresponding order according to the category of the compressed backup data.

[0050] When it is necessary to restore compressed data from an external tablespace, this embodiment retrieves the recovery files sequentially from the backup set and restores them according to the order of the backup set chain.

[0051] It should be noted that in this embodiment, the incremental data recovery flow of the data file is the opposite of the backup flow, starting from the end of the backup set. As an optional example, in this embodiment, when restoring incremental data of a data file, starting from the end of the backup set, the corresponding metadata of the data file is searched, and the page number to be restored is retrieved sequentially according to the information in the metadata. It is checked whether the page has been restored by the previous backup set. If not, the file page of the current backup set is restored, and the restored page number is recorded, until all backup sets have been traversed.

[0052] For example, the write size of the metadata is used to determine whether there is content to be recovered in the data file. If the write size in the metadata is greater than 0 and is not an invalid value, it means that there is content to be recovered. Then, the page numbers to be recovered are retrieved in sequence according to the page header information, the list of modified pages and the corresponding compressed data are read, the compressed data is decompressed and written to the target file according to the page number.

[0053] Figure 7 This diagram illustrates the technical principle of recovering compressed data from an external tablespace according to an embodiment of the present invention. Figure 7 As shown, there is currently a backup chain consisting of full backup, incremental backup 1, and incremental backup 2 waiting to be restored. In terms of backup, the time flow is from full backup to incremental backup, and finally to incremental backup 2; while during restoration, the data files are restored from incremental backup 2, incremental backup 1, and finally full backup.

[0054] Each backup saves the data of file1. Incremental backup 2 saves the modified data in red. During restoration, the red pages are restored first, and their page numbers are recorded. Incremental backup 1 saves the modified pages in green, and two of these pages have already been processed in the restoration of incremental backup 2. Therefore, only the remaining modified pages are restored. Of the purple pages recorded in the full backup, 6 have already been restored and processed previously, so only the remaining pages need to be processed.

[0055] This recovery process, with only bitmap-level overhead, avoids multiple disk flushes of the same page by recording the modified pages that have been recovered, effectively reducing the amount of IO data.

[0056] In practical applications, the openGauss database external tablespace compression backup and recovery method and system of this embodiment have the following beneficial technical effects: 1. Implement full and incremental compressed backup functions for external tablespace data in openGauss database, effectively solving the limitation of existing technologies that only support compressed backup of the main directory ($PGDATA), and can accurately track the modification status of external tablespace data pages; 2. Based on LSN and enhanced CBM mechanisms, a precise backup strategy is implemented that only backs up data pages that have changed since the last backup. Through innovative tablespace OID path mapping, the physical path of external files is converted into a standard logical path that the database can recognize, ensuring accurate identification and matching of incrementally modified pages. Combined with an invalid value identification mechanism, unmodified files can be skipped, significantly improving the efficiency and accuracy of compressed backups. 3. Through a reverse chain recovery process, the recovery operation is performed in reverse from the latest incremental backup to the full backup. By maintaining a global bitmap to record the number of recovered pages, duplicate data flushing is effectively avoided. Actual tests show that the amount of IO operations during the recovery process is reduced by more than 60%. Compressed data is stored in header+page format, and page-level atomic recovery is supported. When the recovery process is interrupted, the location of the unrecovered page can be accurately located, and the fault recovery time is controlled within seconds. 4. It achieves an end-to-end automated process from automatic path identification and intelligent compression backup to efficient recovery. While maintaining full compatibility with the openGauss native backup tool, it supports a hybrid backup mode for the main directory and external tablespaces. The system adopts a modular architecture design, supporting both independent deployment and seamless integration into cloud environments to meet deployment needs of different scales.

[0057] like Figure 8 As shown, the present invention also provides a device including a processor 210, a communication interface 220, a memory 230 for storing processor-executable computer programs, and a communication bus 240. The processor 210, communication interface 220, and memory 230 communicate with each other via the communication bus 240. The processor 210 implements the aforementioned openGauss database external tablespace compression backup and recovery method by running the executable computer program.

[0058] The computer program in memory 230, when implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0059] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected based on actual needs to achieve the purpose of this embodiment. Those skilled in the art can understand and implement this without any creative effort.

[0060] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0061] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for compressed backup and recovery of external tablespaces in an OpenGauss database, characterized in that, The method includes: Step S1: Identify the external tablespace directory, filter and mark the data files in the external tablespace directory that meet the compression conditions; Step S2: Perform full or incremental compression backup on data files that meet the compression conditions according to the compression attributes; Step S3: When it is necessary to restore compressed data from an external tablespace, restore the files in the corresponding order according to the category of the compressed backup data; In step S2, perform a full compression backup of the data files that meet the compression conditions as follows: Determine if the data file has not been changed since the last backup. If not, skip the backup and write an invalid value flag. If the data file has been modified since the last backup, obtain all page numbers of the data file and pull the page data from the database source according to the page number; Check if the fetched page has the corresponding compression attribute, and compress the fetched page data using the corresponding compression attribute; Save the page header information and corresponding compressed page data of the retrieved page in the corresponding folder; In step S2, by mounting the modified page information of the external tablespace directory file, the modified page is obtained from the source and compressed and saved to achieve incremental compression backup of the external tablespace data. In step S2, based on the OID of the external tablespace directory, the link path of the data file in the external directory relative to the database directory is obtained. The file path is matched with the CBM tracking and the modified page information of the external tablespace data file is obtained. When the modified page information of the data file is identified, all modified pages are received from front to back and stored in the format of page header information and page number until all modified pages are obtained.

2. The method for compressed backup and recovery of external tablespaces in an OpenGauss database according to claim 1, characterized in that, In step S1, by comparing the database tablespace path with the external directory path entered by the user, the external tablespace directory that needs to be compressed and backed up is identified, and the OID of the external tablespace directory is recorded.

3. The method for compressed backup and recovery of external tablespaces in an OpenGauss database according to claim 2, characterized in that, In step S1, the tablespace path list is obtained by connecting to the database. The external directory paths entered by the user are traversed, and each external directory path is compared with the tablespace path list item by item. When the string of the external directory path entered by the user completely matches the path in the tablespace path list, the external directory path is marked as an external tablespace directory, and the parent-child containment relationship between directories is excluded during the comparison.

4. The method for compressed backup and recovery of external tablespaces in an OpenGauss database according to claim 1, characterized in that, In step S1, based on the directory information and file name information of the data files, the data files that conform to the database naming rules in the external tablespace directory are obtained. The data files are marked as meeting the compression conditions by marking the metadata of the identified data files. The compression attributes parsed from the user compression command are used to compress and back up the data files that meet the compression conditions in the external tablespace directory at the page level and save them.

5. The method for compressed backup and recovery of external tablespaces in an OpenGauss database according to claim 1, characterized in that, In step S3, when it is necessary to restore the full compressed backup data of the external tablespace, the recovery files are retrieved sequentially from the backup set and restored according to the order of the backup set chain.

6. The method for compressed backup and recovery of external tablespaces in an OpenGauss database according to claim 1, characterized in that, In step S3, when it is necessary to restore the incremental compressed backup data of the external tablespace, starting from the end of the backup set, the corresponding metadata of the data file is searched, and the page number to be restored is retrieved in sequence according to the information in the metadata. It is checked whether the page has been restored by the previous backup set. If it has not been restored, the file page of the current backup set is restored, and the restored page number is recorded, until all backup sets are traversed.

7. An openGauss database external tablespace compression backup and recovery system, the system being used to implement the method as described in any one of claims 1 to 6, characterized in that, The system includes a compressed backup and recovery server, which includes: The identification module is used to identify external tablespace directories, filter and mark data files in external tablespace directories that meet compression conditions; The compression backup module is used to perform full or incremental compression backups on data files that meet the compression conditions based on compression attributes. The recovery module is used to restore files in the corresponding order according to the category of compressed backup data when it is necessary to restore compressed data from external tablespaces.

Images