A page merging method, product, medium and device

Abstract

This invention relates to the field of database technology, and in particular to a page merging method, product, medium, and device. The page merging method includes: identifying source pages to be merged in a table file and matching target pages to the source pages; migrating all active tuples in the source pages to the target pages, and setting a forwarding pointer for each migrated active tuple, wherein the forwarding pointer points from the original physical location of the active tuple in the source page to its new physical location in the target page; recording the mapping relationship between the original physical location and the new physical location of each active tuple in a preset index repair queue; and updating the index entries pointing to the migrated active tuples according to the index repair queue. This invention decouples tuple migration and index updates, and utilizes forwarding pointers to ensure the correctness of concurrent access after migration, achieving efficient merging of sparse table pages and index updates during system runtime, thereby ensuring high availability of the database system.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a page merging method, product, medium, and device. Background Technology

[0002] In existing relational database systems, multi-version concurrency control (MVCC) is commonly used to achieve transaction isolation and improve concurrency performance. Under this mechanism, data update operations are actually combinations of deleting old tuples and inserting new tuples, which generates a large number of dead tuples. To reclaim the space occupied by these dead tuples, database systems provide cleanup mechanisms (such as the VACUUM command).

[0003] In relational database systems, the standard VACUUM operation typically only reclaims table space, marking dead tuples as free and returning the space they occupy to the Free Space Map (FSM) for reuse in subsequent insert operations. However, because the standard VACUUM operation does not physically move tuples to merge sparse table pages, frequent update and delete operations leave a large number of holes or fragments (i.e., table pages with extremely low space utilization) in the table file. This causes the table file to continuously swell, wasting a significant amount of disk storage space and reducing the performance of full table scans.

[0004] To completely resolve the issue of table file space bloat, existing cleanup solutions typically employ the VACUUM FULL or CLUSTER operation. By locking the target table, the entire table is rewritten into a new file, thereby eliminating all file gaps.

[0005] However, while existing cleanup methods can solve the space issue, they require acquiring exclusive locks, which completely blocks read and write operations on the table during the operation, causing prolonged downtime for maintenance. Furthermore, since moving a tuple changes its physical location (Tuple Identifier, or CTID), existing cleanup methods also require a full table rewrite. Since the CTID is a pointer to the tuple from all indexes (such as B-Tree indexes), once the CTID changes, all index entries pointing to that tuple must be updated, making runtime execution unsafe. Summary of the Invention

[0006] In view of the above problems, a page merging method, product, medium and device are proposed to overcome the above problems or at least partially solve the above problems.

[0007] One object of the present invention is to provide a page merging method to merge sparse table pages during system operation, thereby improving the storage space utilization of table files.

[0008] A further objective of this invention is to improve the redirection efficiency of concurrent index scans.

[0009] Another further objective of this invention is to ensure high availability of services.

[0010] Specifically, according to one aspect of the present invention, a page merging method is provided, comprising: Identify the source pages to be merged in the table file and match the target pages for the source pages; Migrate all active tuples from the source page to the target page, and set a forwarding pointer for each migrated active tuple, where the forwarding pointer points from the original physical location of the active tuple in the source page to the new physical location of the active tuple in the target page. Record the mapping relationship between the original physical location and the new physical location of each active tuple to the preset index repair queue; Update the index entries pointing to the migrated active tuples based on the index repair queue.

[0011] Optionally, the forwarding pointer is a system tuple containing the new physical location of the active tuple in the target page; and The steps for setting forwarding pointers for each migrated active tuple include: Modify each migrated active tuple in the source page to a system tuple in place.

[0012] Optionally, the steps of migrating all active tuples from the source page to the target page include: Iterate through all active tuples on the source page; Copy each active tuple and insert it into the target page, and obtain the new physical position of the active tuple in the target page.

[0013] Optionally, before the step of updating the index entries pointing to the migrated active tuples according to the index repair queue, the page merging method further includes: In response to concurrent access requests hitting the original physical location of the active tuple in the source page, obtain the forwarding pointer; Jump to the new physical location of the active tuple on the target page based on the forwarding pointer to retrieve the active tuple's data.

[0014] Optionally, the step of updating the index entries pointing to the migrated active tuples according to the index repair queue includes: Read the mapping from the index repair queue; Delete the index entries that point to the original physical location based on the mapping relationship, and insert new index entries that point to the new physical location.

[0015] Optionally, after updating the index entries pointing to the migrated active tuples according to the index repair queue, the page merging method further includes: Mark the forwarding pointer as cleanable; In response to the cleanup operation, reclaim the forwarding pointers that are marked as cleanupable; After all forwarding pointers in the source page have been reclaimed, the source page is returned to the free space mapping table of the table file.

[0016] Optionally, the steps of identifying the source pages to be merged in the table file and matching the target pages for the source pages include: The merged task is woken up based on a preset hybrid scheduling strategy; Based on the free space mapping table and table statistics in the table file, find the table pages in the table file whose space utilization is lower than the preset merge trigger threshold, and use them as source pages; Find the table page in the same table file that has enough remaining space to accommodate all active tuples in the source page, and use it as the target page.

[0017] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of any of the above-described page merging methods.

[0018] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the steps of any of the above-described page merging methods.

[0019] According to another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the page merging methods described above.

[0020] The page merging method of this invention identifies the source page to be merged and matches it with the target page, migrating active tuples from the source page to the target page. This achieves the merging of sparse table pages, effectively reclaiming internal fragmentation of the table file, preventing unlimited expansion of the table file, and thus improving the storage space utilization of the table file. Simultaneously, the page merging method of this invention ensures the correctness of concurrent access after migration by setting a forwarding pointer for each migrated active tuple and making the forwarding pointer point to the new physical location of the active tuple in the target page. Furthermore, the page merging method of this invention records the mapping relationship between the original and new physical locations of each active tuple after migration and updates the index entries according to the mapping relationship, achieving asynchronous index repair. This decouples tuple migration and index updates, avoiding long-term locking and transaction storms caused by simultaneous index updates during tuple migration. Therefore, it achieves efficient merging of sparse table pages and index updates during system runtime, thereby ensuring the high availability of the database system.

[0021] Furthermore, in the page merging method of the present invention, the forwarding pointer is a system tuple containing the new physical location of the active tuple in the target page, and the forwarding pointer is set by modifying the active tuple in the source page in place to the system tuple. This realizes setting the forwarding pointer at the original physical location of the active tuple, without the need for an additional centralized redirection table, reducing memory and disk overhead, improving redirection efficiency during concurrent access, and thus improving database performance.

[0022] Furthermore, the page merging method of the present invention, by obtaining a forwarding pointer in the original physical location of the source page in response to concurrent access requests before repairing the index, and jumping to the new physical location of the active tuple in the target page according to the forwarding pointer to obtain the data of the active tuple, ensures that all concurrent read operations can correctly access the latest data before the index entries are updated, thereby achieving seamless switching of data access and further ensuring the high availability of the database system.

[0023] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0024] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a flowchart illustrating a page merging method according to an embodiment of the present invention; Figure 2This is a control flowchart of a page merging method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a database system according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0025] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0026] To address the above problems, this invention proposes a page merging method. Figure 1 This is a flowchart illustrating a page merging method according to an embodiment of the present invention. Figure 1 As shown, the page merging method in this embodiment generally includes: Step S102 involves identifying the source pages to be merged in the table file and matching them with target pages. It's important to note that a table file is the physical file in the database that stores data rows, typically composed of multiple fixed-size table pages. Furthermore, the source page to be merged (denoted as P_Source) refers to a table page in the table file that becomes sparse after removing dead tuples, while the target page (denoted as P_Target) refers to a table page in the same table file that still has sufficient remaining space. Therefore, this step selects P_Source and P_Target, thus completing the decision on the merge target.

[0027] Step S104 involves migrating all active tuples from the source page to the target page and setting a forwarding pointer for each migrated active tuple. The forwarding pointer points from the active tuple's original physical location (CTID_old) in the source page to its new physical location (CTID_new) in P_Target. It's important to note that an active tuple refers to a data row in the table page that has not been deleted and is visible to the current transaction. Furthermore, the forwarding pointer (ForwardingPointer) is a special system tuple, also known as a redirected tuple, used to record the CTID_new after the tuple migration. Thus, this step achieves online tuple migration and CTID forwarding.

[0028] Step S106 records the mapping relationship between the original physical location and the new physical location of each active tuple to a preset index fixup queue. It should be noted that the index fixup queue can be a persistent queue, with persistence and crash safety guaranteed by the database's write-ahead log (WAL). Alternatively, the index fixup queue can be a shared memory queue to improve throughput. Thus, this step achieves the recording of pending tasks.

[0029] Step S108 involves updating the index entries pointing to the migrated active tuples according to the index repair queue. In other words, the background worker process continuously pulls tasks from the index repair queue and performs a hot update (HOTUpdate) style index entry repair for each index. Specifically, this step uses a standard index concurrency control mechanism, which does not block reads or writes, thus achieving asynchronous index repair.

[0030] The page merging method of this invention identifies the P_Source to be merged and matches it with P_Target, then migrates the active tuples in P_Source to P_Target, thereby merging sparse table pages, effectively reclaiming internal fragments of the table file, preventing the table file from expanding indefinitely, and thus improving the storage space utilization of the table file.

[0031] Meanwhile, the page merging method in this embodiment of the invention ensures the correctness of concurrent access after migration by setting a forwarding pointer for each migrated active tuple and making the forwarding pointer point to the CTID_new of the active tuple in P_Target.

[0032] In addition, the page merging method in this embodiment of the invention achieves asynchronous index repair by recording the mapping relationship between CTID_old and CTID_new of each active tuple after migrating the tuples and updating the index entries according to the mapping relationship.

[0033] Therefore, the page merging method of this invention decouples tuple migration and index update, avoiding long-term locking and transaction storms caused by the need to update the index simultaneously when migrating tuples. This enables efficient merging of sparse table pages and index updates during system runtime, thereby ensuring the high availability of the database system.

[0034] In some embodiments, step S102 may include the following steps: identifying the P_Source to be merged in the table file and matching the P_Source with the P_Target includes: waking up the merging task based on a preset hybrid scheduling strategy; searching for table pages in the table file with space utilization lower than a preset merging trigger threshold according to the FSM and table statistics of the table file, and using them as P_Sources; and searching for table pages in the same table file with sufficient remaining space to accommodate all active tuples in the P_Source, and using them as P_Targets.

[0035] It should be noted that the hybrid scheduling strategy can include one or more of the following: periodic scheduling strategy, semaphore-triggered strategy, and read / write threshold-triggered strategy. Specifically, the periodic scheduling strategy wakes up the merge task at configurable time intervals (e.g., table_merge_check_interval); the semaphore-triggered strategy actively wakes up the merge task after the standard VACUUM operation is completed; and the read / write threshold-triggered strategy wakes up the merge task when the cumulative number of deleted / updated tuples in the table file exceeds a preset high-water mark threshold.

[0036] In one specific embodiment, the step of waking up the merge task based on a preset hybrid scheduling strategy can be specifically executed as follows: waking up the merge task periodically at configurable time intervals; actively waking up the merge task after the cleanup operation for any table file has been completed; and / or monitoring database statistics, waking up the merge task when the cumulative number of deleted / updated tuples in any table file exceeds a preset high-water mark threshold. After waking up the merge task, the search process for P_Source is automatically triggered.

[0037] In this embodiment, space utilization refers to the proportion of used space (including ordinary tuples, forwarding pointers, and header overhead) in a table page to the total size of the table page. A preset merge trigger threshold can be pre-set and stored in the database to determine whether a page is sufficiently sparse to require merging. Specifically, the preset merge trigger threshold can be selected from any value between 20% and 50%. In one specific embodiment, the preset merge trigger threshold can be selected as 30%.

[0038] For example, first, based on the table file's FSM and table statistics, a table page with a space utilization rate of 10% is found in the table file. This is determined to be below the preset merge trigger threshold, and this table page is designated as P_Source. Second, based on the table file's FSM and table statistics, another table page in the same table file is found with a space utilization rate of 80%. This is determined to have sufficient remaining space to accommodate all active tuples in P_Source, and this table page is designated as P_Target, so that all active tuples in P_Source can be migrated to the remaining space in P_Target.

[0039] Therefore, the page merging method of this invention, by finding pages with space utilization rates below a preset threshold as P_Source and finding pages with sufficient remaining space in the same table file as P_Target, achieves automatic identification of sparse pages and accurate matching of P_Target, ensuring the effectiveness and rationality of the merging operation, avoiding unnecessary page migration, and thus reducing system resource consumption.

[0040] In some embodiments, the step of migrating all active tuples in P_Source to P_Target in step S104 above may include the following steps: traversing all active tuples in P_Source; copying and inserting each active tuple into P_Target, and obtaining the CTID_new of the active tuple in P_Target.

[0041] Specifically, all active tuples in P_Source can include tuple T1, tuple T2, ..., tuple Tn. Accordingly, after copying and inserting tuple T1, tuple T2, ..., tuple Tn into the P_Target page one by one, these tuples acquire new CTIDs (e.g., CTID_new1, CTID_new2, ..., CTID_new n).

[0042] Therefore, the page merging method of this invention achieves fine-grained migration of tuples by traversing all active tuples in P_Source and copying and inserting them one by one into P_Target to obtain CTID_new, thus ensuring data integrity.

[0043] In some embodiments, the forwarding pointer can be a system tuple containing the active tuple in P_Target with CTID_new. Based on this, the step of setting the forwarding pointer for each migrated active tuple in step S104 above may include the following step: modifying each migrated active tuple in P_Source in place to a system tuple.

[0044] In other words, after copying and inserting tuples T1, T2, ..., Tn into the P_Target page, the tuples T1, T2, ..., Tn in P_Source are not immediately deleted, but are instead modified in place to a forwarding pointer. For example, for tuple T1, after copying and inserting tuple T1 into the P_Target page, tuple T1 in P_Source is modified in place to a forwarding pointer, which only contains: T1->CTID_new1.

[0045] Therefore, in the page merging method of this embodiment of the invention, the forwarding pointer is a system tuple containing the active tuple in P_Target, which is CTID_new. The forwarding pointer is set by modifying the active tuple in P_Source in place to the system tuple. This realizes setting the forwarding pointer in the CTID_old of the active tuple, without the need for an additional centralized redirection table, reducing memory and disk overhead, improving redirection efficiency during concurrent access, and thus improving database performance.

[0046] In some embodiments, the forwarding pointer may also contain a mapping relationship between the CTID_old and CTID_new of the active tuple. Based on this, the step of setting a forwarding pointer for each migrated active tuple in step S104 above may further include the following step: recording the forwarding pointer of each migrated active tuple in a centralized redirection table.

[0047] It's important to note that the redirection table is a global mapping table maintained by the database kernel. It ensures persistence and crash safety through the database write-ahead log (WAL) and is used to uniformly record the mapping relationship between CTID_old and CTID_new for all migrated tuples. During concurrent access, the system first checks if a mapping entry for the current CTID exists in the redirection table. If it does, it jumps to the target location of the mapping to retrieve the data for the active tuple.

[0048] Therefore, the page merging method of this invention improves the stability of the database by recording forwarding pointers in a centralized redirection table, thereby achieving centralized management and fast querying of mapping relationships.

[0049] In some embodiments, prior to step S104 above, the page merging method of the present invention may further include the following step: acquiring page locks for P_Source and P_Target. Accordingly, after completing the migration and forwarding pointer setting of all active tuples in P_Source, the page merging method of the present invention may further include the following step: releasing page locks for P_Source and P_Target.

[0050] Specifically, the page lock can be a page-level lightweight lock (Buffer Lock) to prevent concurrent read and write operations on these two pages. Furthermore, after releasing the page locks on P_Source and P_Target, P_Source only contains a forwarding pointer; logically, it no longer contains ordinary tuple data, but it can still be used for redirection of concurrent access.

[0051] Therefore, the page merging method of this invention achieves fine-grained concurrent control of P_Source and P_Target by acquiring a page-level lightweight lock before tuple migration and releasing the page-level lightweight lock after tuple migration, avoiding long-term blocking caused by table-level locks. At the same time, the forwarding pointer in P_Source can still support the redirection of concurrent access after the migration is completed, thereby improving the database's concurrent processing capability and system stability.

[0052] In some embodiments, prior to step S108 above, the page merging method of the present invention may further include the following steps: in response to a concurrent access request hitting the CTID_old of the active tuple in P_Source, obtaining a forwarding pointer; and jumping to the CTID_new of the active tuple in P_Target according to the forwarding pointer to obtain the data of the active tuple.

[0053] It should be noted that concurrent access requests refer to read operations on active tuples initiated by other transactions or queries during or after tuple migration but before index updates are completed.

[0054] Specifically, before the index update is complete, any index scan based on the old index entry will first access CTID_old of P_Source. Since this position has been overwritten with the forwarding pointer, CTID_new in the forwarding pointer is read to relocate to the corresponding position in P_Target and read the tuple data.

[0055] For example, if a concurrent index scan hits the old index entry of tuple T1, it can access CTID_old1 of P_Source; it finds that CTID_old1 is a forwarding pointer, and the content of the forwarding pointer is T1->CTID_new1; according to the guidance of the forwarding pointer, it transparently jumps to access CTID_new1 of P_Target and obtains the data of tuple T1.

[0056] Therefore, the page merging method of this embodiment of the invention obtains a forwarding pointer in CTID_old in P_Source in response to concurrent access requests, and jumps to CTID_new in P_Target according to the forwarding pointer to obtain the data of the active tuple. This ensures that before the index entry is updated, all concurrent read operations will not fail or read old data, and the latest data can still be accessed correctly. This achieves seamless switching of data access and further ensures the high availability of the database system.

[0057] In some embodiments, in step S106 above, the index repair queue may include multiple tasks, each task including a mapping relationship of an active tuple and an object identifier (table_oid) of the table file to which it belongs. For example, for tuple T1, the corresponding task in the index repair queue is (T1_table_oid, CTID_old1, CTID_new1).

[0058] Further, step S108 above may include the following steps: reading the mapping relationship from the index repair queue; deleting the index entry pointing to CTID_old according to the mapping relationship, and inserting a new index entry pointing to CTID_new.

[0059] Specifically, the background process iteratively pulls tasks from the index repair queue to read each mapping sequentially. For example, given a task (T1_table_oid, CTID_old1, CTID_new1), it queries the system directory based on T1_table_oid to find all indexes (Index1, Index2, ..., Index n) on the table file. For each index, it deletes the old index entry pointing to CTID_old1 and inserts a new index entry pointing to CTID_new1. This operation uses standard index concurrency control mechanisms and does not block reads or writes.

[0060] Additionally, once each index update is complete, the task is removed from the index repair queue or marked as completed.

[0061] Therefore, the page merging method of this invention, by reading the mapping relationship from the index repair queue, deleting old index entries and inserting new index entries, achieves accurate updating of index entries, ensuring that all indexes pointing to the migrated tuples can correctly point to CTID_new, thereby avoiding query errors caused by index failure and ensuring the correctness of the database system.

[0062] In some embodiments, after step S108 above, the page merging method of the present invention may further include the following steps: marking forwarding pointers as cleanable; in response to a cleanup operation, reclaiming forwarding pointers marked as cleanable; and returning P_Source to the FSM of the table file after all forwarding pointers in P_Source have been reclaimed.

[0063] Specifically, once it's confirmed that all indexes have been updated to point to CTID_new1, the forwarding pointer at CTID_old1 can be marked as cleanable. During future standard VACUUM operations scanning P_Source, the forwarding pointer marked as cleanable can be safely reclaimed. Furthermore, once all forwarding pointers on P_Source have been cleaned up, P_Source becomes an empty page. At this point, P_Source can be completely returned to FSM for eventual reuse by the system or truncated at the end of the table file. For example, if the empty page is at the end of the table file, a file truncation operation is performed to physically delete the empty page from the table file; otherwise, the empty page is retained in FSM for reuse in subsequent insert operations.

[0064] Therefore, the page merging method of this invention, by marking the forwarding pointer as cleanable after the index entry is updated, reclaiming the forwarding pointer in response to the cleanup operation, and returning P_Source to FSM after all forwarding pointers have been reclaimed, realizes the automatic reclamation of forwarding pointers and P_Source, completely releases the storage space occupied by sparse table pages, effectively reclaims internal table fragments that cannot be reclaimed by standard VACUUM, thereby preventing the table file from expanding indefinitely and improving the utilization of storage space.

[0065] Figure 2 This is a control flowchart of a page merging method according to an embodiment of the present invention. The following is in conjunction with… Figure 2 The process steps of page merging in this embodiment will be described in detail.

[0066] Step S202: Wake up the merged task based on a preset hybrid scheduling strategy. It should be noted that the hybrid scheduling strategy can consist of a periodic scheduling strategy, a semaphore triggering strategy, and a read / write threshold triggering strategy.

[0067] Step S204: Based on the FSM and table statistics, find pages in the table file whose space utilization is lower than the preset merge trigger threshold, and use them as P_Source.

[0068] Step S206: Locate a page in the same table file with sufficient remaining space to hold all active tuples in P_Source, and designate it as P_Target.

[0069] Step S208: Iterate through all active tuples in P_Source.

[0070] Step S210: Copy the active tuples one by one and insert them into P_Target, and obtain the CTID_new of the active tuples in P_Target.

[0071] Step S212: Modify each migrated active tuple in P_Source in-place to a forwarding pointer. It should be noted that the forwarding pointer changes from the active tuple's CTID_old in P_Source to the active tuple's CTID_new in P_Target.

[0072] Step S214: Record the mapping relationship between CTID_old and CTID_new for each active tuple to a preset index repair queue.

[0073] Step S216: Read the mapping relationship from the index repair queue.

[0074] Step S218: Delete the index entry pointing to CTID_old according to the mapping relationship, and insert a new index entry pointing to CTID_new.

[0075] Step S220: Mark the forwarding pointer as cleanable.

[0076] Step S222, in response to the cleanup operation, reclaim the forwarding pointers that were marked as cleanupable.

[0077] In step S224, after all forwarding pointers in P_Source have been reclaimed, P_Source is returned to the FSM. This completes the page merging process, and the current procedure ends.

[0078] The page merging method of this invention identifies the P_Source to be merged and matches it with P_Target, then migrates the active tuples in P_Source to P_Target, thereby merging sparse table pages, effectively reclaiming internal fragments of the table file, preventing the table file from expanding indefinitely, and thus improving the storage space utilization of the table file.

[0079] Meanwhile, the page merging method in this embodiment of the invention ensures the correctness of concurrent access after migration by setting a forwarding pointer for each migrated active tuple and making the forwarding pointer point to the CTID_new of the active tuple in P_Target.

[0080] In addition, the page merging method in this embodiment of the invention achieves asynchronous index repair by recording the mapping relationship between CTID_old and CTID_new of each active tuple after migrating the tuples and updating the index entries according to the mapping relationship.

[0081] Therefore, the page merging method of this invention decouples tuple migration and index update, avoiding long-term locking and transaction storms caused by the need to update the index simultaneously when migrating tuples. This enables efficient merging of sparse table pages and index updates during system runtime, thereby ensuring the high availability of the database system.

[0082] Based on the same inventive concept, this invention also proposes a database system. Figure 3 This is a schematic diagram of a database system according to an embodiment of the present invention. Figure 3 As shown, the database system 100 of the present invention generally includes a table statistical analysis module 101, a page merging decision module 102, a tuple migration execution module 103, and an index asynchronous repair module 104. These modules exist in the database kernel in the form of background processes.

[0083] In this embodiment, the table statistics analysis module 101 is used to identify the P_Sources to be merged in the table file. That is, the table statistics analysis module 101 is responsible for monitoring and scheduling, and identifying sparse table pages. Specifically, the table statistics analysis module 101 can receive the signal sent by the standard VACUUM operation upon completion, and read the FSM and table statistics information. Furthermore, the table statistics analysis module 101 can employ a hybrid scheduling strategy to wake up the merge task, identify the P_Sources, and send the identified P_Sources to the page merge decision module 102.

[0084] The page merge decision module 102 is used to match P_Source with P_Target. That is, the page merge decision module 102 is responsible for formulating the page merge plan and selecting a P_Target that matches the P_Source. Specifically, after the table statistics analysis module 101 initiates the merge task, the page merge decision module 102 finds a suitable P_Target for the identified P_Source, and after finding a matching P_Source and P_Target, submits the merge task to the tuple migration execution module 103.

[0085] The tuple migration execution module 103 is used to migrate all active tuples in P_Source to P_Target, set a forwarding pointer for each migrated active tuple, and record the mapping relationship between CTID_old and CTID_new of each active tuple to a preset index repair queue. It should be noted that the forwarding pointer points from the CTID_old of the active tuple in P_Source to the CTID_new of the active tuple in P_Target. In other words, the tuple migration execution module 103 is responsible for performing online tuple migration and CTID forwarding.

[0086] Specifically, after receiving the merge task, the tuple migration execution module 103 first acquires the page locks of P_Source and P_Target, then iterates through all active tuples in P_Source, copies each active tuple to P_Target, and obtains the CTID_new of the active tuple in P_Target. After each active tuple is migrated from P_Source, the tuple migration execution module 103 does not immediately delete the active tuple from P_Source, but instead modifies it in place to a forwarding pointer and records the mapping relationship between CTID_old and CTID_new of the active tuple in a preset index repair queue. Finally, after completing the migration and forwarding pointer setting of all active tuples in P_Source, the page locks of P_Source and P_Target are released.

[0087] The asynchronous index repair module 104 updates index entries pointing to the migrated active tuples according to the index repair queue. Specifically, the asynchronous index repair module 104 is one or more background worker processes that continuously pull tasks from the index repair queue to update index entries one by one. Once the asynchronous index repair module 104 confirms that all indexes have been updated to point to CTID_new1, it marks the forwarding pointer of CTID_old1 as cleanable, and this marked forwarding pointer can be safely reclaimed during future standard VACUUM operations scanning P_Source. Finally, after all forwarding pointers on P_Source have been cleaned up, P_Source becomes an empty page, allowing it to be completely returned to the FSM.

[0088] Therefore, the database system of this embodiment utilizes four modules as background processes scheduled by the kernel. The table statistics and analysis module 101 and the page merging decision module 102 are responsible for analysis and decision-making. The tuple migration execution module 103 performs tuple migration and writes to the queue. The index asynchronous repair module 104 reads the queue and repairs the index. This decouples tuple migration and index update, avoiding long-term locking and transaction storms caused by the need to update the index simultaneously when migrating tuples. As a result, it achieves efficient merging of sparse table pages and index updates during system operation, thereby ensuring the high availability of the database system.

[0089] This embodiment also provides a computer program product 10, a computer-readable storage medium 20, and a computer device 30. Figure 4 This is a schematic diagram of a computer program product 10 according to an embodiment of the present invention. Figure 5 This is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. Figure 6This is a schematic block diagram of a computer device 30 according to an embodiment of the present invention.

[0090] Computer program product 10 includes computer program 11, which, when executed by processor 32, implements the steps of any of the page merging methods described above. Computer-readable storage medium 20 stores the computer program 11 thereon, which, when executed by processor 32, implements the steps of the page merging method of any of the above embodiments. Computer device 30 may include memory 31, processor 32, and computer program 11 stored in memory 31 and running on processor 32.

[0091] The computer program 11 used to perform the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages.

[0092] Computer program 11 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of the invention, electronic circuitry including, for example, programmable logic circuitry, a Field-Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA) may execute computer-readable program instructions to personalize the electronic circuitry by utilizing state information of the computer-readable program instructions.

[0093] For the purposes of this embodiment, computer program product 10 is a related product that includes computer program 11.

[0094] For the purposes of this embodiment, computer-readable storage medium 20 is a tangible device capable of holding and storing computer program 11. It can be any device capable of containing, storing, communicating, propagating, or transmitting program 811 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage medium 20 include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, and any suitable combination thereof.

[0095] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method of page merging, characterized by include: Identify the source pages to be merged in the table file and match the target pages for the source pages; Migrate all active tuples from the source page to the target page, and set a forwarding pointer for each migrated active tuple, wherein the forwarding pointer points from the original physical location of the active tuple in the source page to the new physical location of the active tuple in the target page. The mapping relationship between the original physical location and the new physical location of each active tuple is recorded in a preset index repair queue; The index entries pointing to the migrated active tuples are updated according to the index repair queue.

2. The page merging method of claim 1, wherein, The forwarding pointer is a system tuple containing the new physical location of the active tuple in the target page; and The step of setting a forwarding pointer for each of the migrated active tuples includes: Each active tuple that was migrated in the source page is modified in place to the system tuple.

3. The page merging method according to claim 2, characterized in that, The step of migrating all active tuples from the source page to the target page includes: Iterate through all active tuples on the source page; The active tuples are copied one by one and inserted into the target page, and the new physical position of the active tuples in the target page is obtained.

4. The page merging method according to claim 1, characterized in that, Prior to the step of updating the index entries pointing to the migrated active tuples according to the index repair queue, the page merging method further includes: In response to a concurrent access request hitting the original physical location of the active tuple in the source page, obtain the forwarding pointer; The forwarding pointer will redirect to the new physical location of the active tuple on the target page to obtain the data of the active tuple.

5. The page merging method according to claim 1, characterized in that, The step of updating the index entry pointing to the migrated active tuple according to the index repair queue includes: Read the mapping relationship from the index repair queue; Delete the index entry pointing to the original physical location according to the mapping relationship, and insert a new index entry pointing to the new physical location.

6. The page merging method according to claim 1, characterized in that, After the step of updating the index entries pointing to the migrated active tuples according to the index repair queue, the page merging method further includes: Mark the forwarding pointer as cleanable; In response to the cleanup operation, the forwarding pointers marked as cleanable are reclaimed; After all the forwarding pointers in the source page have been reclaimed, the source page is returned to the free space mapping table of the table file.

7. The page merging method according to claim 1, characterized in that, The steps of identifying the source pages to be merged in the identification table file and matching the target pages for the source pages include: The merged task is woken up based on a preset hybrid scheduling strategy; Based on the free space mapping table and table statistics of the table file, the table page with a space utilization rate lower than the preset merge trigger threshold is searched in the table file and used as the source page; The target page is a table in the same table file with sufficient remaining space to accommodate all active tuples in the source page.

8. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the page merging method according to any one of claims 1 to 7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the page merging method according to any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the page merging method according to any one of claims 1 to 7.

Images