Database branch processing method and apparatus

Abstract

Embodiments of the present specification provide a database branch processing method and device, wherein the database branch processing method comprises: in response to a branch creation request for a target database, writing a first memory data page in a shared buffer pool to a disk, setting an inventory data page in the disk to a read-only state, and obtaining a baseline data source for representing a database state before branch creation; determining a first branch identifier of a first database branch to be created, establishing an association relationship between the first branch identifier and the baseline data source, and creating the first database branch; creating a first incremental file corresponding to the first branch identifier, the first incremental file being used to store a first incremental data page, the first incremental data page being a data page that has changed under the first database branch; and processing a database task under the first database branch based on the baseline data source and the first incremental file. In this way, the creation overhead of the database branch is reduced, and the branch processing efficiency is improved.

Description

Technical Field

[0001] This specification relates to the field of database branching technology, and in particular to a database branching method and apparatus. Background Technology

[0002] With the rapid development of cloud-native database technology, distributed clusters and multi-tenant architectures place higher demands on database versioning management and isolation. In modern cloud database systems, code-based management of data infrastructure has become a core trend. To meet this evolving technological need, the system requires support for database branch management functions similar to code version control, including the ability to create database branches, conduct independent development and testing, and merge them back into the main branch after verification, thereby improving the flexibility and efficiency of database development and operation.

[0003] Currently, database environment isolation primarily relies on physical copying. Specifically, when creating a database branch, all data files in the original database are completely copied, and the copied data files become independent data files for the new branch. Each branch reads and writes its own corresponding data file. This approach can meet basic isolation requirements when the data volume is small, but it has significant shortcomings in large-scale data scenarios: firstly, the data copying process is time-consuming, taking several hours for data volumes ranging from GB to TB, failing to meet the need for rapid branch creation within seconds; secondly, this method requires additional storage space to store complete data copies, resulting in high storage overhead. Therefore, there is an urgent need for a database branching method that supports rapid database branch creation and reduces storage overhead. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a database branching method. One or more embodiments of this specification also relate to a database branching apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a database branching method is provided, comprising: In response to a branch creation request for the target database, the first memory data page in the shared buffer pool is written to disk, and the existing data pages on disk are set to read-only state to obtain the baseline data source, which is used to characterize the database state before the branch is created. Determine the first branch identifier of the first database branch to be created, establish the association between the first branch identifier and the baseline data source, and create the first database branch; Create a first incremental file corresponding to the first branch identifier, wherein the first incremental file is used to store the first incremental data page, and the first incremental data page is the data page that has changed under the first database branch; Based on the baseline data source and the first incremental file, the database tasks under the first database branch are processed.

[0006] According to a second aspect of the embodiments of this specification, a database branching processing apparatus is provided, comprising: The write module is configured to write the first memory data page in the shared buffer pool to disk in response to a branch creation request for the target database, and set the existing data pages on the disk to read-only state to obtain the baseline data source, wherein the baseline data source is used to characterize the database state before the branch is created. The first creation module is configured to determine the first branch identifier of the first database branch to be created, establish the association between the first branch identifier and the baseline data source, and create the first database branch. The second creation module is configured to create a first incremental file corresponding to the first branch identifier, wherein the first incremental file is used to store the first incremental data page, and the first incremental data page is the data page that has changed under the first database branch; The processing module is configured to process database tasks under the first database branch based on the baseline data source and the first incremental file.

[0007] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the database branching method described above.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the database branching method described above.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the database branching method described above.

[0010] The database branching method provided in this specification, in response to a branch creation request, writes a first memory data page from the shared buffer pool to the disk and sets the existing data pages on the disk to a read-only state to obtain a baseline data source characterizing the database state before branch creation, thereby ensuring consistency between the baseline data source and the actual database state. Furthermore, by establishing an association between the first database branch and the baseline data source, and creating a first incremental file for the first database branch solely for recording data pages that have changed after branch creation, the first database branch can be managed based on a hierarchical storage method of "baseline data source + incremental file," eliminating the need to copy the complete database data during branch creation. This reduces storage resource consumption during database branch creation and shortens branch creation time, enabling rapid creation and efficient management of database branches. Simultaneously, since the first incremental file directly stores the changed data pages under the first database branch, when accessing the first database branch, the corresponding changed data pages can be directly obtained based on the first incremental file, thereby improving the real-time performance and processing efficiency of branch data access. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the architecture of a database branching system provided in one embodiment of this specification; Figure 2 This is a flowchart illustrating a database branching method provided in one embodiment of this specification; Figure 3 This is a flowchart illustrating a database branch creation process according to one embodiment of this specification; Figure 4 This is a schematic diagram of an incremental file storage format provided in one embodiment of this specification; Figure 5 This is a data reading flowchart provided in one embodiment of this specification; Figure 6 This is a data writing flowchart provided in one embodiment of this specification; Figure 7 This is a flowchart illustrating a database branch merging process as provided in one embodiment of this specification; Figure 8 This is a flowchart illustrating a database branch switching process according to one embodiment of this specification; Figure 9 This is a schematic diagram of the structure of a database branching processing device provided in one embodiment of this specification; Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0012] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0015] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0016] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0017] Branching (database branching operations): refers to the mechanism of creating a new database branch based on the current database state at a certain point in time, similar to the Git branching concept in version control systems. After the branch is created, the original branch and the new branch are logically independent, and can provide read and write services to the outside world respectively, and each maintains subsequent data changes.

[0018] WAL (Write-Ahead Log): The write-ahead log mechanism means that before a database actually modifies a data page, it must first record the corresponding modification operation in the log. This mechanism ensures the durability of transactions and the recoverability after a database crash.

[0019] LSN (Log Sequence Number): A monotonically increasing number used in a database to uniquely identify the location of a WAL (Write-Ahead Log) record. A new LSN is generated for each data modification operation, reflecting the chronological order of the data changes.

[0020] branching_lsn (Branch Baseline Log Sequence Number): The LSN value recorded when a branch operation occurs, used as the time baseline for the branch. All modifications with an LSN less than or equal to branching_lsn belong to the shared baseline data before the branch was created, while modifications with an LSN greater than branching_lsn belong to the incremental data generated independently by each branch.

[0021] Data Page: A data page is the basic unit for data storage and management in a database, and it is also the unit for database I / O operations. It is usually 8KB in size.

[0022] A shared buffer is a memory area in a database system used to cache disk data pages. By maintaining this buffer in memory, the database reduces the number of disk I / O accesses, thereby improving data access performance. The buffer is shared by multiple database processes and is typically managed using page replacement algorithms such as LRU (Least Recently Used) or Clock-Sweep.

[0023] Dirty Page: A dirty page is a data page that has been modified in the shared buffer pool but has not yet been flushed back to disk.

[0024] Flush Dirty Page: This refers to the operation of writing modified data pages from the shared buffer pool back to disk. This process can be triggered by a background process (such as the background writer) or a checkpointer.

[0025] Checkpoint is a consistency control mechanism in databases that forces all dirty pages in the shared buffer pool to be flushed to disk and records the consistency position, thereby determining the starting position of WAL replay during system crash recovery.

[0026] In-place update refers to modifying data directly at the original data storage location, which is the opposite of copy-on-write.

[0027] COW (Copy-on-Write): The copy-on-write mechanism refers to a data consistency strategy that ensures the original data remains unchanged by first making a copy of the data and then modifying the copy when it is necessary to modify shared data.

[0028] Sparse File: A file system feature that allows a file's logical size to be much larger than its actual disk space usage. Unwritten data areas are not allocated actual disk blocks, thus saving storage space.

[0029] This specification provides a database branching method, and also relates to a database branching apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0030] Figure 1 This is a schematic diagram of the architecture of a database branch processing system 100 provided in an embodiment of this application.

[0031] like Figure 1 As shown, the database branch processing system 100 of this application embodiment includes an application layer / SQL layer 102, a branch metadata management layer 104, a shared buffer pool 106, a storage management layer 108, a baseline data source 110, and an incremental file 112.

[0032] Among them, the application layer / SQL layer 102 is the entry point for external database services, and is usually responsible for SQL (Structured Query Language) parsing and execution, transaction scheduling, query request distribution, etc.

[0033] The branch metadata management layer 104 provides metadata for the database branch processing system 100. It is used to record the branch identification (ID) of the currently active branch, maintain the branch tree structure (record the parent-child relationship between branches), and record the branching_lsn of each branch. Specifically, the LSN recorded when performing the branching operation serves as the baseline point of the branch to distinguish between the baseline data source and the incremental data in the incremental file (all data with LSN <= branching_lsn belongs to the shared baseline data before the branch was created, and all data with LSN > branching_lsn belongs to the incremental data unique to each branch).

[0034] The shared buffer pool 106 is the core of database memory access. In this embodiment, a single-active branch optimization design is adopted: the shared buffer pool 106 only caches the data pages of the currently active branch, thereby avoiding the cache conflict problem caused by multiple branches coexisting.

[0035] The storage management layer 108 is used to route data access requests and control the read and write paths of data pages based on the context information of the currently active branch.

[0036] The baseline data source 110 stores baseline data shared by all branches; the incremental file 112 stores incremental data unique to the corresponding database branch (corresponding to the data pages that have changed under the corresponding database branch).

[0037] See Figure 2 , Figure 2 A flowchart of a database branching method according to an embodiment of this specification is shown, specifically including the following steps 202-208.

[0038] Step 202: In response to the branch creation request for the target database, write the first memory data page in the shared buffer pool to disk and set the existing data pages on disk to read-only state to obtain the baseline data source, wherein the baseline data source is used to characterize the database state before the branch is created.

[0039] In this embodiment, the database branching system can receive a branch creation request initiated by a user for a target database, thereby triggering the database branch creation process. The branch creation request can be a BRANCH command in the database branching system, a branch creation interface call, or other operation requests for creating a database branch. Through the branch creation request, multiple logically independent database branches can be created based on the current database state.

[0040] The first memory data page is a data page that was modified before the branch was created but not written to disk (i.e., a dirty page).

[0041] It should be noted that, in the embodiments of this application, during the operation of the target database, some frequently accessed or prefetched data pages will be cached in the shared buffer pool in memory to reduce disk I / O overhead and improve data access efficiency.

[0042] Reading mechanism: When a read request for a specific data page in the target database is received, if the data page is already in the shared buffer pool, the database branch processing system will directly read the data page from memory.

[0043] Write mechanism: When a write request for a specific data page in the target database is received, if the data page is already in the shared buffer pool, an in-place update is performed directly in memory, and it is marked as the first in-memory data page (i.e., a dirty page). If the data page is not in the shared buffer pool, the target data page is first loaded into the buffer pool before modification. During the modification process, the system synchronously records the corresponding change operation in the Write-Ahead Log (WAL) and generates a unique Log Sequence Number (LSN).

[0044] Normally, the database branch processing system will start the dirty flushing process when preset trigger conditions are met (e.g., the proportion of the first memory data page in the buffer pool reaches a preset threshold, or the preset checkpoint cycle is reached), to synchronize the changes in memory to the disk.

[0045] Because database data pages typically use an in-place update mechanism, and the shared buffer pool only maintains the latest version of the data pages, if a forced flush is not performed when a branch is created, dirty pages in the buffer pool that have not yet been written to disk (i.e., modifications made before the branch was created) may be directly overwritten by subsequent write operations on the current branch after the branch is created, resulting in the permanent loss of the historical version at the branch's baseline.

[0046] Based on this, in this embodiment of the application, in response to a branch creation request for the target database, the database branch processing system will perform a forced flushing operation.

[0047] Specifically, flushing dirty data refers to the process of writing the first memory data page (i.e., the dirty page) that has been modified but not yet synchronized to disk in the shared buffer pool back to the disk storage medium. Under normal database operation, this process can be triggered by the background checkpointer process at a preset period, or by the background write process based on memory pressure.

[0048] In the branch creation scenario of this application embodiment, by actively calling or simulating the triggering logic of the checkpoint process, all first memory data pages before the current branch is created are forcibly synchronized to the disk, thereby ensuring that the existing data on the disk achieves a dual consistency state of physical and logical consistency.

[0049] This dirty flushing mechanism, driven by branch creation requests, essentially forces a data consistency breakpoint at the critical point of branch switching. It ensures that the baseline data source on disk can fully contain all changes prior to the time of the branch request, fixing them at the log sequence number (LSN) position closest to the request time.

[0050] In this embodiment of the application, "existing data pages on the disk" refers to the full physical data files belonging to the original database (or parent branch / target database) that are stored on the disk at the moment the branch creation request is triggered.

[0051] Through the aforementioned forced flushing operation, these existing data pages are solidified into a set of baseline data sources representing the complete and consistent data state at the time of branch creation. This baseline data source serves as the common starting data for all subsequent derived branches.

[0052] In this embodiment, the baseline data source refers to the set of physical data stored on the disk at a specific point in time when the branch creation request is executed in the target database. This baseline data source includes all data pages synchronized via the forced flushing operation, which can completely and accurately characterize the physical state of the database at the moment of branch creation. From a physical storage distribution perspective, the baseline data source consists of the original database data files and system files, and by setting the access permissions of these files to read-only, the physical state is "solidified" at the instant of branch creation.

[0053] In a tiered storage architecture, the baseline data source serves as the common data foundation for multiple database branches. Once the branch creation process is complete, the baseline data source is locked in a read-only state. This feature offers the following technical benefits: Physical anti-overwrite (immutability): By setting it to read-only, it forcibly ensures that the original data pages are not updated in-place by any subsequent write operations (including write requests from the parent branch or child branches). This "state freeze" mechanism ensures that no matter how the data evolves, the baseline data source can completely restore the original physical data at the time of branch creation, thus providing each branch with a physical starting point that has deterministic and version traceability capabilities.

[0054] Logical zero-copy (global sharing): Because the baseline data remains static, multiple parallel database branches (such as the first database branch, the second database branch, etc.) can simultaneously mount and access the same physical file through the storage manager. This mechanism eliminates the need for a full physical copy for each branch, decoupling the branch creation overhead from the total data volume.

[0055] The baseline data source provides a unified addressing reference for subsequently generated incremental files. In this scheme, all change operations after branch creation are redirected to the associated incremental file, while data pages that have not been changed are always retrieved from the baseline data source. Based on this, the system constructs clear data spatiotemporal boundaries: Baseline layer (existing data): Represents the static data state before the branch is created, and is stored in the baseline data source; Incremental layer (change data): Represents the dynamic evolution data after the branch is created, and is stored in the incremental file.

[0056] In this embodiment, by converting existing disk data into a read-only baseline data source, the technical shortcomings of "long creation time" and "storage space redundancy" caused by full copying in traditional database branch management are solved. This design achieves "logical zero copy" through "physical read-only," allowing the system to support multiple independent database branches that do not interfere with each other and evolve independently while sharing the same baseline data source, based on the maintenance of lightweight incremental files, thus improving the management efficiency of database resources in a cloud-native environment.

[0057] Step 204: Determine the first branch identifier of the first database branch to be created, establish the association between the first branch identifier and the baseline data source, and create the first database branch.

[0058] In this embodiment, the database branch is not a physical clone of the physical data file on the disk, but a logical entity (logical existence) based on metadata definition.

[0059] The process of creating the first database branch is essentially registering the branch's logical description information in the branch metadata management layer.

[0060] Specifically, in response to a branch creation request for the target database, the system first obtains and records the current system log sequence number as the branch baseline log sequence number (branching_lsn).

[0061] The `branching_lsn` acts as the logical delimiter for the corresponding branch. The system dynamically determines the data read path by comparing the log sequence number (LSN) of the data page with the `branching_lsn`. If the LSN of the data page is less than or equal to branching_lsn, the system determines that the data belongs to the existing state before the branch was created and extracts the data from the read-only baseline data source. If the LSN of a data page is greater than branching_lsn, it is determined that the page belongs to the changed state after the branch was created, and the data is extracted from the corresponding private incremental file.

[0062] Furthermore, the first branch identifier is registered and maintained in a tree-like topology (branch tree). This structure records the evolution path and inheritance relationship between branches by establishing parent-child node indexes. With the help of this topology, the first database branch can achieve logical inheritance of the baseline data source; at the same time, by associating independent incremental file storage spaces, it ensures strong physical isolation and independent evolution of each branch's change operations.

[0063] By completing the registration of branch identifiers, binding of branching_lsn, and updating of the branch tree structure at the branch metadata management layer, the system only needs to process lightweight metadata information to complete the logical creation of the first database branch.

[0064] This process avoids the bottleneck of large-scale data replication in traditional solutions, ensures the atomicity and instantaneity of branch creation operations, and provides architectural guarantees for achieving efficient concurrent management of multiple branches.

[0065] Step 206: Create a first incremental file corresponding to the first branch identifier, wherein the first incremental file is used to store the first incremental data page, and the first incremental data page is the data page that has changed under the first database branch.

[0066] In the embodiments of this application, the first incremental data page refers to a data page whose data changes after the creation of the first database branch and during the operation of the first database branch. That is, data pages whose data changes after the creation of the first database branch and during the operation of the first database branch are all referred to as first incremental data pages in the embodiments of this specification. It may be consistent with the original data page in the baseline data source in terms of logical page number (Page ID), but there are differences in data content and log sequence number (LSN).

[0067] Specifically, in this embodiment, any data page generated by an operation whose LSN is greater than the branching log sequence number (branching_lsn) of the first database branch and belongs to the first database branch is defined as a first incremental data page. It represents the incremental change of that branch relative to the baseline data source.

[0068] The first incremental file is a separate physical file specifically created for the first database branch and used to persistently store the first incremental data pages.

[0069] The first incremental file is completely isolated from the baseline data source and incremental files of other branches (such as the second database branch) at the physical storage level. This isolation ensures that when different branches perform write operations, the data is routed to their own dedicated storage areas, fundamentally avoiding the data overwrite risk caused by in-place updates.

[0070] The first incremental file only records the changed data of the first database branch itself. Data pages that have not been modified within that branch are not redundantly stored in the first incremental file. This incremental storage strategy ensures that the physical space occupied by a branch is only related to the actual amount of changes that have occurred, thereby significantly reducing the total storage overhead in multi-branch scenarios.

[0071] During the execution of the first database branch, the storage manager, through the coordinated scheduling of the first incremental file and the baseline data source, presents the complete data state of that branch: Write path: When the first database branch performs a write operation and modifies a data page, the storage management layer intercepts the write based on the currently active branch context. Specifically, the storage management layer prohibits the modified data page (i.e., the first incremental data page) from being written back to the original physical offset in the baseline data source, and instead redirects the write request to the first incremental file associated with the first branch identifier.

[0072] Read path (priority retrieval mechanism): When the system receives a read request for the first database branch, the storage manager adopts an "incremental priority" retrieval strategy, that is, it prioritizes the retrieval of the first incremental file.

[0073] If the target data page exists in the first incremental file (i.e., the target page has been changed), the data is extracted and returned directly from that incremental file; If the target data page is not found in the first incremental file, the storage manager will automatically fall back to the baseline data source to retrieve the corresponding original data page.

[0074] This direct addressing mechanism based on "incremental priority" ensures high timeliness of branch data access. Since the first incremental file stores complete, ready data pages, the system can obtain the latest data status of the branch without complex log backtracking or version merge calculations during reading.

[0075] It should be noted that in the hierarchical storage architecture, the timing of the creation of the first incremental file is flexible and can be adjusted according to the system's resource allocation strategy requirements: the system can either create the first incremental file synchronously when responding to a branch creation request to achieve immediate reservation of physical space, or adopt a delayed creation strategy, that is, only register metadata in the early stage of branch creation, and then dynamically create the corresponding first incremental file when the branch first generates a write operation and triggers data redirection.

[0076] In this embodiment, the branch creation request for the target database is used to request the establishment of a specific database branch based on the original target database. When the database branch processing system receives this request, the system creates a first database branch (the corresponding branch identifier can be set to b1) and simultaneously confirms or forms a second database branch (the corresponding branch identifier can be set to b0). The first and second database branches are logically parallel branches with the same source. The second database branch (b0) is typically used to inherit and continue the original logical link of the target database before the branch creation time (such as mainline evolution), while the first database branch (b1) serves as an independent development or testing link derived from the state at that time. Both share the same baseline data source. Simultaneously, to ensure that data evolution does not interfere with each other, the first and second database branches are each configured with their own independent incremental files. This architecture ensures that the two branches have completely consistent data states when reading existing data, and when performing write operations, their respective changes are recorded in their own incremental files, thereby achieving strong isolation at the physical storage level.

[0077] To facilitate a concise description and logical consistency of the technical solution, the embodiments in this application will be described in detail using the first database branch (b1) as an example. It should be understood that the creation, metadata registration, and data processing of the second database branch (b0) can all refer to the relevant description of the first database branch.

[0078] For example, Figure 3 This is a flowchart illustrating a database branch creation process according to one embodiment of this specification. Specifically, it includes the following steps 302-316.

[0079] Step 302: Receive a branch creation request for the target database.

[0080] Step 304: Check if the first memory data page exists in the shared buffer pool.

[0081] Upon receiving a branch creation request for the target database, the database branch processing system can check if a first memory data page exists in the shared buffer. This first memory data page is a data page that has been modified before the branch creation but has not been written to disk.

[0082] Since some data pages in the database may only exist in the shared buffer pool, creating a database branch directly based on existing data pages on the disk may result in the data state on the disk being inconsistent with the current actual state of the database. Therefore, it is necessary to check the first memory data page in the shared buffer pool before creating a database branch.

[0083] Step 306: Perform a checkpoint operation to write the first memory data page to disk.

[0084] If a first memory data page is detected in the shared buffer pool, the database system can perform a checkpoint operation to write the first memory data page in the shared buffer pool to disk.

[0085] In this embodiment of the application, by performing a checkpoint operation before the database branch is created, it can be ensured that the existing data on the disk is consistent with the current state of the database, thereby ensuring that the subsequently generated baseline data source can accurately represent the database state corresponding to the time of database branch creation, and avoiding data inconsistency problems caused by the existence of unwritten data in the shared buffer pool.

[0086] Step 308: Record the current LSN as the branch baseline log sequence number.

[0087] After the checkpoint operation is completed, the database system can record the log sequence number (LSN) corresponding to the current write-ahead log (WAL) and use this LSN as the branching base log sequence number (branching_lsn).

[0088] The branch baseline log sequence number is used to characterize the log position corresponding to the time of database branch creation, and to distinguish between the shared baseline data (data in the baseline data source) before the database branch is created and the incremental data (data in the incremental file) after the database branch is created.

[0089] Step 310: Set the existing data pages on the disk to read-only mode to obtain the baseline data source.

[0090] After recording the branch baseline log sequence number, the database system can set the existing data pages on the disk to read-only status in order to obtain the baseline data source.

[0091] In this embodiment, the baseline data source is used to characterize the database state before the database branch is created and serves as the data foundation shared by multiple database branches. Furthermore, after obtaining the baseline data source, subsequent data modifications to the database branch are no longer directly written to the baseline data source, but rather written to the incremental file of the corresponding database branch.

[0092] Step 312: Create the first incremental file for the first database branch and the second incremental file for the second database branch.

[0093] It should be noted that in this embodiment, a single branch creation request triggers the generation of two peer branches (first database branch b1 and second database branch b0). The original target database is "frozen" as a read-only baseline data source at the branch point. To ensure that the original task mainline can continue to evolve without interference from the new branch, the system automatically generates the second database branch (b0) to inherit and continue the logical link of the original database; simultaneously, it generates the first database branch (b1) as an independent derived link. By synchronously creating independent incremental files for the two branches, it ensures that both the "continuing the mainline" operation and the "new branch development" operation follow a unified "baseline + incremental" access mechanism. This symmetrical design simplifies the routing logic of the storage management layer, enabling the mainline and derived branches to have completely consistent isolation and access performance at the physical level.

[0094] Step 314: Construct a branch tree based on the first branch identifier of the first database branch and the second branch identifier of the second database branch, and establish the association relationship between the first branch identifier, the second branch identifier and the baseline data source and the branch baseline log sequence number respectively.

[0095] After creating a database branch, the database system can construct a branch tree based on the first branch identifier corresponding to the first database branch and the second branch identifier corresponding to the second database branch, and establish the association between the first branch identifier, the second branch identifier and the baseline data source and the branch baseline log sequence number respectively.

[0096] Among them, the branch tree is used to represent the inheritance relationship between multiple database branches, the branch identifier is used to uniquely identify the corresponding database branch, and the branch baseline log sequence number is used to represent the creation time of the corresponding database branch.

[0097] Furthermore, by establishing a relationship between branch identifiers and baseline data sources, subsequent data access, branch switching, and branch management for database branches can be supported.

[0098] After completing the construction of the branch tree and establishing the relevant relationships, the database branch creation is completed.

[0099] In subsequent data access for database branches, the database system can process the database branch based on the baseline data source and incremental files associated with the corresponding database branch.

[0100] Step 208: Process the database tasks under the first database branch based on the baseline data source and the first incremental file.

[0101] In the embodiments of this specification, the database task refers to all physical and logical operations that require access to, status determination of or status change of the database's memory buffer (such as a shared buffer pool), disk storage space (such as a baseline data source or incremental file), or routing addressing structure (such as navigation information) during the operation cycle of the target database. These operations include, but are not limited to, data read operations, data write operations, data delete operations, database branch delete operations, database branch merge operations, and data migration operations.

[0102] In practical applications, there are various storage formats for incremental files, which can be flexibly configured according to specific application scenarios and system performance requirements. This specification does not impose any restrictions on this.

[0103] In one optional implementation of this embodiment, the first incremental file adopts a self-describing single-layer storage structure, which includes only a first data area. This first data area consists of multiple consecutively arranged self-describing data pages, each containing a data header and a data payload.

[0104] Data header: directly embedded at the beginning of each data page, used to store the first incremental data page identifier (such as logical page number Page ID) of that data page.

[0105] Data payload: Following the header, it stores the specific changes to the first incremental data page.

[0106] Based on the baseline data source and the first incremental file containing only the first data area, the system processes the database tasks under the first database branch through streaming scanning or appending to the end: When the first database branch changes, the storage manager directly encapsulates the logical page number in the data page header and appends it sequentially to the end of the first incremental file. This approach reduces the overhead of maintaining metadata for independent navigation areas.

[0107] When the system receives a read request, the storage manager scans the headers of data pages in the first incremental file from back to front to match the target page identifier. If a match is found, the page payload is read directly; if no match is found at the beginning of the file, the system falls back to the baseline data source for reading.

[0108] In another optional embodiment of this example, the first incremental file includes a first navigation area and a first data area. The first data area is used to store the first incremental data page, and the first navigation area is used to store the first navigation information. The first navigation information is used to indicate the first incremental data page identifier of the first incremental data page and the first position information of the first incremental data page in the first data area. Based on the baseline data source and the first incremental file, the database tasks under the first database branch are processed, including: Based on the baseline data source, as well as the first navigation area and the first data area, the database tasks under the first database branch are processed.

[0109] The first incremental file is logically divided into a first navigation area and a first data area. Through the coordinated operation of the two, a non-in-place update storage mechanism for the first database branch is implemented. First data area (storage medium): Used to persistently store the complete change page (i.e., the first incremental data page) generated by the first database branch. This area carries data through sequential appending or slot allocation, avoiding direct rewriting of the baseline data source.

[0110] First navigation area (index logic): Used to store first navigation information. This information establishes a mapping between changed pages and their physical storage locations, specifically including: The first incremental data page identifier can be the logical page number (Page ID) of the data page, used to identify the specific logical page modified by this branch.

[0111] The first location information corresponds to the physical offset or slot number of the first incremental data page in the first data area, and is used to indicate the physical access path of the data.

[0112] In this embodiment, based on the baseline data source, the first navigation area, and the first data area, the system can achieve efficient access scheduling for the first database branch: When the system receives an access request for the first database branch, it first searches the first navigation area. If a target page identifier exists in the navigation information, the latest changed page is directly extracted from the first data area based on the corresponding first location information.

[0113] Baseline rollback: If the target page identifier is not recorded in the first navigation area, the system will automatically roll back to the baseline data source to obtain the corresponding original stock data.

[0114] The storage distribution of the first data area is highly flexible and has high resource utilization. Specifically, the data area can be pre-allocated, dynamically allocated, or constructed based on a sparse strategy.

[0115] For example, the system allocates physical storage slots in the data area only when a write operation occurs in the first database branch and the first data page is generated, rather than pre-occupying the entire space.

[0116] The representation of the first data area includes, but is not limited to: Contiguous page space: Multiple fixed-size (e.g., 8KB) data pages are arranged contiguously on the disk to create a physical cluster similar to a miniature data file, in order to optimize sequential access performance.

[0117] Sparse Blocks: Utilizing the sparse file feature of the file system, the logical offset of the file points to a large space, but physically only occupies a disk block at the offset position where the actual data is written, thereby achieving extremely low storage redundancy.

[0118] For example, Figure 4 This is a schematic diagram of an incremental file storage format provided in one embodiment of this specification.

[0119] like Figure 4 As shown, an incremental file includes a file header, an index area, and a data area.

[0120] File Header: The file header is used to store the attribute information of the incremental file.

[0121] Specifically, magic_number (4B): a magic identifier used to identify the file format. The system first verifies this value when loading a file to ensure that the file is indeed an incremental file defined in this database, preventing misreading.

[0122] version (4B): Version number, recording the current incremental file structure version. When the system storage architecture is upgraded, backward compatibility is achieved through the version number.

[0123] branch_id (4B): Branch identifier, which specifies which logical branch the incremental file belongs to. This is the core basis for the storage manager to perform branch isolation.

[0124] page_count (4B): A page counter that records the total number of data pages written to the current incremental file. Since incremental files are written sequentially, this value also indicates the end of the data area.

[0125] branching_lsn (8B): Branch baseline log sequence number, recording the global LSN at the time the branch was created. It defines the logical boundary between the baseline data source and the current incremental file.

[0126] Reserved: Reserved space for alignment. This ensures that subsequent PageIndex areas can start from specific physical alignment boundaries, optimizing disk I / O performance.

[0127] The index area is used to store all logical page numbers that have been modified under this branch.

[0128] The index area is fixed at 2MB (can be set manually), and each entry has an 8B (64-bit) page_id.

[0129] The index area and the data area are strictly one-to-one in storage order, and it serves as the search directory for incremental files.

[0130] Index cap: Based on the calculation result of 2MB / 8B=262,144, a single incremental file can support changes to approximately 260,000 pages. This design ensures efficient single-file retrieval while accommodating the change scale of the vast majority of development branches.

[0131] The data area is the physical area that actually stores the changed content, and it consists of multiple physical pages (page_data) arranged in a continuous sequence with a fixed size of 8KB.

[0132] Unlike the baseline data source, the data area is not stored in order of PageID, but is appended in the order in which changes occur.

[0133] In addition, the size of the data area expands dynamically. As page_count increases, the file length also increases, with an upper limit of approximately 2GB.

[0134] The position of page_id[i] in the index area determines the physical offset of the corresponding page_data[i] in the data area. They are bound together by the same index i.

[0135] For example, suppose that the incremental file has stored 3 change pages at this time, with logical page numbers 100, 500 and 20 respectively.

[0136] PageIndex status: page_100[0], page_500[1], page_20[2]: PageData status: Slot 0 stores 8KB of data from page_100; Slot 1 stores 8KB of data for page_500; Slot 2 stores 8KB of data from page_20.

[0137] The process of reading page number page_500: The storage management layer searches the index area of ​​the incremental file and finds that page_500 appears at index i=1.

[0138] Address calculation: Skip the file header and the 2MB index area, then offset forward by 1×8KB.

[0139] Direct Read: The system directly locates and retrieves the 8KB of data at that position and returns it without accessing the baseline data source.

[0140] In the embodiments described in this specification, an incremental file format consisting of a first navigation area and a first data area is designed to achieve efficient storage and fast indexing of branch change data. Specifically, since the first data area is dedicated to persistently storing complete change pages generated by the first database branch, this "incremental independent storage" architecture ensures that when performing write operations, the system can directly convert logical page changes into physical writes to the first incremental file, avoiding rewriting contention of the baseline data source and reducing write latency during branch evolution. Simultaneously, by centrally maintaining the mapping relationship between the first incremental data page identifier and the first location information in the first navigation area, the system can achieve direct location of the changed data page with minimal metadata overhead, avoiding linear backtracking of log files or complex version merging calculations in traditional solutions, thus improving the timeliness of branch data reading.

[0141] In practical applications, there are multiple ways to process database tasks under the first database branch based on the benchmark data source, the first navigation area, and the first data area. The specific method can be selected according to actual needs, and the embodiments in this specification do not limit this.

[0142] In one optional implementation of this embodiment, the process of processing database tasks under the first database branch based on the baseline data source, the first navigation area, and the first data area can also be achieved through linear traversal comparison: The system reads the first navigation information from the first navigation area and constructs a sequentially arranged index list (such as an array or linked list) in memory. Each index unit sequentially stores the first incremental data page identifier and the corresponding first position information.

[0143] When the system receives a read request for the first database branch, it performs a full traversal matching. Starting from the beginning of the index list, the system extracts the data page identifiers from each index cell and compares them with the page identifier to be read.

[0144] If the match is successful, the first location information of the cell record is extracted, and the data is read from the first data area. If no matching identifier is found after traversing the entire index list, it is determined that the page has not been modified, and the process is reversed to read from the baseline data source.

[0145] In another optional implementation of this embodiment, based on the baseline data source, and the first navigation area and the first data area, the database tasks under the first database branch are processed, including: Based on the first navigation information, a first index unit is constructed, wherein the first index unit is used to store the first incremental data page identifier of the corresponding first incremental data page, and the first position information of the corresponding first incremental data page in the first data area; Based on the baseline data source, the first index unit, the set position conversion relationship, and the first data area, the database tasks under the first database branch are processed. The position conversion relationship is used to map the position information of the corresponding index storage unit according to the data page identifier.

[0146] When the first database branch is activated or first accessed, the storage management layer first scans the first index area (Page Index) in the first incremental file. This area records the identifiers (Page IDs) of all modified data pages in that branch and their logical writing order. The system traverses the records in the first index area, constructing a hash table in memory consisting of multiple first index units. From a data structure perspective, each first index unit is essentially a key-value pair: Key: Stores the identifier of the first incremental data page (such as the logical page number Page ID), which serves as the unique basis for retrieval; Value: Stores the first location information corresponding to this data page.

[0147] For example: If the system reads the first index area and finds that page_id[0] is 100 and page_id[1] is 500, then store in the memory hash table: Unit A (Key: 100, Value: 0) and Unit B (Key: 500, Value: 1).

[0148] Here, Value corresponds to its physical slot number in the first data area. Through this structured mapping, the system transforms the first index area, which originally required linear scanning, into a memory addressing layer with O(1) lookup efficiency, laying the foundation for extremely fast response of random reads.

[0149] The defined location conversion relationship refers to the algorithmic logic for converting the logical page identifier into the storage address of the first index unit in memory. In this embodiment, it is preferentially implemented through a hash mapping mechanism.

[0150] When the system receives a request to read changed data for the first database branch, the specific addressing and routing logic is as follows: First, the system uses the position conversion relationship (i.e., the preset hash function) to perform a mapping operation on the target data page identifier, and directly calculates the precise storage address (such as slot[i]) of the first index unit corresponding to the target data page in the memory hash table.

[0151] Subsequently, the system accesses the storage address and extracts the key value therein for identifier matching and validity verification: it checks whether the Key stored in slot[i] is completely consistent with the identifier of the page to be read. If a hash collision occurs (i.e., the slot currently stores the identifier of another data page), it continues to probe and search along the corresponding collision chain according to the preset collision resolution method (e.g., chaining or open addressing) until the corresponding target first index unit is accurately matched. Compared with traditional tree index or linear scan, this position conversion relationship completely eliminates multiple additional disk I / O overheads or complex tree traversal time consumption, and realizes the extreme simplification of the data page addressing path. Once the identifier is successfully matched, the system extracts the record's Value (i.e., physical slot index) from the target first index unit and performs physical jump and address conversion. The storage management layer calculates the physical offset of the target data page in the first incremental file according to the following physical offset formula (1): Formula (1) in, This indicates the header size of the first incremental file. This represents the total size of the first index area. This indicates the physical size of a standard data page (optionally 8KB in this embodiment).

[0152] For example: Suppose we are reading Page_100. The system uses a hash operation to locate slot[i], confirms that key=100, and reads its value as 0. Based on this, the storage management layer skips the file header and index area, directly locates the corresponding offset in the disk file, and extracts the 8KB changed data page through a single physical I / O (Input / Output).

[0153] It should be noted that if a tree-like topology is used for hierarchical addressing, the time complexity of the lookup is logarithmically related to the number of changed data pages N within the branch. When a branch experiences high-frequency concurrent writes and the size of the changed data pages N is large, the number of memory pointer hierarchical jumps and condition comparisons in the tree-like retrieval mechanism also increases continuously. However, the embodiment in this specification uses a hash mapping structure customized for branches, which directly maps the identifier of the target data page to be written to the hash function. Its addressing lookup time complexity remains constant at O(1) under the full changed data size. This means that no matter how the size of the changed data pages N under the first database branch grows (for example, from accumulating from the 10^2 level to the 10^6 level), the number of pointer jumps required for memory to perform addressing routing remains constant at 1, thereby achieving a leap in addressing efficiency of tens of times and fundamentally eliminating the memory-level retrieval throughput bottleneck of branch read / write routing.

[0154] In the embodiments described in this specification, by extracting the first navigation information into the first index unit in memory, the system realizes the transformation from "physical storage layout" to "logical structured index". This design eliminates the need for random read operations on the first database branch to rely on linear scanning of the first index area. Instead, it achieves instantaneous positioning with O(1) time complexity through position transformation, greatly shortening the data addressing path. At the same time, this scheme maintains the association between page identifier and physical offset, ensuring that under the complex "baseline + incremental" hierarchical architecture, the system can quickly determine whether the target data exists in the incremental file. This ensures that while the database branch has the ability to create in seconds, it still has read and write efficiency close to that of the native database, improving the system response speed and resource scheduling efficiency in large-scale concurrent branch scenarios.

[0155] In one optional implementation of this embodiment, the database task includes a branch data reading task; Based on the baseline data source, as well as the first navigation area and the first data area, the database tasks under the first database branch are processed, including: In response to a read request for the first data page to be read, determine whether a first data page identifier for the first data page to be read exists in the first navigation information; If the first navigation information contains a first data page identifier, then based on the first navigation information, the reading position information of the first data page is determined, and based on the reading position information of the first data page, the first data page is read from the first data area. If the first data page identifier is not present in the first navigation information, then the first data page to be read is read from the baseline data source.

[0156] In this embodiment of the application, for data read requests from the first database branch (active branch), the system follows the principle of "memory priority, incremental priority, and baseline fallback," and the specific process is as follows: When the system receives a read request for the first data page (Page X) of the first database branch, it first searches the shared buffer. If a match is found, the page data in memory is returned directly; if a miss occurs (a buffer miss happens), the disk read process is triggered.

[0157] In the event of a buffer miss, the system needs to determine if the page exists in the incremental file of the current branch. At this point, the system retrieves the first navigation information associated with the first database branch: If the identifier of the data page to be read exists in the first navigation information, it means that the page has been modified after the branch was created, and the latest version needs to be obtained from the incremental file. Specifically, based on the read location information determined by the navigation information (such as slot index i), the system locates the first data area of ​​the first incremental file and accurately reads the changed data page.

[0158] If this identifier is not present in the navigation information, it means that the page has not changed since the branch was created and must be obtained from the original baseline data source. Specifically, the system calculates the offset based on the original page number and reads the initial data of that page from the baseline data source.

[0159] In practical applications, there are multiple ways to determine whether there is a first data page identifier for the first data page to be read in the first navigation information. The specific method can be selected according to the implementation requirements. This specification does not limit this method in the embodiments.

[0160] Example A: Linear scan scheme (one-to-one comparison).

[0161] When a branch is activated, the system traverses the first index area in O(N) complexity, comparing the identifier of PageX with the data page identifiers stored in the first index area one by one.

[0162] This approach is simple to implement, requires no additional hash space overhead, and is suitable for scenarios with very small branch changes.

[0163] Example B: Hash mapping scheme (high-performance addressing).

[0164] The system builds a hash table in memory.

[0165] Search process: The system performs a hash operation on PageX to directly locate the specific slot[i]. Slot index i is found. The system calculates the physical offset (e.g., i × 8KB) and extracts data from the incremental file testb1. If not found: The system jumps directly to the baseline data source and addresses it using the original page_id × 8KB.

[0166] For example, Figure 5 This is a data reading flowchart provided by one embodiment of this specification. Specifically, it includes the following steps 502 to 518.

[0167] Step 502: Receive the read request for the first data page to be read.

[0168] This process is triggered when the system is running under the first database branch (the currently active branch) and an upper-layer application initiates access to a specific logical page identifier (such as Page X).

[0169] Step 504: Determine whether the first page of data to be read is cached in the shared buffer pool.

[0170] If the first data page to be read is cached in the shared buffer pool, proceed to step 506; if the first data page to be read is not cached in the shared buffer pool, proceed to step 508.

[0171] The system first searches the shared buffer pool. If the target page is already cached in memory, it returns directly to avoid disk I / O overhead. If the page is not found in the buffer pool (i.e., a buffer miss occurs), the system proceeds to the disk tiered read phase.

[0172] Step 506: Directly read the cached data in the shared buffer pool.

[0173] Step 508: Use a hash function to find the identifier of the first data page to be read in the memory hash table.

[0174] If a first index unit is matched, proceed to steps 510-512; if no first index unit is matched, proceed to steps 514-516.

[0175] The system uses a hash function to perform an instantaneous lookup in the in-memory hash table based on the identifier of the data page to be read, with a complexity of O(1). Hit status: If the corresponding first index unit is found (i.e., the corresponding slot[i] exists and the PageID matches), it means that the data page has changed since the branch was created.

[0176] Missed state: If the corresponding first index unit is not found, it means that the page has not been modified under the current branch, and the data still retains the initial state at the time of branch creation.

[0177] Step 510: Extract the first position information and calculate the offset of the first data area.

[0178] Step 512: Read the first data page to be read from the first incremental file according to the first data area offset.

[0179] Step 514: Calculate the baseline data source offset based on the identifier of the first data page to be read.

[0180] If the index is matched, the system extracts the first location information (i.e., slot index i) from the first index unit. The storage management layer calculates the offset of the page on the disk (e.g., i×8KB plus Header_Size and PageIndex_Size) based on the physical layout of the incremental file, and accurately extracts the latest changed page data from the first data area of ​​the first incremental file.

[0181] Step 516: Read the first data page to be read from the reference data source based on the reference data source offset.

[0182] If the index is not found, the storage management layer calculates the offset based on the original data page identifier and directly locates the baseline data source to obtain the existing data of that page.

[0183] Step 518: Return to the first page of data to be read.

[0184] The storage management layer returns the page data it retrieves (whether from incremental files or baseline data sources) to the application layer via a shared buffer pool. This mechanism ensures that the system can synthesize the complete and up-to-date data state of the first database branch at the current moment with extremely low latency.

[0185] In the embodiments described in this specification, by combining "incremental priority" with "baseline fallback," the system can instantly identify the distribution status of data pages with extremely low time overhead. When the navigation information is matched, the system can directly locate the first data area to extract the latest page image, avoiding the complex log replay or version merging calculations required in traditional branch management. When the navigation information is not matched, the system accesses the baseline data source through a transparent rollback mechanism, ensuring the integrity of the data view. This mechanism improves the data access throughput and response speed under the hierarchical storage architecture while ensuring physical isolation between branches.

[0186] In one optional implementation of this embodiment, the database task includes a data writing task; Based on the baseline data source, as well as the first navigation area and the first data area, the database tasks under the first database branch are processed, including: In response to a write request for the target data page to be written, determine whether the target data page identifier exists in the first navigation information; If the first navigation information contains a target data page identifier, then based on the first navigation information, the write location information of the target data page is determined, and based on the write location information, the data of the target data page identifier is written to the corresponding storage location of the first data area. If the target data page identifier is not present in the first navigation information, the read-only state of the reference data source remains unchanged. A target storage location is allocated for the target data page in the first data area, the target data page is written to the target storage location, and the target data page identifier and the location information of the target storage location are added to the first navigation information.

[0187] In the embodiments described in this specification, when the system receives a write request for the target data page (Page X) to be written to the first database branch (active branch), the storage management layer follows the principle of "baseline read-only, incremental append, and dynamic index update", and the specific process is as follows: The system first obtains the identifier (Page ID) of the target data page to be written, and retrieves the first navigation information corresponding to the current branch to determine whether the page has been modified by this branch: If the target data page identifier exists in the first navigation information, it means that the page already exists in the incremental file. The system extracts the corresponding write position information (such as slot index i), calculates the physical offset, directly locates the corresponding position in the first data area of ​​the first incremental file, and performs in-place overwrite writing on the existing data page.

[0188] If the target data page identifier is not found in the first navigation information, it means that the page has been modified for the first time since the branch was created. The system allocates a new storage slot (e.g., slot N) at the end of the first data area. The data of PageX is written to the Data[N] position of the data area, and the first index area (PageIndex) of the disk incremental file is updated synchronously.

[0189] In practical applications, the operation of determining whether there is a target data page identifier in the first navigation information can adopt different comparison addressing schemes according to the system memory overhead requirements and access performance requirements. This specification does not limit this.

[0190] An alternative implementation is based on a one-to-one alignment scheme using linear scanning (O(n)).

[0191] In this implementation, the system achieves identifier matching by linearly traversing the first navigation information on the disk. Specifically, in response to a write request for the target data page to be written, the storage management layer sequentially reads the first navigation area (Page Index) of the first incremental file. The system compares the identifier of the page to be written (Page X) with each page identifier stored in the navigation area one by one. If a matching identifier is found during the traversal, the scanning stops, and the slot index i corresponding to the identifier is extracted, and an overwrite write is performed; if no matching item is found after traversing all the recorded identifiers in the navigation area, it is determined that the identifier does not exist, and a new slot is allocated at the end of the file and the record is appended. This scheme does not require maintaining a complex index structure in memory, which greatly saves system memory overhead and is suitable for application scenarios with limited memory resources or small amounts of data for branch changes.

[0192] In another alternative implementation, when the first database branch is activated or access is requested for the first time, the system translates the first navigation information on the disk into a high-performance addressing layer in memory.

[0193] First navigation information (PageIndex): This refers to the index data persistently stored in the first incremental file on disk. This area records, in the form of a linear array, the identifiers (Page IDs) of all modified data pages in this branch and their logical order in the file (i.e., slot index i).

[0194] Construction of the Hash Table: The system scans the first navigation area on the disk and extracts the Page ID and its corresponding logical location. The system traverses these records and constructs a hash table in memory consisting of at least one first index unit. If the system reads the first navigation area on the disk and finds that page_id[0] is 100 and page_id[1] is 500, then it stores in the memory hash table: Unit A (Key: 100, Value: 0) and Unit B (Key: 500, Value: 1).

[0195] In response to a write request for the target data page (Page X), the system first determines whether the page identifier already exists using a memory hash table: if the index already exists, it means the page has previously been allocated a position in the incremental file. The system extracts the slot index i from the memory index unit and, based on the calculated physical offset, overwrites the corresponding position in the first incremental file on disk with the new data. If the index does not exist, it means the page is being modified for the first time. The system allocates a new slot N at the end of the first incremental file on disk and performs the following synchronization operations: writes the data of Page X to Data[N] in the disk data area and writes X to PageIndex[N] in the disk navigation area; adds a new mapping (Key: X, Value: N) in the memory hash table to ensure visibility for subsequent reads.

[0196] For example, Figure 6 This is a data writing flowchart provided in one embodiment of this specification. Specifically, it includes the following steps 602 to 612.

[0197] Step 602: Receive a write request for the target data page to be written.

[0198] The system received a write request for the first database branch (the active branch). Because this solution uses a hierarchical storage architecture, all write operations are performed on the incremental files of the currently active branch, and in-place updates to the read-only baseline data source are strictly prohibited.

[0199] Step 604: Retrieve the page identifier in the memory index unit (i.e., the index unit, such as a Hashtable).

[0200] The storage management layer uses a predefined location transformation relationship (such as a hash function) to search in memory index units (such as a memory hash table) based on the identifier of the page to be written (Page X). This step aims to determine whether Page X has been modified within the current branch's lifecycle.

[0201] If the search page identifier is found in the memory index unit, proceed to step 606; if the search page identifier is not found in the memory index unit, proceed to steps 608 to 612.

[0202] Step 606: Overwrite the target data page to be written to the storage location corresponding to the incremental file data area.

[0203] If the memory index matches a page identifier and returns slot index i, it means that the page has been modified previously. The system directly extracts index i and calculates the physical offset. The latest data content is then overwritten to the corresponding offset address in the first data area of ​​the first incremental file.

[0204] Step 608: Allocate a new storage slot (Slot N) in the data area of ​​the incremental file and obtain the corresponding slot index (i).

[0205] Step 610: Write the target data page to be written to the incremental file data area Data[N], and synchronously update the incremental file index area PageIndex[N].

[0206] If the corresponding page identifier is not found in the memory index, it indicates that the page is the first modification after the branch was created. The system allocates a new free storage slot N at the end of the first data area of ​​the first incremental file. The data page content is written to the Data[N] position of the first data area of ​​the first incremental file, and the page identifier X is recorded in the PageIndex[N] of the disk navigation area.

[0207] Step 612: Add a new mapping (X->N) in the memory index unit.

[0208] For a page that has been modified for the first time, after the disk write is complete, the system needs to synchronously insert a new key-value pair (Key:X, Value:N) into the memory index unit (memory hash table). This step ensures that in subsequent read requests, the system can immediately locate the latest data page using the memory index.

[0209] In the embodiments of this specification, by responding to write requests and determining whether the target data page identifier exists in the first navigation information, the system can accurately distinguish between two scenarios: "overwriting existing changes" and "appending new pages." This ensures that all modifications to the first database branch are redirected to an independent first data area, avoiding in-situ destruction of the read-only baseline data source and guaranteeing the consistency and sharing of the baseline data. Secondly, for identifiers already existing in the navigation information, point-to-point updates are performed using the recorded write location information, effectively controlling the storage volume of incremental files and improving write positioning efficiency. For identifiers that do not exist, by dynamically allocating target storage locations in the first data area and synchronously updating the first navigation information, on-demand expansion and real-time indexing of branch data are achieved, ensuring data integrity and traceability of subsequent readings during branch evolution, and greatly reducing the complexity and resource overhead of data writing in a multi-branch environment.

[0210] In an optional implementation of this embodiment, before determining whether a target data page identifier for a target data page exists in the first navigation information, the method further includes: Obtain the write transaction identifier corresponding to the write request. The write transaction identifier is used to indicate the position of the write request in the write-ahead log, and the write transaction identifier has a monotonically increasing trend. The write-ahead log is used to record change operations in the target database. Determine whether the write transaction flag is greater than the first branch creation flag corresponding to the creation of the first database branch. The first branch creation flag is used to indicate the cutoff position of the write-ahead log at the time of creation of the first database branch. If the write transaction identifier is greater than the first branch creation identifier, then the step of determining whether the target write data page identifier exists in the first navigation information is executed.

[0211] In cloud-native databases, write transaction identifiers are typically represented by Log Sequence Numbers (LSNs) or transaction timestamps. It is a strictly monotonically increasing value that persists over time and with each operation, possessing globally unique temporal sequence characteristics.

[0212] Write-ahead logging (such as WAL in cloud-native databases) is a core mechanism in database systems used to ensure data durability and consistency. Any physical or logical changes to data (such as adding, deleting, or modifying data) must be recorded sequentially in the write-ahead log before being actually written to disk data pages.

[0213] First Branch Creation Sequence Number (Branching LSN): This refers to the current cutoff position (or latest log sequence number) of the target database's write-ahead log recorded by the system at the moment the first database branch is created. It represents the boundary between the new branch and the baseline data source. Write-ahead log entries with values ​​less than or equal to the first branch creation sequence number represent data changes that have been permanently saved to the historically shared baseline data source; while write-ahead log entries with values ​​greater than the first branch creation sequence number correspond to independent incremental changes generated after the first database branch was launched.

[0214] In the embodiments described in this specification, when the system receives any write request to the first database branch, the storage management layer does not immediately write the request to disk. Instead, it first retrieves the latest, monotonically increasing write transaction identifier corresponding to the request from the WAL log. The system then compares this identifier with the first branch creation identifier at the time the branch was initially created.

[0215] If the value is greater than 1, it means that the write operation occurred after the branch was created and is an independent change under the first database branch. In this case, the target data page to be written will be written into the first incremental file dedicated to the first database branch.

[0216] If the value is not greater than: it indicates that it is a historical transaction and does not belong to the independent lifecycle of this branch, so it will not be written to the first incremental file of the first database branch.

[0217] For example, the dummy target database has reached position 1000 of the write-ahead log (WAL). If a test branch (the first database branch) was created in the target database, the system records: First branch creation identifier = 1000.

[0218] If a user record is subsequently modified and a write request is triggered, the system will assign the latest write transaction identifier of 1025 to that request.

[0219] The storage management layer performs a numerical comparison: Since 1025 (write transaction identifier) ​​> 1000 (first branch creation identifier), the system confirms that this is an incremental change that occurred after the branch was launched, and then writes the corresponding data page to the first incremental file of the first database branch.

[0220] In the embodiments described in this specification, since the first branch creation identifier indicates the cutoff position of the pre-written log when the branch is created, the system can define the physical boundary between historical shared data (baseline data source) and branch-independent incremental changes. This not only effectively prevents the implicit contamination of newly launched branches by historical residual transactions, but also ensures that the first navigation information can accurately capture, and only capture, valid change page identifiers belonging to the current branch cycle, thereby providing reliable data support for database branch management.

[0221] In one optional implementation of this embodiment, the database task includes a branch deletion task; Based on the baseline data source and the first incremental file, the database tasks under the first database branch are processed, including: In response to a deletion request for the first database branch, the baseline data source remains unchanged, and it is determined whether the first database branch is currently the active branch; If the first database branch is not the currently active branch, then delete the first incremental file; If the first database branch is the currently active branch, then switch the currently active branch to another branch, and after the switch is completed, delete the first incremental file.

[0222] In the embodiments described in this specification, when the system receives a deletion request for the first database branch (b1), and it is verified that the branch is not currently active, the following process is executed: The system directly deletes all incremental files associated with this branch on the disk (such as testb1, testb1.1, etc.). Since these files only record incremental data specific to this branch, the deletion operation will not affect the shared baseline data source.

[0223] In addition, the metadata records for the branch (including the branch identifier, branching_lsn, and the corresponding node in the branch tree) need to be removed from the branch management layer (BranchManager).

[0224] If the system verifies that the first database branch (b1) is the currently active branch, it will adopt a protective strategy of switching first and then deleting: The system first forces a switch to another valid branch (such as the main branch or parent branch). This step is to disconnect the current process from the incremental file of that branch.

[0225] After the switch is complete, confirm that b1 is in an inactive state, and then follow the steps for the "inactive branch" described above to delete the physical incremental file and remove the metadata in BranchManager.

[0226] The baseline data source is unaffected during the deletion process.

[0227] In the embodiments of this specification, firstly, by determining whether the branch is currently active before deletion and performing branch switching when necessary, memory access conflicts or system crashes caused by deleting resources that are being used are effectively avoided, ensuring the security of branch lifecycle management; secondly, by performing deletion operations only on the first incremental file while keeping the baseline data source unaffected, precise cleanup of specific branch changes is achieved without damaging the shared data base, greatly reducing the storage space maintenance cost in multi-branch scenarios.

[0228] In one optional implementation of this embodiment, the database task includes a branch merging task; Based on the baseline data source and the first incremental file, the database tasks under the first database branch are processed, including: In response to a merge request for the first database branch, a second memory data page in the shared buffer pool is written to the first incremental file, wherein the second memory data page is a data page that has changed during the operation of the first database branch and has not been written to disk before the branch merge; Read the first incremental data page stored in the first incremental file and write the first incremental data page to the corresponding storage location in the baseline data source; Delete the first incremental file and restore the baseline data source to a writable state.

[0229] It should be noted that, in the embodiments of this application, when reading the first incremental data page stored in the first incremental file, the first incremental data page stored in the first incremental file includes both the data pages already stored in the first incremental file before the branch merge request is received, and the second memory data pages written into the first incremental file after the branch merge request is received.

[0230] In this embodiment of the application, when a user initiates a merge request for the first database branch (b1), intending to merge the development results of that branch back into the main line (baseline data source), the system performs the following operations: The system first responds to the merge request by identifying the second memory data page (i.e., the memory page that has been modified but has not yet been written to the first incremental file) in the shared buffer that belongs to the first database branch.

[0231] The system forces these changed pages in memory (second memory data pages) to be flushed to the corresponding first incremental file, thereby ensuring that the incremental file on disk contains all the latest modifications since the branch was created, preventing the loss of memory data during the merge process.

[0232] The storage manager sequentially traverses the first navigation area (PageIndex) of the first incremental file, extracting the first incremental data page identifier (Page ID) corresponding to each slot. Subsequently, the system calculates its global physical offset in the baseline data source based on the page identifier, and accurately overwrites the corresponding storage location of the first incremental data page in the first incremental file with the data of the first incremental data page in the corresponding slot in the baseline data source.

[0233] After data backfilling is complete, the system performs cleanup to end the branch's lifecycle: the first incremental file is physically deleted, freeing up disk space. The baseline data source, previously in a read-only locked state, is restored to a writable state. At this point, the database reverts to standard mode, supporting in-place updates. The identifier and related metadata of the first database branch are removed from the BranchManager.

[0234] For example, Figure 7 This is a flowchart illustrating a database branch merging process according to one embodiment of this specification. Specifically, it includes the following steps 702 to 712.

[0235] Step 702: Receive the merge request for the first database branch.

[0236] The system receives a merge command initiated by the user and prepares to integrate all differential changes made to the first database branch since its creation into a shared baseline data source.

[0237] Step 704: Write the second memory data page from the shared buffer pool to the first incremental file.

[0238] The system first identifies second memory data pages in the shared buffer pool that belong to the first database branch and have not yet been persisted. By forcibly flushing these memory changes to the first incremental file, the system ensures that the incremental data on disk is a complete and up-to-date logical view of the branch, thus avoiding data loss during the merge process.

[0239] Step 706: Scan the first navigation area (PageIndex) of the first incremental file to obtain the first incremental data page identifiers of all first incremental data pages.

[0240] The storage management layer sequentially scans the first navigation area of ​​the first incremental file. The system extracts the first incremental data page identifiers (such as Page IDs) of all modified first incremental data pages recorded within this area.

[0241] Step 708: Based on the first incremental data page identifier, write the corresponding first incremental data page to the corresponding storage location of the baseline data source.

[0242] The system calculates the global physical offset of each extracted first incremental data page identifier in the benchmark data source, extracts the corresponding physical page data from the first incremental file, and accurately overwrites and writes it to the corresponding storage location of the benchmark data source, thereby completing the physical-level data alignment.

[0243] Step 710: Delete the first incremental file and update the branch metadata.

[0244] After the data backfilling is complete, the system performs logical-level cleanup: Physical cleanup: Delete the first incremental file that is no longer needed to free up disk space.

[0245] Metadata update: In the branch management layer (BranchManager), deregister the information of the first database branch, update the branch tree structure, and mark the status of the branch as merged or removed.

[0246] Step 712: Restore the baseline data source to a writable state.

[0247] Release the read-only lock on the baseline data source, restoring it to a writable state. At this point, the database exits the branch hierarchical access mode and reverts to the standard in-place update mode.

[0248] In the embodiments described in this specification, firstly, by archiving the second memory data page in memory, the physical integrity of the merged data is ensured; secondly, precise backfilling based on the first incremental data page identifier ensures the accuracy of the baseline data source update; finally, by simultaneously updating the branch metadata while deleting the first incremental file, the real-time consistency between the system logical view and physical storage is ensured, preventing addressing errors caused by outdated branch information. Combined with the recovery of the writable state of the baseline data source, the database can smoothly switch from the complex branch management mode back to the high-performance standard operating mode, improving the system's operational reliability.

[0249] In an optional embodiment of this example, the database branching method further includes: In response to a request to create a subbranch for the first database branch, the subbranch identifier of the database subbranch to be created is determined, the association between the subbranch identifier and the baseline data source is established, and the database subbranch is created. Create a second incremental file corresponding to the database sub-branch. The second incremental file is used to store the second incremental data page, which is the data page that has changed under the database sub-branch. Based on the baseline data source, the first incremental file, and the second incremental file, the database subtasks under the database sub-branch are processed.

[0250] In this embodiment, the system supports creating sub-branches based on existing branches, thereby forming a multi-level branch tree structure. Taking the creation of a sub-branch under the first database branch as an example, the specific processing logic is as follows: When the system receives a request to create a subbranch for the first database branch, it first determines the subbranch identifier of the database subbranch to be created. The system does not copy any actual data; instead, it establishes a relationship between the subbranch identifier and the original baseline data source through metadata management. At this point, the branch management layer (BranchManager) records the subbranch as a descendant node of the first database branch, thus logically completing the creation of the subbranch.

[0251] The system allocates independent storage space for this database sub-branch, i.e., creates a second incremental file. This second incremental file is only used to store the second incremental data pages that have changed under its own operations after the sub-branch was created.

[0252] With this design, write operations to sub-branches are completely confined to the second incremental file, neither polluting the first incremental file of the first database branch nor changing the read-only baseline data source.

[0253] In the embodiments described in this specification, a sub-branch processing mechanism supporting multi-level incremental inheritance is constructed to achieve rapid evolution and simplified storage of database branches. First, by establishing an association between the sub-branch identifier and the baseline data source and creating an independent second incremental file, "zero-copy" sub-branch creation is achieved. Even in a multi-layered nested environment, the time consumption and storage overhead of branch creation remain independent of the total data volume, greatly improving branch scheduling efficiency in complex development scenarios. Second, utilizing a hierarchical processing mechanism based on the baseline data source, the first incremental file, and the second incremental file, sub-branches can transparently inherit the data views of the parent branch and the baseline source. While ensuring data logical continuity, incremental storage at the physical level is achieved, effectively avoiding the waste of storage resources caused by repeated data copying in traditional solutions.

[0254] In an optional implementation of this embodiment, before processing the database subtasks under the database sub-branch based on the baseline data source, the first incremental file, and the second incremental file, the following steps are also included: In response to a database branch switching request, a third memory data page in the shared buffer pool is written to the first incremental file. The database branch switching request is used to indicate that the currently active branch is switched from the first database branch to the database sub-branch. The third memory data page is a data page that has been changed during the operation of the first database branch and has not been written to disk before the branch switching. After the third memory data page is written, the data page cached in the shared buffer pool is released, and the currently active branch is switched to the database sub-branch. The shared buffer pool is used to cache the data pages that have changed under the currently active branch.

[0255] In this embodiment, the third memory data page refers to a data page (dirty page) that has been modified during the operation of the first database branch but has not been written to disk before the branch switch.

[0256] Before executing the switch command, these pages must be forcibly flushed to the first incremental file. This ensures that all the latest changes to the first database branch are safely saved, preventing data loss from the original branch due to memory flushing during the switch.

[0257] Releasing the data pages cached in the shared buffer pool refers to the management behavior of releasing the branch transactions of the first database branch from occupying the shared buffer pool, thereby allowing the shared buffer pool to be reused by other database branches.

[0258] In the embodiments described in this specification, the release can be performed by clearing the cache or invalidating the cache status flag: When using the cache clearing method, the system directly performs physical erasure and space stripping of cached data pages belonging to the first database branch from the shared buffer pool; When the cache status flag is set to invalid, the system changes the flag of the cached data page from active to invalid, so that the shared buffer pool can be reused by other database branches immediately.

[0259] The embodiments in this specification employ a single-active model, where only one "database branch" is active at any given time, and only this active database branch (the currently active branch) can write new changed data pages to the shared buffer pool. This eliminates the memory conflict problem caused by multiple branches writing simultaneously and overwriting each other in the large memory pool at the source.

[0260] Furthermore, the shared buffer pool is used to cache the data pages that have changed under the current active branch (i.e., the incremental data in the incremental file of the current active branch), so that when the current active branch performs high-frequency data modification and reading, it can directly hit the memory, thereby avoiding the physical input / output (I / O) overhead of frequently reading and writing incremental files on the disk.

[0261] For example, when the system receives a request to switch the currently active branch (such as the first database branch b1) to the target branch (such as the database sub-branch b2), the following process is executed: The system first initiates a targeted checkpoint operation on the original active branch, flushing all third-memory data pages in the shared buffer pool to the first incremental file. This step ensures the data integrity of the "departing" branch.

[0262] To prevent residual data from the original branch from contaminating access requests to the new branch, the system performs an `InvalidateSharedBuffer` operation. This marks all data pages in the shared buffer pool as invalid, ensuring that the target branch must reload data from its corresponding incremental file or baseline source after a switch.

[0263] Release the hash table that the first database branch built in memory.

[0264] Scan the incremental file index area of ​​the target branch (database sub-branch), reconstruct the corresponding hash addressing structure in memory, and thus achieve O(1) new branch addressing capability.

[0265] The system clears the memory caches associated with the metadata of the specific branch, including the relational cache (RelCache) and the system directory cache (CatCache). This is because the table structure versions and metadata information under different branches may differ, requiring resynchronization. Finally, the system sets the "currently active branch" flag in the global variable to the target sub-branch. From this point on, all subsequent read and write requests will be processed entirely according to the hierarchical addressing logic of the target sub-branch.

[0266] For example, Figure 8 This is a flowchart illustrating a database branch switching process according to one embodiment of this specification. Specifically, it includes the following steps 802 to 812.

[0267] Step 802: Receive database branch switching request (switching from the first database branch to a database sub-branch).

[0268] Step 804: Perform a targeted checkpoint and write the third memory data page to the first incremental file.

[0269] Step 806: Perform a memory invalidation operation to clear the shared buffer pool.

[0270] Step 808: Unload the original branch memory index and build the memory hash addressing structure of the database sub-branch.

[0271] Step 810: Clear the relationship cache and directory cache.

[0272] Step 812: Set the currently active branch as the database sub-branch.

[0273] In the embodiments of this specification, firstly, by archiving the third memory data page in the shared buffer pool to the original incremental file, the data persistence security during the branch switching process is ensured, and the hidden danger of memory data loss is eliminated; secondly, by clearing the shared buffer pool, cross-contamination of data between different branches is completely eliminated, and the system stability in the branch switching scenario is improved.

[0274] In one optional implementation of this embodiment, the database subtask includes a sub-branch data reading task; Based on the baseline data source, the first incremental file, and the second incremental file, the database subtasks under the database sub-branch are processed, including: In response to a read request for a second data page to be read, determine whether a second data page identifier exists in the second incremental file. If a second data page identifier exists in the second incremental file, then read the second data page from the second incremental file. If the second data page identifier does not exist in the second incremental file, then determine whether the second data page identifier exists in the first incremental file. If the second data page identifier exists in the first incremental file, then read the second data page from the first incremental file. If the second data page identifier is not present in the second incremental file, the second data page is read from the baseline data source.

[0275] When the system receives a data read request for a sub-branch of the database, the storage management layer performs a route search across multiple physical media layers according to the priority of "newest to oldest, nearest to furthest". First priority: Retrieval from the shared buffer pool.

[0276] The system first checks the shared buffer pool in memory. Since the database is actively running, the most recently modified data pages in the sub-branch may still reside in memory. If a match is found, it returns immediately; this is the fastest part of the process.

[0277] Second priority: Second increment file (current sub-branch increment).

[0278] If a memory miss occurs, the system searches the second incremental file. If the identifier of the second data page to be read exists, it means that the data page has been modified after the sub-branch was created. The system then extracts the data from the second incremental file.

[0279] Third priority: First incremental file (parent branch incremental).

[0280] If the page does not exist in the second incremental file, the system executes the "backtracking" logic to retrieve the first incremental file corresponding to its parent branch. If a match is found, it means that the data was generated by the modification of the parent branch, and the child branch logically inherits this modification, so the data is extracted from the first incremental file.

[0281] Fourth priority: Baseline data source (global foundation).

[0282] If the page identifier is not found in any of the above incremental levels, it means that the page has never been modified since the creation of any branch. The system eventually reverts to the underlying baseline data source and reads the initial data at the original offset.

[0283] In the embodiments described in this specification, a hierarchical routing mechanism of "memory cache - multi-level incremental - baseline base" is constructed to achieve efficient logical inheritance of data from parent branches to child branches. First, by setting the shared buffer pool as the addressing layer, extremely fast access to hot data within child branches is ensured, reducing physical I / O frequency. Second, through sequential backtracking addressing of "second incremental file → first incremental file → baseline data source," child branches can obtain a complete data view containing changes to the parent branch with minimal storage overhead (only storing their own increments), achieving a balance between physical isolation and logical inheritance. Finally, this hierarchical routing mechanism eliminates the need for data merging or full copying in multi-level branch environments, enabling the system to maintain fast branch creation speed and stable read performance even when supporting complex branch tree structures, greatly enhancing the flexibility of the database in version management and parallel development.

[0284] In one optional implementation of this embodiment, the first incremental file includes a first navigation area and a first data area. The first data area is used to store a first incremental data page, and the first navigation area is used to store first navigation information. The first navigation information is used to indicate a first incremental data page identifier of the first incremental data page and a first position information of the first incremental data page in the first data area. The second incremental file includes a second navigation area and a second data area. The second data area is used to store a second incremental data page, and the second navigation area is used to store second navigation information. The second navigation information is used to indicate a second incremental data page identifier of the second incremental data page and a second position information of the second incremental data page. Database subtasks include sub-branch data reading tasks; based on the baseline data source, the first incremental file, and the second incremental file, the database subtasks under the database sub-branch are processed, including: In response to a read request for the second data page to be read, determine whether a second data page identifier for the second data page exists in the second navigation information; If the second navigation information contains a second data page identifier, then based on the second navigation information, the reading position information of the second data page is determined, and based on the reading position information of the second data page, the second data page is read from the second data area. If the second navigation information does not contain a second data page identifier, then determine whether the first navigation information contains a second data page identifier. If the first navigation information contains an identifier for a second data page to be read, then based on the first navigation information, the reading position information of the second data page to be read is determined, and based on the reading position information of the second data page to be read, the second data page to be read is read from the first data area. If the identifier of the second data page to be read is not found in the first navigation information, then the second data page to be read is read from the baseline data source.

[0285] In this embodiment, the second data area is responsible for persistently storing all change pages (i.e., the second incremental data pages) generated after the creation of the database sub-branch.

[0286] The second navigation area is used to store critical second navigation information, and it establishes a mapping relationship between "second incremental data page identifier (PageID)" and "second location information (physical offset)".

[0287] When the system receives a data read request for a sub-branch of the database, the storage management layer performs a route search across multiple physical media layers according to the priority of "newest to oldest, nearest to furthest". First priority: Retrieval from the shared buffer pool.

[0288] The system first checks the shared buffer pool in memory. Since the database is actively running, the most recently modified data pages in the sub-branch may still reside in memory. If a match is found, it returns immediately; this is the fastest step in the process.

[0289] Second priority: Second increment file (current sub-branch increment).

[0290] If a memory miss occurs, the system retrieves the second navigation information corresponding to the second incremental file. If the target page identifier (the identifier of the second data page to be read) exists, it means that the data page has been modified after the sub-branch was created. The system extracts data from the second incremental file based on the corresponding physical location information.

[0291] Third priority: First incremental file (parent branch incremental).

[0292] If the page does not exist in the second incremental file, the system executes the "backtracking" logic to retrieve the first incremental file corresponding to its parent branch. If a match is found, it means that the data was generated by the modification of the parent branch, and the child branch logically inherits this modification, so the data is extracted from the first data area.

[0293] Fourth priority: Baseline data source (global foundation).

[0294] If the page identifier is not found in any of the above incremental levels, it means that the page has never been modified since the creation of any branch. The system eventually reverts to the underlying baseline data source and reads the initial data at the original offset.

[0295] In summary, the embodiments described in this specification achieve native awareness and fine-grained management of database branch states by sinking database branching capabilities from the "infrastructure layer" to the "storage engine layer" and constructing a layered storage architecture of "baseline files + incremental files." Specifically, by freezing the original data files as read-only baselines and using independent incremental files to record branch changes, true "zero-copy" branch creation is achieved. Since the branch creation overhead under this architecture is only related to the checkpoint synchronization speed and completely decoupled from the total amount of data in the database, the system can still achieve second-level environment cloning even when processing data of hundreds of terabytes in scale. This feature provides extremely fast underlying support for DevOps workflows in cloud-native environments, supporting high-frequency branching operation needs such as preview deployments.

[0296] Meanwhile, compared to traditional file system snapshots or storage-side cloning, this solution, by moving down to the storage engine level, allows the database to directly perceive and control the logical boundaries and transaction states of branches. The system provides more precise consistency guarantees and richer branch operation semantics (such as branch switching and dynamic merging), effectively avoiding problems like metadata inconsistencies or ambiguous recovery points caused by the disconnect between the underlying storage and the upper-level database logic in traditional solutions, thus greatly improving the reliability and security of branch data.

[0297] Corresponding to the above method embodiments, this specification also provides embodiments of a database branching processing device. Figure 9 A schematic diagram of a database branching processing apparatus according to one embodiment of this specification is shown. Figure 9 As shown, the device includes: The write module 902 is configured to write the first memory data page in the shared buffer pool to disk in response to a branch creation request for the target database, and set the existing data pages on the disk to read-only state to obtain the baseline data source, wherein the baseline data source is used to characterize the database state before the branch is created. The first creation module 904 is configured to determine the first branch identifier of the first database branch to be created, establish the association between the first branch identifier and the baseline data source, and create the first database branch; The second creation module 906 is configured to create a first incremental file corresponding to the first branch identifier, wherein the first incremental file is used to store the first incremental data page, and the first incremental data page is the data page that has changed under the first database branch; Processing module 908 is configured to process database tasks under the first database branch based on the baseline data source and the first incremental file.

[0298] Optionally, the first incremental file includes a first navigation area and a first data area. The first data area is used to store the first incremental data page, and the first navigation area is used to store first navigation information. The first navigation information is used to indicate the first incremental data page identifier of the first incremental data page and the first position information of the first incremental data page in the first data area. Processing module 908 is further configured as follows: Based on the baseline data source, as well as the first navigation area and the first data area, the database tasks under the first database branch are processed.

[0299] Optionally, the processing module 908 is further configured to: Based on the first navigation information, a first index unit is constructed, wherein the first index unit is used to store the first incremental data page identifier of the corresponding first incremental data page, and the first position information of the corresponding first incremental data page in the first data area; Based on the baseline data source, the first index unit, the set position conversion relationship, and the first data area, the database tasks under the first database branch are processed. The position conversion relationship is used to map the position information of the corresponding index storage unit according to the data page identifier.

[0300] Optionally, the database task includes a branch data reading task; the processing module 908 is further configured to: In response to a read request for the first data page to be read, determine whether a first data page identifier for the first data page to be read exists in the first navigation information; If the first navigation information contains a first data page identifier, then based on the first navigation information, the reading position information of the first data page is determined, and based on the reading position information of the first data page, the first data page is read from the first data area. If the first data page identifier is not present in the first navigation information, then the first data page to be read is read from the baseline data source.

[0301] Optionally, the database task includes a data writing task; the processing module 908 is further configured to: In response to a write request for the target data page to be written, determine whether the target data page identifier exists in the first navigation information; If the first navigation information contains a target data page identifier, then based on the first navigation information, the write location information of the target data page is determined, and based on the write location information, the data of the target data page identifier is written to the corresponding storage location of the first data area. If the target data page identifier is not present in the first navigation information, the read-only state of the reference data source remains unchanged. A target storage location is allocated for the target data page in the first data area, the target data page is written to the target storage location, and the target data page identifier and the location information of the target storage location are added to the first navigation information.

[0302] Optionally, the processing module 908 is further configured to: Obtain the write transaction identifier corresponding to the write request. The write transaction identifier is used to indicate the position of the write request in the write-ahead log, and the write transaction identifier has a monotonically increasing trend. The write-ahead log is used to record change operations in the target database. Determine whether the write transaction flag is greater than the first branch creation flag corresponding to the creation of the first database branch. The first branch creation flag is used to indicate the cutoff position of the write-ahead log at the time of creation of the first database branch. If the write transaction identifier is greater than the first branch creation identifier, then the step of determining whether the target write data page identifier exists in the first navigation information is executed.

[0303] Optionally, the database task includes a branch deletion task; the processing module 908 is further configured to: In response to a deletion request for the first database branch, the baseline data source remains unchanged, and it is determined whether the first database branch is currently the active branch; If the first database branch is not the currently active branch, then delete the first incremental file; If the first database branch is the currently active branch, then switch the currently active branch to another branch, and after the switch is completed, delete the first incremental file.

[0304] Optionally, the database task includes a branch merging task; processing module 908 is further configured to: In response to a merge request for the first database branch, a second memory data page in the shared buffer pool is written to the first incremental file, wherein the second memory data page is a data page that has changed during the operation of the first database branch and has not been written to disk before the branch merge; Read the first incremental data page stored in the first incremental file and write the first incremental data page to the corresponding storage location in the baseline data source; Delete the first incremental file and restore the baseline data source to a writable state.

[0305] Optionally, the above-mentioned database branch processing device further includes a sub-branch processing module, configured as follows: In response to a request to create a subbranch for the first database branch, the subbranch identifier of the database subbranch to be created is determined, the association between the subbranch identifier and the baseline data source is established, and the database subbranch is created. Create a second incremental file corresponding to the database sub-branch. The second incremental file is used to store the second incremental data page, which is the data page that has changed under the database sub-branch. Based on the baseline data source, the first incremental file, and the second incremental file, the database subtasks under the database sub-branch are processed.

[0306] Optionally, the sub-branch processing module is further configured as follows: In response to a database branch switching request, a third memory data page in the shared buffer pool is written to the first incremental file. The database branch switching request is used to indicate that the currently active branch is switched from the first database branch to the database sub-branch. The third memory data page is a data page that has been changed during the operation of the first database branch and has not been written to disk before the branch switch. After the third memory data page is written, the data page cached in the shared buffer pool is released, and the currently active branch is switched to the database sub-branch. The shared buffer pool is used to cache the data pages that have changed under the currently active branch.

[0307] Optionally, the database subtask includes a sub-branch data reading task; the sub-branch processing module is further configured as follows: In response to a read request for a second data page to be read, determine whether a second data page identifier exists in the second incremental file. If a second data page identifier exists in the second incremental file, then read the second data page from the second incremental file. If the second data page identifier does not exist in the second incremental file, then determine whether the second data page identifier exists in the first incremental file. If the second data page identifier exists in the first incremental file, then read the second data page from the first incremental file. If the second data page identifier is not present in the second incremental file, the second data page is read from the baseline data source.

[0308] The above is a schematic scheme of a database branching processing device according to this embodiment. It should be noted that the technical solution of this database branching processing device and the technical solution of the database branching processing method described above belong to the same concept. For details not described in detail in the technical solution of the database branching processing device, please refer to the description of the technical solution of the database branching processing method described above.

[0309] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0310] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0311] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0312] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0313] The processor 1020 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the database branching method described above.

[0314] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the database branching processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the database branching processing method described above.

[0315] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the database branching method described above.

[0316] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the database branching method described above. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the database branching method described above.

[0317] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the database branching method described above.

[0318] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the database branching processing method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the database branching processing method described above.

[0319] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0320] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0321] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0322] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0323] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A database branching method, comprising: In response to a branch creation request for the target database, the first memory data page in the shared buffer pool is written to the disk, and the existing data pages in the disk are set to read-only state to obtain a baseline data source, wherein the baseline data source is used to characterize the database state before the branch is created. Determine the first branch identifier of the first database branch to be created, establish the association between the first branch identifier and the baseline data source, and create the first database branch; Create a first incremental file corresponding to the first branch identifier, wherein the first incremental file is used to store the first incremental data page, and the first incremental data page is the data page that has changed under the first database branch; Based on the baseline data source and the first incremental file, the database tasks under the first database branch are processed.

2. The method according to claim 1, wherein the first incremental file includes a first navigation area and a first data area, the first data area is used to store the first incremental data page, the first navigation area is used to store first navigation information, the first navigation information is used to indicate a first incremental data page identifier of the first incremental data page, and a first position information of the first incremental data page in the first data area; The process of processing database tasks under the first database branch based on the baseline data source and the first incremental file includes: Based on the aforementioned baseline data source, as well as the first navigation area and the first data area, the database tasks under the first database branch are processed.

3. The method according to claim 2, wherein processing the database tasks under the first database branch based on the benchmark data source, the first navigation area, and the first data area includes: Based on the first navigation information, a first index unit is constructed, wherein the first index unit is used to store the first incremental data page identifier of the corresponding first incremental data page, and the first position information of the corresponding first incremental data page in the first data area; Based on the baseline data source, the first index unit, the set location conversion relationship, and the first data area, the database tasks under the first database branch are processed, wherein the location conversion relationship is used to map the location information of the corresponding index storage unit according to the data page identifier.

4. The method according to claim 2, wherein the database task includes a branch data reading task; The process of processing database tasks under the first database branch based on the baseline data source and the first incremental file includes: In response to a read request for the first data page to be read, determine whether a first data page identifier for the first data page to be read exists in the first navigation information; If the first navigation information contains the identifier of the first data page to be read, then based on the first navigation information, the reading position information of the first data page to be read is determined, and based on the reading position information, the first data page to be read is read from the first data area; If the identifier of the first data page to be read is not present in the first navigation information, then the first data page to be read is read from the reference data source.

5. The method according to claim 2, wherein the database task includes a data writing task; The process of processing database tasks under the first database branch based on the baseline data source and the first incremental file includes: In response to a write request for the target data page to be written, determine whether the target data page identifier of the target data page to be written exists in the first navigation information; If the target data page identifier exists in the first navigation information, then based on the first navigation information, the write location information of the target data page is determined, and based on the write location information, the data of the target data page identifier is written to the corresponding storage location of the first data area. If the target data page identifier is not present in the first navigation information, the read-only state of the reference data source remains unchanged. A target storage location is allocated for the target data page in the first data area, the target data page is written to the target storage location, and the target data page identifier and the location information of the target storage location are added to the first navigation information.

6. The method according to claim 5, further comprising, before determining whether the first navigation information contains a target data page identifier for the target data page to be written, the method includes: Obtain the write transaction identifier corresponding to the write request, wherein the write transaction identifier is used to indicate the position of the write request in the write-ahead log and the write transaction identifier has a monotonically increasing trend, and the write-ahead log is used to record the change operations of the target database; Determine whether the write transaction identifier is greater than the first branch creation identifier corresponding to the creation of the first database branch, wherein the first branch creation identifier is used to indicate the cutoff position of the write-ahead log at the time of creation of the first database branch; If the write transaction identifier is greater than the first branch creation identifier, then the step of determining whether the target write data page identifier of the target write data page exists in the first navigation information is executed.

7. The method according to claim 1, wherein the database task includes a branch deletion task; The process of processing database tasks under the first database branch based on the baseline data source and the first incremental file includes: In response to a deletion request for the first database branch, the baseline data source remains unchanged, and it is determined whether the first database branch is currently an active branch; If the first database branch is not currently the active branch, then delete the first incremental file; If the first database branch is the currently active branch, then the currently active branch is switched to another branch, and after the switch is completed, the first incremental file is deleted.

8. The method according to claim 1, wherein the database task includes a branch merging task; The process of processing database tasks under the first database branch based on the baseline data source and the first incremental file includes: In response to a merge request for the first database branch, a second memory data page from the shared buffer pool is written to the first incremental file, wherein the second memory data page is a data page that has changed during the operation of the first database branch and has not been written to disk before the branch merge; Read the first incremental data page stored in the first incremental file and write the first incremental data page into the corresponding storage location in the reference data source; Delete the first incremental file and restore the baseline data source to a writable state.

9. The method according to claim 1, further comprising: In response to a sub-branch creation request for the first database branch, the sub-branch identifier of the database sub-branch to be created is determined, and the association between the sub-branch identifier and the baseline data source is established to create the database sub-branch. Create a second incremental file corresponding to the database sub-branch, wherein the second incremental file is used to store the second incremental data page, and the second incremental data page is the data page that has changed under the database sub-branch; Based on the baseline data source, the first incremental file, and the second incremental file, the database subtasks under the database sub-branch are processed.

10. The method according to claim 9, further comprising, before processing the database subtasks under the database sub-branch based on the baseline data source, the first incremental file, and the second incremental file: In response to a database branch switching request, a third memory data page in the shared buffer pool is written to the first incremental file, wherein the database branch switching request is used to indicate switching the currently active branch from the first database branch to the database sub-branch, and the third memory data page is a data page that has been changed during the operation of the first database branch and has not been written to disk before the branch switching; After the third memory data page is written, the data page cached in the shared buffer pool is released, and the currently active branch is switched to the database sub-branch. The shared buffer pool is used to cache the data pages that have changed under the currently active branch.

11. The method according to claim 9, wherein the database subtask includes a sub-branch data reading task; The process of processing the database subtasks under the database sub-branch based on the baseline data source, the first incremental file, and the second incremental file includes: In response to a read request for a second data page to be read, determine whether a second data page identifier for the second data page to be read exists in the second incremental file. If the second data page identifier for the second data page to be read exists in the second incremental file, then read the second data page to be read from the second incremental file. If the second data page identifier does not exist in the second incremental file, then it is determined whether the second data page identifier exists in the first incremental file. If the second data page identifier exists in the first incremental file, then the second data page is read from the first incremental file. If the identifier of the second data page to be read does not exist in the second incremental file, then the second data page to be read is read from the baseline data source.

12. A database branching processing device, comprising: The write module is configured to, in response to a branch creation request for the target database, write the first memory data page in the shared buffer pool to the disk and set the existing data pages in the disk to read-only state to obtain a baseline data source, wherein the baseline data source is used to characterize the database state before the branch is created. The first creation module is configured to determine the first branch identifier of the first database branch to be created, establish the association between the first branch identifier and the baseline data source, and create the first database branch. The second creation module is configured to create a first incremental file corresponding to the first branch identifier, wherein the first incremental file is used to store a first incremental data page, and the first incremental data page is a data page that has changed under the first database branch; The processing module is configured to process database tasks under the first database branch based on the baseline data source and the first incremental file.

13. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the database branching method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the database branching method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the database branching method according to any one of claims 1 to 11.

Images