Data management method and electronic device based on B+ tree

Abstract

The present application provides a data management method and electronic device based on a B+ tree, relating to the field of data processing technology. The method comprises: in response to a first write operation on node data of a B+ tree, writing the node data to an original page of a disk; in response to a first write operation on a target entry in the original page, creating a first incremental record in an incremental page based on the original page; the first incremental record is used to record first data corresponding to the first write operation, and the first pointer of the target entry indicates first address information of the first incremental record; in response to a second write operation on the target entry, creating a second incremental record in the incremental page; the second incremental record is used to record second data corresponding to the second write operation; the first pointer of the target entry indicates second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record. Embodiments of the present application can reduce disk space waste and lower read and write complexity.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data management method and electronic device based on a B+ tree. Background Art

[0002] The B+ tree is a balanced tree data structure. As a multi-branch sorted tree, each node typically has multiple child nodes. B+ trees are widely used in databases and file systems due to their efficient range queries and sequential access capabilities. By organizing node data into fixed-size disk pages, B+ trees enable efficient storage and persistence of large amounts of data.

[0003] In practical applications, B+ trees often use Copy on Write (COW) technology to ensure data consistency and support multi-version transactions. Specifically, when data in a node is modified, the node is first copied to a new location, and then the modification is performed on the copy. However, even though the data modification only involves a small update, the entire disk page must be copied, wasting disk space and increasing read and write complexity. Summary of the Invention

[0004] According to various embodiments of the present application, a data management method and electronic device based on a B+ tree are provided, which can reduce disk space waste and lower reading and writing complexity.

[0005] In a first aspect, the present application provides a data management method based on a B+ tree, the method comprising:

[0006] In response to the first write operation on the node data of the B+ tree, the node data is written to the original page of the disk; the original page contains an original pointer; in response to the first write operation on the target entry in the original page, a first incremental record is created in the incremental page based on the original page; the first incremental record is used to record the first data corresponding to the first write operation, and the first pointer of the target entry in the original page indicates the first address information of the first incremental record; in response to the second write operation on the target entry, a second incremental record is created in the incremental page; the second incremental record is used to record the second data corresponding to the second write operation; the first pointer of the target entry in the original page indicates the second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record; wherein the original pointer is used to indicate the incremental page associated with the original page.

[0007] Through the above method, the target entry of the original page includes a first pointer. When a write operation is performed on the data of the target entry, the update operation of the data of the original page is recorded in the form of an incremental page; and after each write operation, the first pointer in the target entry based on the original page always indicates the address information of the latest incremental page. When updating the data, there is no need to traverse and copy the linked list data of the entire disk page, and the latest incremental data can be directly located, thereby reducing disk space waste and reducing the complexity of read and write operations; it has strong ease of use and practicality.

[0008] In a possible implementation of the first aspect, the method further includes:

[0009] When the node data is written to the original page of the disk, a mapping table corresponding to the original page is created; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

[0010] In a possible implementation of the first aspect, after creating the first incremental record in the incremental page, the method further includes:

[0011] In the mapping table, a mapping relationship between the target entry and the first incremental record is recorded; the mapping relationship includes the first address information of the first incremental record;

[0012] After creating the second incremental record in the incremental page, the method further includes:

[0013] The first address information in the mapping relationship is updated to the second address information of the second incremental record.

[0014] In a possible implementation of the first aspect, the method further includes:

[0015] When the number of the mapping relationships recorded in the mapping table is greater than a first threshold, the data recorded on the original page and the data recorded on the incremental page are merged to obtain a new original page.

[0016] In a possible implementation of the first aspect, the method further includes:

[0017] When the size of the incremental page is greater than a second threshold, the data recorded in the original page and the data recorded in the incremental page are merged to obtain a new original page.

[0018] In a possible implementation of the first aspect, merging the data recorded on the original page with the data recorded on the incremental page includes:

[0019] The data recorded in the latest incremental record corresponding to the target entry in the incremental page is merged into the original page to obtain a new original page.

[0020] In a possible implementation of the first aspect, merging the data recorded on the original page with the data recorded on the incremental page includes:

[0021] Determining the latest incremental record corresponding to the target entry in the incremental page based on the address information recorded in the mapping table;

[0022] The data recorded in the latest incremental record is merged into the original page to obtain a new original page.

[0023] In a possible implementation of the first aspect, after merging the data recorded on the original page with the data recorded on the incremental page to obtain a new original page, the method further includes:

[0024] When the size of the new original page is greater than a third threshold, splitting the node data of the current node;

[0025] Writing the data separated based on the split operation into the newly added node;

[0026] In the parent node of the current node, the index information of the newly added node is updated.

[0027] In a second aspect, the present application provides a data management device based on a B+ tree, comprising:

[0028] A first processing unit is configured to, in response to a first write operation on node data of the B+ tree, write the node data to an original page of the disk; the original page includes an original pointer;

[0029] a second processing unit, configured to, in response to a first write operation on a target entry in the original page, create a first incremental record in an incremental page based on the original page; the first incremental record is configured to record first data corresponding to the first write operation, and a first pointer to the target entry in the original page indicates first address information of the first incremental record;

[0030] a third processing unit, configured to create a second incremental record in the incremental page in response to a second write operation on the target entry; the second incremental record is configured to record second data corresponding to the second write operation; the first pointer of the target entry in the original page indicates second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record;

[0031] The original pointer is used to indicate the incremental page associated with the original page.

[0032] In a third aspect, the present application provides an electronic device, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute any one of the methods described in the first aspect.

[0033] In a fourth aspect, the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute the method described in the first aspect.

[0034] In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when the computer program is executed by a processor.

[0035] In a sixth aspect, the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect above.

[0036] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A schematic diagram of the architecture of the B+ tree storage model provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of the structure of a single linked list of target entries in an incremental page provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of the implementation process of the B+ tree-based data management method provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of the overall architecture of the update data provided in the embodiment of the present application;

[0042] Figure 5A schematic diagram of the structure of a data management device based on a B+ tree provided in an embodiment of the present application;

[0043] Figure 6 A schematic structural diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following embodiments of the technical solution of the present application will be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application and are therefore only examples and are not intended to limit the scope of protection of the present application.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0046] In the description of the embodiments of this application, the technical terms "first" and "second" are used only to distinguish different objects and should not be understood to indicate or imply relative importance or implicitly specify the quantity, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, the meaning of "plurality" is more than two, unless otherwise clearly and specifically defined.

[0047] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] In the description of the embodiments of this application, the term "and / or" is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0049] By organizing node data into fixed-size disk pages, B+ trees achieve efficient storage and persistence for large-scale data. In practical applications, B+ trees often use Copy on Write (COW) to ensure data consistency and support multi-version transactions. However, data modifications only involve a small number of updates, but require copying entire disk pages, which leads to write amplification and generates a large amount of redundant data, resulting in wasted disk space. Each time a new version of data is generated, the process of copying and writing node data to disk increases the burden on read and write performance. Furthermore, each update to node data affects the parent and root nodes, resulting in the rewriting of multiple layers of nodes and increasing write costs.

[0050] Currently, related technologies include using delta pages to record update operations on disk page data, and appending the delta page after each update to the original page, thereby reducing disk input / output (I / O) operations and simplifying data management during subsequent data merging.

[0051] Based on the above approach, every time data is modified during a disk operation, a new current incremental page is generated, all updated data is written to the current incremental page, and the incremental page identifier field in the original page is updated with the identifier of the current incremental page. When data is updated frequently, and when multiple data items are updated at a time, the generated incremental pages form a long chain. Each time data is read, the entire incremental page chain must be traversed to construct the latest data structure view, significantly increasing the complexity of the read operation and resulting in a decrease in read performance. Subsequently, read performance is improved by merging the data in the incremental pages. However, when the amount of data in these incremental pages is large, resource contention may occur, resulting in a transient decrease in system performance.

[0052] In response to the above technical problems, an embodiment of the present application provides a data management method based on a B+ tree, which stores updated data in an incremental page manner, and in the storage model of the B+ tree, adds a pointer field to each entry in the original page to indicate the latest incremental record in the incremental page. When updating data, there is no need to copy the data of the entire disk page. When reading data, the latest data can be quickly located based on the indication of the pointer field of each entry in the original page, thereby reducing the read and write complexity and improving the read and write performance of the disk.

[0053] The following describes the specific implementation process of the B+ tree-based data management method provided in the embodiment of the present application.

[0054] Please refer to Figure 1 , Figure 1This is a schematic diagram of the B+ tree storage model architecture provided in an embodiment of the present application. During B+ tree data storage, B+ tree node data is stored on disk in the form of pages. A node can be an abstraction of a disk page in the B+ tree, representing a data structure that has been deserialized and loaded into memory.

[0055] The B+ tree, as a balanced tree structure, is used for indexing in databases and file systems. Disk pages are the physical unit of data storage. In databases, data is stored on disk in fixed-size pages, such as 4KB or 8KB. A B+ tree node is a logical unit within the B+ tree structure, storing key values and pointers to child nodes. In memory, a disk page is read from disk and loaded into memory, abstracted as a B+ tree node. When a B+ tree node needs to be persisted to disk, it is serialized into a byte stream and then stored in one or more disk pages. When a B+ tree node is read from disk, the corresponding disk page is read into memory and deserialized into the B+ tree node data structure. Due to limited memory space, only the node being accessed or its adjacent nodes are loaded into memory. When accessing a node that is not in memory, the database first reads the corresponding page (the physical storage of the node) from disk and then deserializes it into the in-memory node data structure.

[0056] In the B+ tree storage model, when a node's data is first written from memory to persistent storage on disk, the node's data is solidified into a fixed base state, called the base page. Subsequent data updates and other operations are expanded and managed based on the base page.

[0057] like Figure 1 As shown, the original page may include a newly added field (dp), the current page representation (id), the number of key-value pairs (count), the number of overflow pages (overflow), and the byte array (data) storing the page content. The newly added field (dp), the current page representation (id), the number of key-value pairs (count), and the number of overflow pages (overflow) constitute the page header data (PageHeader) of the original page. The newly added field is used to represent the original pointer indicating the incremental page; the byte array (data) includes all stored entries, such as the first entry (inode-1), the second entry (inode-1), etc.

[0058] For example, the data structure of a B+ tree node may include the disk page identifier (pgid) corresponding to the current node, the keyword (or key value) of the current node, and all entries (inodes) stored in the current node. An inode represents a single entry stored in a node, and the data structure of a single entry includes a pointer field (next), a stored key (key) value, an identifier indicating the target page (pgid), a flag for the current node (flags), and a value (value) associated with the keyword. Entries are used to provide index information for related data.

[0059] Among them, the pointer field is used to indicate the address information of the incremental page; the saved key value is used to indicate sorting according to the key value, thereby supporting efficient search, insertion and deletion of nodes; the identifier indicating the target page is used to record the child node indicated by the internal node; the mark of the current node is used to mark whether the current node is a leaf node or a non-leaf node; if the current node is a leaf node, the value field correspondingly stores the value associated with the key. If it is an internal node (non-leaf node), the value field is not used, and the data is saved in the child node through the identifier (pgid) indicating the target page.

[0060] Accordingly, if Figure 1 In the structures corresponding to the original page on disk and the storage model shown, the fields in each entry (such as next, key, pgid, flags, and value) have the same references as those in the node data structure. The header data of the original page corresponds to the original page's attributes, and the header data of the incremental page corresponds to the incremental page's attributes.

[0061] like Figure 1 As shown, the newly added field (dp) in the original page indicates the associated delta page, which stores all incrementally updated data on the node. When the data of an entry in the original page changes (for example, the data of the first entry inode-1 changes), a new delta record (for example, inode-1´) is created in the delta page. The pointer field (next) of the first entry in the original page is set to indicate the location information of the delta record in the delta page, such as the starting offset (Offset Position).

[0062] When data is modified multiple times, if the data of the first entry changes again, another incremental record (such as inode-1) is added to the incremental page, and the address information indicated by the pointer of the first entry inode-1 is updated to indicate the address information of the incremental record inode-1, such as the starting offset (OffsetPosition) of the incremental record inode-1. Similarly, when the data of the first entry changes again, another incremental record (such as inode-1) is added to the incremental page, and the pointer of the first entry inode-1 is updated to indicate the position information (such as the starting offset) of inode-1. This ensures that the pointer of the first entry always points to the data in the latest incremental record.

[0063] Accordingly, with multiple change operations, the updated data corresponding to each entry in the incremental page will form a single linked list structure, and each incremental record in the linked list is linked to the previous version; for example, the pointer of inode-1´´ indicates the starting offset of inode-1´´, and the pointer of inode-1´´ indicates the offset of inode-1´. Figure 2 The first entry inode-1 in the original page shown corresponds to the single linked list structure in the incremental page. Since the next pointer of inode-1 always points to the latest incremental record, the read operation does not need to traverse all the incremental page hierarchical linked lists, but can directly locate the data of the latest incremental record, significantly improving read efficiency.

[0064] Exemplarily, when updating the data of other entries in the original page, the implementation principle is the same as the above.

[0065] Based on the architecture of the above storage model, the specific implementation process of the data management method based on the B+ tree is further introduced below through embodiments.

[0066] See Figure 3 , Figure 3 This is a schematic diagram of the implementation process of the B+ tree-based data management method provided in an embodiment of the present application. The execution subject of the method can be an electronic device, which includes a database or a file system; the method may include the following steps:

[0067] S301 , in response to a first write operation on node data of a B+ tree, write the node data to an original page of a disk; the original page includes an original pointer.

[0068] In the embodiment of the present application, in a B+ tree, each tree node contains a set of key-value pairs, the key is used to sort and retrieve data, and the value is the corresponding data, such as Figure 4As shown in the figure, the node memory structure corresponding to the B+ tree node architecture. Node 3 contains the key a (data item or object). The corresponding value data includes the corresponding disk page identifier pgid = 3, the pointer field next, the node type flags, and the specific data value = 123. In memory, the node data can be stored in the form of the aforementioned data structure.

[0069] For example, before writing node data from memory to disk for the first time, the structure of the node data in memory is serialized into a continuous byte stream, and the disk is divided into pages of fixed size; using the input / output I / O operation of the database or file system, the continuous byte stream is written to the corresponding disk page, and the node data is solidified into a fixed basic state, which is called the state of the original page of data storage; the original page is the persistent representation of the node data on the disk, such as Figure 4 As shown in the original page, it contains the key in the node data (key=3), the corresponding disk page identifier pgid=3, the pointer field next, the node type flags and the specific data value=123.

[0070] The size of a node can be smaller than or equal to the size of a disk page, ensuring that the node data can be completely stored on a disk page. When data is subsequently updated (such as when writing data), data expansion and management are performed based on the original page.

[0071] In some embodiments, the method further comprises:

[0072] When writing node data to the original page of the disk, a mapping table corresponding to the original page is created; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

[0073] For example, Figure 4 The mapping table shown in the figure represents the correspondence between each entry in the original page (e.g., key=a) and the incremental record in the incremental page in the form of key and value pairs. The value in this correspondence includes the address information of the incremental record corresponding to the entry in the incremental page; this address information can be represented by the next pointer field.

[0074] When the data of the original page is not updated, the fields included in the value in the mapping table may be empty; correspondingly, after a data update operation occurs, the mapping relationship corresponding to the updated entry is recorded in the mapping table.

[0075] S302 : In response to a first write operation on a target entry in an original page, create a first incremental record in an incremental page based on the original page.

[0076] In the embodiment of the present application, an entry is a data structure used to manage files or directories in a file system and is used to store information related to the file or directory; Figure 1 As shown, the first entry inode-1 and the second entry inode-2 in the original page contain various field information; the target entry is any entry of the original page with data to be updated.

[0077] Exemplarily, when an electronic device receives a data update operation corresponding to a target entry of an original page, such as a first write operation to write first data to a target entry, it creates an incremental page corresponding to the target entry based on the original page, and creates an incremental record in the incremental page, such as the first incremental record inode-1´.

[0078] The first incremental record is used to record the first data corresponding to the first write operation, and the first pointer of the target entry in the original page indicates the first address information of the first incremental record. Figure 1 As shown, after the first incremental record inode-1' is created, the first pointer next of the target entry inode-1 in the original page is pointed to the first address information of the first incremental record, that is, the starting offset of inode-1'.

[0079] In some embodiments, after creating the first delta record in the delta page, the method further includes:

[0080] In the mapping table, a mapping relationship between the target entry and the first incremental record is recorded; the mapping relationship includes the first address information of the first incremental record.

[0081] For example, Figure 4 As shown, if a write operation is performed on the target entry (such as the key of the target entry key=a), the target entry a is searched in the node memory corresponding to node 3 of the B+ tree structure. If it does not exist, it is necessary to further determine whether there is a record of the target entry a in the mapping table.

[0082] Among them, if the target entry a does not exist in node 3 and does not exist in the mapping table, it means that the data written to the target entry a is the first newly added data, then the first incremental record inode-1' is created in the incremental page, and a mapping relationship between the target entry and the first incremental record is added to the mapping table, and the next pointer in the mapping relationship indicates the address information of the first incremental record (such as the starting offset); Figure 4 The next pointer in the mapping table shown in indicates the starting offset of the first incremental record inode-1 in the incremental page (as shown in Figure 4 (as indicated by the solid arrow in the mapping table).

[0083] Alternatively, in response to the first write operation, the target entry a is added to the mapping table, and the first data corresponding to the target entry a is written in the incremental page, such as value=456, recorded as the first incremental record, and the starting offset of the first incremental record is saved in the mapping table and associated with the target entry a.

[0084] For example, if the target entry a exists in node 3 but does not exist in the mapping table, it means that the target entry a has not been updated. In this case, a first incremental record inode-1′ is created in the incremental page, and a mapping relationship between the target entry and the first incremental record is added to the mapping table. The next pointer in the mapping relationship indicates the address information (such as the starting offset) of the first incremental record. Figure 4 The next pointer in the mapping table shown in indicates the starting offset of the first incremental record inode-1 in the incremental page (as shown in Figure 4 (as indicated by the solid arrow in the mapping table).

[0085] S303 , in response to a second write operation on the target entry, create a second incremental record in the incremental page; the second incremental record is used to record second data corresponding to the second write operation.

[0086] In an embodiment of the present application, when the electronic device receives multiple update operations on the data corresponding to the target entry of the original page, such as the second write operation to write the first data to the target entry, it creates another incremental record, such as the second incremental record inode-1´´, in the incremental page corresponding to the created target entry.

[0087] The second incremental record is used to record the second data corresponding to the second write operation, and the first pointer of the target entry in the original page indicates the second address information of the second incremental record. Figure 1 As shown, after the second incremental record inode-1´´ is created, the first pointer next of the target entry inode-1 in the original page is pointed to the second address information of the second incremental record, that is, the starting offset of the second incremental record inode-1´´; the second pointer of the second incremental record is pointed to the first address information of the first incremental record, that is, the starting offset of the first incremental record inode-1´.

[0088] In some embodiments, after creating the second incremental record in the incremental page, the method further includes: updating the first address information in the mapping relationship to the second address information of the second incremental record.

[0089] Exemplarily, the first incremental record is the previous data update version of the second incremental record. Figure 4 As shown, if the target entry (such as the key of the target entry key=a) is updated (such as a write operation), the target entry a is searched in the node memory corresponding to node 3 of the B+ tree structure. If it does not exist, it is necessary to further determine whether there is a record of the target entry a in the mapping table.

[0090] Among them, if the target entry a does not exist in node 3, but exists in the mapping table, it means that the data written to the target entry a is new data, and then a second incremental record inode-1 is created in the incremental page; and the mapping relationship between the target entry and the first incremental record in the mapping table is updated to a mapping relationship with the second incremental record, that is, the next pointer in the mapping relationship indicates the address information of the second incremental record (such as the starting offset); Figure 4 The next pointer in the mapping table shown in the figure indicates the starting offset of the second incremental record inode-1'' in the incremental page (e.g. Figure 4 (as indicated by the dotted arrow in the mapping table).

[0091] For example, if the target entry a exists in node 3 and the mapping relationship record corresponding to the target entry a exists in the mapping table, it means that the data of a has been updated. At this time, when the write operation is performed again for update, a second incremental record inode-1´´ is added to the incremental page, and the pointer of the second incremental record inode-1´´ indicates the address information of the previous version of the incremental record, that is, the starting offset of the first incremental record inode-1´; the next pointer in the mapping table is updated to indicate the starting offset of the second incremental record inode-1´´; thereby, the mapping relationship in the mapping table indicates the latest version of the data corresponding to the entry.

[0092] In some embodiments, the method further comprises:

[0093] When the number of mapping relationships recorded in the mapping table is greater than a first threshold, the data recorded on the original page is merged with the data recorded on the incremental page to obtain a new original page. Alternatively, when the size of the incremental page is greater than a second threshold, the data recorded on the original page is merged with the data recorded on the incremental page to obtain a new original page.

[0094] Exemplarily, in order to optimize reading efficiency and reduce merging overhead, a global merge operation is performed based on the number of entries of the mapping relationship recorded in the mapping table or the size of the data stored in the incremental page; when the number of entries of the mapping relationship recorded in the mapping table is greater than a first threshold (such as 100); or when the incremental page corresponding to the entry is greater than a second threshold (such as 4KB), the data stored in the incremental page of the entry is merged into the data of the original page.

[0095] Illustratively, during the merging process, entries may be sorted based on the keys of the original entries to prevent data with duplicate key values.

[0096] In some embodiments, the data recorded on the original page and the data recorded on the incremental page are merged, including: merging the data recorded on the latest incremental record corresponding to the target entry in the incremental page into the original page to obtain a new original page.

[0097] Exemplarily, the latest version of data corresponding to each entry in the original page in the incremental page is merged into the original page so that the data of each entry remains up to date, generating a new original page; and the historical data corresponding to each entry in the incremental page is deleted, thereby reducing disk space usage.

[0098] In some embodiments, the data recorded on the original page and the data recorded on the incremental page are merged, including: determining the latest incremental record corresponding to the target entry in the incremental page based on the address information recorded in the mapping table; merging the data recorded in the latest incremental record into the original page to obtain a new original page.

[0099] Exemplarily, the incremental page also includes newly added data (data that does not exist in the node corresponding to the original page), and the address information corresponding to the newly added data is recorded in the mapping table. During the merging process of the incremental page corresponding to the entry, the address information corresponding to the entry recorded in the mapping relationship in the mapping table (such as the starting offset indicated by the next pointer field) can be combined to obtain the data recorded in the latest incremental record corresponding to the newly added data corresponding to the address information, and the data is merged into the original page to obtain a new original page. In this way, the latest incremental record corresponding to the entry in the incremental page can be directly located, avoiding traversing the linked list of incremental pages layer by layer, and improving the merging efficiency; a global merge is performed on the data keys (such as key=a) in the original page, incremental page, and mapping table to ensure that the generated new original page data is complete and ordered.

[0100] In some embodiments, after merging the data recorded on the original page with the data recorded on the incremental page to obtain a new original page, the method further includes:

[0101] When the size of the new original page is greater than the third threshold, the node data of the current node is split; the data separated based on the split operation is written into the newly added node; and the index information of the newly added node is updated in the parent node of the current node.

[0102] For example, if the size of the newly generated original page exceeds a third threshold (e.g., 4KB), a split operation is triggered on the current node, and the data separated after the split operation is written to the newly added node; for example, data exceeding 4KB is written to the new node. In this split operation, the entries in the new original page can be sorted based on their key values, and the data corresponding to the sequence exceeding 4KB is written to the newly added node.

[0103] For example, the storage threshold of a page is 4 entries, and the data stored in the original page includes data corresponding to entries a, b, e, and f respectively. The data stored in the incremental page includes data corresponding to entries a´, c, d, and g respectively. When data is merged based on the original page, incremental page, and mapping table, data corresponding to entries a´, b, c, d, e, f, and g respectively can be obtained. Since the page storage threshold corresponding to each node is 4 entries, the merged data is split, that is, the data corresponding to entries a´, b, c, and d respectively are stored in the disk page of the first node, and e, f, and g are stored in the disk page corresponding to the second node; the index information of the first node and the index information of the second node are added to the parent node; wherein the first node and the second node are child nodes of the parent node, and the first node and the second node are nodes at the same level in the B+ tree structure.

[0104] Accordingly, after writing the partitioned data to the newly added node, the index information of the newly added node is updated in the root node corresponding to the current node. This index information includes the disk page ID (pgid) of the newly added node. The index information in the parent node is also updated incrementally, avoiding duplicate writes and unnecessary disk I / O operations.

[0105] Through the embodiments of the present application, the latest incremental record is directly located through a pointer, avoiding the high overhead of traversing the entire incremental page linked list; based on the index support provided by the mapping table, the efficiency of querying data is optimized; when updating data, only the incremental record is written, and there is no need to repeatedly write the entire disk page, reducing the overhead of disk I / O operations; based on threshold judgment, delayed merging, and pointers based on the mapping table, the latest incremental data is quickly located, reducing the high cost of layer-by-layer merging and reducing unnecessary traversal and merging during the data merging process; based on the mapping table and the linked list structure corresponding to each entry in the incremental page, it can support tracing of multiple versions of data of the same key and high concurrency updates; the organization method of incremental data of the single linked list structure corresponding to the entry and dynamic delayed merging are also applicable to the read and write requirements of high concurrency scenarios, reducing the conflict between storage and performance; through the incremental update organization method of the single linked list, dynamic expansion is supported and write amplification problems are reduced; pointer positioning is combined during reading and writing to avoid frequent rewriting and improve read and write performance; keys are sorted during the merging process to ensure data consistency and avoid repeated traversal and merging.

[0106] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0107] Corresponding to the data management method based on B+ tree provided in the above embodiment, Figure 5 As shown, an embodiment of the present application provides a data management device based on a B+ tree. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0108] The device includes:

[0109] A first processing unit 51 is configured to write the node data to an original page of a disk in response to a first write operation on the node data of the B+ tree; the original page includes an original pointer;

[0110] A second processing unit 52 is configured to, in response to a first write operation on a target entry in the original page, create a first incremental record in an incremental page based on the original page; the first incremental record is configured to record first data corresponding to the first write operation, and a first pointer to the target entry in the original page indicates first address information of the first incremental record;

[0111] A third processing unit 53 is configured to create a second incremental record in the incremental page in response to a second write operation on the target entry; the second incremental record is configured to record second data corresponding to the second write operation; the first pointer of the target entry in the original page indicates second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record;

[0112] The original pointer is used to indicate the incremental page associated with the original page.

[0113] In one possible implementation, the first processing unit 51 is also used to create a mapping table corresponding to the original page when writing the node data to the original page of the disk; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

[0114] In one possible implementation, the first processing unit 51 is also used to record the mapping relationship between the target entry and the first incremental record in the mapping table; the mapping relationship includes the first address information of the first incremental record; the second processing unit 52 is also used to update the first address information in the mapping relationship to the second address information of the second incremental record.

[0115] In a possible implementation, the device further includes a merging processing unit configured to merge the data recorded on the original page with the data recorded on the incremental page to obtain a new original page when the number of the mapping relationships recorded in the mapping table is greater than a first threshold.

[0116] In a possible implementation, the merging processing unit is further configured to merge the data recorded on the original page with the data recorded on the incremental page to obtain a new original page when the size of the incremental page is greater than a second threshold.

[0117] In a possible implementation, the merge processing unit is further configured to merge the data recorded in the latest incremental record corresponding to the target entry in the incremental page into the original page to obtain a new original page.

[0118] In one possible implementation, the merge processing unit is also used to determine the latest incremental record corresponding to the target entry in the incremental page based on the address information recorded in the mapping table; and merge the data recorded in the latest incremental record into the original page to obtain a new original page.

[0119] In one possible implementation, the device also includes a splitting processing unit, which is used to split the node data of the current node when the size of the new original page is greater than a third threshold; write the data separated based on the splitting operation into the newly added node; and update the index information of the newly added node in the parent node of the current node.

[0120] Through the embodiments of the present application, the latest incremental record is directly located through a pointer, avoiding the high overhead of traversing the entire incremental page linked list; based on the index support provided by the mapping table, the efficiency of querying data is optimized; when updating data, only the incremental record is written, and there is no need to repeatedly write the entire disk page, reducing the overhead of disk I / O operations; based on threshold judgment, delayed merging, and pointers based on the mapping table, the latest incremental data is quickly located, reducing the high cost of layer-by-layer merging and reducing unnecessary traversal and merging during the data merging process; based on the mapping table and the linked list structure corresponding to each entry in the incremental page, it can support tracing of multiple versions of data of the same key and high concurrency updates; the organization method of incremental data of the single linked list structure corresponding to the entry and dynamic delayed merging are also applicable to the read and write requirements of high concurrency scenarios, reducing the conflict between storage and performance; through the incremental update organization method of the single linked list, dynamic expansion is supported and write amplification problems are reduced; pointer positioning is combined during reading and writing to avoid frequent rewriting and improve read and write performance; keys are sorted during the merging process to ensure data consistency and avoid repeated traversal and merging.

[0121] Figure 6 A schematic diagram of the hardware structure of the electronic device 6 is shown.

[0122] like Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown), a memory 61, wherein the memory 61 stores a computer program 62 that can be run on the processor 60. When the processor 60 executes the computer program 62, the steps in the above method embodiment are implemented, such as Figure 3 Alternatively, when the processor 60 executes the computer program 62, the functions of the modules / units in the above-mentioned device embodiments are realized.

[0123] It should be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 6. In other embodiments of the present application, the electronic device 6 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0124] The electronic device 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of electronic device 6 and does not constitute a limitation of electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include an input sending device, a network access device, a bus, etc.

[0125] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0126] Processor 60 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 60 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 60. If processor 60 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 60 latency, and thus improves system efficiency.

[0127] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard drive or memory of the electronic device 6. The memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the memory 61 may include both an internal storage unit of the electronic device 6 and an external storage device. The memory 61 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of a computer program. The memory 61 may also be used to temporarily store data that has been sent or is about to be sent.

[0128] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0129] It should be noted that the structure of the above-mentioned electronic device is only illustrative, and based on different application scenarios, it may also include other physical structures, and the physical structure of the electronic device is not limited here.

[0130] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0131] An embodiment of the present application also provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to implement the steps in the above-mentioned method embodiments.

[0132] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0133] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0134] If the integrated modules / units are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above-mentioned embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned method embodiments. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. Computer-readable media can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0135] The devices, electronic devices, chip systems, computer storage media, and computer program products provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0136] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application. Based on the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, certain steps in each embodiment of the above detection method may be unnecessary, or certain new steps may be added. Or a combination of any two or any multiple embodiments described above. Such modifications, changes, or combined solutions also fall within the scope of the embodiments of the present application.

[0137] It should also be understood that the division of the modes, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various modes, categories, situations and embodiments can be combined without contradiction.

[0138] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0139] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0140] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0142] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

[0143] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data management method based on B+ tree, characterized in that: The method comprises: In response to a first write operation on node data of the B+ tree, the node data is written to an original page of the disk; the original page includes an original pointer; In response to a first write operation on a target entry in the original page, a first incremental record is created in an incremental page based on the original page; the first incremental record is used to record first data corresponding to the first write operation, and a first pointer to the target entry in the original page indicates first address information of the first incremental record; In response to a second write operation on the target entry, a second incremental record is created in the incremental page; the second incremental record is used to record second data corresponding to the second write operation; the first pointer of the target entry in the original page is updated from the first address information to the second address information indicating the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record; The original pointer is used to indicate the incremental page associated with the original page.

2. The method according to claim 1, characterized in that The method further comprises: When the node data is written to the original page of the disk, a mapping table corresponding to the original page is created; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

3. The method according to claim 2, characterized in that After creating the first incremental record in the incremental page, the method further includes: In the mapping table, a mapping relationship between the target entry and the first incremental record is recorded; the mapping relationship includes the first address information of the first incremental record; After creating the second incremental record in the incremental page, the method further includes: The first address information in the mapping relationship is updated to the second address information of the second incremental record.

4. The method according to claim 3, characterized in that The method further comprises: When the number of the mapping relationships recorded in the mapping table is greater than a first threshold, the data recorded on the original page and the data recorded on the incremental page are merged to obtain a new original page.

5. The method according to claim 1, wherein The method further comprises: When the size of the incremental page is greater than a second threshold, the data recorded in the original page and the data recorded in the incremental page are merged to obtain a new original page.

6. The method according to claim 4 or 5, characterized in that The merging of the data recorded on the original page with the data recorded on the incremental page includes: The data recorded in the latest incremental record corresponding to the target entry in the incremental page is merged into the original page to obtain a new original page.

7. The method according to claim 4, characterized in that The merging of the data recorded on the original page with the data recorded on the incremental page includes: Determining the latest incremental record corresponding to the target entry in the incremental page based on the address information recorded in the mapping table; The data recorded in the latest incremental record is merged into the original page to obtain a new original page.

8. The method according to claim 4 or 5, characterized in that After merging the data recorded on the original page with the data recorded on the incremental page to obtain a new original page, the method further includes: When the size of the new original page is greater than a third threshold, splitting the node data of the current node; Writing the data separated based on the split operation into the newly added node; In the parent node of the current node, the index information of the newly added node is updated.

9. An electronic device, characterized in that: The electronic device includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 8.

10. A computer program product, characterized in that When the computer program product is run on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 8.

Images