Data management method based on B + tree and electronic equipment

Abstract

The invention provides a data management method based on a B + tree and electronic equipment. Relates to the technical field of data processing. The method comprises the following steps: in response to a first write operation on node data of a B + tree, writing the node data into an original page of a disk; in response to a first write operation on a target entry in the original page, creating a first increment record in the increment page based on the original page; the first increment record is used for recording first data corresponding to the first write operation, and a first pointer of the target entry indicates first address information of the first increment record; in response to a second write operation on the target entry, creating a second increment record in the increment page; the second incremental record is used for recording second data corresponding to the second write operation; the first pointer of the target entry indicates the second address information of the second increment record, and the second pointer of the second increment record indicates the first address information of the first increment record; according to the embodiment of the invention, disk space waste and read-write complexity can be reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data management method and electronic device based on a B+ tree. Background Art

[0002] B+ tree is a balanced tree data structure. As a multi-branch sorted tree, each node usually includes multiple child nodes. B+ tree is widely used in databases and file systems due to its efficient range query and sequential access capabilities. By organizing the data of the node into fixed-size disk pages, B+ tree achieves efficient storage and persistence of large-scale data.

[0003] In practical applications, B+ trees often use Copy on Write (COW) technology to ensure data consistency and support multi-version transactions; that is, when the data in a node is modified, the node is first copied to a new location, and then the modification is completed on the copy. While data modification only involves a small amount of updates, it requires copying the entire disk page, resulting in a waste of disk space and increased read and write complexity. Summary of the invention

[0004] According to various embodiments of the present application, a data management method and electronic device based on a B+ tree are provided, which can reduce disk space waste and lower reading and writing complexity.

[0005] In a first aspect, the present application provides a data management method based on a B+ tree, the method comprising: In response to the first write operation on the node data of the B+ tree, the node data is written to the original page of the disk; the original page contains an original pointer; in response to the first write operation on the target entry in the original page, a first incremental record is created in the incremental page based on the original page; the first incremental record is used to record the first data corresponding to the first write operation, and the first pointer of the target entry in the original page indicates the first address information of the first incremental record; in response to the second write operation on the target entry, a second incremental record is created in the incremental page; the second incremental record is used to record the second data corresponding to the second write operation; the first pointer of the target entry in the original page indicates the second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record; wherein the original pointer is used to indicate the incremental page associated with the original page.

[0006] Through the above method, the target entry of the original page includes a first pointer. When a write operation is performed on the data of the target entry, the update operation on the data of the original page is recorded in the form of an incremental page; and after each write operation, the first pointer in the target entry based on the original page always indicates the address information of the latest incremental page. When the data is updated, there is no need to traverse and copy the linked list data of the entire disk page, and the latest incremental data can be directly located, thereby reducing the waste of disk space and reducing the complexity of read and write operations; it has strong ease of use and practicality.

[0007] In a possible implementation manner of the first aspect, the method further includes: When the node data is written to the original page of the disk, a mapping table corresponding to the original page is created; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

[0008] In a possible implementation manner of the first aspect, after creating the first incremental record in the incremental page, the method further includes: In the mapping table, a mapping relationship between the target entry and the first incremental record is recorded; the mapping relationship includes the first address information of the first incremental record; After creating the second incremental record in the incremental page, the method further includes: The first address information in the mapping relationship is updated to the second address information of the second incremental record.

[0009] In a possible implementation manner of the first aspect, the method further includes: When the number of the mapping relationships recorded in the mapping table is greater than a first threshold, the data recorded in the original page is merged with the data recorded in the incremental page to obtain a new original page.

[0010] In a possible implementation manner of the first aspect, the method further includes: When the size of the incremental page is greater than a second threshold, the data recorded in the original page is merged with the data recorded in the incremental page to obtain a new original page.

[0011] In a possible implementation manner of the first aspect, the merging of the data recorded in the original page with the data recorded in the incremental page includes: The data recorded in the latest incremental record corresponding to the target entry in the incremental page is merged into the original page to obtain a new original page.

[0012] In a possible implementation manner of the first aspect, the merging of the data recorded in the original page with the data recorded in the incremental page includes: Based on the address information recorded in the mapping table, determining the latest incremental record corresponding to the target entry in the incremental page; The data recorded in the latest incremental record is merged into the original page to obtain a new original page.

[0013] In a possible implementation manner of the first aspect, after merging the data recorded in the original page with the data recorded in the incremental page to obtain a new original page, the method further includes: When the size of the new original page is greater than a third threshold, splitting the node data of the current node; Writing the data separated based on the split operation into the newly added node; In the parent node of the current node, the index information of the newly added node is updated.

[0014] In a second aspect, the present application provides a data management device based on a B+ tree, comprising: A first processing unit, configured to write the node data to an original page of a disk in response to a first write operation on the node data of the B+ tree; the original page includes an original pointer; a second processing unit, configured to create a first incremental record in an incremental page based on the original page in response to a first write operation on a target entry in the original page; the first incremental record is used to record first data corresponding to the first write operation, and a first pointer to the target entry in the original page indicates first address information of the first incremental record; a third processing unit, configured to create a second incremental record in the incremental page in response to a second write operation on the target entry; the second incremental record is used to record second data corresponding to the second write operation; the first pointer of the target entry in the original page indicates second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record; The original pointer is used to indicate the incremental page associated with the original page.

[0015] In a third aspect, the present application provides an electronic device, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute any one of the methods described in the first aspect.

[0016] In a fourth aspect, the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method described in the first aspect.

[0017] In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0018] In a sixth aspect, the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the method described in the first aspect above.

[0019] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A schematic diagram of the architecture of the storage model of the B+ tree provided in the embodiment of the present application; Figure 2 A schematic diagram of the structure of a single linked list of target entries in an incremental page provided in an embodiment of the present application; Figure 3 A schematic diagram of the implementation process of the data management method based on the B+ tree provided in the embodiment of the present application; Figure 4 A schematic diagram of the overall architecture of updating data provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a data management device based on a B+ tree provided in an embodiment of the present application; Figure 6 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following embodiments of the technical solution of the present application are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0024] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0025] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0026] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0027] By organizing the node data into fixed-size disk pages, the B+ tree achieves efficient storage and persistence of large-scale data. In practical applications, B+ trees often use Copy on Write (COW) to ensure data consistency and support multi-version transactions. However, data modification only involves a small number of updates, but the entire disk page needs to be copied, which has the problem of write amplification and generates more redundant data, resulting in a waste of disk space; each time a new version of data is generated, the process of copying and writing node data to disk increases the burden on read and write performance; and each time the node data is updated, it will affect the parent node and root node, resulting in the rewriting of multiple layers of nodes and increasing the write cost.

[0028] At present, related technologies include using delta pages to record update operations on disk page data, and appending the incremental page after each update to the original page, thereby reducing input / output I / O operations on the disk and simplifying data management during subsequent data merging.

[0029] Based on the above method, when data modification occurs in each disk operation, a new current incremental page will be generated, all the current updated data will be written to the current incremental page, and the incremental page identification field in the original page will be updated to the current incremental page identification. When data is updated frequently, and when multiple data are updated each time, the generated incremental pages form a long chain. Each time the data is read, it is necessary to traverse the entire incremental page chain list to build the latest data structure view, which significantly increases the complexity of the read operation and leads to a decrease in read performance; the read performance will be improved later by merging the data of the incremental pages. When the amount of data in the above incremental pages is huge, resource contention may occur, resulting in a decrease in the instantaneous performance of the system.

[0030] In response to the above technical problems, an embodiment of the present application provides a data management method based on a B+ tree, which stores updated data in an incremental page manner, and in the storage model of the B+ tree, adds a pointer field to each entry in the original page to indicate the latest incremental record in the incremental page. When updating data, there is no need to copy the data of the entire disk page. When reading data, the latest data can be quickly located based on the indication of the pointer field of each entry in the original page, thereby reducing the read and write complexity and improving the read and write performance of the disk.

[0031] The following describes the specific implementation process of the B+ tree-based data management method provided in the embodiment of the present application.

[0032] Please refer to Figure 1 , Figure 1 A schematic diagram of the storage model of the B+ tree provided in the embodiment of the present application. In the data storage process of the B+ tree, the node data of the B+ tree is stored in the disk in the form of pages. The node can be an abstraction of a disk page in the B+ tree, representing a data structure that has been deserialized and loaded into the memory.

[0033] Among them, B+ tree is a balanced tree structure used for indexing databases and file systems. Disk page is the physical unit of data storage. In the database, data is stored on the disk in pages. The size of the page is fixed, such as 4KB, 8KB, etc. The node of the B+ tree is a logical unit in the B+ tree structure, which is used to store key values ​​and pointers indicating child nodes. In memory, a disk page is read from the disk and loaded into the memory, and the disk page is abstracted as a node of the B+ tree. When the node of the B+ tree needs to be persisted to the disk, it will be serialized into a byte stream and then stored in one or more disk pages; when the node data of the B+ tree is read from the disk, the corresponding disk page is read into the memory and deserialized into the node data structure of the B+ tree. Due to the limited memory space, only the accessed node or its adjacent nodes will be loaded into the memory. When a node needs to be accessed, if the node is not in the memory, the database will first read the corresponding page (that is, the physical storage form of the node) from the disk, and then deserialize it into the node data structure in the memory.

[0034] In the storage model of the B+ tree, when the data of a node is first written from memory to the persistent storage disk, the data of the node is solidified into a fixed basic state, which can be called the base page. Subsequent data updates and other operations are expanded and managed based on the base page.

[0035] like Figure 1 As shown, the original page may include a newly added field (dp), the current page representation (id), the number of key-value pairs (count), the number of overflow pages (overflow), and a byte array (data) storing the page content. Among them, the newly added field (dp), the current page representation (id), the number of key-value pairs (count), and the number of overflow pages (overflow) constitute the page header data (PageHeader) of the original page, and the newly added field is used to indicate the original pointer indicating the incremental page; the byte array (data) includes all stored entries, such as the first entry (inode-1), the second entry (inode-1), etc.

[0036] Exemplarily, the data structure of a node of a B+ tree may include the disk page identifier (pgid) corresponding to the current node, the keyword (or key value) of the current node, and all entries (inodes) stored in the current node. Inode represents a single entry stored in a node, and the data structure of a single entry includes a pointer field (next), a saved key value, an identifier (pgid) indicating the target page, a flag (flags) of the current node, and a value (value) associated with the keyword; the entry is used to provide index information of related data.

[0037] Among them, the pointer field is used to indicate the address information of the incremental page; the saved key value is used to indicate sorting according to the key value, so as to support efficient search, insertion and deletion of nodes; the identifier indicating the target page is used to record the child node indicated by the internal node; the mark of the current node is used to mark whether the current node is a leaf node or a non-leaf node; if the current node is a leaf node, the value field correspondingly stores the value associated with the key; if it is an internal node (non-leaf node), the value field is not used, and the data is saved in the child node by indicating the identifier (pgid) of the target page.

[0038] Accordingly, if Figure 1 In the structure of the original page and storage model on disk shown in the figure, the references of the fields in each entry (such as next, key, pgid, flags, and value) are the same as those in the node data structure. The page header data of the original page corresponds to the attributes of the original page, and the page header data of the incremental page corresponds to the attributes of the incremental page.

[0039] like Figure 1 As shown in the figure, the newly added field (dp) in the original page is used to indicate the incremental page associated with it, and the incremental page is used to store all incremental updated data on the node. When the data of an entry in the original page changes (such as the data of the first entry inode-1 changes), a new incremental record (such as inode-1') is created in the incremental page, and the pointer field (next) of the first entry in the original page indicates the location information of the incremental record of the incremental page, such as the starting offset (Offset Position).

[0040] When the data is changed multiple times, if the data of the first entry is changed again, an incremental record (such as inode-1´´) is added to the incremental page, and the address information indicated by the pointer of the first entry inode-1 is updated to indicate the address information of the incremental record inode-1´´, such as the starting offset (OffsetPosition) of the incremental record inode-1´´; and so on, when the data of the first entry is changed again, an incremental record (such as inode-1´´) is added to the incremental page, and the pointer of the first entry inode-1 is updated to indicate the position information (such as the starting offset) of inode-1´´. This makes the pointer of the first entry always point to the data in the latest incremental record.

[0041] Accordingly, with multiple change operations, the update data corresponding to each entry in the incremental page will form a single linked list structure, and each incremental record in the linked list is linked to the previous version; for example, the pointer of inode-1´´´ indicates the starting offset of inode-1´´, and the pointer of inode-1´´ indicates the offset of inode-1´. Figure 2 The first entry inode-1 in the original page shown corresponds to the single linked list structure in the incremental page; since the next pointer of inode-1 always points to the latest incremental record, the read operation does not need to traverse all the incremental page hierarchical linked lists, but directly locates the data of the latest incremental record, which significantly improves the reading efficiency.

[0042] Exemplarily, when the data of other items in the original page are updated, the implementation principle is the same as the above.

[0043] Based on the architecture of the above storage model, the specific implementation process of the data management method based on the B+ tree is further introduced below through embodiments.

[0044] See also Figure 3 , Figure 3 The following is a flow chart of the implementation of the B+ tree-based data management method provided in the embodiment of the present application. The execution subject of the method may be an electronic device, which includes a database or a file system; the method may include the following steps: S301, in response to the first write operation on the node data of the B+ tree, the node data is written into an original page of the disk; the original page includes an original pointer.

[0045] In the embodiment of the present application, in the B+ tree, each tree node contains a set of key-value pairs, the key is used to sort and retrieve data, and the value is the corresponding data, such as Figure 4 As shown in the figure, the node memory structure corresponding to the node architecture of the B+ tree, node 3 includes the key: a (data entry or object); the data corresponding to the value includes: the corresponding disk page identifier pgid = 3, the pointer field next, the node type flags and the specific data value = 123. In the memory, the node data can exist in the form of the aforementioned data structure.

[0046] Exemplarily, before writing the node data from the memory to the disk for the first time, the structure of the node data in the memory is serialized into a continuous byte stream, and the disk is divided into pages of fixed size; using the input / output I / O operation of the database or file system, the continuous byte stream is written to the corresponding disk page, and the node data is solidified into a fixed basic state, which is called the state of the original page of data storage; the original page is a persistent representation of the node data on the disk, such as Figure 4As shown in the original page, it contains the key in the node data (key=3), the corresponding disk page identifier pgid=3, the pointer field next, the node type flags and the specific data value=123.

[0047] The size of a node can be less than or equal to the size of a disk page, thereby ensuring that the node data can be completely stored in a disk page. When the data is subsequently updated (such as writing data), data expansion and management are performed based on the original page.

[0048] In some embodiments, the method further comprises: When the node data is written to the original page of the disk, a mapping table corresponding to the original page is created; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

[0049] For example, Figure 4 The mapping table shown in the figure represents the correspondence between each entry in the original page (such as key=a) and the incremental record in the incremental page in the form of a key and value pair. The value in the correspondence includes the address information of the incremental record corresponding to the entry in the incremental page; the address information can be represented by the next pointer field.

[0050] When the data of the original page is not updated, the fields included in the value in the mapping table may be empty; correspondingly, after a data update operation occurs, the mapping table records the mapping relationship corresponding to the updated entry.

[0051] S302 , in response to a first write operation on a target entry in an original page, create a first incremental record in an incremental page based on the original page.

[0052] In the embodiment of the present application, an entry is a data structure used in a file system to manage files or directories and is used to store information related to the files or directories; Figure 1 As shown, the first entry inode-1 and the second entry inode-2 in the original page contain various field information; the target entry is any entry of the original page with data to be updated.

[0053] Exemplarily, when an electronic device receives a data update operation corresponding to a target entry of an original page, such as a first write operation to write first data to a target entry, it creates an incremental page corresponding to the target entry based on the original page, and creates an incremental record in the incremental page, such as the first incremental record inode-1´.

[0054] The first incremental record is used to record the first data corresponding to the first write operation, and the first pointer of the target entry in the original page indicates the first address information of the first incremental record. Figure 1 As shown, after the first incremental record inode-1' is created, the first pointer next of the target entry inode-1 in the original page indicates the first address information of the first incremental record, that is, the starting offset of inode-1'.

[0055] In some embodiments, after creating the first delta record in the delta page, the method further includes: In the mapping table, a mapping relationship between the target entry and the first incremental record is recorded; the mapping relationship includes the first address information of the first incremental record.

[0056] For example, Figure 4 As shown, if a write operation is performed on a target entry (such as the key of the target entry key=a), the target entry a is searched in the node memory corresponding to node 3 of the B+ tree structure. If it does not exist, it is necessary to further determine whether there is a record of the target entry a in the mapping table.

[0057] Among them, if the target entry a does not exist in node 3 and does not exist in the mapping table, it means that the data written corresponding to the target entry a is the first newly added data, then the first incremental record inode-1' is created in the incremental page, and a mapping relationship between the target entry and the first incremental record is added to the mapping table, and the next pointer in the mapping relationship indicates the address information of the first incremental record (such as the starting offset); Figure 4 The next pointer in the mapping table shown in indicates the starting offset of the first incremental record inode-1' in the incremental page (such as Figure 4 (as indicated by the solid arrow in the mapping table).

[0058] Alternatively, in response to the first write operation, the target entry a is added to the mapping table, and the first data corresponding to the target entry a is written in the incremental page, such as value=456, recorded as the first incremental record, and the starting offset of the first incremental record is saved in the mapping table and associated with the target entry a.

[0059] For example, if the target entry a exists in node 3 and does not exist in the mapping table, it means that the target entry a has not been updated, and the first incremental record inode-1' is created in the incremental page, and a mapping relationship between the target entry and the first incremental record is added to the mapping table, and the next pointer in the mapping relationship indicates the address information of the first incremental record (such as the starting offset); Figure 4The next pointer in the mapping table shown in indicates the starting offset of the first incremental record inode-1' in the incremental page (such as Figure 4 (as indicated by the solid arrow in the mapping table).

[0060] S303, in response to a second write operation on the target entry, create a second incremental record in the incremental page; the second incremental record is used to record second data corresponding to the second write operation.

[0061] In an embodiment of the present application, when the electronic device receives multiple update operations on the data corresponding to the target entry of the original page, such as a second write operation to write the first data to the target entry, it creates an incremental record again in the incremental page corresponding to the created target entry, such as the second incremental record inode-1´´.

[0062] The second incremental record is used to record the second data corresponding to the second write operation, and the first pointer of the target entry in the original page indicates the second address information of the second incremental record. Figure 1 As shown, after creating the second incremental record inode-1´´, the first pointer next of the target entry inode-1 in the original page indicates the second address information of the second incremental record, that is, the starting offset of the second incremental record inode-1´´; the second pointer of the second incremental record indicates the first address information of the first incremental record, that is, the starting offset of the first incremental record inode-1´.

[0063] In some embodiments, after creating the second incremental record in the incremental page, the method further includes: updating the first address information in the mapping relationship to the second address information of the second incremental record.

[0064] Exemplarily, the first incremental record is a data update version of the previous second incremental record. Figure 4 As shown, if the target entry (such as the key key=a of the target entry) is updated (such as a write operation), the target entry a is searched in the node memory corresponding to node 3 of the B+ tree structure. If it does not exist, it is necessary to further determine whether there is a record of the target entry a in the mapping table.

[0065] Among them, if the target entry a does not exist in node 3, but exists in the mapping table, it means that the data written to the target entry a is new data, and a second incremental record inode-1'' is created in the incremental page; and the mapping relationship between the target entry in the mapping table and the first incremental record is updated to a mapping relationship with the second incremental record, that is, the next pointer in the mapping relationship indicates the address information (such as the starting offset) of the second incremental record; Figure 4The next pointer in the mapping table shown in indicates the starting offset of the second incremental record inode-1'' in the incremental page (such as Figure 4 (as indicated by the dashed arrow in the mapping table).

[0066] Exemplarily, if the target entry a exists in node 3 and the mapping relationship record corresponding to the target entry a exists in the mapping table, it means that the data of a has been updated. At this time, when the write operation is performed again for update, a second incremental record inode-1´´ is added to the incremental page, and the pointer of the second incremental record inode-1´´ indicates the address information of the previous version of the incremental record, that is, the starting offset of the first incremental record inode-1´; the next pointer in the mapping table is updated to indicate the starting offset of the second incremental record inode-1´´; thereby, the mapping relationship in the mapping table indicates the latest version of the data corresponding to the entry.

[0067] In some embodiments, the method further comprises: When the number of mapping relationships recorded in the mapping table is greater than a first threshold, the data recorded in the original page is merged with the data recorded in the incremental page to obtain a new original page. Alternatively, when the size of the incremental page is greater than a second threshold, the data recorded in the original page is merged with the data recorded in the incremental page to obtain a new original page.

[0068] Exemplarily, in order to optimize reading efficiency and reduce merging overhead, a global merge operation is performed based on the number of entries of the mapping relationship recorded in the mapping table or based on the size of the data stored in the incremental page; when the number of entries of the mapping relationship recorded in the mapping table is greater than a first threshold (such as 100); or when the incremental page corresponding to the entry is greater than a second threshold (such as 4KB), the data stored in the incremental page of the entry is merged into the data of the original page.

[0069] Exemplarily, during the merging process, entries may be sorted based on the keys of the original entries to prevent data with duplicate key values ​​from appearing.

[0070] In some embodiments, the data recorded in the original page and the data recorded in the incremental page are merged, including: merging the data recorded in the latest incremental record corresponding to the target entry in the incremental page into the original page to obtain a new original page.

[0071] Exemplarily, the latest version of data corresponding to each entry in the original page in the incremental page is merged into the original page so that the data of each entry remains up to date, generating a new original page; and the historical data corresponding to each entry in the incremental page is deleted, thereby reducing disk space usage.

[0072] In some embodiments, the data recorded in the original page and the data recorded in the incremental page are merged, including: determining the latest incremental record corresponding to the target entry in the incremental page based on the address information recorded in the mapping table; merging the data recorded in the latest incremental record into the original page to obtain a new original page.

[0073] Exemplarily, the incremental page also includes newly added data (data that does not exist in the node corresponding to the original page), and the address information corresponding to the newly added data is recorded in the mapping table. In the process of merging the incremental page corresponding to the entry, the address information corresponding to the entry recorded in the mapping relationship in the mapping table (such as the starting offset indicated by the next pointer field) can also be combined to obtain the data recorded in the latest incremental record corresponding to the newly added data corresponding to the address information, and merge the data into the original page to obtain a new original page. In this way, the latest incremental record corresponding to the entry in the incremental page can be directly located, avoiding traversing the linked list of incremental pages layer by layer, and improving the merging efficiency; the original page, incremental page and data keys in the mapping table (such as key=a) are globally merged to ensure that the generated new original page data is complete and ordered.

[0074] In some embodiments, after merging the data recorded in the original page with the data recorded in the incremental page to obtain a new original page, the method further includes: When the size of the new original page is greater than the third threshold, the node data of the current node is split; the data separated based on the split operation is written into the newly added node; and the index information of the newly added node is updated in the parent node of the current node.

[0075] Exemplarily, if the size of the generated new original page exceeds the third threshold (such as 4KB), a split operation on the current node is triggered, and the data separated after the split operation is written into the newly added node; for example, data exceeding 4KB is written into the new node. In this split operation, the key value of each entry in the new original page can be sorted, and the data corresponding to the sequence exceeding 4KB is written into the newly added node.

[0076] For example, the storage threshold of a page is 4 entries, the data stored in the original page includes data corresponding to entries a, b, e, and f, respectively, and the data stored in the incremental page includes data corresponding to entries a´, c, d, and g, respectively. When data is merged based on the original page, incremental page, and mapping table, data corresponding to entries a´, b, c, d, e, f, and g can be obtained. Since the page storage threshold corresponding to each node is 4 entries, the merged data is split, that is, the data corresponding to entries a´, b, c, and d are stored in the disk page of the first node, and e, f, and g are stored in the disk page corresponding to the second node; the index information of the first node and the index information of the second node are added to the parent node; wherein the first node and the second node are child nodes of the parent node, and the first node and the second node are nodes at the same level in the B+ tree structure.

[0077] Accordingly, after writing the separated data in the newly added node, the index information of the newly added node is updated in the root node corresponding to the current node, and the index information includes the disk page ID (pgid) of the newly added node. The update of the index information in the parent node also adopts the incremental update method, avoiding repeated writing and unnecessary disk I / O operations.

[0078] Through the embodiments of the present application, the latest incremental record is directly located through a pointer, avoiding the high overhead of traversing the entire incremental page linked list; based on the index support provided by the mapping table, the efficiency of querying data is optimized; when updating data, only the incremental record is written, and there is no need to repeatedly write the entire disk page, reducing the overhead of disk I / O operations; based on threshold judgment, delayed merging, and pointers based on the mapping table, the latest incremental data is quickly located, reducing the high cost of layer-by-layer merging, and reducing unnecessary traversal and merging in the data merging process; based on the linked list structure corresponding to the mapping table and each entry in the incremental page, it can support the tracing of multiple versions of data of the same key and high-concurrency updates; the organization method of incremental data of the single linked list structure corresponding to the entry and the dynamic delayed merging are also applicable to the read and write requirements of high-concurrency scenarios, reducing the conflict between storage and performance; through the incremental update organization method of the single linked list, dynamic expansion is supported to reduce write amplification problems; pointer positioning is combined when reading and writing to avoid frequent rewriting and improve read and write performance; keys are sorted during the merging process to ensure data consistency and avoid repeated traversal and merging.

[0079] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0080] Corresponding to the data management method based on B+ tree provided in the above embodiment, Figure 5As shown, an embodiment of the present application provides a data management device based on a B+ tree. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0081] The device includes: A first processing unit 51 is used for writing the node data to an original page of a disk in response to a first write operation on the node data of the B+ tree; the original page includes an original pointer; A second processing unit 52 is configured to create a first incremental record in an incremental page based on the original page in response to a first write operation on a target entry in the original page; the first incremental record is used to record first data corresponding to the first write operation, and a first pointer to the target entry in the original page indicates first address information of the first incremental record; A third processing unit 53 is configured to create a second incremental record in the incremental page in response to a second write operation on the target entry; the second incremental record is used to record second data corresponding to the second write operation; the first pointer of the target entry in the original page indicates second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record; The original pointer is used to indicate the incremental page associated with the original page.

[0082] In a possible implementation, the first processing unit 51 is also used to create a mapping table corresponding to the original page when writing the node data to the original page of the disk; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

[0083] In a possible implementation, the first processing unit 51 is also used to record, in the mapping table, a mapping relationship between the target entry and the first incremental record; the mapping relationship includes the first address information of the first incremental record; and the second processing unit 52 is also used to update the first address information in the mapping relationship to the second address information of the second incremental record.

[0084] In a possible implementation, the device further includes a merging processing unit for merging the data recorded on the original page with the data recorded on the incremental page to obtain a new original page when the number of the mapping relationships recorded in the mapping table is greater than a first threshold.

[0085] In a possible implementation, the merging unit is further configured to merge the data recorded in the original page with the data recorded in the incremental page to obtain a new original page when the size of the incremental page is greater than a second threshold.

[0086] In a possible implementation, the merge processing unit is further used to merge the data recorded in the latest incremental record corresponding to the target entry in the incremental page into the original page to obtain a new original page.

[0087] In a possible implementation, the merge processing unit is also used to determine the latest incremental record corresponding to the target entry in the incremental page based on the address information recorded in the mapping table; and merge the data recorded in the latest incremental record into the original page to obtain a new original page.

[0088] In one possible implementation, the device also includes a split processing unit, which is used to split the node data of the current node when the size of the new original page is greater than a third threshold; write the data separated based on the split operation into the newly added node; and update the index information of the newly added node in the parent node of the current node.

[0089] Through the embodiments of the present application, the latest incremental record is directly located through a pointer, avoiding the high overhead of traversing the entire incremental page linked list; based on the index support provided by the mapping table, the efficiency of querying data is optimized; when updating data, only the incremental record is written, and there is no need to repeatedly write the entire disk page, reducing the overhead of disk I / O operations; based on threshold judgment, delayed merging, and pointers based on the mapping table, the latest incremental data is quickly located, reducing the high cost of layer-by-layer merging, and reducing unnecessary traversal and merging in the data merging process; based on the linked list structure corresponding to the mapping table and each entry in the incremental page, it can support the tracing of multiple versions of data of the same key and high-concurrency updates; the organization method of incremental data of the single linked list structure corresponding to the entry and the dynamic delayed merging are also applicable to the read and write requirements of high-concurrency scenarios, reducing the conflict between storage and performance; through the incremental update organization method of the single linked list, dynamic expansion is supported to reduce write amplification problems; pointer positioning is combined when reading and writing to avoid frequent rewriting and improve read and write performance; keys are sorted during the merging process to ensure data consistency and avoid repeated traversal and merging.

[0090] Figure 6 A schematic diagram of the hardware structure of the electronic device 6 is shown.

[0091] like Figure 6 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown in the figure), a memory 61, in which a computer program 62 that can be run on the processor 60 is stored. When the processor 60 executes the computer program 62, the steps in the above method embodiment are implemented, such as Figure 3Alternatively, when the processor 60 executes the computer program 62, the functions of the modules / units in the above-mentioned device embodiments are realized.

[0092] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 6. In other embodiments of the present application, the electronic device 6 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0093] The electronic device 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation of the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include an input sending device, a network access device, a bus, etc.

[0094] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0095] The processor 60 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 60 is a cache memory. The memory may store instructions or data that the processor 60 has just used or circulated. If the processor 60 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 60, and thus improves the efficiency of the system.

[0096] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. The memory 61 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 61 may include both an internal storage unit and an external storage device of the electronic device 6. The memory 61 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program. The memory 61 may also be used to temporarily store data that has been sent or is to be sent.

[0097] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] It should be noted that the structure of the above-mentioned electronic device is only illustrative, and based on different application scenarios, it may also include other physical structures, and the physical structure of the electronic device is not limited here.

[0099] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] An embodiment of the present application also provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device implements the steps in the above-mentioned method embodiments.

[0101] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0102] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0103] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0104] The devices, electronic devices, chip systems, computer storage media, and computer program products provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0105] It should be understood that the above is only to help those skilled in the art to better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application. According to the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, some steps in each embodiment of the above detection method may be unnecessary, or some steps may be newly added. Or a combination of any two or any multiple embodiments of the above. Such modifications, changes or combined schemes also fall within the scope of the embodiments of the present application.

[0106] It should also be understood that the division of the methods, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various methods, categories, situations and embodiments can be combined without contradiction.

[0107] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0108] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

[0112] Finally, it should be noted that the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data management method based on B+ tree, characterized in that: The method comprises: In response to a first write operation on node data of the B+ tree, the node data is written to an original page of the disk; the original page includes an original pointer; In response to a first write operation on a target entry in the original page, a first incremental record is created in an incremental page based on the original page; the first incremental record is used to record first data corresponding to the first write operation, and a first pointer to the target entry in the original page indicates first address information of the first incremental record; In response to a second write operation on the target entry, a second incremental record is created in the incremental page; the second incremental record is used to record second data corresponding to the second write operation; the first pointer of the target entry in the original page indicates second address information of the second incremental record, and the second pointer of the second incremental record indicates the first address information of the first incremental record; The original pointer is used to indicate the incremental page associated with the original page.

2. The method according to claim 1, characterized in that The method further comprises: When the node data is written to the original page of the disk, a mapping table corresponding to the original page is created; the mapping table is used to record the correspondence between the data of the target entry in the original page and the address information in the incremental page.

3. The method according to claim 2, characterized in that After creating the first incremental record in the incremental page, the method further includes: In the mapping table, a mapping relationship between the target entry and the first incremental record is recorded; the mapping relationship includes the first address information of the first incremental record; After creating the second incremental record in the incremental page, the method further includes: The first address information in the mapping relationship is updated to the second address information of the second incremental record.

4. The method according to claim 3, characterized in that The method further comprises: When the number of the mapping relationships recorded in the mapping table is greater than a first threshold, the data recorded in the original page is merged with the data recorded in the incremental page to obtain a new original page.

5. The method according to claim 1, characterized in that The method further comprises: When the size of the incremental page is greater than a second threshold, the data recorded in the original page is merged with the data recorded in the incremental page to obtain a new original page.

6. The method according to claim 4 or 5, characterized in that: The merging of the data recorded in the original page with the data recorded in the incremental page includes: The data recorded in the latest incremental record corresponding to the target entry in the incremental page is merged into the original page to obtain a new original page.

7. The method according to claim 4, characterized in that The merging of the data recorded in the original page with the data recorded in the incremental page includes: Based on the address information recorded in the mapping table, determining the latest incremental record corresponding to the target entry in the incremental page; The data recorded in the latest incremental record is merged into the original page to obtain a new original page.

8. The method according to claim 4 or 5, characterized in that: After merging the data recorded in the original page with the data recorded in the incremental page to obtain a new original page, the method further includes: When the size of the new original page is greater than a third threshold, splitting the node data of the current node; Writing the data separated based on the split operation into the newly added node; In the parent node of the current node, the index information of the newly added node is updated.

9. An electronic device, characterized in that: The electronic device includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 8.

10. A computer program product, characterized in that When the computer program product runs on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 8.

Images