B+ tree-based chunk backup index method, system, device and storage medium

Abstract

The application relates to a B+ tree-based block backup index method, system, device and storage medium, and belongs to the technical field of disaster recovery backup. The method comprises the following steps: intercepting a QEMU block write request and extracting an IO descriptor, writing block data into a disk and obtaining a physical offset, initializing a two-dimensional B+ tree index structure, inserting the block IO descriptor and offset information into the two-dimensional B+ tree, querying a block data position according to a logical offset and / or a timestamp range, and deleting and updating an index node according to a specified range. The system comprises a block data interception module, a block data writing module, a B+ tree initialization module, a block data insertion module, a block data searching module and a block data deleting module. Through the two-dimensional hierarchical index structure of the logical offset and the timestamp, the application realizes joint efficient positioning in the time and space dimensions, supports sharing of the timestamp leaf node between the local and global B+ trees, effectively avoids data redundancy, and improves the range query efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of disaster recovery backup, and relates to a block backup index method, system and device based on a B+ tree and a storage medium. BACKGROUND

[0002] With the continuous expansion of data scale, block-level storage has been widely used in cloud computing, virtualization platforms, database systems and distributed file systems, and has become one of the important infrastructures. The block-level storage divides the data into blocks of fixed length, and directly addresses by offset. The offset locates the starting position of the data, and the length specifies the data amount of the operation, which together determines the read-write range, not only avoids the path resolution overhead in the traditional file system, but also is more suitable for supporting high-frequency random read-write operations.

[0003] In a huge block set, efficient data access depends on the system's ability to quickly locate the target block position. Therefore, the data backup system usually constructs a structured index mechanism, which assigns a unique identifier to each block data, such as a hash value, a timestamp or a logical offset address, and establishes a mapping relationship between it and the corresponding physical storage location, so as to realize the quick and accurate retrieval of the target block.

[0004] Among many index structures, B+ tree is widely used in block-level backup systems to establish the mapping relationship between block data identifier and physical address. B+ tree is a multi-way balanced search tree, whose non-leaf nodes are used for navigation, and all data is stored in leaf nodes and connected in sequence by a linked list. This structure not only supports efficient point lookup, but also can quickly perform range query and sequential scanning, and is widely used in block data index scenarios, especially suitable for disk block alignment to reduce IO. However, existing systems often construct single-dimensional index structures, such as indexing based only on logical offset or write time. Although this approach can meet the basic block positioning needs, it is difficult to support more complex joint queries in the multi-version scenario, such as filtering data based on spatial location and time information at the same time, resulting in low efficiency in handling overlapping writes, snapshot recovery and other needs.

[0005] Therefore, it has become a key problem to design a structure that supports spatial and temporal joint indexing to improve block data retrieval efficiency. SUMMARY

[0006] The application provides a block backup index method, system, device and storage medium based on B+ tree to solve the technical problems in the background art.

[0007] The technical solution of the application to solve the above technical problems is as follows:

[0008] In a first aspect, a block backup index method based on a B+ tree is provided, and the method comprises the following steps:

[0009] A block data interception step intercepts an IO request at an entrance of a block write function of QEMU, and extracts a corresponding block IO descriptor and block data of the IO request, the block IO descriptor comprising a timestamp, a logical offset and a data length;

[0010] A block data writing step writes the block data in the IO request to a disk, and obtains a historical block physical offset of the block data in the disk;

[0011] A B+ tree initialization step assigns the block IO descriptor and the historical block physical offset to a two-dimensional B+ tree leaf node for encapsulation, and initializes the two-dimensional B+ tree, the two-dimensional B+ tree comprising an offset B+ tree, a global timestamp B+ tree and a local timestamp B+ tree, each offset B+ tree leaf node pointing to a local timestamp B+ tree, wherein the offset B+ tree leaf node comprises a logical offset, a data length, a local timestamp B+ tree offset and an offset sequence pointer; the global timestamp B+ tree and the local timestamp B+ tree share a timestamp B+ tree leaf node, and the timestamp B+ tree leaf node comprises a timestamp, a logical offset, a data length, a historical block physical offset, a local sequence pointer and a global sequence pointer;

[0012] A block data insertion step inserts corresponding block data into the disk according to a block IO descriptor to be inserted, and creates a corresponding leaf node in the initialized two-dimensional B+ tree structure;

[0013] A block data searching step searches in the two-dimensional B+ tree according to a specified logical offset range and / or a timestamp range, obtains a historical block physical offset of target block data in the disk, and reads the target block data;

[0014] A block data deletion step positions and deletes a matching leaf node in the two-dimensional B+ tree according to a specified logical offset range and / or a timestamp range, and then updates the two-dimensional B+ tree leaf node.

[0015] In a second aspect, a block backup index system based on a B+ tree is provided, and the system comprises:

[0016] A block data interception module intercepts an IO request at an entrance of a block write function of QEMU, and extracts a corresponding block IO descriptor and block data of the IO request, the block IO descriptor comprising a timestamp, a logical offset and a data length;

[0017] A block data writing module writes the block data in the IO request to a disk, and obtains a historical block physical offset of the block data in the disk;

[0018] The B+ tree initialization module encapsulates the block IO descriptor and the historical block physical offset into a two-dimensional B+ tree leaf node, initializes the two-dimensional B+ tree, and the two-dimensional B+ tree comprises an offset B+ tree, a global timestamp B+ tree and a local timestamp B+ tree, each offset B+ tree leaf node points to a local timestamp B+ tree, wherein the offset B+ tree leaf node comprises a logical offset, a data length, a local timestamp B+ tree offset and an offset sequence pointer; the global timestamp B+ tree shares a timestamp B+ tree leaf node with the local timestamp B+ tree, and the timestamp B+ tree leaf node comprises a timestamp, a logical offset, a data length, a historical block physical offset, a local sequence pointer and a global sequence pointer;

[0019] The block data insertion module inserts the corresponding block data into the disk according to the block IO descriptor to be inserted, and newly creates corresponding leaf nodes in the initialized two-dimensional B+ tree structure.

[0020] The block data searching module searches in the two-dimensional B+ tree according to the specified logical offset range and / or timestamp range, obtains the historical block physical offset of the target block data in the disk, and reads the target block data.

[0021] The block data deletion module positions and deletes the matched leaf nodes in the two-dimensional B+ tree according to the specified logical offset range and / or timestamp range, and then updates the two-dimensional B+ tree leaf nodes.

[0022] In a third aspect, an electronic device is provided, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to execute the above-mentioned B+ tree-based block backup index method.

[0023] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the above-mentioned B+ tree-based block backup index method.

[0024] The present application has the following advantages:

[0025] (1) The present application realizes the joint positioning of the time dimension and the space dimension of the block data through the two-dimensional index structure of the logical offset and the timestamp, significantly improves the query flexibility and the retrieval efficiency;

[0026] (2) The present application uses the hierarchical division and the regional index of the offset and the timestamp to quickly locate the local regional data, significantly shortens the range searching and data recovery time;

[0027] (3) The present application adopts the sharing mechanism of the timestamp leaf node between the local and the global B+ tree, compared with the traditional multi-tree independent index structure, effectively avoids the data redundancy, saves the storage resources and simplifies the updating operation. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of the block backup indexing method based on B+ tree in Embodiment 1 of the present invention.

[0030] Figure 2 This is a schematic diagram of the two-dimensional B+ leaf node structure in Embodiment 1 of the present invention.

[0031] Figure 3 This is a schematic diagram of the structure of a two-dimensional B+ tree in Embodiment 1 of the present invention.

[0032] Figure 4 This is a flowchart of the block backup indexing method based on B+ tree with real-time block physical offset field in Embodiment 2 of the present invention.

[0033] Figure 5 This is a structural diagram of the block backup index system based on B+ tree in Embodiment 3 of the present invention.

[0034] Figure 6 This is a structural diagram of the block data insertion module in Embodiment 3 of the present invention.

[0035] Figure 7 This is a structural diagram of the block data search module in Embodiment 3 of the present invention.

[0036] Figure 8 This is a structural diagram of the block data deletion module in Embodiment 3 of the present invention.

[0037] Figure 9 This is a schematic diagram of the structure of the electronic device provided in Embodiment 4 of the present invention.

[0038] The attached diagram lists the components represented by each number as follows:

[0039] 301, block data interception module; 302, block data writing module; 303, B+ tree initialization module; 304, block data insertion module; 305, block data search module; 306, block data deletion module; 3041, insertion position judgment unit; 3042, block data writing unit; 3043, descriptor reconstruction unit; 3044, node insertion unit; 3045, disk writing and updating unit; 3051, offset range query unit; 3052, timestamp range query unit; 3053, composite range query unit; 3054, block data reading unit; 3061, to-be-deleted node search unit; 3062, node range deletion unit; 3063, timestamp index reconstruction unit; 3064, offset index reconstruction unit; 3065, node writing unit; 410, processor; 420, communication interface; 430, memory; 440, communication bus. DETAILED DESCRIPTION

[0040] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0041] Embodiment 1

[0042] Among many index structures, B+ tree is widely used in block-level backup systems to establish the mapping relationship between block data identification and physical address. B+ tree is a multi-way balanced search tree, whose non-leaf nodes are used for navigation, and all data is stored in leaf nodes and connected in sequence through a linked list. This structure not only supports efficient point search, but also can quickly execute range query and sequential scanning, and is widely used in block data index scenarios, and is particularly suitable for disk block alignment to reduce IO. However, existing systems often construct a single-dimensional index structure, such as indexing based on only logical offset or write time. Although this way can meet the basic block positioning needs, it is difficult to support more complex joint queries in the multi-version scenario, such as filtering data based on spatial location and time information at the same time, resulting in low efficiency in processing overlapping writes, snapshot recovery and other needs. Therefore, designing a structure supporting spatial and temporal joint indexing to improve block data retrieval efficiency has become a key problem to be solved at present.

[0043] In view of the above problems, the embodiment of the present application provides a block backup index method based on B+ tree. Figure 1 The block backup index method based on B+ tree provided by the embodiment of the present application is shown in the flowchart. The method provided by the embodiment has an application environment: the virtualization platform is QEMU, and the operating system platform is Ubuntu.

[0044] As Figure 1As shown, a block backup index method based on B+ tree is provided, which comprises:

[0045] In step S101, the IO request is intercepted at the entrance of the block write function of QEMU, and the corresponding block IO descriptor and block data are extracted, the block IO descriptor including timestamp, logical offset and data length.

[0046] In this embodiment, in order to facilitate understanding of how to obtain the block I / O descriptor and the corresponding block data, the step S101 further comprises:

[0047] S1011, intercepting the IO system call through the function hook mechanism, and the hooked function is blk_aio_pwritev;

[0048] S1012, extracting the logical offset, data length, timestamp and QEMU IO vector contained in the IO request;

[0049] S1013, parsing the QEMU IO vector to obtain the corresponding block data in the memory.

[0050] It is worth noting that the key metadata fields contained in the IO request include: logical offset (offset), data length (length), timestamp (timestamp) and QEMU IO vector (QEMU IOVector), wherein the logical offset is used to identify the starting position of the write request in the virtual disk, the data length indicates the number of bytes of the written content, the timestamp is used to record the occurrence time of the IO request, and the QEMU IO vector is used to locate and extract the written block data.

[0051] In step S102, the block data in the IO request is written to the disk to obtain the historical block physical offset thereof in the disk.

[0052] In step S103, the block IO descriptor and the historical block physical offset are assigned to the two-dimensional B+ tree leaf node for encapsulation, and the two-dimensional B+ tree is initialized, the two-dimensional B+ tree including offset B+ tree, global timestamp B+ tree and local timestamp B+ tree, each offset B+ tree leaf node pointing to a local timestamp B+ tree, wherein the offset B+ tree leaf node includes: logical offset, data length, local timestamp B+ tree offset and offset order pointer; the global timestamp B+ tree and the local timestamp B+ tree share the timestamp B+ tree leaf node, and the timestamp B+ tree leaf node includes: timestamp, logical offset, data length, historical block physical offset, local order pointer and global order pointer.

[0053] It is worth noting that, as Figure 2As shown, the leaf node of the offset B+ tree contains fields including logical offset, data length, local timestamp B+ tree offset, and offset sequential pointer; the leaf node of the timestamp B+ tree contains fields including timestamp, logical offset, data length, history block physical offset, local sequential pointer, and global sequential pointer. The two-dimensional B+ tree structure of the embodiment avoids data redundancy, reduces storage cost, and improves the efficiency and performance of range query by sharing the timestamp leaf node between the local B+ tree and the global B+ tree.

[0054] In order to more clearly understand the two-dimensional B+ tree structure in the embodiment, we provide a structure example as follows. Figure 3 As shown, the left offset B+ tree root node, the upper global timestamp B+ tree root node, and a plurality of local timestamp B+ tree root nodes are included, and they all have corresponding internal nodes; the root nodes of different types of B+ trees adopt a multi-layer index structure, which is used to point to the internal nodes and maintain index entry information, and the internal nodes are used to record and maintain the segmented range boundary information to support efficient hierarchical positioning of data in the leaf nodes; the offset sequential pointers exist between the leaf nodes of the offset B+ tree, which are used to quickly traverse the continuous logical offset range; the global timestamp B+ tree and the local timestamp B+ tree share the timestamp B+ tree leaf node, so there are two kinds of pointers between the timestamp B+ tree leaf nodes, the local timestamp sequential pointer and the global timestamp sequential pointer, the local timestamp sequential pointer connects the timestamp B+ tree leaf nodes in a certain offset range in time sequence, and the global timestamp sequential pointer connects all the timestamp B+ tree leaf nodes in time sequence.

[0055] In step S104, according to the block IO descriptor to be inserted, the corresponding block data is written to the disk, and the corresponding leaf node is newly created in the initialized two-dimensional B+ tree structure.

[0056] Optionally, the step S104 further includes:

[0057] S1041, according to the block IO descriptor of the data to be inserted, performing a search in the offset B+ tree to obtain an insertion position of the block data index;

[0058] S1042, writing the block data to be inserted to the disk to obtain a physical block base address;

[0059] S1043, judge whether the logical offset range of the block data to be inserted intersects with the logical offset range of the leaf node of the offset B+ tree, for the intersected part, construct the block IO descriptor according to the intersected range; for a number of continuous logical subintervals without intersection, for each subinterval, construct a new block IO descriptor, and create a corresponding offset B+ tree leaf node with the starting logical offset of the subinterval and the data length as the index key, and generate the local timestamp B+ tree pointed by it;

[0060] S1044, calculate the physical offset of the block IO descriptor according to the physical block base address, generate the timestamp B+ tree leaf node combined with the block IO descriptor, and synchronize the local timestamp B+ tree and the global timestamp B+ tree;

[0061] S1045, write the updated two-dimensional B+ tree leaf node to the disk and record its disk offset.

[0062] It should be noted that the logical offset range of the block data to be inserted may intersect with the logical offset range of one or more leaf nodes of the offset B+ tree, for each leaf node with intersection with the range of the block data to be inserted, the block IO descriptor of the local timestamp B+ tree pointed by the leaf node is reconstructed with the intersection range and the original block IO descriptor, so as to ensure accurate indexing of the written data while having the ability of fast positioning of multi-version data.

[0063] Step S105, according to the specified logical offset range and / or timestamp range, find in the two-dimensional B+ tree to obtain the historical block physical offset of the target block data in the disk, and then read the target block data.

[0064] Optionally, in order to more clearly illustrate the use of the logical offset range and / or timestamp range in different scenarios, the step S105 further comprises:

[0065] S1051, when querying according to the specified logical offset range, find the offset B+ tree leaf node with intersection with the specified logical offset range in the offset B+ tree, obtain the local timestamp B+ tree pointed by it; find the timestamp B+ tree leaf node with intersection with the specified logical offset range in the local timestamp B+ tree, and obtain the corresponding historical block physical offset;

[0066] S1052, when querying according to the specified timestamp range, find the timestamp B+ tree leaf node in the global timestamp B+ tree according to the specified timestamp range, and obtain the corresponding historical block physical offset;

[0067] S1053, when querying according to the specified logical offset range and timestamp range, find the offset B+ tree leaf node intersecting with the specified logical offset range in the offset B+ tree, and obtain the local timestamp B+ tree pointed by the leaf node; find the timestamp B+ tree leaf node intersecting with the specified logical offset range and timestamp range in the local timestamp B+ tree, and obtain the corresponding historical block physical offset;

[0068] S1054, read the corresponding block data according to the obtained historical block physical offset.

[0069] It is worth noting that when no leaf node meeting the condition can be found in the offset B+ tree, the local timestamp B+ tree or the global timestamp B+ tree, the query is directly ended.

[0070] Step S106, locate and delete the matched leaf node in the two-dimensional B+ tree according to the specified logical offset range and / or timestamp range, and then update the two-dimensional B+ tree leaf node.

[0071] Optionally, the step S106 further comprises:

[0072] S1061, find the timestamp B+ tree leaf node intersecting with the logical offset range and / or timestamp range to be deleted in the two-dimensional B+ tree;

[0073] S1062, according to the specified logical offset range and / or timestamp range, range delete the timestamp B+ tree leaf node found in the above operation;

[0074] S1063, when the offset in the local timestamp B+ tree leaf node is discontinuous after the range deletion operation, create a new timestamp B+ tree leaf node index discontinuous interval, and insert it into the local timestamp B+ tree and the global timestamp B+ tree; update the logical offset and data length of the timestamp B+ tree leaf node;

[0075] S1064, merge the leaf node logical offset range of the local timestamp B+ tree, when the logical offset range is segmented into discrete sub-ranges due to the deletion operation, create an offset B+ tree leaf node index for each continuous sub-range, and newly create a corresponding local timestamp B+ tree, and insert the corresponding leaf node in the original local timestamp B+ tree into the local timestamp B+ tree; after the deletion operation is completed, when the local timestamp B+ tree is empty, delete the corresponding offset B+ tree leaf node; update the logical offset and data length of the offset B+ tree leaf node;

[0076] S1065, write the updated timestamp B+ tree leaf node and offset B+ tree leaf node back to the disk.

[0077] The embodiment intercepts an IO request in a block writing process, extracts metadata such as a logical offset and a timestamp, combines a historical block physical address, constructs a two-dimensional B+ tree index structure of the logical offset and the timestamp, realizes joint positioning of a block data space position and a timestamp, and further supports efficient multi-version data insertion, query and deletion operations. In particular, a timestamp leaf node is shared between a local and a global B+ tree, which effectively avoids data redundancy, saves storage resources and simplifies update operations compared with a traditional multi-tree independent index structure.

[0078] Embodiment 2

[0079] As shown in the figure, in the embodiment, a block backup index method based on a B+ tree is provided, and the method comprises: Figure 4

[0080] S201, intercepting an IO request at an entrance of a block writing function of QEMU, and extracting a corresponding block IO descriptor and block data of the IO request, wherein the block IO descriptor comprises a timestamp, a logical offset and a data length;

[0081] S202, writing the block data in the IO request to a disk to obtain a historical block physical offset of the block data in the disk;

[0082] S203, assigning the block IO descriptor and the historical block physical offset to a two-dimensional B+ tree leaf node for encapsulation, initializing the two-dimensional B+ tree, and the two-dimensional B+ tree comprising an offset B+ tree, a global timestamp B+ tree and a local timestamp B+ tree, each offset B+ tree leaf node pointing to a local timestamp B+ tree, wherein the offset B+ tree leaf node comprising a logical offset, a data length, a local timestamp B+ tree offset, an offset sequence pointer and a real-time block physical offset; the global timestamp B+ tree and the local timestamp B+ tree sharing a timestamp B+ tree leaf node, and the timestamp B+ tree leaf node comprising a timestamp, a logical offset, a data length, a historical block physical offset, a local sequence pointer and a global sequence pointer;

[0083] S204, according to a block IO descriptor to be inserted, writing corresponding block data to the disk and additionally writing the block data to a position corresponding to a real-time block physical offset, and newly creating a corresponding leaf node in the initialized two-dimensional B+ tree structure;

[0084] S205, when target block data to be obtained is interval latest version data, obtaining an offset B+ tree leaf node of a specified logical offset range in the offset B+ tree, then obtaining a historical block physical offset, and then reading the target block data according to the obtained offset;

[0085] ​S206, when the target block data to be acquired is not the latest version data in the interval, then according to the specified logical offset range and / or timestamp range, the two-dimensional B+ tree is searched to acquire the historical block physical offset of the target block data, and then the target block data is read according to the acquired offset;

[0086] S207, according to the specified logical offset range and / or timestamp range, the matching leaf node is located and deleted in the two-dimensional B+ tree, and then the local timestamp B+ tree leaf node data is merged in timestamp order and updated to the real-time block physical offset of the corresponding offset B+ tree leaf node, and the two-dimensional B+ tree leaf node is updated.

[0087] Unlike the above-mentioned embodiment 1, the key of this embodiment is that the leaf node of the offset B+ tree introduces a real-time block physical offset field, which establishes a direct mapping relationship between the logical offset and the latest physical address, so that the time-consuming version chain backtracking or secondary index query process can be skipped during query, thereby significantly reducing the number of required disk IO operations and ultimately improving the query efficiency and system response performance.

[0088] Embodiment 3

[0089] In this embodiment, as shown in Figure 5 , a block backup index system based on B+ tree is provided, which comprises:

[0090] The block data interception module 301 is used for intercepting the IO request at the block write function entrance of QEMU and extracting the corresponding block IO descriptor and block data, and the block IO descriptor includes timestamp, logical offset and data length;

[0091] The block data writing module 302 is used for writing the block data in the IO request to the disk to obtain the historical block physical offset thereof;

[0092] The B+ tree initialization module 303 is used for assigning the block IO descriptor and the historical block physical offset to the two-dimensional B+ tree leaf node for encapsulation, initializing the two-dimensional B+ tree, and the two-dimensional B+ tree includes offset B+ tree, global timestamp B+ tree and local timestamp B+ tree, each offset B+ tree leaf node points to a local timestamp B+ tree, wherein the offset B+ tree leaf node includes: logical offset, data length, local timestamp B+ tree offset and offset order pointer; The global timestamp B+ tree and the local timestamp B+ tree share the timestamp B+ tree leaf node, and the timestamp B+ tree leaf node includes: timestamp, logical offset, data length, historical block physical offset, local order pointer and global order pointer;

[0093] The block data insertion module 304 is configured to, according to the block IO descriptor to be inserted, write the corresponding block data to disk, and create a corresponding leaf node in the initialized two-dimensional B+ tree structure;

[0094] The block data searching module 305 is configured to, according to the specified logical offset range and / or timestamp range, search in the two-dimensional B+ tree, obtain the historical block physical offset of the target block data in the disk, and read the target block data;

[0095] The block data deletion module 306 is configured to, according to the specified logical offset range and / or timestamp range, locate and delete the matched leaf node in the two-dimensional B+ tree, and update the leaf node of the two-dimensional B+ tree.

[0096] As shown in FIG. 4, in one embodiment, the block data insertion module 304 further includes: Figure 6 The insertion position judging unit 3041 is configured to, according to the block IO descriptor of the data to be inserted, search in the offset B+ tree to obtain the insertion position of the block data index;

[0097] The block data writing unit 3042 is configured to write the block data to be inserted to disk to obtain the physical block base address;

[0098] The descriptor reconstructing unit 3043 is configured to judge whether the logical offset range of the block data to be inserted intersects with the logical offset range of the leaf node of the existing offset B+ tree, construct the block IO descriptor according to the intersection range for the intersected part, and for the several continuous logical subintervals without intersection, construct a new block IO descriptor for each subinterval, create a corresponding offset B+ tree leaf node with the subinterval starting logical offset and data length as the index key, and generate the local timestamp B+ tree pointed by the leaf node;

[0099] The node inserting unit 3044 is configured to calculate the physical offset of the block IO descriptor according to the physical block base address, generate the timestamp B+ tree leaf node in combination with the block IO descriptor, and synchronously insert the local timestamp B+ tree and the global timestamp B+ tree;

[0100] The disk writing and updating unit 3045 is configured to write the updated two-dimensional B+ tree leaf node to disk, and record the disk offset.

[0101] As shown in FIG. 5, in one embodiment, the block data searching module 305 further includes:

[0102] Figure 7 The searching unit 3051 is configured to, according to the specified logical offset range and / or timestamp range, search in the two-dimensional B+ tree to obtain the historical block physical offset of the target block data in the disk, and read the target block data;

[0103] ​Offset range query unit 3051 is used to, when querying based on a specified logical offset range, search for leaf nodes of offset B+ trees that intersect with the specified logical offset range in the offset B+ tree, obtain the local timestamp B+ tree they point to; search for leaf nodes of timestamp B+ trees that intersect with the specified logical offset range in the local timestamp B+ tree, and obtain their corresponding historical block physical offset.

[0104] The timestamp range query unit 3052 is used to search for the leaf node of the timestamp B+ tree in the global timestamp B+ tree according to the specified timestamp range when querying according to the specified timestamp range, and obtain the corresponding historical block physical offset.

[0105] The composite range query unit 3053 is used to, when querying based on a specified logical offset range and a timestamp range, search for leaf nodes of the offset B+ tree that intersect with the specified logical offset range, obtain the local timestamp B+ tree they point to; and search for leaf nodes of the timestamp B+ tree that intersect with the specified logical offset range and timestamp range in the local timestamp B+ tree, and obtain their corresponding historical block physical offset.

[0106] The block data reading unit 3054 is used to read the corresponding block data based on the acquired historical block physical offset.

[0107] like Figure 7 As shown, in one embodiment, the block data deletion module 306 includes:

[0108] The node to be deleted search unit 3061 is used to search for time stamp B+ tree leaf nodes in a two-dimensional B+ tree whose data range intersects with the logical offset range and / or timestamp range to be deleted.

[0109] The node range deletion unit 3062 is used to delete the time stamp B+ leaf nodes found by the above operation based on the specified logical offset range and / or timestamp range.

[0110] The timestamp index reconstruction unit 3063 is used to create a new discontinuous interval of the timestamp B+ tree leaf node index when the offset within the local timestamp B+ tree leaf node is discontinuous after the range deletion operation, and insert it into the local timestamp B+ tree and the global timestamp B+ tree; and update the logical offset and data length of the timestamp B+ tree leaf node.

[0111] The offset index reconstruction unit 3064 is configured to merge the logical offset range of the leaf nodes of the local timestamp B+ tree, create an offset B+ tree leaf node index for each continuous subrange when the logical offset range is segmented into discrete subranges due to a deletion operation, and insert the corresponding leaf nodes of the original local timestamp B+ tree into a newly created local timestamp B+ tree; when the local timestamp B+ tree is empty after the deletion operation is completed, delete the corresponding offset B+ tree leaf node; and update the logical offset and data length of the offset B+ tree leaf node.

[0112] The node writing unit 3065 is configured to write the updated timestamp B+ tree leaf node and offset B+ tree leaf node back to the disk.

[0113] In the following, some comparative experiments are provided to further illustrate the embodiment, and the details are as follows.

[0114] All experiments are run on a virtual machine started in a virtualization environment managed by VMware ESXI. The virtual machine operating system is Ubuntu 22.04, the kernel version is 6.8.0-57-generic, and Table 1 shows the specific parameters of the test environment.

[0115] Table 1 Test environment information

[0116]

[0117] In this experiment, the test objects are: the embodiment and the comparative example, and the comparative example is a conventional B+ tree block data indexing method indexed by timestamps. The test items are: the comparison of the query time for the same timestamp range and the same offset range.

[0118] In this experiment, C language is used to write the code for testing, and the code randomly generates 5 million block data index items. Among them, the timestamp range spans 24 hours, and the offset range spans 100 GB. The embodiment and the comparative example perform performance tests on the lookup time for a timestamp range of 1 hour and an offset range of 5M, 20M, 50M, 100M, and 200M. The results are shown in Table 2.

[0119] Table 2 Comparison of lookup performance of the embodiment and the comparative example

[0120]

[0121] From Table 2, it can be seen that the data lookup performance of the embodiment is greatly improved compared to the comparative example, which can significantly improve the lookup efficiency. Due to different experimental environments, the data results may vary, but the comparative experiment shows that the efficiency improvement of the embodiment remains stable in different environments, which reflects its universality.

[0122] The B+ tree-based block backup index system provided in this embodiment employs a two-dimensional index structure of logical offset and timestamp to achieve joint positioning of block data in both time and space dimensions, making it particularly suitable for complex query scenarios. By supporting rapid positioning of local data areas, it significantly improves query flexibility and retrieval efficiency, and substantially reduces the time overhead of searching within a wide range.

[0123] Example 4

[0124] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 8 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor, communication interface, and memory communicate with each other through the communication bus. The processor can call logical instructions in the memory to execute the following methods: intercepting an IO request at the entry point of the QEMU block write function and extracting its corresponding block IO descriptor and block data. The block IO descriptor includes a timestamp, logical offset, and data length; writing the block data in the IO request to the disk and obtaining its historical block physical offset on the disk; assigning the block IO descriptor and historical block physical offset to a two-dimensional B+ tree leaf node for encapsulation, and initializing the two-dimensional B+ tree. The two-dimensional B+ tree includes an offset B+ tree, a global timestamp B+ tree, and a local timestamp B+ tree. Each offset B+ tree leaf node points to a local timestamp B+ tree, wherein the offset B+ tree leaf node includes: logical offset, data length, and local timestamp B+ tree offset. The system uses offset sequence pointers; the global timestamp B+ tree and the local timestamp B+ tree share timestamp B+ tree leaf nodes. Each timestamp B+ tree leaf node includes: timestamp, logical offset, data length, historical block physical offset, local sequence pointer, and global sequence pointer. Based on the block I / O descriptor to be inserted, the corresponding block data is written to disk, and the corresponding leaf node is created in the initialized two-dimensional B+ tree structure. Based on the specified logical offset range and / or timestamp range, a search is performed in the two-dimensional B+ tree to obtain the historical block physical offset of the target block data on disk, and then the target block data is read. Based on the specified logical offset range and / or timestamp range, the matching leaf node is located and deleted in the two-dimensional B+ tree, and then the two-dimensional B+ tree leaf nodes are updated.

[0125] Moreover, the logic instructions in the memory 430 described above can be implemented in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art, or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium, including a plurality of instructions, to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, optical disk, and various program code storage media.

[0126] Embodiment 5

[0127] In this embodiment, a computer readable storage medium is provided, which stores a computer program. The program is executed by a processor to implement the B+ tree-based block backup index method as described in embodiment 1 or embodiment 2.

[0128] The computer storage medium of the embodiments of the present application can take any combination of one or more computer readable media. Computer readable media can be divided into signal type media and entity type storage media. Entity type storage media specifically refers to entity devices that carry data in physical form, and the implementation basis covers electromagnetic field effect, photoelectric conversion characteristics, semiconductor material characteristics, or the fusion system of the above technologies. Specifically, the performance form of the entity storage medium includes but is not limited to the following technical forms (non-limiting enumeration): such as mechanical hard disk, solid state disk, optical disk, magnetic tape, flash chip, memory card, USB device, EEPROM, and eMMC or UFS memory in embedded systems, etc. These media can store program codes in a tangible manner and be recognized and read by a computing device to achieve program installation, loading or execution. Whether integrated inside the device or connected and used in external form, any medium that can stably save and be used for program running can be considered as a qualified entity storage medium.

[0129] Computer signal type media can carry data signals that propagate in the baseband or as part of a carrier wave. The form of the propagated signal includes but is not limited to electromagnetic signals, optical signals, or any combination thereof. The computer readable signal medium can also be other media besides storage media, which can transmit or send programs for instruction execution systems, devices or equipment.

[0130] Transmission of the program code can be carried out over a variety of media including, but not limited to, wireless networks, wired connections, optical fiber cables, RF, or any combination thereof. The program code can be written in one or more programming languages, including an object oriented language such as Python, Java, C++, and a procedural language such as C, Go. Execution of the program code can occur as a result of software running on a local computer, partially on the local computer, as a standalone software package, partially on the local computer and partially on a remote server, or entirely on the remote server. When executed at a remote server, the remote server can be accessed through any network connection including a local area network (LAN), a wide area network (WAN), or the Internet.

[0131] The embodiments described above are only examples of the implementation of the present application, and are intended to illustrate the implementation approach of the technical solutions, rather than limit the protection scope of the present application. For those skilled in the art, various improvements and adjustments can be made without departing from the technical concept of the present application, and these improvements and adjustments should be considered as the protection scope of the present application. The protection scope of the present application patent is subject to the claims.

Claims

1. A method for block backup indexing based on B+ tree, characterized in that, The method comprises the steps of: a block data interception step, intercepting an IO request at an entrance of a block write function of QEMU, and extracting a corresponding block IO descriptor and block data of the IO request, the block IO descriptor comprising a timestamp, a logical offset and a data length; a block data writing step, writing the block data in the IO request to a disk, and obtaining a historical block physical offset of the block data in the disk; a B+ tree initialization step, assigning the block IO descriptor and the historical block physical offset to a two-dimensional B+ tree leaf node for encapsulation, and initializing the two-dimensional B+ tree, the two-dimensional B+ tree comprising an offset B+ tree, a global timestamp B+ tree and a local timestamp B+ tree, each offset B+ tree leaf node pointing to a local timestamp B+ tree, wherein the offset B+ tree leaf node comprises a logical offset, a data length, a local timestamp B+ tree offset and an offset sequence pointer; the global timestamp B+ tree and the local timestamp B+ tree share a timestamp B+ tree leaf node, and the timestamp B+ tree leaf node comprises a timestamp, a logical offset, a data length, a historical block physical offset, a local sequence pointer and a global sequence pointer; a block data insertion step, according to a block IO descriptor to be inserted, writing corresponding block data to the disk, and creating a corresponding leaf node in the initialized two-dimensional B+ tree structure; a block data searching step, searching in the two-dimensional B+ tree according to a specified logical offset range and / or a timestamp range, obtaining a historical block physical offset of target block data in the disk, and then reading the target block data; a block data deletion step, locating and deleting a matching leaf node in the two-dimensional B+ tree according to a specified logical offset range and / or a timestamp range, and then updating a two-dimensional B+ tree leaf node.

2. The method of claim 1, wherein, The block data insertion step further comprises: performing a search in the offset B+ tree according to a block IO descriptor of the data to be inserted, and obtaining an insertion position of a block data index; writing the block data to be inserted to the disk, and obtaining a physical block base address; judging whether a logical offset range of the block data to be inserted intersects with a logical offset range of a leaf node of an existing offset B+ tree, for the intersected part, constructing a block IO descriptor according to the intersected range, and for a plurality of continuous logical subintervals without intersection, constructing a new block IO descriptor for each subinterval, creating a corresponding offset B+ tree leaf node with a subinterval starting logical offset and a data length as an index key, and generating a local timestamp B+ tree pointed by the offset B+ tree leaf node; calculating a physical offset of the block IO descriptor according to the physical block base address, generating a timestamp B+ tree leaf node in combination with the block IO descriptor, and synchronously inserting the timestamp B+ tree leaf node into the local timestamp B+ tree and the global timestamp B+ tree; writing the updated two-dimensional B+ tree leaf node to the disk, and recording a disk offset of the two-dimensional B+ tree leaf node.

3. The method of claim 1, wherein, The block data searching step further comprises: When the query is based on the specified logical offset range, find the offset B+ tree leaf node intersecting with the specified logical offset range in the offset B+ tree, obtain the local timestamp B+ tree pointed by the offset B+ tree leaf node, find the timestamp B+ tree leaf node intersecting with the specified logical offset range in the local timestamp B+ tree, and obtain the corresponding historical block physical offset; When the query is based on the specified timestamp range, find the timestamp B+ tree leaf node in the global timestamp B+ tree according to the specified timestamp range, and obtain the corresponding historical block physical offset; When the query is based on the specified logical offset range and the timestamp range, find the offset B+ tree leaf node intersecting with the specified logical offset range in the offset B+ tree, obtain the local timestamp B+ tree pointed by the offset B+ tree leaf node, find the timestamp B+ tree leaf node intersecting with the specified logical offset range and the timestamp range in the local timestamp B+ tree, and obtain the corresponding historical block physical offset; According to the obtained historical block physical offset, the corresponding block data is read.

4. The method of claim 1, wherein, The block data deletion step further comprises: In the two-dimensional B+ tree, find the timestamp B+ tree leaf node intersecting with the logical offset range and / or the timestamp range to be deleted; According to the specified logical offset range and / or the timestamp range, range deletion is performed on the timestamp B+ tree leaf node found in the above operation; When the offset in the local timestamp B+ tree leaf node is discontinuous after the range deletion operation, a new timestamp B+ tree leaf node index discontinuous interval is created, and the new timestamp B+ tree leaf node index discontinuous interval is inserted into the local timestamp B+ tree and the global timestamp B+ tree; the logical offset and the data length of the timestamp B+ tree leaf node are updated; The leaf node logical offset range of the local timestamp B+ tree is merged, when the logical offset range is segmented into discrete sub-ranges due to the deletion operation, an offset B+ tree leaf node index is created for each continuous sub-range, and a corresponding local timestamp B+ tree is newly created, and the corresponding leaf node in the original local timestamp B+ tree is inserted into the corresponding local timestamp B+ tree; after the deletion operation is completed, when the local timestamp B+ tree is empty, the corresponding offset B+ tree leaf node is deleted; the logical offset and the data length of the offset B+ tree leaf node are updated; The updated timestamp B+ tree leaf node and the offset B+ tree leaf node are written back to the disk.

5. The method of claim 1, wherein, In the B+ tree initialization step, the leaf node of the offset B+ tree further comprises a real-time block physical offset field: Meanwhile, the block data insertion step further comprises: According to the block IO descriptor to be inserted, the corresponding block data is written to the disk, and the block data is additionally written to the position corresponding to the real-time block physical offset, and a corresponding leaf node is newly created in the initialized two-dimensional B+ tree structure; The block data finding step further comprises: When the target block data to be obtained is the latest version data in the interval, the offset B+ tree leaf node of the specified logical offset range is obtained in the offset B+ tree, the historical block physical offset is obtained, and then the target block data is read according to the obtained offset. When the target block data to be acquired is not the latest version data in the range, the two-dimensional B+ tree is searched according to the specified logical offset range and / or timestamp range to acquire the historical block physical offset of the target block data, and then the target block data is read according to the acquired offset; The block data deletion step further comprises: According to the specified logical offset range and / or timestamp range, the matching leaf nodes are located and deleted in the two-dimensional B+ tree, and then the local timestamp B+ tree leaf node data is merged in timestamp order and updated to the real-time block physical offset of the corresponding offset B+ tree leaf node, and the two-dimensional B+ tree leaf node is updated.

6. A B+ tree based chunk backup index system, characterized by, Comprise: The block data interception module is used for intercepting the IO request at the block write function entrance of QEMU, and extracting the corresponding block IO descriptor and block data, the block IO descriptor comprising a timestamp, a logical offset and a data length; The block data writing module is used for writing the block data in the IO request to the disk to acquire the historical block physical offset thereof in the disk; The B+ tree initialization module is used for assigning the block IO descriptor and the historical block physical offset to the two-dimensional B+ tree leaf node for encapsulation, and initializing the two-dimensional B+ tree, the two-dimensional B+ tree comprising an offset B+ tree, a global timestamp B+ tree and a local timestamp B+ tree, each offset B+ tree leaf node pointing to a local timestamp B+ tree, wherein the offset B+ tree leaf node comprises a logical offset, a data length, a local timestamp B+ tree offset and an offset order pointer; the global timestamp B+ tree and the local timestamp B+ tree share a timestamp B+ tree leaf node, and the timestamp B+ tree leaf node comprises a timestamp, a logical offset, a data length, a historical block physical offset, a local order pointer and a global order pointer; The block data insertion module is used for writing the corresponding block data to the disk according to the block IO descriptor to be inserted, and creating corresponding leaf nodes in the initialized two-dimensional B+ tree structure; The block data searching module is used for searching in the two-dimensional B+ tree according to the specified logical offset range and / or timestamp range to acquire the historical block physical offset of the target block data in the disk, and then reading the target block data; The block data deletion module is used for locating and deleting the matching leaf nodes in the two-dimensional B+ tree according to the specified logical offset range and / or timestamp range, and then updating the two-dimensional B+ tree leaf node.

7. The B+ tree based chunk backup index system according to claim 6, wherein, The block data insertion module further comprises: The insertion position judging unit is used for searching in the offset B+ tree according to the block IO descriptor of the data to be inserted to acquire the insertion position of the block data index; The block data writing unit is used for writing the block data to be inserted to the disk to acquire the physical block base address; The descriptor reconstruction unit is configured to determine whether a logical offset range of the block data to be inserted intersects with a logical offset range of one or more leaf nodes of the existing offset B+ tree; for the intersected part, construct a block IO descriptor according to the intersected range of each leaf node; and for a plurality of continuous logical subintervals without intersection, construct a new block IO descriptor for each subinterval, take the starting logical offset and the data length of the new block IO descriptor as an index key, newly create a leaf node of the timestamp B+ tree pointed by the new block IO descriptor, and generate a local timestamp B+ tree pointed by the leaf node. The node insertion unit is configured to calculate a physical offset of the block IO descriptor according to a physical block base address, generate a leaf node of the timestamp B+ tree in combination with the block IO descriptor, and synchronously insert the leaf node into the local timestamp B+ tree and the global timestamp B+ tree. The disk landing and updating unit is configured to land the updated two-dimensional B+ tree leaf node on a disk and record a disk offset of the two-dimensional B+ tree leaf node.

8. The B+ tree based chunk backup index system of claim 6, wherein, The block data searching module further includes: The offset range query unit is configured to search, in the offset B+ tree, a leaf node of the offset B+ tree intersecting with a specified logical offset range, acquire a local timestamp B+ tree pointed by the leaf node, search, in the local timestamp B+ tree, a leaf node of the timestamp B+ tree intersecting with the specified logical offset range, and acquire a historical block physical offset corresponding to the leaf node of the timestamp B+ tree. The timestamp range query unit is configured to search, in the global timestamp B+ tree, a leaf node of the timestamp B+ tree according to a specified timestamp range, and acquire a historical block physical offset corresponding to the leaf node of the timestamp B+ tree. The composite range query unit is configured to search, in the offset B+ tree, a leaf node of the offset B+ tree intersecting with a specified logical offset range, acquire a local timestamp B+ tree pointed by the leaf node, search, in the local timestamp B+ tree, a leaf node of the timestamp B+ tree intersecting with the specified logical offset range and a specified timestamp range, and acquire a historical block physical offset corresponding to the leaf node of the timestamp B+ tree. The block data reading unit is configured to read corresponding block data according to the acquired historical block physical offset. In addition, the block data deleting module further includes: The to-be-deleted node searching unit is configured to search, in the two-dimensional B+ tree, a leaf node of the timestamp B+ tree intersecting with a data range, a logical offset range to be deleted, and / or a timestamp range to be deleted. The node range deleting unit is configured to, for the leaf node of the timestamp B+ tree found by the operation, perform range deletion according to a specified logical offset range and / or a specified timestamp range. The timestamp index reconstruction unit is configured to, when offsets in the leaf node of the local timestamp B+ tree are discontinuous after the range deletion operation, newly create an index discontinuous interval of the leaf node of the timestamp B+ tree, insert the index discontinuous interval into the local timestamp B+ tree and the global timestamp B+ tree, and update a logical offset and a data length of the leaf node of the timestamp B+ tree. The offset index reconstruction unit is used to merge the leaf node logical offset range of the local timestamp B+ tree. When the logical offset range is segmented into discrete subranges due to a deletion operation, the offset B+ tree leaf node index is created for each continuous subrange, and a corresponding local timestamp B+ tree is newly created. The corresponding leaf nodes in the original local timestamp B+ tree are inserted into the local timestamp B+ tree. The local timestamp B+ tree leaf node data is merged in timestamp order and is updated to the real-time block physical offset of the offset B+ tree leaf node. After the deletion operation is completed, when the local timestamp B+ tree is empty, the corresponding offset B+ tree leaf node is deleted. The logical offset and data length of the offset B+ tree leaf node are updated. The node writing unit is used to write the updated timestamp B+ tree leaf node and offset B+ tree leaf node back to the disk.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the B+ tree-based block backup index method in any one of claims 1 to 5.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the B+ tree-based block backup index method in any one of claims 1 to 5.

Images