An asynchronous incremental recording method based on mergeable interval tree

Abstract

The present invention discloses an asynchronous incremental recording method based on a mergeable interval tree, comprising: S1. establishing an interval tree, wherein the interval tree records the starting address information of incremental data; S2. parsing IO requests issued by a storage system to obtain parsed data, encapsulating the parsed data into incremental data packets, and storing them; S3. maintaining the starting address information of the incremental data packets in the interval tree, wherein the incremental data packets and the interval tree constitute asynchronous incremental recording. The beneficial effects of the present invention are that the interval tree is used to record data increments, and repeated intervals in the interval tree are merged to keep the intervals used for recording incremental data independent of each other, thereby achieving non-repeated incremental data recording. Furthermore, callback data is added to verify whether the incremental data recording is completed in the interval tree, making the incremental data recording more concise and ensuring the completion of the incremental data recording.

Description

Technical Field

[0001] The present invention belongs to the field of computer storage, and in particular relates to an asynchronous incremental recording method based on a mergable interval tree. Background Art

[0002] Incremental data backup is the mainstream data backup method. A data increment is the amount of data change over a period of time. To ensure an up-to-date backup of all data, modified data on the source volume over a period of time can be periodically copied to the backup volume to protect the source volume. These modified data are called data increments. Incremental data is recorded to facilitate quick search within the storage device. Currently, bitmaps are commonly used to record data increments. Bitmaps are implemented by dividing the data volume into blocks of specific sizes and identifying each block with a bit in the bitmap. Therefore, the size of the block determines the granularity of the bitmap cells. The larger the block size, the fewer the number of blocks, and the coarser the granularity of the bitmap cells. Conversely, the smaller the block size, the more blocks there are, and the finer the granularity of the bitmap cells. When a client writes to a block, the state of the corresponding cell in the bitmap changes from 0 to 1, indicating that the block has changed. During the data backup and copying process, the data blocks corresponding to the units with status 1 are copied to the remote backup volume to overwrite the original corresponding data blocks, and the backup and copying of the data increment is completed.

[0003] Bitmaps have the advantage of simple layout, but as the capacity of storage devices increases, bitmaps occupy more memory or even cannot be stored. When the bitmap volume is reduced to a certain extent by increasing the granularity of the units in the bitmap, it will cause the data blocks to contain too much data. When incremental data is updated, not all the data in the data block is changed, and all the data still needs to be copied to the backup volume, resulting in duplicate data backup and bandwidth waste during upload. Summary of the Invention

[0004] In order to solve the problem of duplicate data backup when recording data in the bitmap in the above prior art, the present invention proposes an asynchronous incremental recording method based on a mergeable interval tree, which is used to record incremental data and keep non-duplicate records between incremental data.

[0005] The technical solution of the present invention is:

[0006] An asynchronous incremental recording method based on a mergeable interval tree, comprising:

[0007] S1. Establish an interval tree, which records the starting address information of the incremental data;

[0008] S2. Analyze the IO request issued by the storage system to obtain the parsed data, encapsulate the parsed data into incremental data packets and store them;

[0009] S3. Maintain the starting address information of the incremental data packet into the interval tree, and the incremental data packet and the interval tree constitute an asynchronous incremental record.

[0010] Preferably, in step S2, parsing the IO request sent by the storage system includes the following steps:

[0011] Determine the operation type of the IO request issued by the storage system. If the IO request is a read request, return the IO request to the storage system. If the IO request is a write request, parse the IO request.

[0012] Preferably, in step S2, the incremental data packet includes the address offset and data length of the IO request.

[0013] Preferably, in step S3, the step of maintaining the information of the incremental data packet into the interval tree includes:

[0014] Obtain the interval occupied by the incremental data according to the address offset and data length, and record it as the interval of the incremental data packet;

[0015] Determine whether the interval of the incremental data packet is continuous or overlapped with the interval of the interval tree node;

[0016] If two intervals are continuous or overlapping, the two intervals are merged into one node interval. If the two intervals are discontinuous and non-overlapping, a new node is added to the interval tree to store the incremental data packet information.

[0017] Preferably, the interval of the interval tree node is set to a left-closed and right-open interval.

[0018] Preferably, in step S3, the incremental data packet further includes a callback data field for storing callback data, and the callback data includes the result state of maintaining the starting address information of the incremental data packet into the interval tree.

[0019] Preferably, in step S3, when the result status of maintaining the starting address information of the received incremental data packet into the interval tree is successful, the incremental data packet saves the result status to the callback data field; otherwise, the result status is saved only when the received result status is successful.

[0020] An asynchronous incremental recording module based on a mergeable interval tree is used to execute an asynchronous incremental recording method based on a mergeable interval tree. The asynchronous incremental recording module is connected to a storage system. The asynchronous incremental recording module includes an asynchronous incremental parsing module and an incremental recording module. The asynchronous incremental parsing module is connected to the storage system, and the incremental recording module is connected to the asynchronous incremental parsing module.

[0021] Preferably, the asynchronous incremental parsing module performs the following steps:

[0022] Determine the operation type of the IO request issued by the storage system. If the IO request is a read request, return the IO request to the storage system. If the IO request is a write request, parse the IO request. Parse the IO request issued by the storage system to obtain parsed data, encapsulate the parsed data into incremental data packets, and store them.

[0023] The result status of maintaining the starting address information of the incremental data packet into the interval tree is judged. When the result status of maintaining the starting address information of the incremental data packet into the interval tree is successful, the incremental data packet saves the result status to the callback data field. Otherwise, wait until the received result status is successful before saving the result status.

[0024] Preferably, the incremental recording module performs the following steps:

[0025] Maintaining the starting address information of the incremental data packet into an interval tree;

[0026] Determine whether the interval of the incremental data packet is continuous or overlapping with the interval of the interval tree node. If the two intervals are continuous or overlapping, merge the two intervals into one node interval. If the two intervals are discontinuous and non-overlapping, add a new node in the interval tree to store the incremental data packet information.

[0027] The starting address information of the incremental data packet is generated and maintained in the result state of the interval tree.

[0028] The beneficial effects of the present invention include: using an interval tree for recording data increments, and merging repeated intervals of the interval tree to keep the intervals of the interval tree used to record incremental data independent of each other, achieving non-repetitive incremental data recording, and further adding callback data to verify whether the incremental data recording is completed on the interval tree. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Attachment Figure 1 This is a flowchart of an asynchronous incremental recording method based on a mergeable interval tree in the present invention.

[0031] Attachment Figure 2 This is a composition diagram of an asynchronous incremental recording module based on a mergeable interval tree in the present invention.

[0032] Specific implementation examples

[0033] The following describes the embodiments of the present invention through specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0034] Reference Attachment Figure 1 This embodiment provides an asynchronous incremental recording method based on a mergeable interval tree, including:

[0035] S1. Establish an interval tree, which is used to record the starting address information of incremental data;

[0036] S2. Parse the IO request sent by the storage system to obtain parsed data, encapsulate the parsed data into an incremental data packet and store it; in executing step S2, it is first necessary to judge the operation type of the IO request. If the IO request is a read request, the IO request is returned to the storage system. If the IO request is a write request, the IO request is parsed. The parsed data includes the starting address information of the IO request. Preferably, the parsed data includes the address offset and data length of the IO request, and the parsed data is encapsulated into an incremental data packet.

[0037] S3. Maintain the starting address information in the incremental data packet into the interval tree, and the incremental data packet and the interval tree constitute an asynchronous incremental record. Preferably, the step of maintaining the address offset and data length of the IO request in the incremental data packet into the interval tree includes: obtaining the interval occupied by the incremental data according to the address offset and data length, and recording it as the interval of the incremental data packet; judging whether the interval of the incremental data packet and the interval of the interval tree node are continuous or overlapping; if the two intervals are continuous or overlapping, the two intervals are merged into one node interval; if the two intervals are discontinuous and non-overlapping, a new node is added to the interval tree to store the incremental data packet information. The interval of the interval tree node is set to a left-closed and right-open interval to facilitate the judgment of whether the two intervals are continuous.

[0038] Preferably, the incremental data packet also includes a callback data field for storing callback data. The callback data includes the address offset and data length of the incremental data packet and the result status of maintaining the information in the interval tree. If the result status of maintaining the information in the interval tree after receiving the incremental data packet is successful, the incremental data packet saves the result status in the callback data field. Otherwise, the incremental data packet waits until the received result status is successful before saving the result status.

[0039] Reference Attachment Figure 2 This embodiment also provides an asynchronous incremental recording module based on a mergeable interval tree, which is used to execute an asynchronous incremental recording method based on a mergeable interval tree. The asynchronous incremental recording module is connected to the storage system. The asynchronous incremental recording module includes an asynchronous incremental parsing module and an incremental recording module. The asynchronous incremental parsing module is connected to the logical volume in the storage system, and the incremental recording module is connected to the asynchronous incremental parsing module.

[0040] The asynchronous incremental parsing module has two interfaces for receiving and sending information. The asynchronous incremental parsing module performs the following steps:

[0041] Receive IO requests from the storage system;

[0042] Determine the operation type of the IO request issued by the storage system. If the IO request is a read request, return the IO request to the storage system. If the IO request is a write request, parse the IO request. Parse the IO request issued by the storage system to obtain parsed data, encapsulate the parsed data into incremental data packets, and store them.

[0043] The result status of maintaining the starting address information of the incremental data packet into the interval tree is judged. When the result status of maintaining the starting address information of the incremental data packet into the interval tree is successful, the incremental data packet saves the result status to the callback data field. Otherwise, wait until the received result status is successful before saving the result status.

[0044] The incremental recording module is used to perform the following steps:

[0045] Maintain the starting address information of the incremental data packet into the interval tree;

[0046] The starting address information of the generated incremental data packet is maintained in the result state of the interval tree, and the result state is sent to the asynchronous incremental parsing module.

[0047] When maintaining the information of the incremental data packet into the interval tree, it is necessary to determine whether the interval of the incremental data packet is continuous or overlapping with the interval of the interval tree node. If the two intervals are continuous or overlapping, the two intervals are merged into one node interval. If the two intervals are discontinuous and non-overlapping, a new node is added to the interval tree to store the incremental data packet information;

[0048] The incremental recording module is equipped with an interface that filters out intervals in the interval tree that overlap with the intervals in the incremental data packet. It then merges the intervals in the incremental data packet with the corresponding overlapping intervals to achieve the purpose of identifying overlapping intervals and combining duplicate intervals. The following details the interfaces and their functions in the incremental recording module.

[0049] Incremental data in the incremental recording module is represented by the incremental interval intval[low,high). The interface interval_insert_merge(tree, intval) is defined to operate on the incremental data interval intval[low,high). The interface interval_insert_merge(tree, intval) is used to insert the incremental interval intval[low,high) into the interval tree tree. It also includes two sub-interfaces, interval_search(root, intval, merge_collection) and interval_merge(merge_collection, intval). The interface interval_insert_merge(tree, intval) first calls the interface interval_search to query the merge_collection of all intervals in the interval tree that can be merged with the incremental interval intval. If such an interval exists in the interval tree (that is, merge_collection is not empty), the interface interval_merge is called to merge the intervals in merge_collection, ultimately obtaining the merged interval node. Otherwise, if merge_collection is empty, a new interval tree node, change_node, is created and filled with intval. Finally, the interval tree's native interface, interval_insert, is called to insert change_node into the interval tree. This completes the operation of merging the incremental interval into the interval tree.

[0050] The interface interval_search(root, intval, merge_collection) searches the interval tree from the root node down. The first condition determines whether the current interval tree node meets the merge condition. If so, the current interval tree node is added to the merge_collection. The second condition determines whether the left subtree contains an incremental interval that meets the condition. If so, the function recursively calls itself to search the left subtree. Similarly, the third condition determines whether the right subtree contains an incremental interval that meets the condition. If so, the function recursively searches the right subtree. Finally, the search is completed for all nodes.

[0051] The interface interval_merge(merge_collection, intval) first selects the retained interval tree node select_node from the passed mergeable interval tree node set merge_collection, then merges the intervals of other nodes in the collection and the interval intval into the select_node node, then updates the max value of select_node, and then deletes the other nodes in the collection from the interval tree. If there are no retained nodes that meet the conditions, a new interval tree node select_node is created, and the intervals of all nodes in merge_collection are merged into the interval of select_node. The nodes in the collection are then deleted from the interval tree, and finally the interval tree native interface interval_insert is called to insert select_node into the interval tree.

[0052] The pseudo codes of the three interfaces interval_insert_merge(tree, intval), interval_search(root, intval, merge_collection) and interval_merge(merge_collection, intval) are provided below.

[0053] Interface name: interval_insert_merge(tree, intval) pseudo code flow:

[0054] interval_insert_merge(tree, intval)

[0055] root = tree.root

[0056] merge_collection = NULL

[0057] interval_search(root, intval, merge_collection)

[0058] if merge_collection != NULL

[0059] change_node = interval_merge(merge_collection, intval)

[0060] else

[0061] change_node = new TreeNode()

[0062] change_node.intval = intval

[0063] interval_insert(tree, change_node)

[0064] return change_node

[0065] Pseudocode process of the interface name: interval_search(root, intval, merge_collection):

[0066] interval_search(root, intval, merge_collection)

[0067] x = root

[0068] if (x.low <= intval.high and x.high >= intval.low) or

[0069] (x.high >= intval.low and x.low <= intval.high)

[0070] add x to merge_collection

[0071] if (x.left != NULL) and (x.left.max >= intval.low)

[0072] interval_search(x.left, intval, merge_collection)

[0073] if (x.right != NULL) and (x.right.max >= intval.low) and

[0074] (x.right.key <= intval.high)

[0075] interval_search(x.right, intval, merge_collection)

[0076] Pseudocode process of the interface name: interval_merge(merge_collection, intval):

[0077] interval_merge(tree, merge_collection, intval)

[0078] select_node = NULL

[0079] for merge_node in merge_collection

[0080] if intval.low >= merge_node.low

[0081] select_node = merge_node

[0082] break

[0083] if select_node == NULL

[0084] select_node = new TreeNode()

[0085] select_node.intval = intval

[0086] for merge_node in merge_collection

[0087] merge merge_node.intval into select_node.intval

[0088] interval_delete(tree, merge_node)

[0089] interval_insert(tree, select_node)

[0090] else

[0091] remove select_node from merge_colletion

[0092] merge intval into select_node.intval

[0093] for merge_node in merge_collection

[0094] merge merge_node.intval into select_node.intval

[0095] select_node.max = max(select_node.left.max,

[0096] select_node.right.max,

[0097] select_node.intval.high)

[0098] interval_delete(tree, merge_node)

[0099] return select_node

[0100] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes or substitutions that are not conceived through creative work should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection defined in the claims.

Claims

1. An asynchronous incremental recording method based on a mergeable interval tree, characterized in that: include: S1. Establish an interval tree, which records the starting address information of the incremental data; S2. Analyze the IO request issued by the storage system to obtain the parsed data, encapsulate the parsed data into incremental data packets and store them; S3. The starting address information of the incremental data packet is maintained into the interval tree, and the incremental data packet and interval tree constitute asynchronous incremental records; In step S2, parsing the IO request sent by the storage system includes the following steps: Determine the operation type of the IO request issued by the storage system. If the IO request is a read request, return the IO request to the storage system. If the IO request is a write request, parse the IO request. In step S2, the incremental data packet includes the address offset and data length of the IO request; In step S3, the step of maintaining the information of the incremental data packet into the interval tree includes: Obtain the interval occupied by the incremental data according to the address offset and data length, and record it as the interval of the incremental data packet; Determine whether the interval of the incremental data packet is continuous or overlapped with the interval of the interval tree node; If two intervals are continuous or overlapping, the two intervals are merged into one node interval. If the two intervals are discontinuous and non-overlapping, a new node is added to the interval tree to store the incremental data packet information; The interval of the interval tree node is set to a left-closed and right-open interval.

2. The asynchronous incremental recording method based on a mergeable interval tree according to claim 1, characterized in that: In step S3, the incremental data packet further includes a callback data field for storing callback data, and the callback data includes the result state of maintaining the starting address information of the incremental data packet into the interval tree.

3. The asynchronous incremental recording method based on a mergeable interval tree according to claim 2, characterized in that: In step S3, when the result status of maintaining the starting address information of the received incremental data packet into the interval tree is successful, the incremental data packet saves the result status to the callback data field. Otherwise, the result status is saved when the received result status is successful.

4. An asynchronous incremental recording module based on a mergeable interval tree, used to execute the asynchronous incremental recording method based on a mergeable interval tree according to any one of claims 1 to 3, characterized in that: The asynchronous incremental recording module is connected to the storage system, the asynchronous incremental recording module includes an asynchronous incremental parsing module and an incremental recording module, the asynchronous incremental parsing module is connected to the storage system, and the incremental recording module is connected to the asynchronous incremental parsing module.

5. The asynchronous incremental recording module based on a mergeable interval tree according to claim 4, characterized in that: The asynchronous incremental parsing module performs the following steps: Determine the operation type of the IO request issued by the storage system. If the IO request is a read request, return the IO request to the storage system. If the IO request is a write request, parse the IO request. Parse the IO request issued by the storage system to obtain parsed data, encapsulate the parsed data into incremental data packets, and store them. The result status of maintaining the starting address information of the incremental data packet into the interval tree is judged. When the result status of maintaining the starting address information of the incremental data packet into the interval tree is successful, the incremental data packet saves the result status to the callback data field. Otherwise, wait until the received result status is successful before saving the result status.

6. The asynchronous incremental recording module based on a mergeable interval tree according to claim 4, characterized in that: The incremental recording module performs the following steps: Maintaining the starting address information of the incremental data packet into an interval tree; Determine whether the interval of the incremental data packet is continuous or overlapping with the interval of the interval tree node. If the two intervals are continuous or overlapping, merge the two intervals into one node interval. If the two intervals are discontinuous and non-overlapping, add a new node in the interval tree to store the incremental data packet information. The starting address information of the incremental data packet is generated and maintained in the result state of the interval tree.

Images