Balanced tree traversal optimization method for storage
By adding a linear directory entry table and a pending directory entry dictionary for each directory, the problem of poor log-style balanced tree traversal performance is solved, and efficient directory entry traversal and data consistency is achieved.
Patent Information
- Application Number
- CN202411832208.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-04-04
AI Technical Summary
In the prior art, the log-type balanced tree traversal performance is low, especially when traversing directory items in the file system, it is necessary to conduct deep traversal of all node blocks, resulting in increased performance loss and delay.
In persistent storage, add a linear directory entry table to each directory, and record the sequence number of the directory entry in the directory entry table and the last state of the operation to the pending directory entry dictionary in memory to ensure data consistency between the added directory entry table and the balanced tree.
Through the traversal of linear directory entry tables, the performance of directory entry traversal is significantly improved, the deep traversal of all nodes of the balance tree is avoided, the performance cost of disk reading is reduced, and the data consistency problem is solved.
Smart Images
Figure CN119759849A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of file systems and databases, and relates to a balanced tree traversal optimization method, in particular to a balanced tree traversal optimization method for storage. Background Art
[0002] In software technology, such as file systems and databases, balanced trees are the most widely used for indexing large amounts of data.
[0003] To implement directories in a file system, you can use a log-style balanced tree to index directory entries. Given a file name, you can find the directory entry corresponding to the file name through a log-style balanced tree search. However, traversing directory entries on a balanced tree has relatively low performance. Due to the characteristics of a log-style balanced tree, it is not possible to optimize the traversal of leaf nodes by adding "pointers between leaf nodes". To traverse a log-style balanced tree, all node blocks (including intermediate node blocks and leaf node blocks) in the balanced tree must be read from the disk. Moreover, it involves jumping reads between nodes. Given the mechanism of mechanical hard disks, the performance cost of such jump reads is very expensive.
[0004] Therefore, the present invention proposes a balanced tree traversal optimization method for storage, which is used to solve the above problems.
[0005] After searching, no public documents of the prior art identical or similar to the present invention were found. Summary of the invention
[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and to propose a balanced tree traversal optimization method for storage, which can apply a balanced tree to discover performance problems of traversing directory entries in a file system.
[0007] The present invention solves the practical problem by adopting the following technical solutions:
[0008] A balanced tree traversal optimization method for storage includes the following steps:
[0009] Step 1: In persistent storage, add a linear directory entry table for each directory;
[0010] Step 2: Record the sequence number of the directory entry in the directory entry table and the final status of the operation into the pending directory entry dictionary in the memory;
[0011] Step 3: Modify or traverse the directory entry to ensure data consistency between the added directory entry table and the balanced tree.
[0012] Moreover, the specific method of step 1 is:
[0013] In persistent storage, a linear directory entry table is added to each directory. Before modifying a directory entry, the flag word of the corresponding directory entry in the directory entry table is changed to "unknown", and additional changes are made to the content.
[0014] Furthermore, the final status of the operation of step 2 includes: retain or delete.
[0015] Furthermore, the specific steps of modifying the directory entry in step 3 include:
[0016] (1) In a directory modification transaction, any modification to a directory entry must first mark the corresponding directory entry in the directory entry table as "unknown" and make additional changes to the content;
[0017] (2) Then add a pair of key values to the pending directory entry dictionary, where the key is the serial number of the corresponding directory entry and the value is the nature of the operation. If it is a modification or creation, its value is "retain"; for a deletion operation, its value is "delete";
[0018] (3) When the transaction ends, the pending directory entry dictionary in the memory is traversed, and for each directory entry in the dictionary, corresponding modifications are made to the directory entry table according to the directory entry sequence number. For directory entries whose value in the dictionary is "reserved", the flag word is changed to "reserved", and for others, it is changed to "deleted".
[0019] Moreover, the specific steps of traversing the directory entries in step 3 include:
[0020] (1) In the operation of traversing the directory, read the directory entry table in sequence;
[0021] (2) Each directory entry is read out in turn. If the flag word of the directory entry is "reserved", the item is returned to the user. If the flag word is "unknown", the directory entry needs to be "reviewed". The review function returns the value of the flag word of the directory entry. If the result is "reserved", the information of the directory entry is returned to the user. Other branches indicate that the directory entry has been deleted or is not used.
[0022] Furthermore, the specific method for reviewing this catalog item in step 3 (2) is as follows:
[0023] ① Use the file name of the directory entry to be reviewed in the directory entry table to find the corresponding directory entry from the balanced tree. If the corresponding directory entry is found, that is, the name and serial number of the directory entry found on the balanced tree match the directory entry with review; if the corresponding directory entry is not found, go to step ④;
[0024] ② Search the corresponding key-value pair in the pending directory entry dictionary according to the serial number of the directory entry to be reviewed; if the corresponding key-value pair is not found in the pending directory entry dictionary, change the identification word of the directory entry to be reviewed in the directory entry table to "reserved";
[0025] ③ The review process returns to "retain", and the process ends;
[0026] ④ Search the corresponding key-value pair in the pending directory entry dictionary according to the serial number of the directory entry to be reviewed; if the corresponding key-value pair is not found in the pending directory entry dictionary, change the identification word of the directory entry to be reviewed in the directory entry table to "delete";
[0027] ⑤The review process returns to “delete” and the process ends.
[0028] Advantages and beneficial effects of the present invention:
[0029] 1. The present invention proposes a method for optimizing the traversal of a log-type balanced tree for storage in a file system, so that the traversal of directory entries is linear in performance consumption, and the operation of reading directory entry information is basically continuous on the disk. Moreover, the present invention focuses on solving the problem of data consistency between the additional structure and the balanced tree generated thereby.
[0030] 2. Traditional log-based balanced tree traversal requires deep traversal of all nodes in the balanced tree. For a balanced tree with a large number of directory entries, its performance loss leads to a long delay. The present invention converts the process of traversing a balanced tree into traversing a linear directory entry table, and the performance is significantly improved.
[0031] 3. The present invention provides a performance optimization method for traversing a log-type balanced tree index. The present invention provides an additional directory entry file to allow the traversal operation to read the directory entries directly and continuously from the directory file, thereby avoiding traversing the node blocks of the balanced tree on the disk. At the same time, in the process of reading the directory entries of the directory entry file, the legacy work of data consistency with the balanced tree is handled. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a processing flow chart of the review function of the present invention. DETAILED DESCRIPTION
[0033] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings:
[0034] A balanced tree traversal optimization method for storage includes the following steps:
[0035] Step 1: In persistent storage, add a linear directory entry table for each directory;
[0036] The specific method of step 1 is:
[0037] In persistent storage, a linear directory entry table is added to each directory. Before modifying a directory entry, the flag word of the corresponding directory entry in the directory entry table is changed to "unknown", and additional changes are made to the content.
[0038] Step 2: Record the sequence number of the directory entry in the directory entry table and the final status of the operation into the pending directory entry dictionary in the memory;
[0039] The final status of the operation in step 2 includes: retain or delete
[0040] Step 3: Modify or traverse the directory entry to ensure data consistency between the added directory entry table and the balanced tree;
[0041] In this embodiment, the specific steps of modifying the directory entry in step 3 include:
[0042] (1) In a directory modification transaction, any modification to a directory entry must first mark the corresponding directory entry in the directory entry table as "unknown" and make additional changes to the content;
[0043] (2) Then add a pair of key values to the pending directory entry dictionary, where the key is the serial number of the corresponding directory entry and the value is the nature of the operation. If it is a modification or creation, its value is "retain"; for a deletion operation, its value is "delete";
[0044] (3) When the transaction ends, the pending directory entry dictionary in the memory is traversed, and for each directory entry in the dictionary, corresponding modifications are made to the directory entry table according to the directory entry sequence number. For directory entries whose value in the dictionary is "reserved", the flag word is changed to "reserved", and for others, it is changed to "deleted".
[0045] In this embodiment, the specific steps of traversing the directory entries in step 3 include:
[0046] (1) In the operation of traversing the directory, read the directory entry table in sequence;
[0047] (2) Each directory entry is read out in turn. If the flag word of the directory entry is "reserved", the item is returned to the user. If the flag word is "unknown", the directory entry needs to be "reviewed". The review function returns the value of the flag word of the directory entry. If the result is "reserved", the information of the directory entry is returned to the user. Other branches indicate that the directory entry has been deleted or is not used.
[0048] In this embodiment, the specific method for reviewing this directory item is:
[0049] ① Use the file name of the directory entry to be reviewed in the directory entry table to find the corresponding directory entry from the balanced tree. If the corresponding directory entry is found, that is, the name and serial number of the directory entry found on the balanced tree match the directory entry with review; if the corresponding directory entry is not found, go to step ④;
[0050] ② Search the corresponding key-value pair in the pending directory entry dictionary according to the serial number of the directory entry to be reviewed; if the corresponding key-value pair is not found in the pending directory entry dictionary, change the identification word of the directory entry to be reviewed in the directory entry table to "reserved";
[0051] ③ The review process returns to "retain", and the process ends;
[0052] ④ Search the corresponding key-value pair in the pending directory entry dictionary according to the serial number of the directory entry to be reviewed; if the corresponding key-value pair is not found in the pending directory entry dictionary, change the identification word of the directory entry to be reviewed in the directory entry table to "delete";
[0053] ⑤The review process returns to “delete” and the process ends.
[0054] The working principle of the present invention is:
[0055] In the persistent storage, a linear directory entry table is added to each directory. Before changing the directory entry, the flag word of the corresponding directory entry in the directory entry table is changed to "unknown", and the content is modified. At the same time, the sequence number of this directory entry and the final status of the operation (retain or delete) are recorded in the pending directory entry dictionary in the memory. If the transaction ends normally, at the end, the pending directory entry dictionary is traversed, and the flag word of the corresponding directory entry in the directory entry table is changed (retained or deleted) according to the nature of the operation. If the transaction does not end normally (such as program termination), when the operation of traversing the directory entry is performed, a review process is performed to check the status of the directory entry from the balanced tree, and the directory entry table is updated. When traversing the directory entry, only a very small number of directory entries with the flag word "unknown" are generated by the interrupted transaction. The vast majority of other directory entries do not need to access the balanced tree, and the directory entry data is only read from the linear directory entry table, thereby improving performance.
[0056] In a directory indexed by a balanced tree, a serial number and a flag word are marked for each directory entry. Then, the fields of the directory entry include: file name, serial number, flag word and other file attributes (such as: file type, permissions, location of the file content on the disk, etc. This algorithm does not pay attention to other file attributes). A linear space is opened on the disk, called the directory entry table, which also stores directory entries. That is, directory entries are stored on the leaf node blocks of the balanced tree, and directory entries are also stored in the directory entry table. The former stores directory entries at the corresponding positions of the leaf node blocks through the indexing method of the balanced tree. The directory entry table stores the corresponding directory entries according to the serial number position, that is, the byte offset of the directory entry corresponding to this serial number can be calculated by multiplying the serial number by the length of the directory entry (the directory entry has a fixed byte length). When it is necessary to traverse the directory entry, the directory entry can be directly accessed by reading the directory entry file without traversing the balanced tree. In addition to creating a directory entry table for each directory, a dictionary object called "pending directory entry dictionary" needs to be created in memory during each transaction of directory operation. The key is the file number and the value is an operation code, which can be represented by an integer. The operation code has two values: "retain" or "delete".
[0057] Next, we need to focus on solving the problem of data consistency between the added directory entry table and the balanced tree. First, the directory entry table is an additional structure used for performance optimization, and it must maintain data consistency with the functional structure it serves (balanced tree). When data inconsistency occurs, the data on the balanced tree shall prevail. After clarifying the above principles, combined with directory operations, the present invention proposes the following method:
[0058] (1) Handling of “Changes”
[0059] In a transaction that modifies the directory, for any change to a directory entry, the flag word of the corresponding directory entry in the directory entry table must first be marked as "unknown" and additional changes must be made to the content. Then a pair of key values is added to the pending directory entry dictionary. The key is the serial number of the corresponding directory entry, and the value is the nature of the operation. If it is a change or creation, its value is "retained", and for a deletion operation, its value is "delete". When the transaction ends, the pending directory entry dictionary in the memory is traversed, and for each directory entry in the dictionary, the corresponding modification is made to the directory entry table according to the directory entry serial number. For directory entries with a value of "retained" in the dictionary, the flag word is changed to "retained", and others are changed to "delete".
[0060] (2) Processing during the "traversal" process
[0061] In the operation of traversing the directory (the semantics is read, not transactional), the directory entry table is read in order. Because it is linearly stored, the performance is significantly better than traversing the balanced tree. Each directory entry is read out in turn. According to the flag word of the directory entry, if it is "reserved", the item is returned to the user operation; if the flag word is "unknown", the directory entry needs to be "reviewed". The review function returns the value of the flag word of this directory entry. If the result is "reserved", the information of this directory entry is returned to the user operation. Other branches indicate that this directory entry has been deleted or is not used.
[0062] (3) Review process
[0063] like Figure 1 As shown, first, find the corresponding directory entry from the balanced tree with the file name. If the corresponding directory entry is found (the name and serial number are consistent), (if it is not in the pending directory entry dictionary, the flag word of the directory entry traversed in the directory entry file is changed to "reserved",) return "reserved".
[0064] If the corresponding directory entry is not found (for example, if it is not in the pending directory entry dictionary, the flag word of the directory entry traversed in the directory entry file is changed to "delete"), return "delete".
[0065] It should be emphasized that the embodiments of the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation modes. Any other implementation modes derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.
Claims
1. A balanced tree traversal optimization method for storage, characterized in that: The following steps are involved: Step 1: In persistent storage, add a linear directory entry table for each directory; Step 2: Record the sequence number of the directory entry in the directory entry table and the final status of the operation into the pending directory entry dictionary in the memory; Step 3: Modify or traverse the directory entry to ensure data consistency between the added directory entry table and the balanced tree.
2. A balanced tree traversal optimization method for storage according to claim 1, characterized in that: The specific method of step 1 is: In the persistent storage, a linear directory entry table is added to each directory. Before modifying a directory entry, the flag word of the corresponding directory entry in the directory entry table is changed to "unknown", and additional changes are made to the content.
3. A balanced tree traversal optimization method for storage according to claim 1, characterized in that: The final status of the operation in step 2 includes: retention or deletion.
4. The method for optimizing balanced tree traversal for storage according to claim 1, characterized in that: The specific steps of modifying the directory entry in step 3 include: (1) In a directory modification transaction, any modification to a directory entry must first mark the corresponding directory entry in the directory entry table as "unknown" and make additional changes to the content; (2) Then add a pair of key values to the pending directory entry dictionary, where the key is the serial number of the corresponding directory entry and the value is the nature of the operation. If it is a modification or creation, its value is "retain"; for a deletion operation, its value is "delete"; (3) When the transaction ends, the pending directory entry dictionary in the memory is traversed, and for each directory entry in the dictionary, corresponding modifications are made to the directory entry table according to the directory entry sequence number. For directory entries whose value in the dictionary is "reserved", the flag word is changed to "reserved", and for others, it is changed to "deleted".
5. A balanced tree traversal optimization method for storage according to claim 1, characterized in that: The specific steps of traversing the directory entries in step 3 include: (1) In the operation of traversing the directory, read the directory entry table in sequence; (2) Each directory entry is read out in turn. If the flag word of the directory entry is "reserved", the item is returned to the user. If the flag word is "unknown", the directory entry needs to be "reviewed". The review function returns the value of the flag word of the directory entry. If the result is "reserved", the information of the directory entry is returned to the user. Other branches indicate that the directory entry has been deleted or is not used.
6. A balanced tree traversal optimization method for storage according to claim 5, characterized in that: The specific method for reviewing this catalog item in step (2) of step 3 is: ① Use the file name of the directory entry to be reviewed in the directory entry table to find the corresponding directory entry from the balanced tree. If the corresponding directory entry is found, that is, the name and serial number of the directory entry found on the balanced tree match the directory entry with review; if the corresponding directory entry is not found, go to step ④; ② Use the serial number of the directory item to be reviewed to find the corresponding key-value pair in the pending directory item dictionary; If the corresponding key-value pair is not found in the pending directory entry dictionary, the identification word of the pending directory entry in the directory entry table is changed to "reserved"; ③The review process returns to "reserved", and the process ends; ④ Use the serial number of the directory item to be reviewed to find the corresponding key-value pair in the pending directory item dictionary; If the corresponding key-value pair is not found in the pending directory entry dictionary, the identification word of the pending directory entry in the directory entry table is changed to "delete"; ⑤The review process returns to "delete" and the process ends.