A data processing method, file system, device and medium
By constructing a sequential linked list and using the tree node heat value to determine the IO type for data prefetching, the problem of inaccurate IO type determination caused by the variety of directory traversal methods is solved, thus improving the efficiency of directory traversal and data access.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ALIBABA (CHINA) CO LTD
- Filing Date
- 2023-03-03
- Publication Date
- 2026-05-19
AI Technical Summary
In existing technologies, directory traversal methods are diverse and lack unified rules, leading to inaccurate IO type judgment, affecting the accuracy of data prefetching, and thus reducing the efficiency of directory traversal and data access.
By constructing a sequential linked list, connecting the identifiers of tree nodes according to the directory access order, using the popularity value of the tree node to determine the IO type of the IO request, and performing data prefetching when the IO type is specified, compatibility with different rules for directory traversal is achieved.
It improves the performance of directory traversal and data access efficiency, is compatible with different directory traversal methods, and reduces IO access latency.
Smart Images
Figure CN116383142B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of storage technology, and in particular to a data processing method, file system, device, and medium. Background Technology
[0002] When accessing data in a file system, files or directories need to be searched according to certain rules. The search process involves directory traversal, and the performance of directory traversal directly affects the efficiency of data access.
[0003] To improve the performance of directory traversal, data prefetching can be performed, which means loading data from disk or other storage devices into memory before accessing the data. However, in the current technology, due to the variety of directory traversal methods, there is no unified rule to accurately determine the IO type. Inaccurate IO type will lead to inaccurate data prefetching, which in turn will affect the performance of directory traversal and the efficiency of data access. Summary of the Invention
[0004] In view of the above problems, a data processing method, file system, device, and medium are proposed to overcome or at least partially solve the above problems, including:
[0005] A data processing method, wherein the file system adopts a directory tree structure, the directory tree structure having multiple tree nodes, the method comprising:
[0006] Construct a sequential linked list based on the directory access order in the file system. The sequential linked list includes the identifiers of multiple tree nodes that are connected sequentially according to the directory access order.
[0007] Receive IO requests and determine the target tree node accessed by the IO request;
[0008] When the identifier of the target tree node is found in the sequential linked list, the heat value of the target tree node is determined, and the IO type of the IO request is determined based on the heat value of the target tree node.
[0009] When the IO type of the IO request is a specified IO type, data prefetching is performed based on the target tree node.
[0010] Optionally, when the identifier of the target tree node is found in the sequential linked list, the heat value of the target tree node is determined, including:
[0011] If the identifier of the target tree node is found in the sequential linked list, determine the preceding tree node of the target tree node; where the preceding tree node is the tree node in the sequential linked list that is ordered before the target tree node.
[0012] Determine the heat value of the target tree node based on the heat value of the forward tree node.
[0013] Optionally, the heat value of the target tree node is determined based on the heat value of the forward tree node, including:
[0014] Inherit the heat value of the forward tree node and increment it to obtain the heat value of the target tree node.
[0015] Optionally, the heat value of the target tree node is obtained by inheriting the heat value of the forward tree node and incrementing it, including:
[0016] Based on the heat values of multiple forward tree nodes, determine the target forward tree node from the multiple forward tree nodes;
[0017] The heat value of the target tree node is obtained by incrementing the heat value of the target forward tree node.
[0018] Optionally, the IO type of the IO request is determined based on the heat value of the target tree node, including:
[0019] When the heat value of the target tree node is greater than the preset heat value, the IO type of the IO request is determined to be the specified IO type.
[0020] Optionally, when the IO type of the IO request is a specified IO type, data prefetching is performed based on the target tree node, including:
[0021] When the IO type of the IO request is a specified IO type, determine the backward tree node of the target tree node; where the backward tree node is the tree node in the sequential linked list that is ordered after the target tree node;
[0022] Perform data prefetching on the data corresponding to the backward tree nodes.
[0023] Optionally, the backward tree nodes include a preset number of tree nodes in the sequential linked list that are ordered after the target tree node.
[0024] Optionally, the IO type can be specified as sequential IO.
[0025] A file system is characterized in that the file system adopts a directory tree structure, the directory tree structure has multiple tree nodes, the file system includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, the computer program implementing the above-mentioned data processing method when executed by the processor.
[0026] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the data processing method described above.
[0027] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data processing method described above.
[0028] The embodiments of the present invention have the following advantages:
[0029] In this embodiment of the invention, a sequential linked list is constructed according to the directory access order in the file system. The sequential linked list includes identifiers of multiple tree nodes connected sequentially according to the directory access order. When an IO request is received, the target tree node to be accessed by the IO request is determined. If the identifier of the target tree node is found in the sequential linked list, the popularity value of the target tree node is determined, and the IO type of the IO request is determined based on the popularity value of the target tree node. When the IO type of the IO request is a specified IO type, data prefetching is performed based on the target tree node. This realizes the determination of the IO type of the IO request based on the popularity value in the sequential linked list to trigger data prefetching. It can be compatible with directory traversal with different rules, improving the performance of directory traversal and the efficiency of data access. Attached Figure Description
[0030] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1a This is a schematic diagram of a directory tree structure provided in an embodiment of the present invention;
[0032] Figure 1b This is a schematic diagram of directory traversal in a directory tree structure provided by an embodiment of the present invention;
[0033] Figure 1c This is a schematic diagram of directory traversal in another directory tree structure provided by an embodiment of the present invention;
[0034] Figure 2 This is a flowchart of the steps of a data processing method provided in an embodiment of the present invention;
[0035] Figure 3 This is a flowchart of another data processing method provided in an embodiment of the present invention;
[0036] Figure 4 This is a flowchart of another data processing method provided in an embodiment of the present invention. Detailed Implementation
[0037] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0038] With the development of cloud-native technologies, various applications are gradually storing data in the cloud. For big data storage in the cloud, there are storage forms such as file storage and object storage.
[0039] Among them, object storage, as a new storage form, has advantages such as ease of use and high cost performance, and is usually the priority for big data storage. As for file storage, as a traditional storage form, it supports native POSIX (Portable Operating System Interface) semantics and has good application compatibility.
[0040] In real-world scenarios, big data analytics tools offer good support for the POSIX interface. Users expect to access object storage data in the same way they would access file storage data. However, due to architectural flaws in object storage, its traversal interface performance is poor. To accommodate different user habits, object storage can borrow the directory structure of file storage, optimize the flat directory structure, provide a user-space file system, and thus offer users a "folder" display method.
[0041] Therefore, when accessing data using the Posix interface, it is necessary to search for files or directories according to certain rules. The search process involves directory traversal, and the performance of directory traversal directly affects the efficiency of data access.
[0042] In one example, accessing the directory tree through the POSIX interface can be accomplished by multiple function operations forming a function call chain. For instance, accessing a folder requires three steps: 1. Open the folder using the `opendir` command; 2. Traverse the directory using the `readdirplus` command to obtain metadata information for all subdirectories or subfiles in the current directory; 3. Close the folder using the `releasedir` command. Similarly, accessing a file requires three steps: 1. Open the file using the `open` command; 2. Read or write the file using the `read` or `write` command; 3. Close the file using the `release` command.
[0043] To improve the performance of directory traversal, data prefetching can be performed, which means loading data from disk or other storage devices into memory before accessing the data. The IO type of the IO request can be determined based on the access order, i.e., whether it is sequential IO or random IO, and then data prefetching can be triggered according to the IO type of the IO request.
[0044] However, the access order differs depending on the rules used for directory traversal. Some rules distinguish between files and directories, processing directories before processing files, while others do not. This results in inaccurate judgment of the IO type of IO requests based on the access order, which in turn affects the performance of directory traversal and the efficiency of data access.
[0045] For example Figure 1a Taking the directory structure as an example, this directory structure is a directory tree structure. A directory tree is a tree-like organizational structure used in a file system to manage folders and files. It can have multiple tree nodes (inodes). It is a data structure in Unix and Linux operating systems, used to store basic information about directories and files, including filename, file size, time, user, and group. It serves as the index node for file systems and hard disk storage. Figure 1a The middle circle represents a tree node, where the number is the unique identifier of the tree node within the file system, such as Inode0 to Inode24.
[0046] In a directory tree structure, there are root nodes, parent nodes, child nodes, and leaf nodes. Leaf nodes correspond to files, and non-leaf nodes correspond to folders (directories). Figure 1a In the inode, inode0 is the root node, inode1, inode5, inode8, inode13, and inode17 are folders, and the other nodes are ordinary files. Inode0 is the parent node, which can include child nodes such as inode1, inode2, inode3, inode4, inode5, inode6, inode7, inode8, inode9, inode10, and inode11, and so on.
[0047] For access models employing partial rules, it's necessary to distinguish between files and directories, processing directories first and then files. This involves using tools on Linux systems to capture system calls and obtain the access order of the directory tree. Figure 1a Based on the directory tree, the access order is as follows: Figure 1b As shown below:
[0048] inode0, inode1, inode13, inode5, inode17, inode8, inode12, inode21, inode22, inode14, inode2, inode3, inode4, inode15, inode16, inode23, inode24, inode6, inode7, inode18, inode19, inode20, inode9, inode10, inode11.
[0049] The access order of this access model can be summarized as: preorder traversal of the directory tree, traversing directories first and then files. For example, the zip command in the Linux system follows this access principle.
[0050] For access models employing partial rules, which do not distinguish between files and directories, the access order of the directory tree is obtained by capturing system calls using tools on the Linux system. Figure 1a Based on the directory tree, the access order is as follows: Figure 1c As shown below:
[0051] inode0, inode1, inode12, inode13, inode21, inode22, inode14, inode2, inode3, inode4, inode5, inode15, inode16, inode17, inode23, inode24, inode6, inode7, inode8, inode18, inode19, inode20, inode9, inode10, inode11.
[0052] The access order of this access model can be summarized as: preorder traversal of the directory tree, without distinguishing between directories and files. For example, the commands tar, cp, ls, ll, du, and rm -rf in the Linux system all follow this access principle.
[0053] It is evident that the difference between access models using different rules lies in the fact that the former distinguishes between folders and directories at a single directory level, prioritizing the use of the opendir command to open all directories, and then using the open command to open all files.
[0054] The differences in access models based on different rules mean that the normalization scheme for different access models cannot treat directories and files differently. Instead, the type of IO request can be determined by the hit rate (i.e., the hotness value) of the temporary table, and then the decision to start data prefetching can be made based on the IO request, avoiding the need to determine whether to prefetch based on the access order.
[0055] Based on this, the embodiments of the present invention determine the IO type of the IO request by using the heat value in the sequential linked list. This enables accurate identification of whether directory IO is sequential or random IO on traditional kernel-mode and cloud-native user-space file system (FUSE) clients, thereby triggering prefetching of the metadata and file data of the directory to be accessed, thus reducing IO access latency, greatly improving directory operation performance, and being compatible with directory traversal rules of different types.
[0056] The following provides further explanation:
[0057] Reference Figure 2 The diagram illustrates a flowchart of a data processing method according to an embodiment of the present invention. The file system can adopt a directory tree structure, such as... Figure 1a A directory tree is a tree-like organizational structure in a file system used to manage folders and files. It can have multiple tree nodes (inodes) and is a data structure used in Unix and Linux operating systems to store basic information about directories and files, including filenames, file sizes, timestamps, users, and groups. It serves as the index node for file systems and hard drive storage. Figure 1a The middle circle represents a tree node, where the number is the unique identifier of the tree node within the file system, such as Inode0 to Inode24.
[0058] In a directory tree structure, there are root nodes, parent nodes, child nodes, and leaf nodes. Leaf nodes correspond to files, and non-leaf nodes correspond to folders (directories). Figure 1a In the inode, inode0 is the root node, inode1, inode5, inode8, inode13, and inode17 are folders, and the other nodes are ordinary files. Inode0 is the parent node, which can include child nodes such as inode1, inode2, inode3, inode4, inode5, inode6, inode7, inode8, inode9, inode10, and inode11, and so on.
[0059] Specifically, it may include the following steps:
[0060] Step 201: Construct a sequential linked list according to the directory access order in the file system. The sequential linked list includes the identifiers of multiple tree nodes connected sequentially according to the directory access order.
[0061] During the directory traversal of the file system, the directory access order can be determined, that is, the order in which tree nodes are traversed during the access process. Then, according to the directory access order, the identifiers of the tree nodes traversed in sequence can be connected to form a sequential linked list. As the directory traversal continues to move forward, the sequential linked list will be continuously built forward.
[0062] Specifically, when traversing a directory using the Readdirplus command, the traversed tree nodes are inserted after the target tree node, and when the folder is closed using the Releasedir command, they are removed from the linked list.
[0063] In one example, the file system has a globally unique sequential linked list. The information managed by the linked list is stored in random access memory. Since it is a temporary linked list, it can be reset even if it is lost due to power failure.
[0064] Step 202: Receive the IO request and determine the target tree node accessed by the IO request.
[0065] The IO request can be generated when accessing data via the POSIX interface. The data to be accessed can be stored in files or in object storage. For data stored in objects, the directory structure can be optimized by referencing the directory structure of file storage, providing a user-space file system and thus offering users a "folder" display method.
[0066] When an IO request is received, the system can perform directory traversal in the file system based on the IO request. By continuously traversing the directory tree, the system can find the target tree node that the IO request needs to access, and then access the target tree node to obtain the required data and provide feedback to the user.
[0067] It should be noted that in the process of finding the target tree node to be accessed, multiple tree nodes may be traversed until the target tree node to be accessed is found. The sequential linked list can be updated in real time according to the directory traversal process, that is, the tree nodes traversed in real time during the directory traversal are added to the sequential linked list.
[0068] Step 203: When the identifier of the target tree node is found in the sequential linked list, determine the heat value of the target tree node, and determine the IO type of the IO request based on the heat value of the target tree node.
[0069] For IO types, it can be determined whether the access order of IO requests within the most recent preset time period meets the consistency requirement with the sequential list. That is, it is determined whether the access order is completely consistent with the sequential list or the degree of consistency is greater than a preset degree. For example, if 90% of the access order is consistent with the sequential list, and the access order is completely consistent with the sequential list or the degree of consistency is greater than the preset degree, then the IO type is characterized as sequential IO. If the access order is not completely consistent with the sequential list or the degree of consistency is less than or equal to the preset degree, then the IO type is characterized as random IO.
[0070] Specifically, to prevent interference from random I / O, a popularity value can be set. The popularity value can be determined by the number of times a tree node is accessed. The number of times a node is accessed can be within a preset time range, such as the number of times it has been accessed in the last 5 minutes. The higher the number of times a node is accessed, the higher the popularity value, and the lower the number of times a tree node is accessed, the lower the popularity value.
[0071] The popularity value of tree nodes can be used to measure the reliability of IO order. The popularity value is positively correlated with the order of IO requests and negatively correlated with the randomness of IO requests. That is, the lower the popularity value, the fewer times the same one or more tree nodes are accessed, the lower the order and the higher the randomness. The higher the popularity value, the more times the same one or more tree nodes are accessed, the higher the order and the lower the randomness.
[0072] After accessing the target tree node, it can be determined whether the identifier of the target tree node can be found in the sequential linked list. If the identifier of the target tree node cannot be found in the sequential linked list, the IO type of the IO request may be random IO, and no further operation is required. If the identifier of the target tree node is found in the sequential linked list, the popularity value of the target tree node can be further determined. Then, based on the popularity value of the target tree node, the IO type of the IO request, i.e., whether it is random IO or sequential IO, can be determined.
[0073] In one embodiment of the present invention, when the identifier of the target tree node is found in the sequential linked list, determining the heat value of the target tree node includes:
[0074] If the identifier of the target tree node is found in the sequential linked list, determine the preceding tree node of the target tree node; based on the popularity value of the preceding tree node, determine the popularity value of the target tree node.
[0075] When the identifier of the target tree node is found in the sequential linked list, the preceding tree node of the target tree node can be determined. The preceding tree node can be a tree node that is sorted before the target tree node in the sequential linked list. Specifically, the preceding tree node can be a preset number of tree nodes sorted before the target tree node, such as 10. These can correspond to the most recently visited tree nodes. Then, the popularity value of the target tree node can be determined based on the popularity value of the preceding tree node.
[0076] In one embodiment of the present invention, determining the heat value of the target tree node based on the heat value of the forward tree node includes:
[0077] Inherit the heat value of the forward tree node and increment it to obtain the heat value of the target tree node.
[0078] Specifically, the popularity value of a target tree node is related not only to the number of times the target tree node itself is visited, but also to the number of times its preceding tree nodes are visited. Therefore, the popularity value of the target tree node can be obtained by inheriting the popularity value of the preceding tree nodes and incrementing it, such as by 1. This can represent the number of times the target tree node and its preceding tree nodes are visited.
[0079] Taking metadata in the file system as an example, when operating on metadata, you can start with the getattr command. If the obtained tree node is found in the sequential linked list, the tree node can inherit the heat value from the previous tree node and increment it, such as by 1.
[0080] Taking data in the file system as an example, when operating on the data, you can start with the read or write command. If the obtained tree node is found in the sequential linked list, the tree node can inherit the heat value from the previous tree node and increment it, such as by 1.
[0081] In one embodiment of the present invention, the heat value of the target tree node is obtained by inheriting the heat value of the forward tree node and incrementing it, including:
[0082] Based on the heat values of multiple forward tree nodes, the target forward tree node is determined from the multiple forward tree nodes; the heat value of the target forward tree node is obtained by incrementing the heat value of the target forward tree node.
[0083] In practical applications, the forward tree node with the highest heat value can be determined from multiple forward tree nodes and used as the target forward tree node. Then, the heat value of the target forward tree node can be incremented, such as by 1, to obtain the heat value of the target tree node.
[0084] In one embodiment of the present invention, determining the IO type of the IO request based on the heat value of the target tree node includes:
[0085] When the heat value of the target tree node is greater than the preset heat value, the IO type of the IO request is determined to be the specified IO type.
[0086] As an example, the IO type is specified as sequential IO.
[0087] After determining the popularity value of the target tree node, it is possible to further determine whether the popularity value of the target tree node is greater than the preset popularity value. For example, if the preset popularity value is 6, and the popularity value of the target tree node is less than or equal to the preset popularity value, then the IO type can be determined to be a non-specified IO type, i.e., random IO. If the popularity value of the target tree node is greater than the preset popularity value, then the IO type of the IO request can be determined to be a specified IO type, i.e., sequential IO.
[0088] Step 204: When the IO type of the IO request is a specified IO type, perform data prefetching based on the target tree node.
[0089] When the IO type of the IO request is a specified IO type, that is, when the IO type is sequential IO, it indicates that the tree node to be accessed later is related to the target tree node. Therefore, data prefetching can be performed on the target tree node before subsequent data access, and the data can be loaded from the disk or other storage devices into memory.
[0090] In one embodiment of the present invention, when the IO type of the IO request is a specified IO type, data prefetching is performed based on the target tree node, including:
[0091] When the IO type of the IO request is a specified IO type, determine the backward tree node of the target tree node; and perform data prefetching on the data corresponding to the backward tree node.
[0092] When the IO type of the IO request is a specified IO type, such as sequential IO, the backward tree node of the target tree node can be determined. The backward tree node can be a tree node in the sequential linked list that is sorted after the target tree node. Specifically, in order to avoid memory consumption, the backward tree node can be a preset number of tree nodes that are sorted after the target tree node, such as 20. Then, data prefetching can be performed on the data corresponding to the backward tree node.
[0093] Taking metadata in the file system as an example, when operating on metadata, you can start by using the getattr command. When the operation via the getattr command is recognized as sequential I / O, the metadata can be prefetched.
[0094] Taking data in a file system as an example, when operating on the data, you can start with a read or write command. When the read or write command operation is recognized as sequential I / O, the metadata can be prefetched.
[0095] In one example, for data smaller than a preset size, such as less than 1MB or 2MB, it can be prefetched directly into memory. For data larger than or equal to the preset size, the header data can be prefetched into memory, thus preventing memory waste.
[0096] In this embodiment of the invention, a sequential linked list is constructed according to the directory access order in the file system. The sequential linked list includes identifiers of multiple tree nodes connected sequentially according to the directory access order. When an IO request is received, the target tree node to be accessed by the IO request is determined. If the identifier of the target tree node is found in the sequential linked list, the popularity value of the target tree node is determined, and the IO type of the IO request is determined based on the popularity value of the target tree node. When the IO type of the IO request is a specified IO type, data prefetching is performed based on the target tree node. This realizes the determination of the IO type of the IO request based on the popularity value in the sequential linked list to trigger data prefetching. It can be compatible with directory traversal with different rules, improving the performance of directory traversal and the efficiency of data access.
[0097] Reference Figure 3 The diagram illustrates a flowchart of another data processing method provided by an embodiment of the present invention, which may specifically include the following steps:
[0098] Step 301: Construct a sequential linked list according to the directory access order in the file system. The sequential linked list includes the identifiers of multiple tree nodes connected sequentially according to the directory access order.
[0099] Step 302: Receive the IO request and determine the target tree node accessed by the IO request.
[0100] Step 303: When the identifier of the target tree node is found in the sequential linked list, determine the preceding tree node of the target tree node; wherein, the preceding tree node is the tree node in the sequential linked list that is ordered before the target tree node.
[0101] Step 304: Determine the target forward tree node from the multiple forward tree nodes based on the heat values of the multiple forward tree nodes.
[0102] Step 305: Increment the heat value of the target tree node based on the heat value of the target forward tree node to obtain the heat value of the target tree node.
[0103] Step 306: When the heat value of the target tree node is greater than the preset heat value, determine that the IO type of the IO request is the specified IO type.
[0104] Step 307: When the IO type of the IO request is a specified IO type, perform data prefetching based on the target tree node.
[0105] Reference Figure 4 The diagram illustrates a flowchart of another data processing method provided by an embodiment of the present invention, which may specifically include the following steps:
[0106] Step 401: Construct a sequential linked list according to the directory access order in the file system. The sequential linked list includes the identifiers of multiple tree nodes connected sequentially according to the directory access order.
[0107] Step 402: Receive the IO request and determine the target tree node accessed by the IO request.
[0108] Step 403: When the identifier of the target tree node is found in the sequential linked list, determine the heat value of the target tree node, and determine the IO type of the IO request based on the heat value of the target tree node.
[0109] Step 404: When the IO type of the IO request is a specified IO type, determine the backward tree node of the target tree node; wherein, the backward tree node is the tree node in the sequential linked list that is ordered after the target tree node.
[0110] Step 405: Perform data prefetching on the data corresponding to the backward tree nodes.
[0111] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.
[0112] An embodiment of the present invention also provides a file system. The file system adopts a directory tree structure with multiple tree nodes. The file system includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the data processing method described above.
[0113] An embodiment of the present invention also provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the above-described data processing method.
[0114] An embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the above-described data processing method.
[0115] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.
[0116] It should be noted that the user-related data involved in the embodiments of the present invention are all processed in accordance with the relevant data protection laws and policies of the country where the user is located, and with the authorization granted by the owner of the relevant device.
[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0119] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.
[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0122] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.
[0123] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0124] The above provides a detailed description of the data processing method, file system, device, and medium provided. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A data processing method, wherein the file system adopts a directory tree structure, the directory tree structure having multiple tree nodes, the method comprising: Based on the directory access order in the file system, a sequential linked list is constructed, wherein the sequential linked list includes the identifiers of multiple tree nodes connected sequentially according to the directory access order; Receive IO requests and determine the target tree node accessed by the IO request; When the identifier of the target tree node is found in the sequential linked list, the popularity value of the target tree node is determined, and the IO type of the IO request is determined based on the popularity value of the target tree node; wherein, the popularity value is determined based on the number of times the target tree node is accessed within a preset time range; the popularity value is positively correlated with the order of the IO request and negatively correlated with the randomness of the IO request; the IO type includes random IO and sequential IO; When the IO type of the IO request is a specified IO type, data prefetching is performed based on the target tree node.
2. The method according to claim 1, characterized in that, When the identifier of the target tree node is found in the sequential linked list, determining the heat value of the target tree node includes: If the identifier of the target tree node is found in the sequential linked list, the preceding tree node of the target tree node is determined; wherein, the preceding tree node is the tree node in the sequential linked list that is ordered before the target tree node; The heat value of the target tree node is determined based on the heat value of the forward tree node.
3. The method according to claim 2, characterized in that, Determining the heat value of the target tree node based on the heat value of the forward tree node includes: The heat value of the target tree node is obtained by inheriting the heat value of the forward tree node and incrementing it.
4. The method according to claim 3, characterized in that, The process of inheriting the heat value of the forward tree node and incrementing it to obtain the heat value of the target tree node includes: Based on the heat values of the multiple forward tree nodes, the target forward tree node is determined from the multiple forward tree nodes; The heat value of the target tree node is obtained by incrementing the heat value of the target forward tree node.
5. The method according to claim 1, characterized in that, Determining the IO type of the IO request based on the heat value of the target tree node includes: When the heat value of the target tree node is greater than the preset heat value, the IO type of the IO request is determined to be the specified IO type.
6. The method according to any one of claims 1 to 5, characterized in that, When the IO type of the IO request is a specified IO type, data prefetching based on the target tree node includes: When the IO type of the IO request is a specified IO type, the backward tree node of the target tree node is determined; wherein, the backward tree node is the tree node in the sequential linked list that is ordered after the target tree node; Data prefetching is performed on the data corresponding to the backward tree nodes.
7. The method according to claim 6, characterized in that, The backward tree node includes a preset number of tree nodes in the sequential linked list that are ordered after the target tree node.
8. The method according to claim 1, characterized in that, The specified IO type is sequential IO.
9. A file system, characterized in that, The file system adopts a directory tree structure with multiple tree nodes. The file system includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the data processing method as described in any one of claims 1 to 8.
10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the data processing method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the data processing method as described in any one of claims 1 to 8.