Metadata Management Method, Apparatus, Device, Medium and Product

Abstract

The present application discloses a metadata management method, apparatus, device, medium and product. By determining the write priority of metadata nodes based on the cache space utilization rate and node hit rate, metadata nodes that occupy a large cache space and have a low utilization rate are written, data that is not frequently accessed is written to the disk in a timely manner, and data that is more frequently accessed is retained in the cache. This reasonable write strategy avoids excessive occupation of the cache space. The metadata nodes to be written are aggregated and then written to the disk instead of writing each node separately, so that multiple small I / O operations can be combined into fewer large I / O operations. Reducing the number of I / O operations can effectively reduce the burden on the disk, improve the disk write efficiency, and reduce disk write amplification.

Description

Technical Field

[0001] This application relates to the technical field of servers, and particularly to a method, apparatus, device, medium, and product for metadata management. Background Art

[0002] All-flash storage systems use solid-state drives as the main storage medium. In an all-flash storage system, metadata access is a key area for optimization because the effective management of metadata directly affects the performance of data access and the stability of the storage system. Metadata includes the directory structure of the file system, file attributes, index information, etc., and it is crucial for quickly locating and accessing data.

[0003] In related technologies, metadata is usually maintained through a B+Tree. Each metadata node orderly maintains multiple mapping relationships from logical addresses to physical addresses. Since the amount of metadata far exceeds the amount of random access memory, the metadata nodes in the metadata need to periodically organize metadata flushing to disk. For example, when modifying a certain metadata during data flushing and inserting it into the corresponding leaf node for flushing to disk, the mapping relationships from the leaf node to the root node will all change. Therefore, all nodes (root node, intermediate nodes, and leaf nodes) need to be flushed to disk. However, this way of handling metadata flushing to disk requires reorganizing and associating the merged leaf node metadata to determine their hierarchical relationships and storage order, etc., which increases the process of metadata flushing, generates additional data, exacerbates the write amplification phenomenon, and results in a low disk write efficiency of the metadata nodes.

[0004] Therefore, how to improve the disk write efficiency of metadata nodes is an urgent problem to be solved currently. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for metadata management to at least solve the problem of low disk write efficiency of metadata nodes in related technologies.

[0006] This application provides a method for metadata management, and the method includes:

[0007] When the metadata stored in the cache meets the condition for writing to disk, obtain the flushing priorities of each metadata node according to the cache space utilization rate and node hit rate of each metadata node;

[0008] Determine the metadata nodes to be flushed according to the flushing priorities of each metadata node;

[0009] Perform aggregation processing on the metadata nodes to be flushed to obtain a metadata block;

[0010] Write the metadata block to disk.

[0011] The present application also provides a metadata management device, which includes:

[0012] An acquisition module, configured to obtain the writing priorities of each metadata node according to the cache space utilization rate and node hit rate of each metadata node when the metadata stored in the cache meets the condition of being written to the disk;

[0013] A determination module, configured to determine the metadata nodes to be written according to the writing priorities of each metadata node;

[0014] An aggregation module, configured to perform aggregation processing on the metadata nodes to be written to obtain a metadata block;

[0015] A writing module, configured to write the metadata block to the disk.

[0016] The present application also provides an electronic device, including: a memory, configured to store a computer program; a processor, configured to implement the steps of any of the above metadata management methods when executing the computer program.

[0017] The present application also provides a computer-readable storage medium, in which a computer program is stored, and wherein the computer program implements the steps of any of the above metadata management methods when executed by a processor.

[0018] The present application also provides a computer program product, including a computer program, and the computer program implements the steps of any of the above metadata management methods when executed by a processor.

[0019] Through the present application, when the metadata stored in the cache meets the condition of being written to the disk, the writing priorities of each metadata node are obtained according to the cache space utilization rate and node hit rate of each metadata node, the metadata nodes to be written are determined according to the writing priorities of each metadata node, the metadata nodes to be written are subjected to aggregation processing to obtain a metadata block, and the metadata block is written to the disk. By determining the writing priorities of metadata nodes through the cache space utilization rate and node hit rate, the metadata nodes that occupy a large amount of cache space and have a low utilization rate are written, the data that is not frequently accessed is written to the disk in a timely manner, and the data that is more frequently accessed is retained in the cache. This reasonable writing strategy avoids excessive occupation of the cache space. After aggregating the metadata nodes to be written and then writing to the disk, instead of writing each node separately, multiple small I / O operations can be combined into fewer large I / O operations. Reducing the number of I / O operations can effectively reduce the burden on the disk, improve the disk writing efficiency, and reduce disk write amplification. Description of the Drawings

[0020] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0021] To more clearly illustrate the embodiments of the present application, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0022] Figure 1 It is a schematic flowchart of a metadata management method provided for an embodiment of the present application;

[0023] Figure 2 It is a schematic structural diagram of a metadata management device provided for an embodiment of the present application. Detailed implementation manners

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.

[0025] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0026] To enable those skilled in the art of this technology to better understand the solution of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0027] Term explanation:

[0028] LBA: Logical Block Addressing, the logical block address, is a method of addressing data storage on a hard disk. LBA is a linear address allocation method that divides the storage space on the hard disk into logical blocks of the same size (usually each logical block is 512 bytes or 4096 bytes), and assigns a unique number (address) to each logical block. These numbers start from 0 and increase sequentially. For example, if a hard disk has 1000 logical blocks, then the LBA address range of these logical blocks is from 0 to 999.

[0029] PBA: Physical Block Addressing, the physical block address, refers to the actual storage location of data in a storage device (such as a hard disk, solid-state drive, etc.). It corresponds to LBA (Logical Block Addressing), and is the address used inside the storage device to identify physical storage units.

[0030] Metadata: Metadata is data that describes other data, used to provide information such as the structure, attributes, and location of the data. For example, in a file system, metadata may include information such as the size of the file, creation time, modification time, permissions, file type, storage location, etc. In a database, metadata includes table structure, index information, data types, etc. The file system can quickly locate the storage location of a file on the disk through metadata, and the database system can quickly query table structure and index information through metadata.

[0031] Metadata node: In a full-flash storage system or related distributed system environment, a metadata node usually refers to a node responsible for storing and managing metadata, with functions such as storing, processing, and providing metadata. The metadata node provides metadata access services for other components or clients in the entire storage system. When an application needs to access the metadata of a file, it sends a request to the metadata node, and the metadata node will extract relevant information from the stored metadata according to the request content and return it to the application, enabling the application to understand the relevant attributes and data storage location of the file, and then perform subsequent operations.

[0032] Flushing to disk: It means forcing the data or metadata stored in volatile storage media such as memory to be written into non-volatile storage media such as flash memory to ensure that the data will not be lost when the system powers off or fails. A full-flash storage system usually sets a cache in memory to temporarily store frequently accessed data and metadata to improve the read and write performance of the system. When the data in the cache reaches a certain threshold (for example, the cache is full or the residence time of some data in the cache exceeds the set period), or the system receives a specific instruction (such as a regular disk flushing instruction, the system is about to shut down, etc.), the flushing to disk operation will be triggered.

[0033] The cache hit rate refers to the proportion of the number of requests for which the required data exists in the cache to the total number of requests. When the cache hit rate for metadata read and write is low, it means that most read and write requests for metadata cannot find the required data in the cache and still need to be retrieved from the original storage location (disk).

[0034] Low effective utilization of the cache means that the cache is not fully utilized to store valuable metadata. It is possible that there is a lot of metadata in the cache that is rarely accessed or almost never accessed again, while the frequently accessed metadata is not cached or not updated in the cache in a timely manner, resulting in waste of cache space and inability to achieve the best performance improvement effect.

[0035] A multi-level queue is a data structure composed of multiple independent queues. Each queue usually has a different priority, and data items can be assigned to different queues according to their priorities, access frequencies, or other business rules. This structure allows the system to classify and prioritize data items, thus achieving more effective management and scheduling.

[0036] In a cache system, a multi-level queue can be used to manage the cache policy of metadata nodes. By keeping frequently accessed data items in the high-priority queue, the cache hit rate can be improved. When a cache hit occurs, the corresponding metadata node may be moved to a higher-priority queue to reflect its popularity.

[0037] Cache eviction: When the cache space is insufficient, the system may start evicting metadata nodes from the lowest-priority queue.

[0038] Prefetching and warm-up: The system can preload the potentially required data into the cache according to the priorities and access patterns of metadata nodes.

[0039] Meta Cache: Metadata cache is a cache mechanism used to store and manage metadata. By caching frequently accessed metadata, the number of accesses to the underlying storage device is reduced, thereby improving the read performance and response speed of the system.

[0040] All-flash storage system: It is a high-performance storage system built entirely based on flash memory technology. It abandons the traditional hard disk drive (HDD) and only uses solid-state drives (SSDs) as the storage medium, thus significantly improving the read and write performance, response speed, and reliability of the storage system.

[0041] Based on the above problems, the embodiments of the present application provide a metadata management method, which will be described in detail in combination with the execution process of the metadata management method.

[0042] Refer to Figure 1As shown in the figure, the metadata management method provided by the embodiment of the present invention includes the following steps:

[0043] S11. When the metadata stored in the cache meets the condition of being written to the disk, obtain the write priority of each metadata node according to the cache space utilization rate and node hit rate of each metadata node.

[0044] Specifically, when the metadata stored in the cache meets the condition of being written to the disk, obtain the write priority of each metadata node according to the cache space utilization rate and node hit rate of each metadata node.

[0045] Optionally, the above step S11 can be implemented in the following manner:

[0046] (1). Based on a preset data structure, classify each metadata node to determine the initial write priority of each metadata node.

[0047] Among them, the initial write priorities of the metadata nodes from high to low are: leaf nodes, intermediate nodes, and root nodes.

[0048] Specifically, the default order of the initial write priorities is leaf nodes > intermediate nodes > root nodes. Since the leaf nodes directly store the actual metadata information, flushing them to the disk first can persist the latest data faster, ensure data security and consistency, and reduce the risk of data loss. While the intermediate nodes and root nodes are mainly used for indexing and organizing the leaf nodes, and their updates are relatively less urgent, so their write priorities are lower.

[0049] (2). Calculate the write priority weights of each metadata node according to the cache space utilization rate and node hit rate of each metadata node.

[0050] Among them, the node hit rate refers to the ratio of the number of accesses to a certain node to the total number of accesses within a certain period of time. A high hit rate indicates that the node is frequently accessed and may be hot data. If the space utilization rate of the hot data is also low, it means that it occupies more cache space but has relatively less data volume. At this time, lower its disk flushing priority to keep it in the cache so as to quickly respond to subsequent access requests, reduce the number of reads from the disk, and improve system performance.

[0051] The node space utilization rate refers to the ratio of the cache space occupied by the node to the total space allocated to it. A high space utilization rate indicates that the node fully utilizes the cache space to store data. When the node hit rate is low but the space utilization rate is high, it means that the node stores more data that is not frequently accessed and occupies valuable cache resources. At this time, the disk flushing priority of the node should be increased to flush its data to the disk to release the cache space for more needed nodes.

[0052] Optionally, step (2) above can be implemented as follows:

[0053] Obtain the cache space utilization rate and node hit rate of each metadata node;

[0054] Determine the cache space utilization rate weight and node hit rate weight of each metadata node;

[0055] Calculate the write priority weight of each metadata node according to the cache space utilization rate, the cache space utilization rate weight, the node hit rate, and the node hit rate weight of each metadata node.

[0056] Specifically, use the formula: write priority weight = node hit rate × node hit rate weight + space utilization rate × space utilization rate weight to calculate the write priority weight of each node. The node hit rate weight and space utilization rate weight are parameters preset according to the actual requirements and characteristics of the system, and are used to adjust the influence degree of the node hit rate and space utilization rate on the weight. For example, if the system has high requirements for the access performance of hot data, the node hit rate weight can be appropriately increased; if the cache resources of the system are relatively tight and more attention needs to be paid to the space utilization rate, the space utilization rate weight can be increased.

[0057] (3) Based on the change of the write priority weight of each metadata node, adjust the initial write priority of each metadata node to determine the write priority of each metadata node.

[0058] Specifically, the higher the node hit rate and the lower the space utilization rate, the lower the write priority weight of the node at this time, and the corresponding disk write priority of the node is reduced; the lower the node hit rate and the higher the space utilization rate, the higher the write priority weight of the node at this time, and the corresponding disk write priority of the node is increased.

[0059] A possible situation is that when the cache resources are insufficient, increasing the weight of the space utilization rate is to more preferentially write the data of those nodes with high space utilization rate but low hit rate to the disk, so as to timely release the cache space, reduce the waiting time of metadata in the cache, and avoid data loss or system performance degradation caused by cache overflow. In this way, the disk write priority can be dynamically adjusted according to the usage of cache resources, so that the system can maintain good performance and stability under different load conditions.

[0060] Exemplarily, assume there are three metadata nodes A, B, and C in the system. The hit rate of node A is 80% and the space utilization rate is 30%; the hit rate of node B is 30% and the space utilization rate is 70%; the hit rate of node C is 50% and the space utilization rate is 50%. Let the weight of the node hit rate be 0.4 and the weight of the space utilization rate be 0.6. Then the weight of node A = 0.8×0.4 + 0.3×0.6 = 0.5; the weight of node B = 0.3×0.4 + 0.7×0.6 = 0.54; the weight of node C = 0.5×0.4 + 0.5×0.6 = 0.5. At this time, the weight of node B is the highest, and the disk write priority is relatively high. If the cache resources are insufficient, increase the weight of the space utilization rate to 0.8 and decrease the weight of the node hit rate to 0.2. Then the weight of node A = 0.8×0.2 + 0.3×0.8 = 0.4; the weight of node B = 0.3×0.2 + 0.7×0.8 = 0.62; the weight of node C = 0.5×0.2 + 0.5×0.8 = 0.5. It can be seen that the weight of node B is further increased, and its disk write priority is also further improved, and it is more likely to be preferentially written to the disk to release the cache space.

[0061] In some embodiments, before determining that the metadata stored in the cache meets the condition for writing to the disk, the following steps are further performed:

[0062] Determine whether the metadata stored in the cache is consistent with the metadata read from the disk;

[0063] If the metadata stored in the cache is inconsistent with the metadata read from the disk, mark the metadata stored in the cache as dirty metadata.

[0064] Specifically, when data is written, it not only involves the storage of actual data, but also generates operations for adding, deleting, and modifying metadata. Since the operations of adding, deleting, and modifying metadata are only written into the cache space first and not immediately persisted to storage devices such as disks, the metadata in the cache is inconsistent with the data state on the disk at this time. This metadata with inconsistent state is called dirty metadata (dirty-meta). That is, dirty metadata refers to the metadata that has been modified in the cache but has not been synchronized to the persistent storage (such as a disk).

[0065] Exemplarily, the metadata of a file has its file size information modified in the cache, but this modification has not been written to the disk yet. Then the metadata of this file is in the state of dirty metadata in the cache.

[0066] Optionally, the condition that the metadata stored in the cache meets the requirement for writing to the disk includes: the proportion of the dirty metadata occupied in the cache is greater than a preset proportion.

[0067] Among them, the preset ratio can be set according to the actual situation. For example, when the dirty metadata stored in the cache reaches 80% of the cache space, or other reasonable values, there is no specific limitation here.

[0068] Specifically, the cache space has a certain capacity limit. When the dirty metadata stored in it reaches a certain water level (that is, a certain ratio or quantity threshold of the cache capacity), a cache write operation will be triggered. Cache writing is to write the accumulated dirty metadata in the cache to a persistent storage device (such as a disk) so that the data in the storage device (such as a disk) is consistent with the data state in the cache.

[0069] Exemplarily, when the dirty metadata occupancy in the cache space reaches 80%, the system will automatically start the cache writing process, batch write these dirty metadata to the disk to release the cache space and ensure data persistence.

[0070] In some embodiments, before determining that the metadata stored in the cache meets the condition for writing to the disk, the following steps are also performed:

[0071] In response to a data write instruction, aggregate the data to be written into target data;

[0072] Allocate continuous space for the target data and sequentially write the target data into the storage units of the data space by way of append write;

[0073] When the target metadata is generated when writing the target data, query whether there is a metadata node corresponding to the target metadata in the preset data structure;

[0074] If there is a metadata node corresponding to the target metadata in the preset data structure, update the metadata node;

[0075] If there is no metadata node corresponding to the target metadata in the preset data structure, apply for a metadata node corresponding to the target metadata.

[0076] Among them, the preset data structure can be a B+ tree. A B+ tree is a balanced tree data structure, often used in storage systems to maintain indexes. It supports fast data search, insertion, and deletion operations.

[0077] Specifically, when writing data, in response to a data writing instruction, the data to be written is aggregated into target data; continuous space is allocated for the target data, and the target data is written into the storage unit of the data space by means of append writing. When target metadata is generated during the writing of the target data, it is queried whether there is a metadata node corresponding to the target metadata in the preset data structure; if there is a metadata node corresponding to the target metadata in the preset data structure, the metadata node is updated; if there is no metadata node corresponding to the target metadata in the preset data structure, a corresponding metadata node is applied for the target metadata.

[0078] When applying for a metadata node (node), a cache space of a preset size will be allocated for it, and the size of the cache space is 4K or 8K. The size of the metadata node is set to 4KB or 8KB (each node can index more child nodes). Compared with the cache space of 512 bytes for each metadata node in the prior art, the cache space of the metadata node in this solution can more easily perform node splitting and merging operations, and is also convenient for caching and pre-reading in memory; larger nodes can reduce the depth of the tree, thereby reducing the number of cache misses required when searching for data; larger nodes can store more key values and pointers, thereby reducing disk I / O operations when searching for data. B+ trees are usually optimized for sequential I / O operations, and larger nodes can make better use of this feature to improve I / O efficiency.

[0079] In a log-structured storage system, all write operations (including data and metadata) are recorded in a log area, which is usually located in the storage device and is written sequentially. This sequential writing method can make full use of the writing performance of the storage device. Log aggregation refers to combining multiple small write operations into one large write operation. The advantage of this is that it can reduce the number of write operations, thereby reducing the wear on the storage device and improving the write efficiency. In a log-structured storage system, these small write operations are first recorded in the log, and then a background process (such as a garbage collector) periodically aggregates these logs and writes them into the data space.

[0080] When writing the aggregated data into the data space, the system allocates continuous space for the aggregated data. This can reduce the addressing time of the storage device when writing data and improve the writing performance.

[0081] Through the above strategies of log aggregation and allocating continuous space, write amplification can be effectively reduced. Specifically, by combining multiple small write operations into one large write operation, the actual amount of data written can be reduced, thereby reducing write amplification.

[0082] In addition, when data is written to an all-flash storage system, new metadata is usually generated. Taking the insertion of the logical address to physical address (LP) mapping as an example, the processing of new metadata through an index B+ tree is described. Herein, the LP mapping is the mapping from the logical address to the physical address, which indicates the actual location of the data on the storage device, and it involves updating nodes to include the logical address and the corresponding physical address.

[0083] When a user or application initiates a data write request, the storage system writes the data to the specified location; associated with the data write operation, the system generates new metadata. For example, in a file system, this may include the creation time, owner, size, etc. of the file; in a database, this may include the index information of the table, the identifier of the row, etc. The system uses a B+ tree to index and query metadata. When a new metadata needs to be inserted, the system first looks up the corresponding metadata node (Node) in the B+ tree. The system queries in the B+ tree whether there is a node related to the new metadata (i.e., the target metadata), for example, it can be achieved by looking up the file name, the logical address of the data block, etc. in the B+ tree. If the corresponding node is found, the system updates the information in the node to reflect the new data write operation. If the corresponding node is not found, the system needs to create a target metadata node and insert it into the B+ tree. After inserting the new LP mapping, the system needs to maintain the balance of the B+ tree, which may involve splitting or merging of nodes. To maintain the performance of the B+ tree, the system needs to ensure that the height of the tree is as low as possible.

[0084] Querying and inserting metadata LP mappings through an index B+ tree is an effective method for managing metadata in a storage system. This method not only supports fast data lookup and update, but also helps to maintain data consistency and integrity. By optimizing the structure of the B+ tree and managing the cache, the performance and reliability of the storage system can be further improved.

[0085] In the embodiments of the present disclosure, the allocation of the metadata space is uniformly responsible for by the data space management system, which can achieve more efficient resource management and scheduling.

[0086] Among them, the metadata memory space refers to the area allocated in the memory for temporarily storing metadata to facilitate quick access and management of metadata. The metadata memory space is used to store metadata, which is used to describe the attributes and locations of data in the data space, including the name, size, creation time, modification time, permissions, storage location, etc. of the file, and can help the system quickly locate and access the data in the data space, maintaining the consistency and integrity of the data. The data in the metadata memory space changes dynamically and is updated as data is added, deleted, modified, or queried. Since the memory access speed is much faster than that of the disk, the metadata access speed can be improved. For frequently updated metadata, caching it in the memory can reduce the number of writes to the disk because multiple write operations can be aggregated in the memory first and then written to the disk uniformly.

[0087] The data space refers to the area in the storage system for storing actual user data. The data space is used to store user data such as file contents, database records, pictures, videos, etc.

[0088] In addition, when data is added, deleted, or modified, the system does not immediately apply these operations directly to the metadata in the storage device. Instead, these operations are first merged and then written to the cache space. The cache space is a region in the memory. Since the read and write speed of the memory is much faster than that of the storage device (such as a solid-state drive), this can reduce the number of direct writes to the storage device and improve the write performance. For example, the add, delete, and modify operations of the metadata of multiple files may be merged into a batch and written to the cache, rather than each operation being written to the storage device separately.

[0089] In some embodiments, the data of the metadata node is synchronously replicated to the mirror node.

[0090] To ensure the security and reliability of the data, the synchronous mirroring method is adopted. That is, the data of the metadata node is synchronously replicated to the mirror node for backup. When the primary node (the node storing the original metadata) fails, such as due to hardware damage, software errors, etc., resulting in data loss or inaccessibility, a complete and consistent metadata copy can be obtained from the mirror node, so as to ensure that the system can continue to operate normally and reduce the risk of data loss. This cache mirroring function is similar to a data redundancy strategy, and by saving the same data on different nodes, the fault tolerance of the system and the availability of the data are improved.

[0091] S12. Determine the metadata node to be flushed according to the flushing priorities of the respective metadata nodes.

[0092] In some embodiments, after performing the above step S12, the following steps may also be performed:

[0093] Within a first preset duration, determine whether the write count of the first metadata node in the preset address space cached is greater than or equal to a preset count;

[0094] If the write count of the first metadata node is greater than or equal to the preset count, then lower the write priority of the first metadata node;

[0095] If the write count of the first metadata node is less than the preset count, then raise the write priority of the first metadata node.

[0096] Specifically, before writing metadata, it is necessary to decide whether to write a certain metadata node based on the hit situation within the local address space and time range. The preset address space refers to the address area adjacent to or related to the metadata node to be judged currently, which includes the metadata node and some surrounding nodes. The first preset duration is a set time period used to count the hit situation of nodes during this period. By comprehensively considering these two factors, the system can more accurately judge which nodes need to be written and which can be temporarily not processed. Within the first preset duration, by determining whether the write count of the first metadata node in the preset address space cached is greater than or equal to the preset count, if the write count of the first metadata node is greater than or equal to the preset count, then lower the write priority of the first metadata node; otherwise, raise the write priority of the first metadata node.

[0097] When it is found that the first metadata node and the related metadata nodes before and after it are written multiple times in a short period, it can be judged that there is hot write data in the preset address space. In some cases, the preset duration can be defined as a few minutes or even a shorter time interval.

[0098] For the related metadata nodes with hot write data, the system will lower their disk write priorities. This is because if frequently updated data is immediately written to disk (write the data in the cache to a persistent storage device such as a disk), it may cause a large number of disk I / O operations, affecting system performance. After lowering the disk write priority, the data of these nodes will stay in the cache for a longer time, waiting for a suitable opportunity to be written to disk to reduce unnecessary I / O overhead.

[0099] Contrary to the case of writing data to hotspots, if the metadata nodes within the preset address space range have not been modified within the first preset duration, the system will increase the write priority of the first metadata node. The first preset duration is also a configurable time parameter, which could be several hours or even days, for example. For such nodes that have not been modified for a long time, writing them to storage devices such as disks in a timely manner can ensure data persistence and free up cache space to cache other required node data. After the write priority is increased, these nodes will be preferentially processed in the cache write operation to complete the disk write operation as soon as possible.

[0100] The strategy of dynamically adjusting the write priority of metadata nodes based on the hit situation within the local address space and time range mainly aims to optimize the data transfer process between the cache and persistent storage (such as disks), improving the overall performance and resource utilization rate of the system. By reasonably arranging which nodes are written first and which are written later, it can not only ensure that hot data is processed more effectively in the cache but also persistently store stable data in a timely manner, avoiding waste of cache space, so that the system can operate efficiently when dealing with different types of data access and updates.

[0101] As metadata is continuously updated and deleted, there may be many unused metadata spaces. If these garbage spaces are not recycled in a timely manner, it will reduce the utilization rate of the storage space. Based on this, the embodiments of the present disclosure provide the following metadata space garbage collection mechanism to ensure the continuous effectiveness of the metadata space.

[0102] Optionally, after performing data aggregation processing on multiple metadata nodes in the cache to form multiple metadata blocks, the following method can also be executed:

[0103] Periodically obtain the space utilization rate of the multiple metadata blocks;

[0104] Recycle the metadata block space of the metadata blocks with a space utilization rate lower than the preset space utilization rate among the multiple metadata blocks.

[0105] Among them, the preset space utilization rate can be set according to the actual situation. For example, it can be set to 50%, 30%, 20%, etc., and no specific limitation is made here.

[0106] Specifically, periodically obtain the space utilization rate of multiple metadata blocks, and recycle the metadata block space of the metadata blocks with a space utilization rate lower than the preset space utilization rate among the multiple metadata blocks. That is, when the usage efficiency of the metadata space drops to a certain level (for example, there is too much free space or it is severely fragmented), the system should initiate metadata garbage collection. Among them, metadata garbage collection includes: identifying unused metadata spaces, marking them as recyclable, and then reorganizing the storage space to release these spaces for subsequent use.

[0107] Optionally, after performing data aggregation processing on multiple metadata nodes in the cache to form multiple metadata blocks, the following method can also be executed:

[0108] Obtain the overall inefficiency of the metadata space, the utilization rate of the metadata space, and the dynamic change speed of the metadata space in real time.

[0109] Determine the preset recycling threshold of the metadata space according to the overall inefficiency of the metadata space, the utilization rate of the metadata space, and the dynamic change speed of the metadata space.

[0110] Among them, the overall inefficiency of the metadata space is used to represent the proportion of the unutilized part existing in the metadata space. For example, some metadata may become irrelevant due to changes in the data structure, but still occupy storage space, which will reduce the overall efficiency of the metadata space. By evaluating the overall inefficiency, it can be known how much potential space can be recycled in the metadata space.

[0111] The utilization rate of the metadata space is used to represent the proportion of the occupied part in the current metadata space to the total space. If the utilization rate of the metadata space is too high, it means that the metadata space may face the pressure of insufficient space, and some space needs to be recycled in time to avoid system failures or performance degradation. On the contrary, if the utilization rate is too low, it may mean that there is over-allocation in the metadata space, and the allocation strategy needs to be adjusted to improve the space utilization rate. By monitoring the utilization rate, the usage status of the metadata space can be understood, providing an important basis for adjusting the recycling threshold.

[0112] The dynamic change speed of the metadata space is used to represent the degree of change of the usage of the metadata space over time. The dynamic change speed includes the frequencies of operations such as addition, deletion, and modification of metadata. If the dynamic change speed of the metadata space is relatively fast, it means that the data activities in the system are relatively frequent, and the recycling threshold may need to be adjusted more flexibly to adapt to the changing space requirements. For example, in some systems with high real-time requirements, metadata may be updated and deleted frequently. At this time, the recycling strategy needs to be adjusted in time according to the dynamic change speed to ensure that the metadata space always maintains an efficient utilization state.

[0113] Specifically, by comprehensively considering the overall inefficiency of the metadata space, the utilization rate of the metadata space, and the dynamic change speed of the metadata space, a preset recycling threshold is determined through a certain algorithm. For example, corresponding weights can be assigned according to the importance of different factors, and then weighted calculation is performed. For example, for a system with high requirements for space utilization, the weight of the overall inefficiency may be set to a relatively high value; while for a system with high requirements for data real-time performance, the dynamic change speed may be more emphasized.

[0114] Through this comprehensive judgment method, a relatively reasonable recycling threshold can be obtained, enabling the metadata space to maintain a high recycling utilization rate under different workloads and system states, while ensuring the stability and performance of the system.

[0115] When it is detected that the utilization rate of the metadata space is greater than or equal to the preset recycling threshold, the space recycling mechanism is triggered.

[0116] The preset recycling threshold of the metadata space adopts a dynamic adjustment strategy, which is flexibly adjusted according to the actual usage of the metadata space, and more efficient metadata space management can be achieved.

[0117] S13. Aggregate the metadata nodes to be rewritten to obtain metadata blocks.

[0118] Specifically, aggregate the metadata nodes to be rewritten to obtain metadata blocks. First, these metadata nodes to be rewritten will be classified and grouped according to the type, relevance, or other attributes of the metadata. During the classification process, it will be checked whether there is redundant or duplicate information in the metadata nodes. If duplicate metadata records are found, merging or removal operations will be performed, and only one valid piece of data will be retained. This can reduce the data volume, improve the storage efficiency, and also avoid repeatedly writing the same data during the rewrite process.

[0119] The aggregation process also includes constructing the logical relationships between metadata nodes. For example, in a tree-structured metadata organization, determine the parent-child relationships, hierarchical relationships, etc. between each node. By clarifying these logical relationships, after the metadata is rewritten to persistent storage, the integrity and correctness of the data structure can be maintained, facilitating subsequent data access and management.

[0120] S14. Write the metadata blocks to the disk.

[0121] Specifically, when the metadata rewrite operation is triggered, first, the metadata nodes to be rewritten will be added to the list to be written. This list to be written can be understood as a task queue, which records the metadata node information that is about to be rewritten to the storage device (such as a disk), so that the system can process it in order.

[0122] After adding the metadata nodes to the above-mentioned list to be written, the system will apply for block space for these metadata. Metadata blocks are the basic units for storing metadata and are used to store metadata information. The process of applying for block space is actually to reserve a continuous storage space on the storage device for subsequent writing of metadata into it.

[0123] To improve storage efficiency and performance, the system will organize the metadata in the form of stripes for flushing. Striping is a technique that divides data into multiple small pieces and distributes them at different storage locations. When constructing a full stripe to flush data, the system will fill the metadata nodes in the destage list into the stripe according to certain rules to make it reach the full stripe state. This can make full use of the read / write bandwidth of the storage device, reduce the number of I / O operations, and improve the flushing efficiency.

[0124] After constructing the full stripe for flushing data, the system needs to allocate physical addresses for each stripe. Physical addresses are the actual storage locations on the storage device. By mapping the stripes to physical addresses, the system can accurately know where to write data to the storage device and where to read data from. The process of allocating physical addresses is usually completed by the address mapping module of the storage system, which will allocate appropriate physical addresses for the stripes according to the layout of the storage device and the available space situation.

[0125] Writing the constructed stripe data into the allocated physical addresses is the flushing operation of the metadata. Once the flushing is completed, the metadata changes from the original dirty state to the clean state. Dirty metadata refers to the data that has been modified in memory but has not been flushed to the storage device yet, while clean metadata refers to the data that has been successfully flushed to the storage device and is consistent with the data on the storage device.

[0126] In some embodiments, after performing the above step S14 (writing the metadata block to the disk), the following steps are further performed:

[0127] 1), Obtain the access popularity of the metadata written to the disk according to the access times and access time of the metadata;

[0128] 2), Dynamically adjust the positions of each metadata in the multi-level queue according to the access popularity of the metadata written to the disk;

[0129] 3), Periodically evaluate the access popularity of the metadata stored in the multi-level queue to identify hot metadata;

[0130] 4), Prefetch the hot data in the multi-level queue and the associated data of the hot data into the cache.

[0131] Among them, the associated data of the hot data includes: metadata adjacent to the hot data and metadata logically related to the hot data.

[0132] Exemplarily, if the currently accessed metadata is a file or directory in a file system, prefetch the adjacent metadata. For example, if a file is accessed, the metadata of other files in the same directory can be prefetched. It is also possible to perform prefetching based on the logical relationship between metadata. For example, if the metadata of a directory is accessed, the metadata of all subdirectories and files under that directory can be prefetched. Also, for example, if a user frequently accesses files in a certain directory, the metadata of other files in that directory can be prefetched.

[0133] Optionally, the multi-level queue includes a first multi-level queue and a second multi-level queue. Among them, the first multi-level queue is used to count the metadata heat of intermediate nodes, and the second multi-level queue is used to count the metadata heat of leaf nodes; the access priority of the first multi-level queue is higher than that of the second multi-level queue; the first multi-level queue and the second multi-level queue each include at least two priority queues.

[0134] Specifically, the multi-level queue can include multiple priority queues, and each priority queue is used to store metadata with different heat levels. In the embodiments of the present disclosure, the multi-level queue includes a first multi-level queue and a second multi-level queue. The first multi-level queue is used to count the metadata heat of intermediate nodes, and the second multi-level queue is used to count the metadata heat of leaf nodes. The access priority of the first multi-level queue is higher than that of the second multi-level queue; the first multi-level queue and the second multi-level queue each include at least two priority queues. For example, both the first multi-level queue and the second multi-level queue can include a low-priority queue, a medium-priority queue, and a high-priority queue. Among them, the low-priority queue is used to store metadata with a low access frequency, the medium-priority queue is used to store metadata with a medium access frequency, and the high-priority queue is used to store metadata with a high access frequency.

[0135] Among them, the access frequency refers to the number of times the metadata is accessed, and the access time refers to the time when the metadata was last accessed.

[0136] Specifically, based on the access count and access time, the access popularity of metadata can be calculated. When the metadata is accessed for the first time, it can be added to the lowest priority queue, and at the same time, the access count and access time of the metadata are recorded. Each time the metadata is accessed, its access count and access time are updated. For example, a counter can be used to record the access count, and the current timestamp can be recorded as the access time. According to the calculated access popularity, the position of the metadata in the multi-level queue is dynamically adjusted. If the access popularity of the metadata exceeds the access popularity threshold, it is promoted from the current queue to a higher priority queue. For example, if the access popularity of the metadata exceeds the threshold of the low priority queue, it is promoted to the medium priority queue; if the access popularity of the metadata exceeds the threshold of the medium priority queue, it is promoted to the high priority queue. Periodically evaluate the popularity of the metadata in the multi-level queue, identify the metadata that may become hotspots, and prefetch these hotspot metadata and their associated metadata into the cache. For example, when the metadata in the high priority queue is accessed, the system considers that these metadata and their associated metadata may be hotspot data and need to be prefetched. Additionally, if the cache space is insufficient, the metadata in the low priority queue is evicted.

[0137] In the embodiments of the present disclosure, by combining the access frequency and time, the popularity of the metadata can be more accurately counted. The structure of the multi-level queue makes cache eviction and prefetching more efficient, improving the cache hit rate. By dynamically adjusting the priority of the metadata, it is ensured that the hotspot data is always retained in the high priority queue. By prefetching the hotspot data and its associated data, it is ensured that these data are in the cache, thereby improving the cache hit rate. The prefetch operation can load the data into the cache in advance, reducing the latency when the user accesses the data. The clean metadata after the write operation is maintained by the read cache. It can be understood that after the write operation is completed, the original write cache becomes a read cache for use. Putting the clean metadata into the cache can enable subsequent read operations on these metadata to directly obtain from the cache without having to read from the storage device again, thereby greatly improving the performance of the system.

[0138] In some embodiments, when the leaf node is flushed to disk, the mapping relationships between the intermediate node, the root node, and the leaf node are updated.

[0139] Specifically, after the metadata write is completed, the mapping relationships between the intermediate node and the root node and the flushed leaf node need to be updated. In the tree structure of the metadata, the leaf node stores the actual metadata information, while the intermediate node and the root node are used to organize and index these leaf nodes. When the leaf node is flushed to disk, its physical position on the storage device may change, so the mapping information in the intermediate node and the root node needs to be updated to ensure that the flushed leaf node can be correctly accessed.

[0140] Since the intermediate nodes and the root node are frequently changed due to the disk flushing of the leaf nodes, they belong to hot data. Hot data refers to data that is frequently accessed or modified. For such hot data, the system sets their disk flushing priority to the lowest. This is because frequent flushing of these nodes will generate a large number of I / O operations, affecting system performance. If the cache space permits, these intermediate nodes and the root node will reside in memory permanently to quickly respond to frequent access requests, reduce the number of reads from the storage device, and improve the overall performance of the system.

[0141] The metadata management method provided by the embodiments of the present disclosure determines the flushing priority of metadata nodes based on the cache space utilization rate and the node hit rate. For metadata nodes that occupy a large cache space and have a low utilization rate, they are flushed, writing data that is not frequently accessed to the disk in a timely manner, and retaining data that is more frequently accessed in the cache. This reasonable flushing strategy avoids excessive occupation of the cache space. After aggregating the metadata nodes to be flushed, they are written to the disk instead of writing individual nodes separately. In this way, multiple small I / O operations can be combined into fewer large I / O operations. Reducing the number of I / O operations can effectively reduce the burden on the disk, improve the disk writing efficiency, and reduce disk write amplification.

[0142] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0143] An embodiment of the present application further provides a metadata management device 200. Figure 2 FIG. is a schematic structural diagram of a metadata management device 200 provided by the present disclosure, including:

[0144] An acquisition module 210, configured to obtain the flushing priority of each metadata node according to the cache space utilization rate and the node hit rate of each metadata node when the metadata stored in the cache meets the condition of being written to the disk;

[0145] A determination module 220, configured to determine the metadata nodes to be flushed according to the flushing priority of each metadata node;

[0146] An aggregation module 230, configured to perform aggregation processing on the metadata nodes to be flushed to obtain a metadata block;

[0147] A writing module 240, configured to write the metadata block to the disk.

[0148] As an optional implementation manner of the embodiments of the present disclosure, the device further includes a judgment module, and the judgment module is configured to:

[0149] Determine whether the metadata stored in the cache is consistent with the original metadata read from the disk;

[0150] If the metadata stored in the cache is inconsistent with the original metadata read from the disk, mark the metadata stored in the cache as dirty metadata.

[0151] As an optional implementation manner of an embodiment of the present disclosure, the disk writing condition includes: the metadata stored in the cache meets the condition for writing to the disk, including: the proportion of the dirty metadata occupied in the cache is greater than a preset proportion.

[0152] As an optional implementation manner of an embodiment of the present disclosure, the device further includes a prefetching module, and the prefetching module includes:

[0153] An obtaining unit, configured to obtain the access popularity of the metadata written to the disk according to the access times and access time of the metadata;

[0154] A statistics unit, configured to dynamically adjust the positions of each metadata in the multi-level queue according to the access popularity of the metadata written to the disk;

[0155] An identification unit, configured to periodically evaluate the access popularity of the metadata stored in the multi-level queue and identify hot metadata;

[0156] A prefetching unit, configured to prefetch the hot data in the multi-level queue and the associated data of the hot data into the cache; the associated data of the hot data includes: the metadata adjacent to the hot data and the metadata logically related to the hot data.

[0157] As an optional implementation manner of an embodiment of the present disclosure, the multi-level queue includes a first multi-level queue and a second multi-level queue; the first multi-level queue is used to count the metadata popularity of intermediate nodes, and the second multi-level queue is used to count the metadata popularity of leaf nodes; the access priority of the first multi-level queue is higher than that of the second multi-level queue; the first multi-level queue and the second multi-level queue each include at least two priority queues.

[0158] As an optional implementation manner of an embodiment of the present disclosure, the device further includes: a write response module, and the write response module is specifically configured to:

[0159] In response to a data writing instruction, aggregate the data to be written into target data;

[0160] Allocate continuous space for the target data, and sequentially write the target data into the storage unit of the data space by means of append writing;

[0161] When generating target metadata by writing the target data, query whether there is a metadata node corresponding to the target metadata in the preset data structure;

[0162] If there is a metadata node corresponding to the target metadata in the preset data structure, update the metadata node;

[0163] If there is no metadata node corresponding to the target metadata in the preset data structure, apply for a metadata node corresponding to the target metadata.

[0164] As an optional implementation manner of an embodiment of the present disclosure, the obtaining module includes:

[0165] A determination unit, configured to perform level division on each metadata node based on a preset data structure to determine an initial write priority of each metadata node; the initial write priorities of the metadata nodes from high to low are: leaf nodes, intermediate nodes, and root nodes;

[0166] A calculation unit, configured to calculate a write priority weight of each metadata node according to the cache space utilization rate and node hit rate of each metadata node;

[0167] An adjustment unit, configured to adjust the initial write priority of each metadata node based on the change in the write priority weight of each metadata node to determine the write priority of each metadata node.

[0168] As an optional implementation manner of an embodiment of the present disclosure, the calculation unit is specifically configured to:

[0169] Obtain the cache space utilization rate and node hit rate of each metadata node;

[0170] Determine the cache space utilization rate weight and node hit rate weight of each metadata node;

[0171] Calculate the write priority weight of each metadata node according to the cache space utilization rate, the cache space utilization rate weight, the node hit rate, and the node hit rate weight of each metadata node.

[0172] As an optional implementation manner of an embodiment of the present disclosure, the apparatus further includes an update module, configured to:

[0173] When the leaf node is flushed to the disk, update the mapping relationship between the intermediate node, the root node, and the leaf node.

[0174] As an optional implementation manner of an embodiment of the present disclosure, the apparatus further includes a priority adjustment module, and the priority adjustment module is configured to:

[0175] Within a first preset time period, determine whether the write count of the first metadata node within the preset address space of the cache is greater than or equal to a preset count;

[0176] If the write count of the first metadata node is greater than or equal to the preset count, then reduce the write priority of the first metadata node;

[0177] If the write count of the first metadata node is less than the preset count, then increase the write priority of the first metadata node.

[0178] As an optional implementation manner of the embodiments of the present disclosure, the apparatus further includes a space recycling module, and the space recycling module is configured to:

[0179] Periodically obtain the space utilization rate of the metadata block;

[0180] When the space utilization rate of the metadata block is lower than a preset space utilization rate, perform metadata block space recycling.

[0181] For the descriptions of the features in the corresponding embodiments of the metadata management apparatus 200, reference may be made to the relevant descriptions of the corresponding embodiments of the metadata management method, which will not be elaborated herein one by one.

[0182] The metadata management apparatus provided by the embodiments of the present disclosure determines the write priority of metadata nodes based on the cache space utilization rate and the node hit rate, flushes metadata nodes that occupy a large amount of cache space and have a low utilization rate, writes data that is not frequently accessed to the disk in a timely manner, and retains data that is more frequently accessed in the cache. This reasonable flushing strategy avoids excessive occupation of the cache space. After aggregating the metadata nodes to be flushed, they are written to the disk instead of writing each node separately, so that multiple small I / O operations can be combined into fewer large I / O operations. Reducing the number of I / O operations can effectively reduce the burden on the disk, improve the disk write efficiency, and reduce the disk write amplification.

[0183] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above embodiments of the metadata management method.

[0184] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the steps in any one of the above embodiments of the metadata management method when running.

[0185] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), mobile hard disks, magnetic disks, or optical discs.

[0186] The embodiments of the present application also provide a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the metadata management method.

[0187] The embodiments of the present application also provide another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the metadata management method.

[0188] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0189] The above provides a detailed introduction to a metadata management method provided by the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A metadata management method, characterized in that, The method includes: When the metadata stored in the cache meets the condition for writing to the disk, obtain the write priority of each metadata node according to the cache space utilization rate and node hit rate of each metadata node; the cache space utilization rate of the node is used to represent the ratio of the cache space occupied by the target node to the total cache space allocated to the target node; the node hit rate is used to represent the ratio of the number of accesses to the target node to the total number of accesses to all metadata nodes within a certain period of time; Determine the metadata node to be flushed according to the write priority of each metadata node; Perform an aggregation process on the metadata node to be flushed to obtain a metadata block; Write the metadata block to the disk; The obtaining of the write priority of each metadata node according to the cache space utilization rate and node hit rate of each metadata node includes: Based on a preset data structure, perform a level division on each metadata node to determine the initial write priority of each metadata node; the initial write priorities of each metadata node from high to low are: leaf node, intermediate node, root node; Calculate the write priority weight of each metadata node according to the cache space utilization rate and node hit rate of each metadata node; Based on the change of the write priority weight of each metadata node, adjust the initial write priority of each metadata node to determine the write priority of each metadata node.

2. The metadata management method according to claim 1, wherein The method further includes: Judge whether the metadata stored in the cache is consistent with the metadata read from the disk; If the metadata stored in the cache is inconsistent with the metadata read from the disk, mark the metadata stored in the cache as dirty metadata.

3. The metadata management method according to claim 2, wherein The condition that the metadata stored in the cache meets the condition for writing to the disk includes: the proportion of the dirty metadata occupied in the cache is greater than a preset proportion.

4. The metadata management method according to claim 1, wherein After writing the metadata block to the disk, the method further includes: Obtain the access popularity of the metadata written to the disk according to the access times and access time of the metadata; Dynamically adjust the positions of each metadata in the multi-level queue according to the access popularity of the metadata written to the disk; Periodically evaluate the access popularity of the metadata stored in the multi-level queue to identify hot metadata; Prefetch the hot data in the multi-level queue and the associated data of the hot data into the cache; the associated data of the hot data includes: the metadata adjacent to the hot data and the metadata logically related to the hot data.

5. The metadata management method according to claim 4, characterized in that, The multi-level queue includes a first multi-level queue and a second multi-level queue; the first multi-level queue is used to count the metadata popularity of the intermediate nodes, and the second multi-level queue is used to count the metadata popularity of the leaf nodes; the access priority of the first multi-level queue is higher than that of the second multi-level queue; the first multi-level queue and the second multi-level queue each include at least two priority queues.

6. The metadata management method according to claim 1, wherein The method further includes: In response to a data write instruction, aggregate the data to be written into target data; Allocate continuous space for the target data, and sequentially write the target data into the storage unit of the data space by means of append write; When generating target metadata by writing the target data, query whether there is a metadata node corresponding to the target metadata in a preset data structure; If there is a metadata node corresponding to the target metadata in the preset data structure, update the metadata node; If there is no metadata node corresponding to the target metadata in the preset data structure, apply for a metadata node corresponding to the target metadata.

7. The metadata management method according to claim 1, wherein The calculating the write priority weights of each metadata node according to the cache space utilization rate and node hit rate of each metadata node includes: Obtain the cache space utilization rate and node hit rate of each metadata node; Determine the cache space utilization rate weights and node hit rate weights of each metadata node; Calculate the write priority weights of each metadata node according to the cache space utilization rate, the cache space utilization rate weight, the node hit rate, and the node hit rate weight of each metadata node.

8. The metadata management method according to claim 1, wherein After writing the metadata block to the disk, the method further includes: When the leaf node is flushed to the disk, update the mapping relationship between the intermediate node, the root node, and the leaf node.

9. The metadata management method according to claim 1, wherein After determining the metadata node to be flushed according to the write priorities of the metadata nodes, the method further includes: Within a first preset time period, determine whether the write count of the first metadata node in a preset address space cached is greater than or equal to a preset count; If the write count of the first metadata node is greater than or equal to the preset count, reduce the write priority of the first metadata node; If the write count of the first metadata node is less than the preset count, increase the write priority of the first metadata node.

10. The metadata management method according to claim 1, wherein After aggregating the metadata nodes to be flushed to obtain a metadata block, the method further includes: Periodically obtain the space utilization rate of the metadata block; When the space utilization rate of the metadata block is lower than a preset space utilization rate, perform metadata block space recycling.

11. A metadata management device, characterized in that, The device includes: An obtaining module, configured to, when the metadata stored in the cache meets the condition of being written to the disk, obtain the write priorities of each metadata node according to the cache space utilization rate and node hit rate of each metadata node; the cache space utilization rate of the node is used to represent the ratio of the cache space occupied by the target node to the total cache space allocated to the target node; the node hit rate is used to represent the ratio of the number of accesses to the target node to the total number of accesses to each metadata node within a certain period of time; A determining module, configured to determine the metadata node to be flushed according to the write priorities of the metadata nodes; An aggregating module, configured to aggregate the metadata nodes to be flushed to obtain a metadata block; A writing module, configured to write the metadata block to the disk; The obtaining module is specifically configured to: the obtaining the write priorities of each metadata node according to the cache space utilization rate and node hit rate of each metadata node includes: Based on a preset data structure, classify each metadata node to determine the initial flashing priority of each metadata node; the initial flashing priorities of the metadata nodes from high to low are: leaf nodes, intermediate nodes, and root nodes; Calculate the flashing priority weights of each metadata node according to the cache space utilization rate and node hit rate of each metadata node; Based on the change of the flashing priority weights of the metadata nodes, adjust the initial flashing priorities of the metadata nodes to determine the flashing priorities of the metadata nodes.

12. An electronic device, characterized in that, Include: A memory for storing a computer program; A processor for implementing the steps of the metadata management method according to any one of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the metadata management method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product comprising a computer program, characterized in that, The computer program implements the steps of the metadata management method according to any one of claims 1 to 10 when executed by a processor.

Images