Metadata access method and apparatus, electronic device, and storage medium

Abstract

The application relates to the technical field of data storage, and discloses a metadata access method and device, electronic equipment and a storage medium. The metadata access method comprises the following steps: receiving a data access request for accessing target metadata, determining a target storage object for storing the target metadata; acquiring a target b+ tree of the target storage object, the target b+ tree being a data structure of the target storage object; determining a target node for storing the target metadata in the target b+ tree according to a keyword of the target metadata; determining a target cache area corresponding to a target layer of the target node in the target b+ tree; and acquiring the target metadata from the target cache area to respond to the data access request. Through the metadata access method, the metadata access efficiency can be effectively improved, and the performance of a storage system is improved.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and more specifically to metadata access methods, devices, electronic devices, and storage media. Background Technology

[0002] Storage systems can pool and manage a large number of various storage media in a unified manner, and provide different service methods such as blocks, files, and objects according to front-end needs. After pooling the media by creating Redundant Arrays of Independent Disks (RAID), the storage system needs to rely on metadata to establish a mapping relationship between the pooled space and the externally provided services. Thus, when performing read and write access on the host side, the appropriate storage media can be read or written through metadata.

[0003] Since spatial mapping relationships are organized and managed using metadata, the performance of a storage system largely depends on the performance of metadata access. However, due to the enormous volume of metadata in a storage system, it is impossible to store all of the metadata in memory, which means that the vast majority of the metadata ultimately needs to be saved to the backend hard disk.

[0004] In related technologies, storage systems utilize caching to improve metadata access performance, thereby shortening the metadata access path and enhancing access efficiency. However, storage systems have limited caching resources; therefore, how to allocate caching resources more rationally remains a problem to be solved. Summary of the Invention

[0005] In view of this, the present invention provides a metadata access method, apparatus, electronic device and storage medium to solve the problem of limited cache resources affecting metadata access performance.

[0006] In a first aspect, the present invention provides a metadata access method, the method comprising:

[0007] Receive data access requests for accessing target metadata and determine the target storage object for storing the target metadata;

[0008] Obtain the target B+ tree of the target storage object, where the target B+ tree is the data structure of the target storage object;

[0009] Based on the keywords of the target metadata, determine the target node in the target B+ tree that stores the target metadata;

[0010] Based on the target node's target layer in the target B+ tree, determine the target cache region corresponding to the target layer;

[0011] Retrieve target metadata from the target cache region in response to a data access request.

[0012] In this approach, the storage resources used by the target storage object can be pre-divided according to the hierarchical structure of the target B+ tree of the target storage object. Then, when it is determined that the target node storing the target metadata is in the target layer of the target B+ tree, the target metadata can be obtained from the corresponding target cache area, thereby effectively improving the metadata access efficiency and achieving the goal of improving the performance of the storage system.

[0013] In one optional implementation, based on the target layer of the target node in the target b+ tree, the target cache region corresponding to the target layer is determined, including:

[0014] Based on the layer structure of the target B+ tree, determine the cache area corresponding to each layer in the layer structure;

[0015] Based on the cache regions corresponding to each layer, determine the target cache region corresponding to the target layer.

[0016] In this approach, different cache regions can be used to cache the corresponding metadata based on the access frequency of each node, so as to make reasonable use of the cache resources of the target storage object. In this way, when accessing the metadata of the target storage object, the utilization rate of the cache resources of the target storage object can be improved, thereby improving the performance of the storage system.

[0017] In one optional implementation, the layer structure of the target b+ tree includes a fixed write cache layer, a variable write cache layer, and a read cache layer. The cache area corresponding to the fixed write cache layer is a fixed write cache area, the cache area corresponding to the variable write cache layer is a variable write cache area, and the cache area corresponding to the read cache layer is a read cache area. The fixed write cache layer is higher than the variable write cache layer, and the variable write cache layer is higher than the read cache layer.

[0018] Based on the cache regions corresponding to each layer, determine the target cache region corresponding to the target layer, including:

[0019] If the target layer is within the fixed write cache layer, then the target cache region is determined to be the fixed write cache region;

[0020] If the target layer is within the variable-write cache layer, then the target cache region is determined to be a variable-write cache region;

[0021] If the target layer is within the read cache layer, then the target cache area is determined to be the read cache area.

[0022] In one alternative implementation, the method further includes:

[0023] Monitor the access parameters of the target B+ tree;

[0024] Based on the changes in access parameters within a specified usage period, adjust the cache resources configured for the target B+ tree. The cache resources include write cache resources and / or read cache resources.

[0025] In this approach, the cache resources configured for the target B+ tree can be specifically adjusted based on the access parameters of the target B+ tree within a specified time period, thereby enabling the cache resources of the target B+ tree to meet the operational requirements of the target B+ tree, thus improving the access performance of metadata and enhancing the performance of the storage system.

[0026] In one optional implementation, the access parameters include the write access frequency of metadata; based on changes in the access parameters over a specified usage period, the cache resources configured for the target B+ tree are adjusted, including:

[0027] Obtain the historical data hit rate of multiple metadata stored in the target B+ tree;

[0028] If the historical data hit rate is greater than or equal to the specified hit rate threshold, and the write access heat of the target B+ tree within the target usage time is greater than or equal to the specified write access heat, then expand the write cache resources;

[0029] If the historical data hit rate is less than the specified hit rate threshold, and the write access heat of the target B+ tree within the target usage time is greater than or equal to the specified write access heat, then reduce the write cache resources.

[0030] In one alternative implementation, the method further includes:

[0031] Obtain the read access frequency of the target B+ tree per unit time.

[0032] Get the write access frequency of the target B+ tree per unit time;

[0033] Configure the cache resources for the target B+ tree based on the ratio between read access frequency and write access frequency.

[0034] In one alternative implementation, the method further includes:

[0035] If the data hit rate of the target B+ tree is higher than that of other B+ trees within a unit of time, then cache resources are expanded, and other B+ trees are the data structures of other storage objects. The target storage object and other storage objects are stored in the same storage system.

[0036] If the data hit rate of the target B+ tree is lower than that of other B+ trees within a unit of time, then reduce the cache resources.

[0037] In a second aspect, the present invention provides a metadata access device, the device comprising:

[0038] The first determining module is used to receive data access requests for accessing target metadata and determine the target storage object for storing target metadata.

[0039] The first acquisition module is used to acquire the target B+ tree of the target storage object, where the target B+ tree is the data structure of the target storage object.

[0040] The second determination module is used to determine the target node storing the target metadata in the target b+ tree based on the keywords of the target metadata.

[0041] The third determining module is used to determine the target cache area corresponding to the target layer based on the target layer of the target node in the target b+ tree;

[0042] The second acquisition module is used to retrieve target metadata from the target cache area in response to data access requests.

[0043] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the metadata access method of the first aspect or any corresponding embodiment described above.

[0044] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the metadata access method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the structure of a metadata access system based on relevant technologies;

[0047] Figure 2 This is a flowchart illustrating a metadata access method according to an embodiment of the present invention;

[0048] Figure 3 This is a flowchart illustrating another metadata access method according to an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of b+ tree node partitioning according to an embodiment of the present invention;

[0050] Figure 5 This is a flowchart illustrating another metadata access method according to an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the cache resource adjustment process according to an embodiment of the present invention;

[0052] Figure 7 This is a structural block diagram of a metadata access device according to an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] In related technologies, storage systems pool storage media to form data storage pools. This allows for the creation of numerous storage objects (e.g., logical volumes (LUNs), files, or objects) based on user needs. These cached objects are then mapped to the host via fiber optic or Internet Protocol (IP) networks using remote protocols (e.g., Fabric, an open-source distributed ledger technology protocol) for users to access according to their requirements.

[0056] like Figure 1 As shown, taking a LUN as an example, the storage system includes n LUNs, where n > 0 and n is a positive integer. When the host writes data to a LUN in the storage system, the storage system writes the data received by that LUN into the data pool and allocates an appropriate write location within the data pool. Finally, metadata is responsible for recording the mapping relationship between these data from the LUN to the data storage pool. The storage system uses caching to store metadata, thereby shortening the metadata access path and improving metadata access performance. However, since the cache resources in the storage system are limited, how to allocate cache resources more rationally becomes a problem to be solved.

[0057] In view of this, embodiments of the present invention provide a metadata access method, comprising: receiving a data access request for accessing target metadata; determining a target storage object storing the target metadata; obtaining a target B+ tree of the target storage object, wherein the target B+ tree is the data structure of the target storage object; determining a target node storing the target metadata in the target B+ tree based on the keyword of the target metadata; determining a target cache area corresponding to the target layer based on the target layer of the target node in the target B+ tree; and retrieving the target metadata from the target cache area to respond to the data access request. Through the metadata access method provided by the present invention, the storage resources used by the target storage object can be pre-divided according to the layer structure of the target B+ tree of the target storage object. Therefore, when it is determined that the target node storing the target metadata is in the target layer of the target B+ tree, the target metadata can be retrieved from the corresponding target cache area, thereby effectively improving metadata access efficiency and achieving the goal of improving storage system performance.

[0058] According to an embodiment of the present invention, a metadata access method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0059] This embodiment provides a metadata access method that can be used in the aforementioned storage system. Figure 2 This is a flowchart of a metadata access method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0060] Step S201: Receive a data access request to access the target metadata and determine the target storage object for storing the target metadata.

[0061] In this embodiment of the invention, the target storage object can be a logical volume, a file, or an object already created in the storage system. Different storage objects correspond to different metadata. To improve the efficiency of metadata access, in response to a received data access request for accessing the target metadata, the target storage object storing the target metadata is determined, so as to determine the storage resources that the target storage object can use.

[0062] Step S202: Obtain the target b+ tree of the target storage object.

[0063] In this embodiment of the invention, the target B+ tree is the data structure of the target storage object. To facilitate the management of the target storage object's metadata, the target B+ tree is used to store and manage the target storage object's metadata, thereby improving the query efficiency of the target metadata.

[0064] Step S203: Determine the target node storing the target metadata in the target b+ tree based on the keywords of the target metadata.

[0065] In this embodiment of the invention, the internal nodes (non-leaf nodes) of the target B+ tree contain multiple key-value pairs (KV, or index entries), each consisting of a key and a pointer. The key is used for lookup and comparison operations between nodes, while the pointer points to the next-level node, a sibling node, or a leaf node. Leaf nodes store the actual metadata content. They contain a key and pointers to the actual metadata. These pointers can point to disk blocks or the target cache area where the data is stored.

[0066] Therefore, in order to facilitate the rapid search and access of target metadata, the target node storing the target metadata in the target B+ tree is determined based on the keyword of the target metadata, so that the target cache area storing the target metadata can be quickly located.

[0067] Step S204: Based on the target layer of the target node in the target b+ tree, determine the target cache area corresponding to the target layer.

[0068] In this embodiment of the invention, based on the layer structure of the target B+ tree, different cache regions are used to store the corresponding metadata for nodes at different layers. Therefore, to improve the access performance of the target metadata, the target cache region corresponding to the target layer is determined based on the target layer of the target node in the target B+ tree.

[0069] Step S205: Obtain target metadata from the target cache area in response to the data access request.

[0070] In this embodiment of the invention, the target cache region can be a portion of the cache region corresponding to the target storage object, or it can be the entire cache region. When it is determined that the target metadata exists in the target cache region, the target metadata is retrieved from the target cache region to respond to the received data access request.

[0071] The metadata access method provided in this embodiment can pre-divide the storage resources used by the target storage object according to the layer structure of the target b+ tree of the target storage object. Then, when it is determined that the target node storing the target metadata is in the target layer of the target b+ tree, the target metadata can be obtained from the corresponding target cache area, thereby effectively improving the metadata access efficiency and achieving the purpose of improving the performance of the storage system.

[0072] This embodiment provides a metadata access method that can be used in the aforementioned storage system. Figure 3 This is a flowchart of a metadata access method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0073] Step S301: Receive a data access request for accessing the target metadata and determine the target storage object storing the target metadata. For details, please refer to [link to relevant documentation]. Figure 1 Step S201 of the illustrated embodiment will not be described again here.

[0074] Step S302: Obtain the target B+ tree of the target storage object. See details below. Figure 1 Step S202 of the illustrated embodiment will not be described again here.

[0075] Step S303: Based on the keywords of the target metadata, determine the target node storing the target metadata in the target B+ tree. For details, please refer to [link to relevant documentation]. Figure 1 Step S203 of the illustrated embodiment will not be described again here.

[0076] Step S304: Based on the target layer of the target node in the target b+ tree, determine the target cache area corresponding to the target layer.

[0077] Specifically, step S304 includes:

[0078] Step S3041: Based on the layer structure of the target b+ tree, determine the cache area corresponding to each layer in the layer structure.

[0079] In this embodiment of the invention, based on the layer structure of the target B+ tree, the number of layers in the target B+ tree can be determined. Therefore, to facilitate the rational use of the available storage resources corresponding to the target storage object, a corresponding cache region is pre-allocated to each node of each layer of the target B+ tree. Adjacent nodes can correspond to the same cache region or different cache regions, and this can be configured according to actual needs.

[0080] Once it is determined that the target metadata is managed by the target B+ tree, the corresponding cache area for each layer in the layer structure is determined so that when the target node is determined, the target cache area for storing the target metadata can be quickly located.

[0081] Step S3042: Determine the target cache area corresponding to the target layer based on the cache areas corresponding to each layer.

[0082] In this embodiment of the invention, after determining the target layer where the target node is located, the target cache area corresponding to the target layer is determined according to the correspondence between each layer and the cache area.

[0083] In an optional implementation, to improve the efficiency of determining the target cache region, the target cache region can be directly determined based on the correspondence between each layer and the cache region.

[0084] In some optional implementations, the layer structure of the target b+ tree includes a fixed write cache layer, a variable write cache layer, and a read cache layer. The cache area corresponding to the fixed write cache layer is a fixed write cache area, the cache area corresponding to the variable write cache layer is a variable write cache area, and the cache area corresponding to the read cache layer is a read cache area. The fixed write cache layer is higher than the variable write cache layer, and the variable write cache layer is higher than the read cache layer. The above step S3042 includes:

[0085] Step a1: If the target layer is within the fixed write cache layer, then the target cache area is determined to be the fixed write cache area.

[0086] Step a2: If the target layer is within the writable cache layer, then the target cache region is determined to be a writable cache region.

[0087] Step a3: If the target layer is within the read cache layer, then the target cache area is determined to be the read cache area.

[0088] In this approach, caching resources used to improve metadata access performance can include read cache resources and write cache resources. Read cache areas provide read cache resources, and write cache areas provide write cache resources. Write cache areas include fixed write cache areas and variable write cache areas.

[0089] Since write cache resources for the target storage object are limited, and the higher the level in the target B+ tree structure, the more frequently the corresponding node is accessed, it is necessary to store the metadata of the most frequently accessed node in a fixed write cache area. Because metadata access is initiated by the root node of the target B+ tree, and the root node and its adjacent upper-level intermediate nodes form the final path for key-value (KV) access, the root node and one or two levels of intermediate nodes below it can be divided into a fixed write cache layer. This avoids performance degradation caused by frequent read / write operations on metadata by high-access nodes.

[0090] Because intermediate nodes are accessed relatively frequently, and changes to metadata key-value pairs can be completed in memory, to avoid frequent disk reads and writes, the layer containing the intermediate nodes in the lower layers can be divided into a variable-write cache layer, and the metadata within the variable-write cache layer can be stored in the variable-write cache area. The variable-write cache layer may include some leaf nodes.

[0091] Since leaf nodes are modified relatively infrequently and the amount of metadata is small, storing them in the write cache area for a long time will reduce the utilization of write cache resources. Therefore, the layer containing leaf nodes can be divided into a read cache layer to store the metadata in the write cache area, thereby helping to improve the read efficiency of metadata.

[0092] The fixed write cache layer represents a write cache that is fixedly configured for the target B+ tree and is a dedicated configuration for the target B+ tree, which cannot be reduced. The variable write cache layer represents a write cache that can be shared by all storage objects in the storage system and their corresponding B+ trees. It is not a dedicated cache resource for a specific B+ tree and can be allocated to the target B+ tree as needed, or it can be reclaimed from the target B+ tree. The read cache layer is used to store the nodes of the target B+ tree, which can reduce node access latency and improve access efficiency. The read cache is also not dedicated to the target B+ tree and can also be allocated to or reclaimed from the target B+ tree as needed.

[0093] Furthermore, if the target layer is within a fixed write cache layer, the target cache area is determined to be a fixed write cache area; if the target layer is within a variable write cache layer, the target cache area is determined to be a variable write cache area; and if the target layer is within a read cache layer, the target cache area is determined to be a read cache area.

[0094] Step S305: Retrieve target metadata from the target cache area in response to the data access request. See details below. Figure 1 Step S205 of the illustrated embodiment will not be described again here.

[0095] The metadata access method provided in this embodiment can cache the corresponding metadata in different cache areas according to the access frequency of each node, so as to make reasonable use of the cache resources of the target storage object. In this way, when accessing the metadata of the target storage object, the utilization rate of the cache resources of the target storage object can be improved, thereby improving the performance of the storage system.

[0096] In some optional implementation scenarios, the target B+ tree structure of the target storage object can be as follows: Figure 4 As shown, the root node and the first-level intermediate nodes are divided into a fixed write cache layer, using a fixed write cache area to store the corresponding metadata. The second-level intermediate nodes are divided into a variable write cache layer, using a variable write cache area to store the corresponding metadata. The leaf nodes are divided into a read cache layer, using a read cache area to store the corresponding metadata.

[0097] In another optional implementation scenario, when a key-value (KV) modification occurs, the data in the leaf node can be modified after locating it from the root node, and then the leaf node can be re-written to disk. However, it's important to note that if the leaf node is written to disk using an overwrite method that overwrites the original location, there's a possibility that, due to faults or other reasons, the data on disk might be partially new, leading to data corruption or even loss. Therefore, the leaf node is re-written to disk using a redirected write (ROW) method. That is, when the metadata content changes and is written to the write pool again, it's written to a new location, and the new address of the changed node is updated in the upper-level intermediate node. Since intermediate nodes don't need to be immediately written to disk in the write cache area, updating the intermediate node data in the write cache area is sufficient. However, if the target B+ tree has insufficient write cache, forcing the lower-level intermediate nodes to be written to disk, the intermediate nodes are also written to a new location on disk using the ROW method, and the new address is updated in the upper-level intermediate node. Similarly, the upper-level intermediate node, being in the write cache, only needs to be updated in the write cache.

[0098] In another optional implementation scenario, the storage system will reclaim nodes that have failed on the disk, thus preventing the disk space from being exhausted due to long-term ROW writes.

[0099] This embodiment provides a metadata access method that can be used in the aforementioned storage system. Figure 5 This is a flowchart of a metadata access method according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps:

[0100] Step S501: Receive a data access request to access the target metadata and determine the target storage object for storing the target metadata.

[0101] Step S502: Obtain the target B+ tree of the target storage object.

[0102] Step S503: Determine the target node storing the target metadata in the target b+ tree based on the keywords of the target metadata.

[0103] Step S504: Based on the target layer of the target node in the target b+ tree, determine the target cache area corresponding to the target layer.

[0104] Step S505: Obtain target metadata from the target cache area in response to the data access request.

[0105] Step S506: Monitor the access parameters of the target b+ tree.

[0106] In this embodiment of the invention, to monitor whether the cache resources allocated to the target B+ tree can meet the access requirements of the metadata of each node during operation, the access parameters of the target B+ tree are monitored. These access parameters may include, but are not limited to, the following types of parameters: the read / write access ratio of the target B+ tree, read / write access heat (read / write access frequency), node hit rate of the read / write cache, and business priority (usually specified by the user according to actual needs, or optional).

[0107] Step S507: Adjust the cache resources configured for the target b+ tree based on the changes in access parameters within a specified usage period.

[0108] In this embodiment of the invention, based on the changes in access parameters within a specified usage period, the access status of each metadata element within the target storage object can be understood. This allows for determination of whether the cache resources currently configured for the target B+ tree are reasonable. When the cache resources configured for the target B+ tree do not meet the calling requirements of the target cache object, timely adjustments can be made to ensure the rational allocation of storage resources, thereby improving the performance of the storage system. The cache resources may include write cache resources and / or read cache resources. The duration of a single usage period can be set in hours or days.

[0109] Specifically, if the access parameters include the write access frequency of metadata, then step S507 above includes:

[0110] Step b1: Obtain the historical data hit rate of multiple metadata stored in the target b+ tree.

[0111] Step b2: If the historical data hit rate is greater than or equal to the specified hit rate threshold, and the write access heat of the target b+ tree within the target usage time is greater than or equal to the specified write access heat, then expand the write cache resources.

[0112] Step b3: If the historical data hit rate is less than the specified hit rate threshold, and the write access heat of the target b+ tree within the target usage time is greater than or equal to the specified write access heat, then the write cache resources will not be adjusted.

[0113] Step b4: If the historical data hit rate is less than the specified hit rate threshold, and the write access heat of the target b+ tree within the target usage time is less than the specified write access heat, then reduce the write cache resources.

[0114] In this approach, the historical data hit rate of multiple metadata items stored in the target B+ tree is obtained to clarify the historical access patterns of these metadata items. If the historical data hit rate is greater than or equal to a specified hit rate threshold, it indicates that the access frequency of the multiple metadata items stored in the target B+ tree is consistently increasing. Furthermore, when the write access frequency of the target B+ tree within the target usage duration is greater than or equal to a specified write access frequency, it indicates that a write cache needs to be added to the target B+ tree in advance. Therefore, the write cache resources configured for the target B+ tree are expanded to meet its write cache requirements.

[0115] If the historical data hit rate is less than the specified hit rate threshold, it indicates that the access frequency of multiple metadata items stored in the target B+ tree is low, and the current cache resources configured for the target B+ tree can meet the demand. However, when the write access frequency of the target B+ tree within the target usage duration is greater than or equal to the specified write access frequency, it indicates that the access frequency of the target B+ tree has increased. However, due to the relatively scattered keywords of the metadata, the actual hit effect may be poor, and even increasing the write cache resources may not achieve the expected effect. Therefore, to avoid resource waste, the write cache resources configured for the target B+ tree can be reduced.

[0116] The metadata access method provided in this embodiment can adjust the cache resources configured for the target B+ tree based on the access parameters of the target B+ tree within a specified time period, thereby enabling the cache resources of the target B+ tree to meet the operational requirements of the target B+ tree, thus improving the metadata access performance and enhancing the performance of the storage system.

[0117] In some optional implementations, the above metadata access method further includes:

[0118] Step S508: Obtain the read access frequency of the target b+ tree within a unit of time.

[0119] In this embodiment of the invention, to facilitate targeted allocation of cache resources called by the target B+ tree and meet the needs of reading and writing, the read access frequency of the B+ tree within a unit of time is first obtained to determine the access status of read metadata. The unit of time can be an hour or a day.

[0120] Step S509: Obtain the write access heat of the target b+ tree within a unit of time.

[0121] In this embodiment of the invention, the write access heat of the target b+ tree within a unit of time is obtained to determine the access status of write metadata.

[0122] Step S510: Configure the cache resources of the target b+ tree based on the ratio between read access frequency and write access frequency.

[0123] In this embodiment of the invention, based on the ratio between read access frequency and write access frequency, it can be determined whether the target B+ tree prefers to use read cache resources or write cache resources during operation, and thus can be configured accordingly. If the target B+ tree prefers to use read cache resources during operation, the proportion of read cache resources is increased when configuring the cache resources of the target B+ tree. If the target B+ tree prefers to use write cache resources during operation, the proportion of write cache resources is increased when configuring the cache resources of the target B+ tree. In extreme cases, only read cache resources or write cache resources can be configured.

[0124] The metadata access method provided in the above embodiments makes the resource configuration of the target b+ tree more targeted.

[0125] In some alternative implementations, the above metadata access method further includes:

[0126] Step S511: If the data hit rate of the target b+ tree is higher than that of other b+ trees within a unit of time, then increase the cache resources.

[0127] In this embodiment of the invention, other B+ trees are data structures for other storage objects, and the target storage object and other storage objects are stored in the same storage system. If the data hit rate of the target B+ tree is higher than that of other B+ trees within a unit of time, it indicates that the access popularity of the target B+ tree is relatively high. Therefore, cache resources are expanded to effectively improve the read and write performance of the target storage object.

[0128] Step S512: If the data hit rate of the target B+ tree is lower than that of other B+ trees within a unit of time, then reduce the cache resources.

[0129] In this embodiment of the invention, if the data hit rate of the target B+ tree is lower than that of other B+ trees within a unit of time, it indicates that the access popularity of the target B+ tree is relatively low and the keywords of the metadata are relatively scattered. At this time, a large number of accesses (especially read operations) interact directly with the disk through the cache, and the cache has little effect on performance improvement. Therefore, the cache resources are reduced to lower the cache ratio and reserve more cache resources for other B+ trees, thereby improving the overall performance of the storage system.

[0130] In the metadata access method provided in the above embodiments, the cache resources in the storage system can be reasonably allocated based on the data hit status of each b+ tree, thereby effectively improving the overall performance of the storage system.

[0131] In some optional implementation scenarios, such as Figure 6As shown, the storage system is equipped with a cache monitoring unit, which can detect the access status of each B+ tree during operation, and then make targeted adjustments to the writable cache resources of each B+ tree, thereby improving the overall performance of the storage system.

[0132] In other alternative implementation scenarios, the B+ trees of each storage object can be managed by different threads, thereby enabling independent maintenance and management of each B+ tree. This effectively avoids thread contention and lock conflicts among the B+ trees during operation, thereby improving the concurrent processing capability of metadata and maintaining multi-threaded load balancing.

[0133] Furthermore, the size and number of B+ trees are not fixed and can be set according to business needs. For example, they can be divided proportionally based on the range of key values, such as forming one B+ tree for key values ​​in the range of 1 to 10000, another for key values ​​in the range of 100001 to 20000, and so on, thus obtaining multiple B+ trees. Alternatively, non-proportional division can be used based on a clear hot / cold remainder relationship in business operations, where each subtree in a hot area covers a smaller key range, and each subtree in a cold area covers a larger key range. For example, in a logical volume, the area with addresses 100 to 500 is considered a hot area, meaning it is frequently accessed. Therefore, when dividing the B+ trees in this area, it can be divided with a granularity of 50, meaning a subtree for addresses 100 to 150, and so on; while in non-hot areas, it can be divided with a granularity of 500, meaning a subtree for addresses 500 to 1000. It's important to note that the address here is equivalent to the key value. Therefore, addresses 100-150 represent metadata for keys 100-150 that are partitioned into a B+ tree. For example, the partitioning can be skipped based on specific patterns in business access, such as grouping non-adjacent key regions into a B+ tree. The key to this partitioning is to base it on the actual business situation and maintain a relatively balanced access distribution across each subtree.

[0134] For the detection and control of cache usage, a relatively simple threshold setting method can be adopted, that is, defining a threshold for each factor and calculating it by weighting multiple factors. This is also a relatively common method. Alternatively, some new AI algorithms can be used, which may make the cache configuration more accurate, but the complexity will also be higher. The choice can be made according to business needs.

[0135] This embodiment also provides a metadata access device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0136] This embodiment provides a metadata access device, such as... Figure 7 As shown, it includes:

[0137] The first determining module 701 is used to receive a data access request for accessing target metadata and determine the target storage object for storing target metadata.

[0138] The first acquisition module 702 is used to acquire the target b+ tree of the target storage object, where the target b+ tree is the data structure of the target storage object.

[0139] The second determining module 703 is used to determine the target node storing the target metadata in the target b+ tree based on the keywords of the target metadata.

[0140] The third determining module 704 is used to determine the target cache area corresponding to the target layer based on the target layer of the target node in the target b+ tree;

[0141] The second acquisition module 705 is used to acquire target metadata from the target cache area in response to a data access request.

[0142] In some optional implementations, the third determining module 704 includes: a first determining unit, configured to determine the cache region corresponding to each layer in the layer structure according to the layer structure of the target b+ tree; and a second determining unit, configured to determine the target cache region corresponding to the target layer according to the cache region corresponding to each layer.

[0143] In some optional implementations, the layer structure of the target B+ tree includes a fixed write cache layer, a variable write cache layer, and a read cache layer. The cache area corresponding to the fixed write cache layer is a fixed write cache area, the cache area corresponding to the variable write cache layer is a variable write cache area, and the cache area corresponding to the read cache layer is a read cache area. The fixed write cache layer is higher than the variable write cache layer, and the variable write cache layer is higher than the read cache layer. The second determining unit includes: a first region determining unit, used to determine the target cache area as a fixed write cache area if the target layer is within the fixed write cache layer; a second region determining unit, used to determine the target cache area as a variable write cache area if the target layer is within the variable write cache layer; and a third region determining unit, used to determine the target cache area as a read cache area if the target layer is within the read cache layer.

[0144] In some optional implementations, the apparatus further includes: a monitoring module for monitoring access parameters of the target b+ tree; and an adjustment module for adjusting the cache resources configured corresponding to the target b+ tree based on changes in the access parameters within a specified usage period, wherein the cache resources include: write cache resources and / or read cache resources.

[0145] In some optional implementations, the access parameters include the write access frequency of metadata; the adjustment module includes: a parameter acquisition unit, used to acquire the historical data hit rate of multiple metadata stored in the target b+ tree; a first adjustment unit, used to increase write cache resources if the historical data hit rate is greater than or equal to a specified hit rate threshold, and the write access frequency of the target b+ tree within the target usage duration is greater than or equal to a specified write access frequency; and a second adjustment unit, used to reduce write cache resources if the historical data hit rate is less than a specified hit rate threshold, and the write access frequency of the target b+ tree within the target usage duration is greater than or equal to a specified write access frequency.

[0146] In some optional implementations, the apparatus further includes: a first heat acquisition module for acquiring the read access heat of the target b+ tree within a unit time; a second heat acquisition module for acquiring the write access heat of the target b+ tree within a unit time; and a first configuration module for configuring the cache resources of the target b+ tree based on the ratio between the read access heat and the write access heat.

[0147] In some optional implementations, the apparatus further includes: a second configuration module, configured to expand cache resources if the data hit rate of the target b+ tree is higher than the data hit rate of other b+ trees within a unit time, wherein the other b+ trees are data structures of other storage objects, and the target storage object and other storage objects are stored in the same storage system; and a third configuration module, configured to reduce cache resources if the data hit rate of the target b+ tree is lower than the data hit rate of other b+ trees within a unit time.

[0148] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0149] In this embodiment, the metadata access device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0150] This invention also provides an electronic device having the above-described features. Figure 7 The metadata access device shown.

[0151] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of the present invention, such as... Figure 8As shown, the electronic device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0152] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0153] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0154] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0155] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0156] The electronic device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0157] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touch screen.

[0158] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0159] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for accessing metadata, characterized in that, The method includes: Receive a data access request for accessing target metadata, and determine the target storage object storing the target metadata; Obtain the target b+ tree of the target storage object, where the target b+ tree is the data structure of the target storage object; Based on the keywords of the target metadata, determine the target node in the target b+ tree that stores the target metadata; Based on the target node's target layer in the target b+ tree, determine the target cache region corresponding to the target layer; Retrieve the target metadata from the target cache region in response to the data access request; The step of determining the target cache region corresponding to the target layer based on the target node in the target b+ tree includes: Based on the layer structure of the target b+ tree, determine the cache region corresponding to each layer in the layer structure; Based on the cache regions corresponding to each layer, determine the target cache region corresponding to the target layer; The layer structure of the target b+ tree includes a fixed write cache layer, a variable write cache layer, and a read cache layer. The cache area corresponding to the fixed write cache layer is a fixed write cache area, the cache area corresponding to the variable write cache layer is a variable write cache area, and the cache area corresponding to the read cache layer is a read cache area. The fixed write cache layer is higher than the variable write cache layer, and the variable write cache layer is higher than the read cache layer. The step of determining the target cache region corresponding to the target layer based on the cache regions corresponding to each layer includes: If the target layer is located within the fixed write cache layer, then the target cache region is determined to be the fixed write cache region; If the target layer is located within the variable write cache layer, then the target cache region is determined to be the variable write cache region; If the target layer is located within the read cache layer, then the target cache region is determined to be the read cache region.

2. The method according to claim 1, characterized in that, The method further includes: Monitor the access parameters of the target b+ tree; Based on the changes in the access parameters within a specified usage period, the cache resources configured for the target B+ tree are adjusted, including write cache resources and / or read cache resources.

3. The method according to claim 2, characterized in that, The access parameters include the write access frequency of metadata; adjusting the cache resources configured for the target B+ tree based on changes in the access parameters within a specified usage period includes: Obtain the historical data hit rate of multiple metadata stored in the target b+ tree; If the historical data hit rate is greater than or equal to the specified hit rate threshold, and the write access heat of the target B+ tree within the target usage time is greater than or equal to the specified write access heat, then the write cache resources are expanded. If the historical data hit rate is less than the specified hit rate threshold, and the write access heat of the target B+ tree within the target usage time is greater than or equal to the specified write access heat, then the write cache resources are reduced.

4. The method according to claim 2 or 3, characterized in that, The method further includes: Obtain the read access frequency of the target b+ tree per unit time. Obtain the write access heat of the target b+ tree per unit time; The cache resources of the target B+ tree are configured based on the ratio between the read access frequency and the write access frequency.

5. The method according to claim 4, characterized in that, The method further includes: If the data hit rate of the target B+ tree is higher than that of other B+ trees within a unit of time, then the cache resources are expanded. The other B+ trees are the data structures of other storage objects. The target storage object and the other storage objects are stored in the same storage system. If the data hit rate of the target B+ tree is lower than that of other B+ trees within a unit of time, then the cache resources are reduced.

6. A metadata access device, characterized in that, The device includes: The first determining module is used to receive a data access request for accessing target metadata and determine the target storage object storing the target metadata. The first acquisition module is used to acquire the target b+ tree of the target storage object, wherein the target b+ tree is the data structure of the target storage object; The second determining module is used to determine the target node storing the target metadata in the target b+ tree based on the keywords of the target metadata; The third determining module is used to determine the target cache region corresponding to the target layer based on the target layer of the target node in the target b+ tree; The second acquisition module is used to acquire the target metadata from the target cache area in response to the data access request; The third determining module includes: a first determining unit, used to determine the cache region corresponding to each layer in the target b+ tree layer structure according to the layer structure of the target b+ tree; a second determining unit, used to determine the target cache region corresponding to the target layer according to the cache regions corresponding to each layer; wherein, the layer structure of the target b+ tree includes a fixed write cache layer, a variable write cache layer, and a read cache layer, the cache region corresponding to the fixed write cache layer is the fixed write cache region, the cache region corresponding to the variable write cache layer is the variable write cache region, the cache region corresponding to the read cache layer is the read cache region, the fixed write cache layer is higher than the variable write cache layer, and the variable write cache layer is higher than the read cache layer; the second determining unit includes: a first region determining unit, used to determine the target cache region as the fixed write cache region if the target layer is within the fixed write cache layer; a second region determining unit, used to determine the target cache region as the variable write cache region if the target layer is within the variable write cache layer; a third region determining unit, used to determine the target cache region as the read cache region if the target layer is within the read cache layer.

7. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the metadata access method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the metadata access method according to any one of claims 1 to 5.

Images