Storage resource quota management method and distributed storage system

Abstract

The embodiment of the invention provides a storage resource quota management method and a distributed storage system. In the embodiment, a token ring structure is introduced for quota management, tokens carrying quota use information are transmitted on virtual nodes in the token ring, and when any virtual node on the token ring receives the token, quota management is carried out on service operation needing to execute quota management, and the token is updated; and the updated token is transmitted to the next virtual node, so that only one virtual node can be ensured to update the same quota use information at the same time, and the problems of updating delay and inconsistency of the existing quota use information are avoided; and the problem that the quota distribution exceeds the quota limit due to the fact that multiple nodes apply for use of the quota at the same time is also avoided, so that the problems that the quota use exceeds the limit and the quota use is insufficient are fundamentally avoided, and the accuracy and reliability of storage resource quota management are improved.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a storage resource quota management method and a distributed storage system. Background Art

[0002] With the continuous development of distributed storage technology, the requirements for storage resource quota management in distributed storage systems are also increasing. Here, storage resource quota can refer to the amount of allocated storage resources used to limit the use of storage resources, such as the allocated disk space size and / or the number of files allowed to be created.

[0003] In current practical applications, distributed storage systems typically utilize an upper-layer quota management service to manage global quotas, such as allocation and monitoring. They utilize lower-layer metadata and data services to manage local quota usage information. These services periodically synchronize quota usage information with the aforementioned quota management service, allowing it to update global quota usage information. However, since quota usage information from each metadata and data service is regularly synchronized with the quota management service during quota management, this can lead to delayed and inconsistent updates of global quota usage information within the quota management service. This can potentially cause problems such as quota overages, impacting the accuracy and reliability of storage resource quota management. Summary of the Invention

[0004] In view of this, the present application provides a storage resource quota management method and a distributed storage system to improve the accuracy and reliability of storage resource quota management.

[0005] An embodiment of the present application provides a storage resource quota management method applied to a distributed storage system, wherein the distributed storage system includes multiple distributedly deployed physical nodes, each physical node is virtualized into at least one virtual node, and each virtual node corresponds to at least one data service and / or metadata service; the method includes:

[0006] In the token ring management mode, a virtual node on any token ring serves as a token ring management node, generates a token according to the quota target indicated by the token ring, and sends the token to the first virtual node on the token ring; the token ring is composed of multiple virtual nodes virtualized in the distributed storage system;

[0007] When any virtual node on the token ring receives a token, it performs quota management on the current metadata service operation and / or current data service operation that needs quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node, updates the token based on the quota management, and transmits the updated token to the next virtual node;

[0008] Among them, when any virtual node in the token ring fails, the failed virtual node is removed from the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring; when the failed virtual node is restored, the restored virtual node is re-added to the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring.

[0009] The embodiment of the present application further provides a distributed storage system, comprising a plurality of distributedly deployed physical nodes, wherein any physical node is virtualized into at least one virtual node, and any virtual node corresponds to at least one data service and / or metadata service;

[0010] In the token ring management mode, a virtual node on any token ring serves as a token ring management node, generates a token according to the quota target indicated by the token ring, and sends the token to the first virtual node on the token ring; the token ring is composed of multiple virtual nodes virtualized in the distributed storage system;

[0011] When any virtual node on the token ring receives a token, it performs quota management on the current metadata service operation and / or current data service operation that needs quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node, updates the token based on the quota management, and transmits the updated token to the next virtual node;

[0012] Among them, when any virtual node in the token ring fails, the failed virtual node is removed from the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring; when the failed virtual node is restored, the restored virtual node is re-added to the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring.

[0013] As can be seen from the above technical solution, in the embodiment of the present application, by introducing a token ring structure for quota management, a token carrying quota usage information is passed on each virtual node in the token ring. Whenever any virtual node on the token ring receives a token, quota management is performed on the current metadata service operation and / or current data service operation that needs to be quota managed and matches the token, and the token is updated based on the quota management, and the updated token is transmitted to the next virtual node. This ensures that only one virtual node updates the same quota usage information, that is, the quota usage information in the same token, at the same time, avoiding the existing quota usage information update delay and inconsistency problems, and also avoiding the quota allocation exceeding the quota limit caused by multiple nodes applying for quota at the same time, thereby fundamentally avoiding the problems of quota overuse and insufficient quota use, and improving the accuracy and reliability of storage resource quota management. In addition, all virtual nodes on the same token ring have the same application and use rights for the quota in the token corresponding to the token ring, which effectively avoids the situation of uneven quota distribution or lax quota restrictions.

[0014] Furthermore, in an embodiment of the present application, when a virtual node in the token ring fails or recovers, a virtual node is removed from or added to the token ring, and the readjusted token ring and the latest system service status version information are broadcast to all virtual nodes. In this way, by dynamically adjusting the token ring in accordance with the service status changes in the distributed storage system, that is, the changes in the virtual nodes, the situation in which the token cannot be transmitted normally due to the failure of a virtual node in the token ring can be avoided, thereby avoiding the impact on quota management. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0016] Figure 1 A schematic diagram of the method flow provided in an embodiment of the present application.

[0017] Figure 2 A schematic diagram of the token ring structure provided in an embodiment of the present application.

[0018] Figure 3 A flowchart of another method provided in an embodiment of the present application.

[0019] Figure 4 A flowchart of another method provided in an embodiment of the present application.

[0020] Figure 5 A schematic diagram of the structure of a distributed storage system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To facilitate understanding of the technical solutions of the embodiments of the present application, the quotas involved in the embodiments of the present application are described below.

[0022] Distributed file storage manages a vast amount of file storage resources, which are divided into different namespaces and managed using a directory tree (DTree) within each namespace. DTree is a tree-like structure that represents the directory hierarchy in a file system.

[0023] Quotas are typically configured at the namespace or directory tree level. A namespace can be understood as a logical unit in a distributed file storage system, used to divide file resources into different groups. It's also a logical unit used to logically isolate and organize storage resources in a distributed storage system. Each namespace can be considered an independent file system with independent quota configuration. This means that file storage resources in different namespaces are isolated from each other, and each namespace has its own quota configuration and storage policy. A directory tree (DTree) can be understood as a tree-like structure that represents the directory hierarchy within a file system. It's a level below the namespace, providing more granular resource control over file storage resources in the form of a directory tree. For example, in a multi-tenant scenario, each tenant can own one or more namespaces. Tenants can further configure independent quotas for the directory trees within their namespaces, providing more granular limits on disk space, file count, and other factors. Within tenants, users and user groups can be further divided, representing organizations (e.g., teams, projects), individuals (e.g., applications), and groups (categories), enabling quota configuration at different levels and with different dimensions.

[0024] The quota configuration may include but is not limited to: quota target, quota type, quota level, quota management method, and quota limit threshold. Among them, quota targets are generally divided into namespaces and directory trees (DTree). Quota types are generally divided into disk space quotas (such as capacity, size) and file quantity quotas (such as number of files). Quota levels are generally divided into user levels and user group levels. Quota management methods are generally divided into hard quotas and soft quotas; the hard quota here is an absolute limit, and the storage resource usage must not exceed the quota limit. The soft quota allows exceeding the configured limit, but an alarm will be issued or further operations will be performed (such as limiting the quota usage time, etc.). It should be noted that if the quota level is not configured, the quota configuration of the namespace or directory tree is for the current namespace or directory tree, and does not distinguish between accessing users and user groups.

[0025] In a distributed file storage system, the namespace and directory tree described above can be stored as metadata in a key-value (KV) format on a metadata server. This metadata is then propagated to all distributed nodes and access nodes (i.e., physical nodes within the distributed storage system) via a distributed consensus protocol (such as Paxos or Raft). Quota configuration, as part of the namespace and directory tree metadata, is recorded within the namespace or directory tree.

[0026] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0027] See also Figure 1 , Figure 1 A flowchart of a method provided in an embodiment of the present application. The method is applied to a distributed storage system. As an embodiment, the distributed storage system may include multiple distributed physical nodes, each of which is configured with at least one metadata service and at least one data service. Based on this, each physical node is virtualized into at least one virtual node, each of which corresponds to at least one data service and / or metadata service.

[0028] Here, metadata services can be understood as a service architecture for processing metadata service operations; metadata service operations include, for example, file creation, file deletion, file renaming, and setting file size, and are not specifically limited here. Data services can be understood as a service architecture for processing data service operations; data service operations include, for example, data storage, data deletion, data update, and data read operations, and are not specifically limited here.

[0029] like Figure 1 As shown, the process may include the following steps:

[0030] Step 101: In the token ring management mode, a virtual node on any token ring serves as a token ring management node, generates a token according to the quota target indicated by the token ring, and sends the token to the first virtual node on the token ring.

[0031] In this embodiment, a token ring is introduced to manage storage resource quotas in a distributed storage system. Figure 2 As shown, the token ring is composed of multiple virtual nodes virtualized in the distributed storage system. The virtual node order of each virtual node in the token ring can be flexibly set based on actual application requirements and is not specifically limited in this embodiment.

[0032] As an embodiment, at least one token ring may be configured in a distributed storage system. Each token ring may correspond to at least one token, and the token corresponding to each token ring may be circulated and passed around the token ring in the order of the virtual nodes in the token ring. In this embodiment, the transfer of tokens on each token ring is independent and independent of each other. This ensures the concurrency of the transfer of different tokens to the same virtual node, improving efficiency, and effectively avoids the situation where multiple tokens on the same virtual node are waiting to be transferred, resulting in a long period of time when the token cannot be transferred back to the token ring management node and is lost.

[0033] The tokens corresponding to each token ring can be generated by a virtual node on the token ring that serves as the token ring management node, based on the quota target indicated by the token ring. Furthermore, the token ring management node can also be responsible for managing the creation, startup, shutdown, and destruction of token rings. The virtual node that serves as the token ring management node in any token ring can be flexibly set based on actual application requirements. For example, it can be the first virtual node in the token ring, and this embodiment does not specifically limit this.

[0034] In this step, a token is generated according to the quota target indicated by the token ring. There are many ways to implement it. For example, as an embodiment, see Figure 3 As shown, in this embodiment, a token is generated according to the quota target indicated by the token ring. The specific implementation may include the following steps:

[0035] Step 301: obtain quota configuration information that matches the quota target, and obtain the latest system service status version information.

[0036] In this embodiment, as an embodiment, a monitoring service node is included in the distributed storage system, and the monitoring service node can be understood as a physical node on which a monitoring service is deployed in the distributed storage system. The monitoring service node can be responsible for detecting the service status of the data service and metadata service on each physical node in the distributed storage system, and when it detects that the service status of a certain data service or metadata service has changed (such as from a normal operating state to a fault state, or from a fault state to a fault recovery state, etc.), the maintained system service status version information is updated, such as incrementing the current system service status version information to obtain the latest system service status version information, wherein the initial system service status version information can be a preset initial value such as 0 or 1. Based on this, after completing the update of the system service status version information, the monitoring service node will propagate the current latest system service status version information to all virtual nodes on the token ring where the virtual node corresponding to the data service or metadata service in the distributed storage system is located through a broadcast mechanism, so that all virtual nodes can obtain the current latest system service status version information.

[0037] Accordingly, the monitoring service node may also be responsible for maintaining global quota configuration information, which is manually configured by the administrator through the configuration process. As described above, each quota configuration information may include, but is not limited to, quota target, quota type, quota level, quota management method, and quota limit threshold. For example, when the administrator changes quota configuration information through the configuration process (such as creating new quota configuration information or adjusting existing quota configuration information), the configuration process will send a configuration request to the monitoring service to notify the monitoring service node to update the global quota configuration information maintained locally. Afterwards, the monitoring service node will propagate the updated global quota configuration information to all metadata services and data services in the distributed storage system, as well as the token ring management nodes in each token ring, through a broadcast mechanism. By subscribing to the monitoring service, each metadata service and data service can receive the latest quota configuration information propagated by the monitoring service node and update its local quota configuration copy based on the received quota configuration information.

[0038] Based on this, as an embodiment, the Token Ring Management Node in the Token Ring can obtain quota configuration information that matches the quota target indicated by the Token Ring from the local quota configuration copy of the corresponding metadata service or data service. The specific method for obtaining quota configuration information that matches the quota target indicated by the Token Ring from the local quota configuration copy of the corresponding metadata service or data service will be described below with examples and will not be elaborated on here.

[0039] Step 302: Generate a token based on the obtained quota configuration information and the latest system service status version information.

[0040] In this embodiment, as an example, the data structure of the token can be, for example: {"token_id":"xxx","token_thread":"xxx","thread_epoch":"xxx","quota_target":"xxx","quota_type":"xxx","quota_level":"xxx","quota_proc":"xxx","quota_limit":"xxx","quota_used":"xxx"}.

[0041] Among them, token_id represents the token identifier; token_thread represents the token ring identifier; thread_epoch represents the token ring version information (that is, the system service status version information); quota_target represents the quota configuration target (such as namespace ID or DTreeID); quota_type represents the quota type; quota_level represents the quota level (if it can be empty, it means that the quota level is not restricted); quota_proc represents the quota management method; quota_limit represents the quota limit threshold; quota_used represents the quota usage value.

[0042] In this embodiment, as an embodiment, when a token is received, the token ring management node in the token ring will record the quota usage value in the token locally as the historical quota usage value corresponding to the token. Based on this, the token is generated based on the obtained quota configuration information and the current latest system service status version information. In specific implementation, for example, if the token is generated for the first time, the token can be generated directly based on the obtained quota configuration information and the current latest system service status version information, and the quota usage value in the token is an initial usage value such as 0; if the token is not generated for the first time, the token can be generated based on the obtained quota configuration information, the current latest system service status version information, and the historical quota usage value corresponding to the token that has been recorded locally.

[0043] This is completed Figure 3 The following is a description of the method flow. Figure 1 The method flow is described below.

[0044] Step 102: When any virtual node on the token ring receives a token, it performs quota management on the current metadata service operation and / or current data service operation that requires quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node, and updates the token based on the quota management, and transmits the updated token to the next virtual node.

[0045] In this embodiment, as an embodiment, in this step, based on the system service status version information carried by the token and the local system service status version information of the virtual node, quota management is performed on the current metadata service operation and / or the current data service operation that needs to execute quota management and matches the token. In the specific implementation, for example, it can be as follows: first check whether the system service status version information carried by the token and the local system service status version information of the virtual node are consistent; if the system service status version information carried by the token and the local system service status version information of the virtual node are consistent, then determine the quota value required for the current metadata service operation and / or the current data service operation, and based on the quota value required, allocate the corresponding quota value from the configuration usage value of the token to the current metadata service operation and / or the current data service operation.

[0046] Based on this, in this step, updating the token based on the quota management can be specifically implemented as follows: based on the quota value required for the current metadata service operation and / or the current data service operation, updating the quota usage value in the token.

[0047] In this embodiment, the update of the quota usage value in the token may include increase and decrease, etc. For example, taking the metadata service operation as an example, if the current metadata service operation is a file creation operation, the quota value required for the current metadata service operation is the number of files to be created, and the quota usage value in the token may be the number of files that have been used, so that the updated quota usage value is the sum of the quota usage value before the update and the quota value required for the current metadata service operation (i.e., corresponding to the above increase). If the current metadata service operation is a file deletion operation, the quota value required for the current metadata service operation is the number of files to be deleted, and the quota usage value in the token may be the number of files that have been used, so that the updated quota usage value is the difference between the quota usage value before the update and the quota value required for the current metadata service operation (i.e., corresponding to the above decrease).

[0048] In this embodiment, as an example, if the system service status version information carried by the token is inconsistent with the system service status version information of the local virtual node, it means that the token has expired and the token can be discarded. Based on this, for the current metadata service operation and / or current data service operation that requires quota management and matches the token on the virtual node, the token will continue to be waited for.

[0049] In this embodiment, as an embodiment, as described above, the monitoring service node in the distributed storage system, when any virtual node in the token ring fails, the failed virtual node is removed from the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined, and the readjusted virtual node order and the current latest system service status version information are broadcast to all virtual nodes in the token ring; and when the failed virtual node is restored, the restored virtual node is re-added to the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined, and the readjusted virtual node order and the current latest system service status version information are broadcast to all virtual nodes in the token ring.

[0050] Based on the above description, as an embodiment, if the virtual node on the token ring, which serves as the token ring management node, does not receive the token within the first set time, it means that the token may have been lost during the transmission process, and the reason for the loss may be packet loss or being discarded in a certain virtual node, etc. In order to ensure the normal operation of quota management, the token needs to be regenerated, so at this time, the step of generating the token according to the quota target indicated by the token ring can be returned.

[0051] Optionally, the first set time may be, for example, a period of time starting from the moment when the Token Ring Management Node sends a token for the last time, which is not specifically limited in this embodiment.

[0052] As another embodiment, if the virtual node on the token ring that serves as the token ring management node receives system service status version information, it also needs to regenerate the token. At this time, it can return to the step of generating the token according to the quota target indicated by the token ring.

[0053] As for how to obtain the current metadata service operation and / or current data service operation that requires quota management and matches the token in this step, the following will give an example and will not be described here.

[0054] So far, completed Figure 1 The process shown.

[0055] pass Figure 1As can be seen from the illustrated process, in the embodiment of the present application, by introducing a token ring structure for quota management, a token carrying quota usage information is passed between each virtual node in the token ring. Whenever any virtual node on the token ring receives a token, quota management is performed on the current metadata service operation and / or current data service operation that requires quota management and matches the token, and the token is updated based on the quota management, and the updated token is transmitted to the next virtual node. This ensures that only one virtual node updates the same quota usage information, that is, the quota usage information in the same token, at the same time, avoiding the existing quota usage information update delay and inconsistency problems, and also avoids the problem of quota allocation exceeding the quota limit caused by multiple nodes applying for quota at the same time, thereby fundamentally avoiding the problems of quota overuse and insufficient quota use, and improving the accuracy and reliability of storage resource quota management. In addition, all virtual nodes on the same token ring have the same application and use rights for the quota in the token corresponding to the token ring, which effectively avoids the situation of uneven quota distribution or lax quota restrictions.

[0056] Furthermore, in an embodiment of the present application, when a virtual node in the token ring fails or recovers, a virtual node is removed from or added to the token ring, and the readjusted token ring and the latest system service status version information are broadcast to all virtual nodes. In this way, by dynamically adjusting the token ring in accordance with the service status changes in the distributed storage system, that is, the changes in the virtual nodes, the situation in which the token cannot be transmitted normally due to the failure of a virtual node in the token ring can be avoided, thereby avoiding the impact on quota management.

[0057] The following describes how to obtain quota configuration information that matches the quota target indicated by the token ring from the local quota configuration copy of the corresponding metadata service or data service in step 301:

[0058] In this embodiment, as an example, the quota configuration information matching the quota target indicated by the token ring is obtained from the quota configuration copy local to the corresponding metadata service or data service. There are many ways to implement this in practice. For example, as an example, the quota configuration information including the configuration target is obtained from the quota configuration copy local to the corresponding metadata service or data service as the quota configuration information matching the quota target indicated by the token ring.

[0059] For another example, if the configuration target is a namespace, the quota configuration information containing the configuration target and the quota configuration information containing the configuration target that has an affiliation with the configuration target are obtained from the local quota configuration copy of the corresponding metadata service or data service as the quota configuration information that matches the quota target indicated by the token ring. The configuration target that has an affiliation with the configuration target can be understood as the target tree DTree under the configuration target.

[0060] The following describes how to obtain the current metadata service operation and / or current data service operation that needs to perform quota management and matches the token in step 102:

[0061] In this embodiment, taking a data service as an example, when the data service receives a data service operation, it checks whether there is quota configuration information matching the data service operation in the local quota configuration copy based on the quota parameters indicated by the data service operation. The quota parameters may include information such as the quota target, quota type, and quota level. If there is, it is determined that the data service operation is subject to quota restrictions. The data service operation can then be sent to the virtual node corresponding to the data service, so that the virtual node caches the received data service operation locally to await quota management. If there is no quota configuration information, it is determined that the data service operation is not subject to quota restrictions and can be processed downstream.

[0062] Based on the above description, when any virtual node on the token ring receives a token, it can obtain the data service operation and / or metadata service operation that matches the quota target in the token from the local data service operation and / or metadata service operation of the virtual node as the current metadata service operation and / or current data service operation that needs to perform quota management and matches the token.

[0063] Furthermore, this embodiment also introduces a dual-copy mechanism. Specifically, as an embodiment, when any metadata service or data service in the distributed storage system first receives the latest quota configuration information propagated by the monitoring service node, it will generate two identical quota configuration copies based on the latest quota configuration information. One quota configuration copy (denoted as copy 1) is used by the metadata service or data service to search for quota configuration information, and the other quota configuration copy (denoted as copy 2) is used for updating quota configuration information. This can avoid conflicts between the search and update operations of quota configuration information and improve update efficiency.

[0064] Optionally, after all metadata services and data services in the distributed storage system have completed updating the quota configuration information for replica 2, a distributed consistency protocol (such as the Paxos protocol or the Raft protocol) can be used to implement consistency updates for replica 1 in all metadata services and data services. This embodiment does not specifically limit how to implement consistency updates through the distributed consistency protocol.

[0065] The following describes how to switch quota management modes in a distributed storage system.

[0066] In this embodiment, as an example, the distributed storage system can adopt two quota management modes for quota management, one is a hierarchical management mode and the other is a token ring management mode. The two hierarchical management modes can be flexibly switched based on actual application requirements.

[0067] The following describes the hierarchical management model:

[0068] In this embodiment, under a hierarchical management model, there are upper-level quota management services and lower-level metadata and data services. The upper-level quota management service is responsible for unified management of global quotas, such as allocation and monitoring, while the lower-level metadata and data services are responsible for local quota allocation and verification. Specifically, taking the lower-level metadata service as an example, when the lower-level metadata service receives a metadata service operation, it will prioritize allocating the required quota value from the local pre-allocated quota value to avoid frequent calls to the upper-level quota management service. When the local pre-allocated quota value reaches a set threshold, the lower-level quota management service will apply for a quota and cache it locally for future use.

[0069] Based on this, as an embodiment, when the current quota management mode is the hierarchical management mode, upon receiving an external mode switching instruction, the quota management service in the distributed storage system will reclaim the pre-allocated but unused quota values ​​locally in all data services and metadata services in the distributed storage system. After the reclaim is completed, a first mode change notification will be sent to all metadata services, all data services, and all token ring management nodes in the distributed storage system. The first mode change notification is used to indicate that the current quota management mode is changed to the token ring management mode.

[0070] Optionally, the quota management service may send the first mode change notification to all metadata services, all data services, and all token ring management nodes in the distributed storage system through the monitoring service node.

[0071] When any Token Ring management node receives the first mode change notification, it will enter the Token Ring management mode. Correspondingly, when any metadata service or data service receives the first mode change notification, it will also enter the Token Ring management mode.

[0072] The following example describes how to reclaim pre-allocated but unused quota values ​​for all data services and metadata services in the distributed storage system:

[0073] In this embodiment, as an example, the quota management service can send a reclaim notification to all metadata services and data services to notify them to return the locally pre-allocated but unused quota value. The reclaim notification can include information such as the quota target requested for reclaim, the quota type, and the quota level, which is not limited here.

[0074] Upon receiving a reclaim notification, the metadata service or data service will search its local database for quota configuration information (also referred to as quota configuration items) that matches the reclaim notification. The matching condition is that the quota configuration item matches the quota target, quota type, and quota level information in the reclaim notification. Optionally, if the quota target to be reclaimed is a namespace, the pre-allocated but unused quota values ​​in the quota configuration items corresponding to all DTrees belonging to the namespace will also be reclaimed.

[0075] For any matching quota configuration item, the pre-allocated remaining value (pre-allocated remaining value = pre-allocated quota value - quota usage value) in the quota configuration item can be updated to zero to indicate that the pre-allocated but unused quota value corresponding to the quota configuration item has been recycled. This pre-allocated remaining value is then reported to the quota management service as the recycled quota value. This prevents subsequent service operations of the metadata service or data service from continuing to use the recycled quota value.

[0076] The quota management service receives the recovery quota values ​​reported by each metadata service and data service. For any quota configuration item, the total recovery quota value belonging to the quota configuration item is calculated: the total recovery quota value = the sum of the received recovery quota values ​​belonging to the quota configuration item. Based on this, the quota management service updates the global allocated quota value belonging to the quota configuration item in the local area: the updated global allocated quota value = the updated global allocated quota value - the total recovery quota value. Correspondingly, the quota management service updates the global quota available value belonging to the quota configuration item in the local area: the updated global quota available value = the global quota available value before the update + the total recovery quota value. This ensures the accuracy of the global quota configuration information.

[0077] As an example, during quota reclaim, before executing a metadata service or service operation, the metadata service or data service will locally search for a pre-allocated remaining value that matches the operation. If the pre-allocated remaining value found is less than the quota value required for the current operation, the metadata service or data service will request the required quota value from the quota management service. Based on this, the metadata service or data service sends a quota request to the quota management service; the quota request includes at least the required quota value and the service identifier of the metadata service or data service, which is not specifically limited here.

[0078] When the quota management service receives a quota request, it checks whether the locally available global quota value meets the quota requirements of the request. If so, the requested quota value is allocated. If not, the request is added to the quota request queue and waits until quota reclaim is complete before being processed.

[0079] Based on this, after the quota recovery is completed, the quota management service will remove the waiting quota application request from the quota application queue and retry to allocate the quota. Specifically, it checks whether the current global quota available value meets the quota requirements of the quota application request. If it does, it will allocate the required quota value and update the global allocated quota value: the updated global allocated quota value = the global allocated quota value before the update + the quota value allocated to the quota application request (recorded as the allocated value), and the updated global quota available value = the global quota available value before the update - the allocated value. If it does not meet the requirements, it means that the global quota available value has reached the quota limit threshold. At this time, the allocation failure is directly returned to the metadata service or data service that sent the quota application request.

[0080] After the quota is recovered, the quota management service will exit the hierarchical management mode and enter the token management mode.

[0081] As an example, under a hierarchical management model, the configuration management service can prioritize or preemptively reclaim quotas from metadata services and / or data services with lower loads based on the load of the underlying metadata services and data services, to avoid impacting the performance of high-load services. Furthermore, in this embodiment, the underlying metadata service or data service can autonomously detect the service's business load and the usage of its locally pre-allocated quota values. When the business load is low and the pre-allocated quota values ​​are being used sparingly, it can proactively return the pre-allocated but unused quota values ​​to the upper-level quota management service.

[0082] The following describes the specific switching process from the token ring management mode to the hierarchical management mode:

[0083] As an embodiment, when the current quota management mode is the token ring management mode, when the quota management service monitors that the local global quota used value is no longer close to the quota limit threshold (such as when the global quota used value is less than or equal to the preset quota used value, this may be due to reasons such as the administrator raising the quota limit threshold through the configuration process, or the user deleting files to release storage resources), in this case, the current quota management mode can be switched from the token ring management mode to the hierarchical management mode. Alternatively, when the quota management service receives an external mode switching instruction, it triggers the switching of the current quota management mode from the token ring management mode to the hierarchical management mode. It should be noted that this embodiment does not specifically limit the triggering timing of the quota management mode switching.

[0084] After the quota management service determines to switch the current quota management mode from the token ring management mode to the hierarchical management mode, it sends a second mode change notification to the monitoring service node, which then sends the second mode change notification to all metadata services, all data services, and all token ring management nodes in the distributed storage system. The second mode change notification is used to indicate that the current quota management mode has been changed to the hierarchical management mode.

[0085] Based on this, when any virtual node on the token ring receives the second mode change notification:

[0086] If this virtual node is not a token ring management node, then when this virtual node holds a token and there is currently no metadata service operation and / or data service operation that requires quota management and matches the token, all pre-allocated remaining values ​​matching the token in the local area will be returned to the token, and the token will be transferred to the next virtual node.

[0087] The pre-allocated remaining value that matches the token may mean that the token and quota parameters such as quota target, quota type, and quota level in the quota configuration item where the pre-allocated remaining value is located are consistent.

[0088] If this virtual node is a token ring management node, then: when the token is received within the second set time, the token is stopped from being transmitted, and when it is found that there is no metadata service operation and / or data service operation that requires quota management and matches the token, the quota message carrying the quota usage value in the token is reported to the quota management service in the distributed storage system; when the token is not received within the second set time, the quota usage value matching the token in the local area of ​​this virtual node is obtained, and the quota message carrying the obtained quota usage value is reported to the quota management service; so that the quota management service updates the global quota configuration information in the local area based on the received quota message.

[0089] The second set time can be flexibly set based on actual needs, such as two token cycles, three token cycles, etc., and is not specifically limited here. A token cycle can be understood as the transfer period of a token in the token ring, that is, the time it takes for a token to be transferred from the token ring management node to each virtual node in the token ring and finally return to the starting node.

[0090] In this embodiment, when the token ring management node receives the second mode change notification, it needs to wait for the second set time to ensure that all tokens on the token ring can return to the token management node to avoid affecting the management of the quota usage value in the token.

[0091] After the virtual node completes reporting of the quota message, it exits the token ring management mode.

[0092] Accordingly, upon receiving the aforementioned second mode change notification, any metadata service or data service will also exit the token ring management mode and enter the hierarchical management mode. It is understood that before the mode change is complete, any metadata service or data service that receives metadata service operations or service operations that require quota management and match the token will still be subject to quota limit management based on the quota usage value in the token. After the mode change is complete, quota management will be implemented in accordance with the hierarchical management mode. For example, metadata service operations or service operations that require quota management can apply to the quota management service for the required quota value.

[0093] It should be noted that in this embodiment, the notification sent through the monitoring service can be implemented by adding corresponding fields in the notification. For example, taking the mode switching notification as an example, a quota management mode field can be added to the notification to indicate the quota management mode to be switched to (such as token ring management mode or hierarchical management mode).

[0094] The following is a further description of the above token ring management mode:

[0095] In this embodiment, a pre-allocated quota mechanism is introduced and combined with the aforementioned token ring management model to implement quota management. Here, the pre-allocated quota mechanism can be understood as a mechanism that pre-allocates a certain quota available value (denoted as the pre-allocated remaining value) to ensure that storage resources are not overused and caches it locally on the virtual node for quota allocation. In this way, when the virtual node processes metadata service operations and / or service operations that require quota management, it can preferentially use the locally cached pre-allocated remaining value for quota allocation without waiting for tokens, thereby improving the efficiency of quota management.

[0096] For example, as an example, see Figure 4As shown, the specific implementation process of quota management using the token ring management mode combined with the pre-allocated quota mechanism may include the following steps:

[0097] Step 401: When any virtual node on the token ring obtains a metadata service operation or data service operation that matches a token, if the virtual node holds the token, the quota usage value in the token is updated based on the quota value usage requirements of the obtained metadata service operation or data service operation.

[0098] In this embodiment, the obtained metadata service operation or data service operation that matches the token may refer to the metadata service operation or data service operation that matches the token in the local token waiting queue of the virtual node.

[0099] In this embodiment, as an embodiment, after updating the quota usage value in the token based on the obtained quota value usage requirements of the metadata service operation or data service operation, this virtual node can also compensate the pre-allocated remaining value matching the token based on the quota usage value in the token, wherein the pre-allocated remaining value after compensation is greater than the pre-allocated remaining value before compensation.

[0100] Optionally, the compensation value may be a fixed value pre-set based on actual application requirements, or may be dynamically determined based on historical compensation values, such as by multiplying the historical compensation value by a preset compensation coefficient.

[0101] Step 402: If the virtual node does not hold the token, then when the pre-allocated remaining value matching the token in the local virtual node meets the quota value usage requirement, the pre-allocated remaining value is updated based on the quota value usage requirement; when the pre-allocated remaining value does not meet the quota value usage requirement, the obtained metadata service operation or data service operation is added to the token waiting queue.

[0102] In this embodiment, the pre-allocated remaining value satisfies the quota value usage requirement, which may mean that the pre-allocated remaining value is greater than or equal to the quota value required by the quota value usage requirement. Correspondingly, the pre-allocated remaining value does not satisfy the quota value usage requirement, which may mean that the pre-allocated remaining value is less than the quota value required by the quota value usage requirement.

[0103] In this embodiment, as an embodiment, any virtual node on the token ring, when the virtual node holds a token, if it is found that the pre-allocated remaining value matching the token in the local area is greater than the preset value and has not changed within at least one token cycle, then the pre-allocated remaining value is returned to the token; wherein the pre-allocated remaining value after the return is less than the pre-allocated remaining value before the return, and the quota usage value in the token after the return is greater than the quota usage value in the token before the return.

[0104] Optionally, the return value of returning the pre-allocated remaining value to the token can be, for example, a fixed value set in advance based on actual application requirements, or the pre-allocated remaining value, or can be dynamically determined based on historical return values, such as the product of historical return values ​​and a preset return coefficient.

[0105] For example, when any virtual node on the token ring receives a metadata service operation or data service operation that matches the token from the metadata service or data service corresponding to the virtual node during the period when it does not hold a token, it checks whether the pre-allocated remaining value that matches the token in the local virtual node meets the quota value usage requirement of the service operation. If it meets the requirement, the pre-allocated remaining value is directly updated: the updated pre-allocated remaining value = the pre-allocated remaining value before the update - the quota usage value required by the current quota value usage requirement. If it does not meet the requirement, the service operation can be added to the token waiting queue to wait for the token to arrive before continuing processing.

[0106] During the period of holding the token, this virtual node takes out the service operation that matches the token from the token waiting queue and tries to allocate the quota again. The specific quota allocation process can be found in the above description and will not be repeated here. After allocating the quota usage value to the service operation, a portion of the quota available value in the token can be compensated to the local pre-allocated remaining value of this virtual node: the allocated remaining value after compensation = the pre-allocated remaining value before compensation + the compensation value. In this way, the quota available value of the token after compensation = the quota available value of the token before compensation - the compensation value. As for the specific setting of the compensation value, please refer to the above description and will not be repeated here.

[0107] While holding a token, if the virtual node finds that the pre-allocated remaining value from the previous round (or multiple rounds) of token cycles has not been used, it can proactively return the pre-allocated remaining value to the token: the local pre-allocated remaining value after return = the local pre-allocated remaining value before return - the return value; thus, the quota available value of the token after return = the quota available value of the token before return + the return value. For the specific setting of the return value, please refer to the relevant description above and will not be repeated here.

[0108] This completes the description of the method provided in the embodiment of the present application. The following describes the system provided in the embodiment of the present application:

[0109] As an embodiment, this embodiment also provides a distributed storage system. Figure 5 , Figure 5 A schematic diagram of a system structure provided in an embodiment of the present application. Figure 5 As shown, the distributed storage system 500 includes a plurality of distributedly deployed physical nodes 501 , and any physical node is virtualized into at least one virtual node 502 , and any virtual node 502 corresponds to at least one data service and / or metadata service.

[0110] In the token ring management mode, a virtual node on any token ring serves as a token ring management node, generates a token according to the quota target indicated by the token ring, and sends the token to the first virtual node on the token ring; the token ring is composed of multiple virtual nodes virtualized in the distributed storage system;

[0111] When any virtual node on the token ring receives a token, it performs quota management on the current metadata service operation and / or current data service operation that needs quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node, updates the token based on the quota management, and transmits the updated token to the next virtual node;

[0112] Among them, when any virtual node in the token ring fails, the failed virtual node is removed from the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring; when the failed virtual node is restored, the restored virtual node is re-added to the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring.

[0113] As an embodiment, generating a token according to the quota target indicated by the token ring includes:

[0114] Obtaining quota configuration information that matches the quota target, and obtaining the latest system service status version information;

[0115] The token is generated based on the obtained quota configuration information and the current latest system service status version information.

[0116] As an embodiment, before entering the token ring management mode, the token ring management node on the token ring receives a first mode change notification, where the first mode change notification is used to indicate that the current quota management mode is changed to the token ring management mode;

[0117] Among them, the first mode change notification is sent by the quota management service in the distributed storage system after reclaiming the pre-allocated but unused quota values ​​of the data service and metadata service in the distributed storage system based on the external mode switching instruction when the current quota management mode is the hierarchical management mode.

[0118] As an embodiment, the performing of quota management on the current metadata service operation and / or the current data service operation that requires quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node includes:

[0119] If the system service status version information carried by the token is consistent with the local system service status version information of the virtual node, then determining the quota value required for the current metadata service operation and / or the current data service operation;

[0120] The updating of the token based on the quota management includes:

[0121] The quota usage value in the token is updated based on the quota value required for the current metadata service operation and / or the current data service operation.

[0122] As an embodiment, if the system service status version information carried by the token is inconsistent with the local system service status version information of the virtual node, the token is discarded;

[0123] If the virtual node on the token ring, which serves as the token ring management node, does not receive the token within the first set time, or if it receives system service status version information, it returns to the step of generating a token according to the quota target indicated by the token ring.

[0124] As an embodiment, when any virtual node on the token ring obtains a metadata service operation or a data service operation that matches the token, if the virtual node holds the token, the quota usage value in the token is updated based on the quota value usage requirement of the obtained metadata service operation or data service operation;

[0125] If this virtual node does not hold the token, then when the pre-allocated remaining value matching the token in the local virtual node meets the quota value usage requirement, the pre-allocated remaining value is updated based on the quota value usage requirement; when the pre-allocated remaining value does not meet the quota value usage requirement, the obtained metadata service operation or data service operation is added to the token waiting queue.

[0126] As an embodiment, after updating the quota usage value in the token based on the obtained quota value usage requirements of the metadata service operation or data service operation, this virtual node compensates the pre-allocated remaining value based on the quota usage value in the token, wherein the pre-allocated remaining value after compensation is greater than the pre-allocated remaining value before compensation.

[0127] As an embodiment, when any virtual node on the token ring holds the token, if it is found that the pre-allocated remaining value in the local area is greater than the preset value and has not changed within at least one token cycle, the pre-allocated remaining value will be returned to the token; wherein, the pre-allocated remaining value after the return is less than the pre-allocated remaining value before the return, and the quota usage value in the token after the return is greater than the quota usage value in the token before the return.

[0128] As an embodiment, when any virtual node on the token ring receives the second mode change notification, the second mode change notification is used to instruct to change the current quota management mode to the hierarchical management mode;

[0129] If the virtual node is not a token ring management node, then when the virtual node holds the token and there is currently no metadata service operation and / or data service operation that requires quota management and matches the token, all pre-allocated remaining values ​​matching the token in the local area are returned to the token, and the token is transmitted to the next virtual node;

[0130] If this virtual node is a token ring management node, then: when the token is received within the second set time, the token is stopped from being transmitted, and when it is found that there is no metadata service operation and / or data service operation that requires quota management and matches the token, the quota message carrying the quota usage value in the token is reported to the quota management service in the distributed storage system; when the token is not received within the second set time, the quota usage value matching the token in the local area of ​​this virtual node is obtained, and the quota message carrying the obtained quota usage value is reported to the quota management service; so that the quota management service updates the global quota configuration information in the local area based on the received quota message;

[0131] After the virtual node completes reporting of the quota message, it exits the token ring management mode.

[0132] So far, completed Figure 5 Structural description of the system shown.

[0133] The implementation process of the functions and effects of each step in the above system is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0134] The above are merely preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A storage resource quota management method applied to a distributed storage system, characterized in that: The distributed storage system includes a plurality of distributedly deployed physical nodes, each physical node is virtualized into at least one virtual node, and each virtual node corresponds to at least one data service and / or metadata service; The method comprises: In the token ring management mode, a virtual node on any token ring serves as a token ring management node, generates a token according to the quota target indicated by the token ring, and sends the token to the first virtual node on the token ring; the token ring is composed of multiple virtual nodes virtualized in the distributed storage system; When any virtual node on the token ring receives a token, it performs quota management on the current metadata service operation and / or current data service operation that needs quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node, updates the token based on the quota management, and transmits the updated token to the next virtual node; Among them, when any virtual node in the token ring fails, the failed virtual node is removed from the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring; when the failed virtual node is restored, the restored virtual node is re-added to the token ring, the order of the virtual nodes in the token ring is readjusted, the current latest system service status version information is determined and broadcast to all virtual nodes in the token ring.

2. The method according to claim 1, characterized in that Generating a token according to the quota target indicated by the token ring includes: Obtaining quota configuration information that matches the quota target, and obtaining the latest system service status version information; The token is generated based on the obtained quota configuration information and the current latest system service status version information.

3. The method according to claim 1, characterized in that Before entering the token ring management mode, the method further comprises: The token ring management node on the token ring receives a first mode change notification, where the first mode change notification is used to instruct to change the current quota management mode to the token ring management mode; Among them, the first mode change notification is sent by the quota management service in the distributed storage system after reclaiming the pre-allocated but unused quota values ​​of the data service and metadata service in the distributed storage system based on the external mode switching instruction when the current quota management mode is the hierarchical management mode.

4. The method according to claim 1, wherein The performing quota management on the current metadata service operation and / or the current data service operation that needs quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node includes: If the system service status version information carried by the token is consistent with the local system service status version information of the virtual node, then determining the quota value required for the current metadata service operation and / or the current data service operation; The updating of the token based on the quota management includes: The quota usage value in the token is updated based on the quota value required for the current metadata service operation and / or the current data service operation.

5. The method according to claim 4, characterized in that The method further comprises: if the system service status version information carried by the token is inconsistent with the local system service status version information of the virtual node, discarding the token; The method further includes: if the virtual node on the token ring serving as the token ring management node does not receive the token within a first set time, or if it receives system service status version information, returning to the step of generating a token according to the quota target indicated by the token ring.

6. The method according to claim 1, characterized in that The method further comprises: When any virtual node on the token ring obtains a metadata service operation or a data service operation that matches the token, if the virtual node holds the token, the quota usage value in the token is updated based on the quota value usage requirement of the obtained metadata service operation or data service operation; If this virtual node does not hold the token, then when the pre-allocated remaining value matching the token in the local virtual node meets the quota value usage requirement, the pre-allocated remaining value is updated based on the quota value usage requirement; when the pre-allocated remaining value does not meet the quota value usage requirement, the obtained metadata service operation or data service operation is added to the token waiting queue.

7. The method according to claim 6, characterized in that After updating the quota usage value in the token based on the obtained quota value usage requirement of the metadata service operation or the data service operation, the method further includes: The virtual node compensates the pre-allocated remaining value based on the quota usage value in the token, wherein the pre-allocated remaining value after compensation is greater than the pre-allocated remaining value before compensation.

8. The method according to claim 6, characterized in that The method further comprises: Any virtual node on the token ring, when the virtual node holds the token, if it is found that the pre-allocated remaining value in the local area is greater than the preset value and has not changed within at least one token cycle, the pre-allocated remaining value will be returned to the token; wherein the pre-allocated remaining value after the return is less than the pre-allocated remaining value before the return, and the quota usage value in the token after the return is greater than the quota usage value in the token before the return.

9. The method according to claim 1, characterized in that The method further comprises: When any virtual node on the token ring receives a second mode change notification, the second mode change notification is used to instruct to change the current quota management mode to a hierarchical management mode; If the virtual node is not a token ring management node, then when the virtual node holds the token and there is currently no metadata service operation and / or data service operation that requires quota management and matches the token, all pre-allocated remaining values ​​matching the token in the local area are returned to the token, and the token is transmitted to the next virtual node; If this virtual node is a token ring management node, then: when the token is received within the second set time, the token is stopped from being transmitted, and when it is found that there is no metadata service operation and / or data service operation that requires quota management and matches the token, the quota message carrying the quota usage value in the token is reported to the quota management service in the distributed storage system; when the token is not received within the second set time, the quota usage value matching the token in the local area of ​​this virtual node is obtained, and the quota message carrying the obtained quota usage value is reported to the quota management service; so that the quota management service updates the global quota configuration information in the local area based on the received quota message; After the virtual node completes reporting of the quota message, it exits the token ring management mode.

10. A distributed storage system, characterized in that: The distributed storage system includes a plurality of distributedly deployed physical nodes, each physical node is virtualized into at least one virtual node, and each virtual node corresponds to at least one data service and / or metadata service; In the token ring management mode, a virtual node on any token ring acts as a token ring management node, generates a token according to the quota target indicated by the token ring, and sends the token to the first virtual node on the token ring; The token ring is composed of multiple virtual nodes virtualized in the distributed storage system; When any virtual node on the token ring receives a token, it performs quota management on the current metadata service operation and / or current data service operation that needs quota management and matches the token based on the system service status version information carried by the token and the local system service status version information of the virtual node, updates the token based on the quota management, and transmits the updated token to the next virtual node; When any virtual node in the token ring fails, the failed virtual node is removed from the token ring, the order of the virtual nodes in the token ring is readjusted, the latest system service status version information is determined and broadcasted to all virtual nodes in the token ring; When the faulty virtual node is restored, the restored virtual node will be added back into the token ring, the virtual node sequence in the token ring will be readjusted, and the latest system service status version information will be determined and broadcasted to all virtual nodes in the token ring.

Images