Method and apparatus for improving accuracy of parallel file system space quota limits

Abstract

This invention provides a method and apparatus for improving the accuracy of space quota limits in a parallel file system. The method includes: a client of the parallel file system distributing data to be written for a managed object to various distributed storage services for storage; each distributed storage service performing write operations on the data to be written for the managed object according to a preset maximum write bandwidth; a centralized management service periodically collecting and summarizing space usage information of the stored data of the managed objects from each distributed storage service, and comparing the space usage information of each managed object with the preset quota limit corresponding to that managed object; based on the comparison results, the centralized management service sending a list of managed objects that have reached their quota limits or an adjusted data write bandwidth to each distributed storage service; and each distributed storage service controlling the write operations on the data to be written for the managed objects. This invention can effectively improve the performance of quota management and avoid writing large amounts of data exceeding the quota limit.

Description

Technical Field

[0001] This invention relates to the field of computers, and in particular to a method and apparatus for improving the accuracy of space quota limits in parallel file systems. Background Technology

[0002] Quota management is a common function in file systems, typically used to limit data usage for users, groups, or folders, and to allocate storage space reasonably. For local file systems such as XFS, it can track every data write in real time and calculate space usage, rejecting further data writes when space usage reaches the quota limit.

[0003] Parallel file systems are distributed file systems where user files are striped and stored across multiple physical nodes' backend storage disks, managed by multiple storage services on those nodes. When a client writes a file, it first obtains the file's distribution information through a metadata service, then communicates directly with the storage service to which each data shard belongs to send the data. There is no unified data path entry point, and real-time statistics on storage space usage are not possible. Therefore, quota limits for parallel file systems cannot be reliably controlled; writing often stops only after far exceeding the set quota, significantly reducing the effectiveness of quota management. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for improving the accuracy of space quota limits in parallel file systems.

[0005] To address the above problems, this invention provides a method for improving the accuracy of space quota limits in parallel file systems, comprising: The client of the parallel file system sends write data requests to distribute the data to be written to the managed object to various distributed storage services for storage. Each distributed storage service performs write operations on the data to be written to the managed object based on the write data request and according to the preset maximum write bandwidth; The centralized management service periodically collects and summarizes the space usage information of the stored data of the managed objects from each distributed storage service, compares the space usage information of each managed object with the preset quota limit corresponding to the managed object, and obtains the comparison result. Based on the comparison results, the centralized management service sends a list of managed objects that have reached their quota limits or an adjusted data write bandwidth to each distributed storage service. Each distributed storage service controls the write operations of data to be written to managed objects based on the list of managed objects that have reached their quota limits or the adjusted data write bandwidth.

[0006] Furthermore, in the above method, the centralized management service periodically collects and summarizes the space usage information of the managed objects' storage data from each distributed storage service, compares the space usage information of each managed object with the preset quota limit corresponding to that managed object, and obtains the comparison result, including: The centralized management service periodically collects and summarizes the storage space usage information of the managed objects from each distributed storage service according to the preset time period Duration for updating space usage information. It then compares the space usage information of each managed object with the preset quota limit corresponding to that managed object to obtain a list of managed objects that have reached the quota limit. Based on the comparison results, the centralized management service sends a list of managed objects that have reached their quota limits to each distributed storage service, including: Based on the comparison results, the centralized management service pushes a list of managed objects that have reached their quota limits to each storage service.

[0007] Furthermore, in the above method, each distributed storage service controls the write operations of data to be written to managed objects based on the list information of managed objects that have reached their quota limits, including: Each storage service receives a list of managed objects that have reached their quota limits from the centralized management service, and updates and saves its own list of managed objects that have reached their quota limits based on the received list of managed objects that have reached their quota limits. Each time a storage service receives a write request from a client of the parallel file system, it compares it with its own list of managed objects that have reached their quota limits. If the managed object requested for writing is in the list, the write request is rejected.

[0008] Furthermore, in the above method, the centralized management service sends the adjusted data write bandwidth to each distributed storage service based on the comparison results, including: Each time the centralized management service collects the space usage information of the managed objects, it calculates the remaining available space of the managed objects that have not reached the quota limit. Based on the remaining available space of the managed object and the preset time period for the managed object to update the space usage information to the centralized management service, it calculates the current corrected maximum write bandwidth of the managed object and sends it to each distributed storage service.

[0009] Furthermore, in the above method, the calculation method for the maximum write bandwidth is modified as follows: Let the quota limit of a certain managed object be Max, the amount of space already used be Used, the preset time period for the managed object to update the space usage to the centralized management service be Duration, and the maximum write bandwidth available to the managed object be BW. Then BW = (Max - Used) / Duration.

[0010] Furthermore, in the above method, each time the centralized management service collects the space usage information of the managed objects, it calculates the remaining available space of the managed objects that have not reached the quota limit. Based on the remaining available space of the managed object and the preset time period for the managed object to update its space usage information to the centralized management service, it calculates the current corrected maximum write bandwidth corresponding to the managed object and sends it to each distributed storage service, including: The centralized management service collects space usage information of managed objects each time and calculates the amount of available space remaining for managed objects that have not reached the quota limit; If the remaining available space exceeds the preset limit, the current corrected maximum write bandwidth of the managed object is calculated based on the remaining available space of the managed object and the preset time period for the managed object to update the space usage information to the centralized management service, and then sent to each distributed storage service.

[0011] Furthermore, in the above method, the preset limit L is calculated using the following formula: L = Max − (Bmax × Duration) Among them, Max is the quota limit for the managed object, Bmax is the preset maximum bandwidth supported by the parallel file system, and Duration is the preset time period for the managed object to update the space usage of the centralized management service.

[0012] Furthermore, in the above method, each distributed storage service controls the write operations of the data to be written to the managed object based on the adjusted data write bandwidth, including: Each distributed storage service controls the speed at which the file system client sends the data to be written for the managed object to each distributed storage service, based on the modified maximum write bandwidth corresponding to the managed object received from the centralized management service.

[0013] According to another aspect of the present invention, a computer-readable storage medium is also provided, having stored thereon computer-executable instructions, wherein when executed by a processor, the computer-executable instructions cause the processor to perform the method described in any of the preceding claims.

[0014] According to another aspect of the present invention, a calculator device is also provided, comprising: Processor; and A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method described in any of the preceding descriptions.

[0015] Compared with existing technologies, this invention uses a parallel file system client to send write data requests, distributing the data to be written for managed objects to various distributed storage services for storage. Each distributed storage service, based on the write data requests and within a preset maximum write bandwidth, performs write operations on the data to be written for the managed objects. A centralized management service periodically collects and summarizes the space usage information of the stored data of managed objects from each distributed storage service, compares the space usage information of each managed object with the preset quota limit corresponding to that managed object, and obtains the comparison result. Based on the comparison result, the centralized management service sends a list of managed objects that have reached their quota limit or an adjusted data write bandwidth to each distributed storage service. Each distributed storage service controls the write operation of the data to be written for the managed objects based on the list of managed objects that have reached their quota limit or the adjusted data write bandwidth. This effectively improves the performance of quota management and avoids writing large amounts of data exceeding the quota limit. Attached Figure Description

[0016] Figure 1 This is a diagram of the line file system and client network connection topology according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the exchange of quota information between centralized management services and storage services according to an embodiment of the present invention. Figure 3 This is a data writing flowchart for a quota management function according to an embodiment of the present invention; Figure 4 This is a flowchart of the centralized management service according to an embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Parallel file systems are distributed file systems where user files are striped and stored across multiple physical nodes on backend disks. These storage services manage the files. When a client writes a file, it first obtains the file distribution information through a metadata service, then directly communicates with the storage service to which each data shard belongs to send the data. There is no unified data path entry point, and storage space usage cannot be monitored in real time. A centralized service can periodically obtain and aggregate data usage information from various storage services, compare it with quotas to determine if the quota limit has been reached, and then push the information of managed objects (users, groups, or folders) that have reached the quota limit to the storage services. The storage services then use this information to reject further writing of data to objects exceeding the quota. This creates a time window for exceeding quota limits. Therefore, the quota limits of parallel file systems usually have a "grace period" that can last for minutes. Since parallel file systems are designed for high-performance I / O, the throughput of their minimum configuration clusters can often exceed 50GB / s. If the grace period is 1 minute, and the write speed is 50GB per second, 3TB of data can be written in excess of the quota limit within the grace period. This makes the quota management of parallel file systems very ineffective.

[0019] The parallel file system and client network connection topology involved in the embodiments of the present invention are as follows: Figure 1 As shown, in Figure 1 In the process, the centralized management service manages the various storage services, provides clients with the distribution information of data fragments after file striping, and collects space usage information from each storage service and then pushes the management object information that has reached or exceeded the quota to each storage service. After multiple clients obtain the distribution information of file data fragments from the centralized management service, they simultaneously write data to multiple backend storage services in parallel.

[0020] The flowchart for the exchange of quota information between centralized management services and storage services can be seen as follows: Figure 2 As shown: 1) Initiate information collection: The centralized management service periodically sends messages to all storage services requesting the collection of data usage information; 2) Local Information Reporting: Each storage service sends its local data usage information to the centralized management service; 3) Information aggregation and calculation: The centralized management service receives data usage information from each storage service and performs aggregation and calculation; 4) Quota matching determination: The centralized management service compares and summarizes the calculation results with the preset quota items in the system to compile a list of management objects that have reached the quota; 5) Result Distribution: The centralized management service sends the list of managed objects that have reached their quota limits to all storage services.

[0021] 6) Local execution control: Each storage service receives and locally saves a list of managed objects that have reached the quota limit. Each time a write data request is received, it checks against this list.

[0022] 7) Cyclic execution: After completing the above process, the next cycle begins, and the centralized management service initiates the collection of data usage information again, repeating the entire process.

[0023] The data writing flowchart for supporting quota management functions can be shown as follows: Figure 3 As shown: 1) Initiating a write request: The client initiates a write data request to the storage service to which each data shard belongs for a specific file; 2) Quota Status Check: The storage service verifies whether the write target has been recorded as having reached the quota limit; 3) If the write target has not reached the quota limit: perform the data write operation and return a "write successful" result to the client; 4) If the write target has reached the quota limit: reject this data write request and return a "insufficient space" message to the client.

[0024] In this embodiment of the invention, the flowchart following the processing steps of the centralized management service can be as follows: Figure 4 As shown: 1) Initiate information collection: The centralized management service periodically sends messages to all storage services requesting the collection of data usage information; 2) Local Information Reporting: Each storage service sends its local data usage information to the centralized management service; 3) Information aggregation and calculation: The centralized management service receives data usage information from each storage service and performs aggregation and calculation. 4) Quota matching determination: The centralized management service compares the calculation results with the preset quota items in the system to compile a list of management objects that have reached the quota; 5) Bandwidth limit setting: The centralized management service sets a bandwidth limit for managed objects that have not yet reached their quota based on the aggregated calculation results; 6) Result Distribution: The centralized management service sends a list of managed objects that have reached their quota limits to all storage services; 7) Local execution control: Each storage service receives and locally saves a list of managed objects that have reached the quota limit. Each time a write data request is received, it checks against this list. 8) Cyclic execution: After completing the above process, the next cycle begins, and the centralized management service initiates the collection of data usage information again, repeating the entire process.

[0025] Example: Suppose the system sets a 20T space usage quota for the managed object—user USER1. Currently, 19T of data has been written. The central management service collects space usage information every 100 seconds. Therefore, Max = 20T, Used = 19T, Duration = 100s. Calculations show that the maximum write bandwidth available to this user in the next cycle is (20T – 19T) / 100s = 10G / s. At the start of the next cycle, the central management service re-collects data and finds that the user has already occupied 19.5T of space. Therefore, Max = 20T, Used = 19.5T, Duration = 100s. Calculations show that the maximum write bandwidth available to this user in the next cycle is (20T – 19.5T) / 100s = 5G / s.

[0026] This invention provides a method for improving the accuracy of space quota limits in parallel file systems, comprising: Step S1: The client of the parallel file system sends a write data request to distribute the data to be written of the managed object to each distributed storage service for storage; each distributed storage service, based on the write data request and according to the preset maximum write bandwidth Bmax, performs the write operation on the data to be written of the managed object. Here, the centralized management service can be responsible for collecting data, calculating quotas and bandwidth, but the actual write requests are sent directly from the client to each storage service. There is no centralized data entry point between the client and each distributed storage service. Step S2: The centralized management service periodically collects and summarizes the space usage information of the stored data of the managed objects from each distributed storage service, compares the space usage information of each managed object with the preset quota limit corresponding to the managed object, and obtains the comparison result. Preferably, step S2 includes: Step S211: The centralized management service periodically collects and summarizes the space usage information of the stored data of the managed objects from each distributed storage service according to the preset time period Duration for updating space usage information. The space usage information of each managed object is compared with the preset quota limit corresponding to the managed object to obtain a list of managed objects that have reached the quota limit. Here, the management objects that have reached the quota limit include: management objects that are greater than or equal to the quota limit, that is, management objects that have just reached the quota limit and management objects that have exceeded the quota limit.

[0027] Preferably, step S2 includes: Step S221: The centralized management service periodically collects and summarizes the storage space usage information of the managed objects from each distributed storage service according to the preset time period Duration for updating the space usage information. The space usage information of each managed object is compared with the preset quota limit corresponding to the managed object to obtain the managed objects that have not reached the quota limit. Step S3: Based on the comparison results, the centralized management service sends a list of managed objects that have reached the quota limit or the adjusted data write bandwidth to each distributed storage service. Step S3 includes: In step S311, the centralized management service pushes a list of managed objects that have reached their quota limits to each storage service based on the comparison results.

[0028] Step S3 includes: Step S321: Each time the centralized management service collects the space usage information of the managed objects, it calculates the remaining available space of the managed objects that have not reached the quota limit. Based on the remaining available space of the managed object and the preset time period for the managed object to update the space usage information to the centralized management service, it calculates the current corrected maximum write bandwidth BW of the managed object and sends it to each distributed storage service.

[0029] A key aspect of this invention is that, for managed objects, a traffic limiting (QoS) function is used to restrict the data write bandwidth of the managed objects. The central management service needs to calculate the maximum allowed write bandwidth BW based on its own periodicity for collecting space usage information and the remaining space quota of the managed objects, and configure the traffic bandwidth limit of the managed objects.

[0030] The specific calculation method is as follows: Let the quota limit of a certain managed object be Max, the amount of space already used be Used, the preset time period for the centralized management service to collect space usage be Duration, and the maximum corrected write bandwidth available to the managed object be BW. Then BW = (Max - Used) / Duration.

[0031] More preferably, step S321 includes: Step S3211: The centralized management service collects the space usage information of the managed objects each time and calculates the amount of available space remaining for the managed objects that have not reached the quota limit. Step S3212: If the remaining available space exceeds a preset limit, calculate the current corrected maximum write bandwidth BW of the managed object based on the remaining available space of the managed object and the preset time period for the managed object to update the space usage information to the centralized management service, and send it to each distributed storage service.

[0032] Here, when the space usage of the managed object does not exceed a certain preset limit L, the traffic limit QoS may not be set for this object.

[0033] Preferably, the preset limit can be calculated from the maximum bandwidth supported by the parallel file system and the remaining available space of the managed object; Here, the preset limit L of a certain managed object is less than the quota limit of this managed object. Let the quota limit of the managed object be Max (total space upper limit); Let the currently used space be Used; Let the remaining available space be S = Max - Used; Let the preset maximum bandwidth supported by the parallel file system be Bmax (unit: bytes / second); Let the preset time period for the managed object to update the space usage to the centralized management service be Duration (unit: seconds, that is, the time interval between two data collections by the centralized management service); The preset limit L is the space usage threshold for triggering the traffic limit QoS, and it satisfies: L = Max - (Bmax × Duration); The remaining space S = Max - L must just be filled up at the maximum bandwidth Bmax within one period Duration: S = Bmax × Duration; Substitute S = Max - L, and we get: Max - L = Bmax × Duration; After arrangement, we get the preset limit formula: L = Max - (Bmax × Duration).

[0034] When Used < L: The remaining space S > Bmax × Duration. Even if writing continuously at the preset maximum bandwidth, it can last for at least one statistical period, and there is no need to set the traffic limit QoS; When Used ≥ L: The remaining space S ≤ Bmax × Duration. If continuing to write at the preset maximum bandwidth, the space will be exhausted before the next statistics. Therefore, it is necessary to start the traffic limit QoS and limit the bandwidth to BW = S / Duration to ensure that it just lasts until the next statistical period.

[0035] For example, assume: Quota limit Max = 100GB; System maximum bandwidth Bmax = 1GB / s; Statistical period Duration = 60s; The preset limit is: L = 100GB − (1GB / s × 60s) = 40GB; When used space < 40 GB and remaining space > 60 GB, full bandwidth writing is possible without QoS requirements; When the used space (Used) is greater than or equal to 40 GB and the remaining space is less than or equal to 60 GB, the current limit should be set to BW = (100 - Used) / 60, in GB / s.

[0036] In step S4, each distributed storage service controls the write operation of the data to be written to the managed objects based on the list of managed objects that have reached the quota limit or the adjusted data write bandwidth.

[0037] Step S4 includes: In step S411, each storage service receives the list of managed objects that have reached the quota limit pushed by the centralized management service, and updates and saves its own list of managed objects that have reached the quota limit based on the received list of managed objects that have reached the quota limit. In step S412, each storage service receives a write data request from the client of the parallel file system and compares it with its own list of managed objects that have reached the quota limit. If the managed object for which the write data request is made is in the list of managed objects, the write is rejected and a "space insufficient" error message is returned.

[0038] Step S4 includes: In step S421, each distributed storage service controls the speed at which the file system client sends the data to be written for the managed object to each distributed storage service based on the current corrected maximum write bandwidth BW corresponding to the managed object received from the centralized management service.

[0039] Here, the bandwidth control (QoS) function of the parallel file system can be used to limit the upper limit of the write bandwidth of the managed object and dynamically adjust the current corrected maximum write bandwidth BW of the managed object to ensure that no data exceeding the quota limit is written within the time period during which each distributed storage service updates the space usage information to the centralized management service.

[0040] Here, if each distributed storage service receives a list of managed objects that have reached their quota, it caches / saves this list locally. Each time a client writes a data request, it first checks whether the managed object to be written is in its own list of managed objects that have reached their quota limit. If the list of managed objects has reached its quota limit, write operations will be rejected directly, and the message "Quota reached / Insufficient space" will be returned. If each storage service receives the adjusted data write bandwidth BW, it binds this bandwidth value to the corresponding management object and limits the rate of all subsequent write traffic to that management object. This controls the speed at which clients send data and the speed at which data is written to disk, ensuring that the write speed does not exceed BW. This allows the remaining space to last until the next quota statistics cycle, smoothly controlling the write speed and preventing the space from being exhausted instantly.

[0041] After obtaining the list of managed objects that have reached their quotas or the adjusted data write bandwidth, each distributed storage service performs the following operations: For managed objects that have reached their quota, when a write request is received from a client for that object, the write operation will be rejected directly. For managed objects that have not reached their quota but require bandwidth limitation, the write speed of the object is limited based on the adjusted data write bandwidth to ensure that the remaining available space is not exhausted before the next update cycle of space usage information.

[0042] In addition, if the managed object is not in the list of managed objects that have reached the quota limit, and there is no corresponding modified maximum write bandwidth BW, then the write operation on the data to be written of the managed object will still be allowed according to the preset maximum write bandwidth Bmax.

[0043] In parallel file systems, client-side data writing directly communicates with multiple distributed storage services in the backend, lacking a unified data entry point. Quota control in parallel file systems can be achieved through a centralized management node periodically collecting information on used space, comparing it to preset quotas, and then issuing quota-reaching management object information to the storage services. However, this approach cannot detect in real-time when used space reaches the threshold and prevent further data writing. Especially given the ultra-high bandwidth characteristics of parallel file systems, data often continues to be written far exceeding the set quota, significantly reducing the effectiveness of quota management. The method and apparatus of this invention for improving the accuracy of space quota limits in parallel file systems can effectively improve quota management and prevent the writing of large amounts of data exceeding quota limits.

[0044] According to another aspect of the present invention, a computer-readable storage medium is also provided, having stored thereon computer-executable instructions, wherein when executed by a processor, the computer-executable instructions cause the processor to perform the method described in any of the preceding claims.

[0045] According to another aspect of the present invention, a calculator device is also provided, comprising: Processor; and A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method described in any of the preceding descriptions.

[0046] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0047] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0048] Obviously, those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the invention. Therefore, if these modifications and variations fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A method for improving the accuracy of space quota limits in parallel file systems, characterized in that, include: The client of the parallel file system sends write data requests to distribute the data to be written to the managed object to various distributed storage services for storage. Each distributed storage service performs write operations on the data to be written to the managed object based on the write data request and according to the preset maximum write bandwidth; The centralized management service periodically collects and summarizes the space usage information of the stored data of the managed objects from each distributed storage service, compares the space usage information of each managed object with the preset quota limit corresponding to the managed object, and obtains the comparison result. Based on the comparison results, the centralized management service sends a list of managed objects that have reached their quota limits or an adjusted data write bandwidth to each distributed storage service. Each distributed storage service controls the write operations of data to be written to managed objects based on the list of managed objects that have reached their quota limits or the adjusted data write bandwidth.

2. The method for improving the accuracy of space quota limits in a parallel file system as described in claim 1, characterized in that, The centralized management service periodically collects and aggregates the storage space usage information of the managed objects from each distributed storage service. It then compares the space usage information of each managed object with the preset quota limit corresponding to that object to obtain the comparison results, including: The centralized management service periodically collects and summarizes the storage space usage information of the managed objects from each distributed storage service according to the preset time period for updating space usage information. It then compares the space usage information of each managed object with the preset quota limit corresponding to that managed object to obtain a list of managed objects that have reached the quota limit. Based on the comparison results, the centralized management service sends a list of managed objects that have reached their quota limits to each distributed storage service, including: Based on the comparison results, the centralized management service pushes a list of managed objects that have reached their quota limits to each storage service.

3. The method for improving the accuracy of space quota limits in a parallel file system as described in claim 2, characterized in that, Each distributed storage service controls write operations on data to be written to managed objects based on a list of managed objects that have reached their quota limits, including: Each storage service receives a list of managed objects that have reached their quota limits from the centralized management service, and updates and saves its own list of managed objects that have reached their quota limits based on the received list of managed objects that have reached their quota limits. Each time a storage service receives a write request from a client of the parallel file system, it compares it with its own list of managed objects that have reached their quota limits. If the managed object requested for writing is in the list, the write request is rejected.

4. The method for improving the accuracy of space quota limits in a parallel file system as described in claim 1, characterized in that, Based on the comparison results, the centralized management service sends the adjusted data write bandwidth to each distributed storage service, including: Each time the centralized management service collects the space usage information of the managed objects, it calculates the remaining available space of the managed objects that have not reached the quota limit. Based on the remaining available space of the managed object and the preset time period for the managed object to update the space usage information to the centralized management service, it calculates the current corrected maximum write bandwidth of the managed object and sends it to each distributed storage service.

5. The method for improving the accuracy of space quota limits in a parallel file system as described in claim 4, characterized in that, The revised method for calculating the maximum write bandwidth is as follows: Let the quota limit of a certain managed object be Max, the amount of space already used be Used, the preset time period for the managed object to update the space usage to the centralized management service be Duration, and the maximum write bandwidth available to the managed object be BW. Then BW = (Max - Used) / Duration.

6. The method for improving the accuracy of space quota limits in a parallel file system as described in claim 4, characterized in that, Each time the centralized management service collects the space usage information of a managed object, it calculates the remaining available space for managed objects that have not reached their quota limits. Based on the remaining available space of the managed object and the preset time period for the managed object to update its space usage information to the centralized management service, it calculates the current corrected maximum write bandwidth for the managed object and sends it to each distributed storage service, including: The centralized management service collects space usage information of managed objects each time and calculates the amount of available space remaining for managed objects that have not reached the quota limit; If the remaining available space exceeds the preset limit, the current corrected maximum write bandwidth of the managed object is calculated based on the remaining available space of the managed object and the preset time period for the managed object to update the space usage information to the centralized management service, and then sent to each distributed storage service.

7. The method for improving the accuracy of space quota limits in a parallel file system as described in claim 6, characterized in that, The preset limit L is calculated using the following formula: L = Max − (Bmax × Duration) Among them, Max is the quota limit for the managed object, Bmax is the preset maximum bandwidth supported by the parallel file system, and Duration is the preset time period for the managed object to update the space usage of the centralized management service.

8. The method for improving the accuracy of space quota limits in a parallel file system as described in claim 6, characterized in that, Each distributed storage service controls the write operations of the data to be written to the managed objects based on the adjusted data write bandwidth, including: Each distributed storage service controls the speed at which the file system client sends the data to be written for the managed object to each distributed storage service, based on the current modified maximum write bandwidth corresponding to the managed object received from the centralized management service.

9. A computer-readable storage medium having stored thereon computer-executable instructions, wherein, When the computer-executable instructions are executed by the processor, the processor causes the processor to perform the method as described in any one of claims 1 to 8.

10. A calculator device, wherein, include: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method as described in any one of claims 1 to 8.

Images