Directory quota management method, storage medium and computer program product

Abstract

The embodiment of the invention discloses a directory quota management method, a storage medium and a computer program product. The method comprises the steps of generating corresponding quota updating information for a target directory of a to-be-processed file meeting a first condition; the target directory is a directory included between a parent directory and a root directory of the to-be-processed file; sending a first request to a server, so that the server updates directory information for setting quotas based on quota updating information carried in the first request; the first request represents a quota updating request. Through the scheme of the embodiment of the invention, the number of layers of step-by-step recursion from the parent directory to the root directory caused by data updating is reduced, the problems of complex calculation of centerless distributed file quota updating and access performance degradation caused by deep-level directory recursion updating are solved, the access pressure on a server side is reduced, and the access performance of a file system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud storage, and particularly relates to a directory quota management method, a storage medium and a computer program product. BACKGROUND

[0002] With the continuous development of the field of cloud storage, the scale of data grows exponentially, and single-machine storage has been difficult to meet the growing storage demand. A distributed file system can effectively solve the storage and management problems of data. In order to realize effective allocation and utilization of resources, corresponding permission settings and quota management need to be performed on different file system directories.

[0003] In related technologies, the directory quota management method in the distributed file system mainly adopts a centralized service management method and a centerless recursive update method. The centralized service is simple and efficient, but needs to separately configure a management service and a data storage service to manage the directory quota data, which will lead to additional resource overhead and increased cluster cost. The centerless recursive update method can reduce the additional resource overhead and cluster cost to a certain extent, but since each quota update request needs to synchronously complete the update of all directory information from the parent directory to the root directory, the file system access performance is prone to large attenuation. SUMMARY

[0004] To solve the technical problems in the related art, the present application provides a directory quota management method, a storage medium and a computer program product.

[0005] To achieve the above-mentioned purposes, the technical solutions of the embodiments of the present application are as follows: In a first aspect, the embodiments of the present application provide a directory quota management method applied to a client, and the method comprises the following steps: For a target directory of a to-be-processed file meeting a first condition, corresponding quota update information is generated; the target directory is a directory included between the parent directory of the to-be-processed file and the root directory; A first request is sent to a server, so that the server updates directory information for setting a quota based on quota update information carried in the first request; the first request represents a quota update request.

[0006] In a second aspect, the embodiments of the present application further provide another directory quota management method applied to a server, and the method comprises the following steps: A first request sent by a client is received; the first request represents a quota update request; Directory information for setting a quota is updated based on quota update information carried in the first request; The quota update information is generated for a target directory of the client for a to-be-processed file meeting a first condition; and the target directory is a directory included between a parent directory of the to-be-processed file and a root directory.

[0007] In a third aspect, an embodiment of the present application further provides a storage medium having a computer program stored thereon, where the computer program, when executed by a processor, implements steps of the client-side directory quota management method according to the embodiment of the present application, or implements steps of the server-side directory quota management method according to the embodiment of the present application.

[0008] In a fourth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program, where the computer program, when executed by a processor, implements steps of the client-side directory quota management method according to the embodiment of the present application, or implements steps of the server-side directory quota management method according to the embodiment of the present application.

[0009] The directory quota management method, the storage medium and the computer program product provided by the embodiment of the present application generate corresponding quota update information for a target directory of a to-be-processed file meeting a first condition; the target directory is a directory included between a parent directory of the to-be-processed file and a root directory; a first request is sent to a server, so that the server updates directory information with a set quota based on quota update information carried in the first request; and the first request represents a quota update request. By using the technical solution of the embodiment of the present application, quota update information is generated at the client side, that is, corresponding quota update information is generated only for a target directory meeting a first condition, and then a first request carrying the quota update information is sent to the server. The server only updates directory information with a set quota in the processing process, that is, the server does not update directory information without a set quota. In this way, by optimizing the recursive update level, that is, reducing the number of levels of recursive update from a parent directory to a root directory caused by data update, the problem of complex update calculation of a distributed file quota without a center and the problem of performance degradation caused by recursive update of a deep level directory are solved, the access pressure on the server is reduced, and the access performance of the file system is improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 Flowchart of the directory quota management method according to the embodiment of the present application Figure One ; Figure 2 Flowchart of quota update request merging according to the embodiment of the present application Figure 3 Flowchart of the directory quota management method according to the embodiment of the present application Figure Two ; Figure 4Flowchart of the directory quota management method of the embodiment of the present application Figure Three ; Figure 5 Flowchart of the directory quota management method of the embodiment of the present application Figure Four ; Figure 6 Flowchart of the quota update information generation of the embodiment of the present application Figure 7 Composition structure of the directory quota management device of the embodiment of the present application Figure One ; Figure 8 Composition structure of the directory quota management device of the embodiment of the present application Figure Two ; Figure 9 Hardware composition structure of the client of the embodiment of the present application Figure 10 Hardware composition structure of the server of the embodiment of the present application Figure 11 Composition structure of the directory quota management system of the embodiment of the present application. DETAILED DESCRIPTION

[0011] The present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application.

[0013] With the continuous development of the field of cloud storage, especially the continuous development of the field of public cloud and large private cloud, the data scale grows exponentially, and single machine storage has been difficult to meet the growing storage demand. Distributed storage architecture (i.e. distributed file system) has been widely used in the field of cloud computing, cloud storage, etc. due to its strong scalability, outstanding performance, and perfect function, which can provide petabyte (PB, Petabyte) or even exabyte (EB, Exabyte) level storage capacity in a global unified namespace after resource expansion. In order to realize effective allocation and utilization of resources under a single namespace, differentiated permission settings and quota management of directories of different user groups and users are required, which requires corresponding permission settings and quota management of different file system directories on the file system.

[0014] At present, the directory quota management method of the distributed file system in the related art mainly includes the following types: (1) Centralized service management: mainly by using centralized data service, setting quota for specified directory of file system and storing in dictionary form, when data in file system directory is modified, directly retrieving and updating quota information of directory through dictionary.

[0015] (2) Decentralized recursive update: using decentralized architecture, since each storage node in the cluster is independent and equal in role, file system directory quota management can only be managed independently. When the client side request needs to modify the data under the specified directory, the corresponding directory quota needs to be updated on the server side. The update method is recursive update layer by layer. Specifically, when modifying file or directory data, update the parent directory information layer by layer after updating the own information, until the root directory of the file system is updated.

[0016] However, the above two directory quota management methods in the related art have the following disadvantages: (1) Centralized service is simple and efficient, but requires separate configuration of management service and data storage service to manage directory quota data, which will result in additional resource overhead and increased cluster cost; (2) Decentralized recursive update can reduce additional resource overhead and cluster cost to some extent, but since each quota update request needs to synchronize all directory information updates from its parent directory to the root directory, this quota update method is prone to cause significant degradation of file system access performance.

[0017] Therefore, the problems of complex decentralized distributed file quota update calculation and deep level directory recursive update leading to access performance degradation in the related art are urgent problems to be solved.

[0018] Based on this, the embodiments of the present application propose a directory quota management method. In various embodiments of the present application, the client side generates quota update information in a targeted manner, that is, only generates corresponding quota update information for target directories that meet the first condition, and then sends a first request carrying the quota update information to the server. The server only updates the directory information for which the quota is set during processing, that is, the server does not update the directory information for which the quota is not set. In this way, by optimizing the recursive update level, that is, reducing the number of layers of recursive update from parent directory to root directory caused by data update, the problems of complex decentralized distributed file quota update calculation and deep level directory recursive update leading to access performance degradation are solved, the access pressure on the server is reduced, and the access performance of the file system is improved.

[0019] The embodiments of the present application provide a directory quota management method, which is applied to a client, Figure 1 Flowchart of the directory quota management method of the embodiments of the present application Figure One For example,Figure 1 As shown, the method comprises: Step 101: generating quota update information corresponding to a target directory of a to-be-processed file meeting a first condition.

[0020] Here, the target directory is a directory included between the parent directory of the to-be-processed file to the root directory. The to-be-processed file can be a distributed file in the field of cloud storage, and accordingly, the directory quota management method proposed in the embodiments of the present application can be applied to a centerless distributed file storage system.

[0021] It should be noted that in the embodiments of the present application, the directory quota management method is based on the centerless recursive update idea in the related art, adopts a directional update strategy, and further optimizes the recursive update level, that is, deeply optimizes the access performance of the centerless distributed file storage under a single namespace, and greatly improves the access performance of the file system.

[0022] In actual application, the client needs to determine the target directory of the to-be-processed file meeting the first condition before generating the quota update information corresponding to the target directory of the to-be-processed file meeting the first condition.

[0023] Based on this, in an embodiment, the method further comprises: When there is a directory with quota setting between the parent directory of the to-be-processed file to the root directory, the directory with quota setting is determined as the target directory meeting the first condition.

[0024] Here, the first condition includes that there is quota setting between the parent directory of the to-be-processed file to the root directory. The embodiments of the present application adopt a directional generation of quota update information corresponding to the target directory on the client side, that is, only generate the corresponding quota update information for the directory (i.e. the target directory) with quota setting between the parent directory of the to-be-processed file to the root directory, and do not perform the generation operation of the quota update information for the directory without quota setting between the parent directory of the to-be-processed file to the root directory. In this way, the number of layers of the recursive level from the parent directory to the root directory can be reduced, so as to reduce the centerless distributed file quota update calculation complexity.

[0025] Here, the quota update information contains all quota directory information between the parent directory and the root directory of the to-be-processed file, a quota type, a related operation type, and an update size, etc. The all quota directory information between the parent directory and the root directory of the to-be-processed file can be represented in the form of a quota directory identification (ID). The quota type includes a to-be-processed file or directory quantity and a to-be-processed file or directory capacity size. The related operation type can be understood as a modification type of the current operation, for example, increasing or decreasing a directory quota. The update size can be understood as a directory quota capacity change size caused by the current operation.

[0026] In actual application, the quota update information is generated according to information carried in the client operation request in response to the client operation request.

[0027] Based on this, in an embodiment, the generating of the corresponding quota update information includes: In response to a second request, quota update information corresponding to each target directory is generated by querying the second request. The second request represents a client operation request.

[0028] Here, the information carried in the client operation request can include client information, full path information of a to-be-processed file or directory, a data block size of the current operation, a file or directory number size of the current operation, a related operation type, etc. It should be noted that the client can determine a directory quota capacity change size caused by the current operation, that is, an update size, based on a change of the data block size of the current operation. The client can also determine the directory quota capacity change size caused by the current operation, that is, the update size, based on the file or directory number size of the current operation. The embodiments of the present application do not limit the way of determining the update size.

[0029] In actual application, in an embodiment, after the quota update information corresponding to each target directory is generated by querying the second request, the method further includes storing the quota update information corresponding to each target directory into an information structure.

[0030] Here, the information structure can also be referred to as a message structure, or can also be referred to as a quota structure message, or can also be referred to as a quota structure information. The embodiments of the present application do not limit the name of the information structure. By defining the information structure, the quota update information corresponding to multiple associated target directories can be organized together, so that these quota update information can be more conveniently managed and operated.

[0031] Here, the client obtains the operation type (i.e., the operation change type) of the requested operation by querying the second request (i.e., the client operation request), generates update flag information based on the operation change type, and fills the update flag information into the information structure. The client obtains the data block size of the current operation or the file or directory number size of the current operation by querying the second request, determines the directory quota capacity change size caused by the current operation based on the data block size of the current operation or the file or directory number size of the current operation, and fills the directory quota capacity change size into the information structure.

[0032] Step 102: sending a first request to the server to update the directory information with a quota based on quota update information carried in the first request.

[0033] In the embodiment of the present application, the first request represents a quota update request. Here, the update of the directory information with a quota by the server can also be understood as that the server does not update the directory information without a quota, that is, the server skips the directory without a quota in the processing process and does not perform quota update on it.

[0034] In actual application, when the client initiates multiple first requests at the same time, in order to improve processing efficiency, the client side can merge multiple concurrent requests of the same type, and then send the aggregated request after merging to the server side for processing.

[0035] Based on this, in an embodiment, the number of first requests is multiple; the method further includes: merging multiple concurrent first requests to obtain one or more aggregated requests; Correspondingly, the sending of the first request to the server to update the directory information with a quota based on quota update information carried in the first request includes: sending the one or more aggregated requests to the server to update the directory information with a quota based on quota update information carried in the one or more aggregated requests.

[0036] Here, the aggregated request can be understood as a request obtained by merging multiple concurrent requests of the same type. In the embodiment of the present application, the client can carry the corresponding quota update information generated for the target directory of the to-be-processed file satisfying the first condition in the aggregated request, and send the aggregated request to the server.

[0037] In actual application, thread pool technology can be introduced to optimize the path of the related requests on the client side, that is, the original quota update requests are sequentially sent to the server side to be optimized to combine similar requests, so as to reduce the access pressure on the server side and improve the access performance of the file system.

[0038] Based on this, in an embodiment, the combining the plurality of concurrent first requests to obtain one or more aggregated requests comprises: sending the plurality of concurrent first requests to a request pool; combining the plurality of concurrent first requests in the request pool by using a request combining strategy to obtain the one or more aggregated requests.

[0039] Figure 2 A flowchart of the quota update request combining of the embodiments of the present application is shown in FIG. 1. Figure 2 As shown in FIG. 1, a client concurrently initiates a large number of quota update requests, such as request 1, request 2, …, request n, where n is a positive integer greater than 1. These quota update requests are sent to a request pool for management. In the request pool, the request 1, request 2, …, request n are classified and combined in the form of a queue to generate one or more aggregated requests, such as aggregated request 1, aggregated request 2, …, aggregated request m, where m is a positive integer greater than 1 and m is less than or equal to n.

[0040] Here, the request combining strategy includes a file or directory dimension combining strategy and a request type dimension combining strategy. The process of combining the plurality of concurrent quota update requests by using the request combining strategy is described below.

[0041] In an embodiment, the combining the plurality of concurrent first requests by using the request combining strategy to obtain the one or more aggregated requests comprises one of the following: when the plurality of concurrent first requests meet a second condition, combining the plurality of concurrent first requests by using the target directory or file name of the request to obtain the one or more aggregated requests; classifying the plurality of concurrent first requests by using the request type to obtain requests of each type that update the same quota directory, and combining the requests of each type that update the same quota directory respectively to obtain the corresponding one or more aggregated requests.

[0042] Here, the second condition includes that the directories or files of the plurality of request operations are the same directory or file. Correspondingly, the plurality of concurrent first requests meeting the second condition can be that the plurality of requests simultaneously operate on the same directory or file.

[0043] Here, when the request merging strategy includes a file or directory dimension merging strategy, for the case that multiple requests operate on the same directory or file at the same time, the client can merge multiple concurrent first requests based on the file or directory dimension merging strategy, specifically, the multiple concurrent first requests are merged by using the target directory or file name of the requests to obtain one or more aggregated requests; when the request merging strategy includes a request type dimension merging strategy, the client can merge multiple concurrent first requests based on the request type dimension merging strategy, specifically, the client can first classify the multiple concurrent first requests to obtain requests of each type of updating the same quota directory, and then merge the requests of each type of updating the same quota directory to obtain one or more corresponding aggregated requests, that is, the embodiment of the application merges only the requests of updating the same quota directory based on the request type dimension, and does not merge the requests of updating non-same quota directories, for example, if two requests (such as request 1 and request 2) request to update different quota directories, the client will not merge request 1 and request 2.

[0044] Here, the client adopts a pooling technology to manage the related requests by path optimization, that is, the client aggregates the request types that cause similar changes in the quota in the request pool due to the current client-side operation request, specifically, the client can merge the request types that cause similar changes in the request pool in the form of a message queue to obtain one or more aggregated requests, and then sends the one or more aggregated requests to the server side (i.e., the server) for processing, thereby reducing the access pressure on the server side and improving the access performance of the file system.

[0045] Correspondingly, the embodiment of the application also provides another directory quota management method, which is applied to a server, Figure 3 The flow of the directory quota management method of the embodiment of the application is shown in Figure Two As shown in the figure, Figure 3 The method comprises the following steps: Step 301: receiving a first request sent by a client.

[0046] In the embodiment of the application, the first request represents a quota update request.

[0047] Here, in actual application, when the client initiates multiple first requests at the same time, in order to improve the processing efficiency, the client side can merge multiple concurrent requests of the same type, and then send the aggregated request obtained after merging to the server side for processing, at this time, the server receiving the first request sent by the client can be the aggregated request obtained by merging multiple concurrent requests of the same type.

[0048] Step 302: updating the directory information with the set quota based on the quota update information carried in the first request.

[0049] In the embodiment of the present application, the quota update information is generated by the client for a target directory of the to-be-processed file meeting the first condition; and the target directory is a directory included between the parent directory of the to-be-processed file and the root directory.

[0050] Here, the to-be-processed file can be a distributed file in the field of cloud storage, and accordingly, the directory quota management method proposed in the embodiment of the present application can be applied to a centerless distributed file storage system.

[0051] It should be noted that, in the embodiment of the present application, the directory quota management method is based on the centerless recursive update idea in the related art, adopts a directional update strategy, and further optimizes the recursive update level, that is, deeply optimizes the access performance of the centerless distributed file storage under a single namespace, and greatly improves the access performance of the file system.

[0052] Here, the first condition includes that there is quota setting between the parent directory of the to-be-processed file and the root directory, and accordingly, the target directory meeting the first condition is the directory with quota setting when there is quota setting between the parent directory of the to-be-processed file and the root directory.

[0053] Here, the quota update information contains all quota directory information between the parent directory of the to-be-processed file and the root directory, quota types, related operation types, and update sizes, etc., wherein the all quota directory information between the parent directory of the to-be-processed file and the root directory can be represented in the form of quota directory ID; the quota types include the number of to-be-processed files or directories and the size of the capacity of to-be-processed files or directories; the related operation type can be understood as the modification type of the current operation, such as increasing or decreasing the directory quota; and the update size can be understood as the size of the change in the capacity of the directory quota caused by the current operation.

[0054] In actual application, the server also needs to verify the quota update information before completing the quota update, and specifically, the verification can be realized by verifying the update flag information.

[0055] Based on this, in an embodiment, before the updating of the directory information with the set quota based on the quota update information carried in the first request, the method further includes: generating update flag information based on the first request; verifying the accuracy of the update flag information to obtain a verification result.

[0056] Here, the first request can also carry an operation change type. After receiving the first request, the server obtains the operation change type by parsing the first request, generates update flag information based on the operation change type, and checks the accuracy of the update flag information. Only when the accuracy check of the update flag information passes, the server performs quota update. If the accuracy check of the update flag information does not pass, the server returns error information to the client.

[0057] Based on this, in an embodiment, the updating of the directory information for which the quota is set based on the quota update information carried in the first request comprises: When the check result indicates that the accuracy check of the update flag information passes, the directory information for which the quota is set is updated based on the quota update information carried in the first request.

[0058] Here, when the check result indicates that the accuracy check of the update flag information passes, the quota update information such as the quota type, the related operation type, and the update size carried in the first request is obtained by parsing the first request, and the directory information for which the quota is set is updated based on the quota update information. Here, the updating of the directory information for which the quota is set by the server can also be understood as that the server does not update the directory information for which the quota is not set, that is, the server skips the directory for which the quota is not set in the processing process and does not perform quota update.

[0059] It should be noted that before updating the quota directory information to be updated, the server can also lock the quota directory to be updated, that is, lock, then update the quota of the directory information, release the lock of the quota directory to be updated after the update is completed, and finally return the quota update result message to the client.

[0060] The present application also provides another directory quota management method, which is a client-server interaction method, Figure 4 The flowchart of the directory quota management method of the present application embodiment is shown in Figure Three As shown in Figure 4 The method comprises: Step 401: The client generates corresponding quota update information for the target directory of the to-be-processed file meeting the first condition.

[0061] In the present application embodiment, the target directory is a directory included between the parent directory of the to-be-processed file and the root directory. The first condition includes that there is quota setting between the parent directory of the to-be-processed file and the root directory, and correspondingly, the target directory meeting the first condition is the directory with quota setting when there is quota setting between the parent directory of the to-be-processed file and the root directory.

[0062] Step 402: the client sends a first request to the server.

[0063] In the embodiment of the present application, the first request represents a quota update request.

[0064] Step 403: the server receives the first request sent by the client.

[0065] Step 404: the server updates the directory information with set quota based on the quota update information carried in the first request.

[0066] It should be noted that the specific processing process of the client and the server for directory quota management has been described in detail above, and will not be repeated here.

[0067] The technical scheme of the embodiment of the present application adopts the method of directional generation of quota update information on the client side, that is, only the target directory that meets the first condition generates the corresponding quota update information, and then sends the first request carrying the quota update information to the server. The server only updates the directory information with set quota in the processing process, that is, the server does not update the directory information without set quota. In this way, by optimizing the recursive update level, that is, reducing the number of layers of recursive update from parent directory to root directory caused by data update, the problem of complex calculation of non-central distributed file quota update and performance degradation caused by deep level directory recursive update is solved, the access pressure on the server is reduced, and the access performance of the file system is improved.

[0068] The present application will be described below in conjunction with application examples.

[0069] In view of the problem of complex calculation of non-central distributed file quota update and performance degradation caused by deep level directory recursive update in the related art, the present application proposes a directory quota management optimization method for non-central distributed file storage system. The method is based on the idea of non-central recursive update and further optimizes the recursive update level to solve the problem of performance degradation caused by deep level directory recursive update.

[0070] The core technical idea of the present application is as follows: in order to reduce the number of layers of recursive update from parent directory to root directory caused by data update, the present application adopts the method of directional generation of corresponding quota update information, that is, the relevant quota update information is generated for the directories with set quota (corresponding to the target directory described above) from parent directory to root directory, and the plurality of concurrent quota update requests (corresponding to the first request described above) are merged to improve the processing efficiency, and then the quota update information is verified, and finally the quota update is completed when the verification is passed.

[0071] From the overall technical idea, the present application is very different from other traditional solutions: 1) the traditional solution adopts to sink all update requests to the storage end (i.e. server end) for processing, and calculates the update type and data size change in the storage end; 2) it completes the update one by one from the parent directory to the root directory in the form of a single queue. While the present application adopts: 1) calculates and generates quota change information (i.e. quota update information) on the client side, and merges multiple concurrent modification requests of the same type (corresponding to the first request) before sending them to the storage end for processing; 2) the storage end only updates the directory information with quota during processing, and skips the directories without quota. Therefore, the quota management method proposed in the present application can significantly improve the access performance of the file system in the field of centerless distributed file storage system.

[0072] The technical solution of the present application mainly includes two parts: quota update information generation and update analysis, and the main process is as shown in Figure 5 The directory quota management method of the present application will be described in detail as follows: Figure 5 I. Generation of quota update information specific process: 1. Respond to the client operation request (corresponding to the second request), which can carry the client information, the full path information of the file or directory, the data size information (such as the data block size of the current operation, the file or directory size of the current operation) and the related operation type, etc.

[0073] 2. Quota update information generation, i.e. generating the update information of the directories with quota settings between the parent directory and the root directory, and storing the quota update information of each target directory in the information structure. The information structure can include all directory information with quota settings between the parent directory and the root directory, the quota type, the directory quota capacity change size caused by the current operation, and the request ID, which guarantees global uniqueness. In addition, the information structure can also include the final data after merging the current multiple concurrent requests (corresponding to one or more aggregated requests).

[0074] Therefore, the present application is essentially different from the traditional directory quota update solution: the traditional solution adopts to update all directories from the parent directory to the root directory one by one from bottom to top, without distinguishing whether the directories have quota or not. While the present application adopts the accurate update strategy, i.e. only records and updates the directory information with quota between the parent directory and the root directory, and skips the directories without quota, thereby improving the access performance of the file system.

[0075] The quota update information generation step and the generation information pseudo code example are as shown in Figure 6 The detailed steps of generating the quota update information are as follows: ​2.1, in response to a client operation request, the client operation request is parsed to obtain the quota directory ID and quota type information carried in the request between the parent directory and the root directory of the file to be operated, the quota update information is generated according to the information carried in the query, and all the quota directories between the parent directory and the root directory are saved as a data linked list and stored in the quota structure message (corresponding to the information structure mentioned above); 2.2, the operation change type (i.e. the modification type of the current operation, such as increase or decrease) carried in the request is obtained, the update flag information is generated, and the update flag information is filled into the information structure generated in step 2.1; 2.3, the data block size of the current operation and the file / directory size of the current operation carried in the request are obtained, the capacity change size (corresponding to the directory quota capacity change size mentioned above) is calculated, and the capacity change size is filled into the information structure generated in step 2.1; 2.4, if the quota update information generated by the current other request is similar, the message merging and delivery processing is performed, and the update request (i.e. the quota update request) merging process is as shown in Figure 2 This step is managed through the request pool, and the request types that cause similar changes in the target quota due to the current client-side operation request are aggregated in the request pool.

[0076] Specifically, as shown in Figure 2 A large number of concurrent requests (e.g. request 1 to request n) of the client arrive at the request pool, and the requests are classified and merged in the form of a queue in the request pool to generate aggregated requests (e.g. aggregated request 1 to aggregated request m, where m≤n). The request merging strategy can be in the file or directory dimension, or in the request type dimension, specifically: 1. file or directory dimension: when multiple requests simultaneously operate on the same directory or file, the target directory or file name is used to filter and merge them; 2. request type dimension: requests updating the same quota directory are merged and processed. This application adopts the pooling technology to merge similar changes in the form of a message queue, and then sends it to the storage end for processing. This method has a significant promoting effect on the access performance optimization of the distributed file storage system.

[0077] II. Analysis and update quota process: (1) verify the accuracy of the message (corresponding to the verification of the accuracy of the update flag information mentioned above), which is specifically divided into verifying whether the quota directory exists, whether the pre-update quota information is correct, and whether the update size meets the quota requirement. If the verification is passed, the quota update is operated; if the verification is not passed, the error information is returned to the client; (2) Parse the quota update information and perform quota update operations to obtain the quota type, related operation type and update size carried in the quota update information; lock the quota directory to be updated (i.e. lock), then update the quota of the directory with the set quota (corresponding to the aforementioned target directory), release the lock after the update is completed, and finally return the quota update result message to the client.

[0078] This application optimizes directory quota management in a decentralized distributed file storage system based on the concept of centralized service quota precise management. It solves the problems of computational complexity in quota updates and computational pressure caused by deep-level directory recursive updates, improving computational efficiency. It also reduces the number of input / output (IO) operations, decreasing the access pressure on the server and thus improving file system access performance. Furthermore, this application introduces thread pool technology. By optimizing the path of related requests on the client side, it transforms the previous method of pushing quota update requests down to the server one by one into merging similar requests, reducing the access pressure on the server and improving file storage access performance.

[0079] Compared with the solutions of related technologies, the solution of this application has the following advantages: 1) The traditional method of updating from the parent directory to the root directory will cause a great loss of access performance as the directory hierarchy increases. However, this application uses a specified quota directory for targeted updates, which greatly optimizes access performance.

[0080] 2) Traditional solutions use a separate update processing method for each quota modification request, which limits the client's concurrent performance. However, this application merges concurrent requests through a request pool, which greatly improves the client's high-concurrency access performance. In summary, the performance optimization of this application has a great advantage for the access performance of decentralized distributed file storage systems.

[0081] This application, based on the existing quota update method for decentralized distributed file storage, incorporates the concept of centralized services and deeply optimizes the number of I / O operations for quota updates. By adopting the approach of targeted updates and request aggregation, it deeply optimizes the access performance of decentralized distributed file storage under a single namespace, greatly improving the access performance of the file system. It can be promoted as a target quota update scheme for distributed file storage systems and is applicable to decentralized distributed file storage systems, providing valuable reference for practical systems.

[0082] To implement the client-side directory quota management method of this application embodiment, this application embodiment also provides a directory quota management device, which is installed on the client. Figure 7 This is a schematic diagram of the composition structure of the catalog quota management device according to an embodiment of this application. Figure OneAs shown in Figure 7 the device comprises: a first generation unit 71 configured to generate quota update information corresponding to a target directory of a to-be-processed file meeting a first condition; the target directory is a directory included between a parent directory of the to-be-processed file and a root directory; a sending unit 72 configured to send a first request to a server, so that the server updates directory information with a quota based on quota update information carried in the first request; the first request represents a quota update request.

[0083] In an embodiment, the device further comprises a determination unit; wherein, the determination unit is configured to determine a directory with a quota set between the parent directory of the to-be-processed file and the root directory as the target directory meeting the first condition.

[0084] In an embodiment, the first generation unit 71 is specifically configured to: generate quota update information corresponding to each target directory in response to a second request; the second request represents a client operation request; Correspondingly, the device further comprises a storage unit; wherein, the storage unit is configured to store the quota update information corresponding to each target directory into an information structure.

[0085] In an embodiment, the number of the first requests is multiple; the device further comprises a merging unit; wherein, the merging unit is configured to merge multiple concurrent first requests to obtain one or more aggregated requests; Correspondingly, the sending unit 72 is specifically configured to: send the one or more aggregated requests to the server, so that the server updates directory information with a quota based on quota update information carried in the one or more aggregated requests.

[0086] In an embodiment, the merging unit is specifically configured to: send the multiple concurrent first requests to a request pool; In the request pool, the multiple concurrent first requests are merged by using a request merging strategy to obtain the one or more aggregated requests.

[0087] In another embodiment, the merging unit is further specifically configured to perform one of the following: When the multiple concurrent first requests meet the second condition, the multiple concurrent first requests are merged using the target directory or file name of the request to obtain the one or more aggregated requests; The multiple concurrent first requests are classified according to their request types to obtain requests for updating the same quota directory in each category; the requests for updating the same quota directory in each category are merged to obtain the corresponding one or more aggregated requests.

[0088] In practical applications, the first generation unit 71 can be implemented by the processor in the catalog quota management device, and the sending unit 72 can be implemented by the communication interface in the catalog quota management device.

[0089] To implement the server-side directory quota management method of this application embodiment, this application embodiment also provides another directory quota management device, which is installed on the server. Figure 8 This is a schematic diagram of the composition structure of the catalog quota management device according to an embodiment of this application. Figure Two ,like Figure 8 As shown, the device includes: Receiving unit 81 is used to receive a first request sent by the client; the first request represents a quota update request; Management unit 82 is used to update the quota setting directory information based on the quota update information carried in the first request; The quota update information is generated by the client for the target directory of the file to be processed that meets the first condition; the target directory is the directory between the parent directory and the root directory of the file to be processed.

[0090] In one embodiment, the apparatus further includes: a second generation unit and a verification unit; wherein, The second generation unit is used to generate update flag information based on the first request; The verification unit is used to verify the accuracy of the updated flag information and obtain a verification result; Accordingly, the management unit 82 is specifically used for: When the verification result indicates that the accuracy of the updated flag information has been verified, the quota setting directory information is updated based on the quota update information carried in the first request.

[0091] In practical applications, the receiving unit 81 can be implemented by the communication interface in the catalog quota management device, and the management unit 82 can be implemented by the processor in the catalog quota management device.

[0092] It should be noted that the directory quota management apparatus provided in the above embodiments is only used for example to illustrate the division of the above program modules when performing directory quota management, and in actual application, the above processing can be completed by different program modules according to needs, that is, the internal structure of the apparatus is divided into different program modules to complete all or part of the above-described processing. In addition, the directory quota management apparatus and the directory quota management method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0093] Based on the hardware implementation of the above program modules, and in order to realize the client-side directory quota management method of the embodiments of the present application, the embodiments of the present application also provide a client, Figure 9 The schematic diagram of the hardware composition structure of the client of the embodiments of the present application is shown in Figure 9 The client 90 includes: The first communication interface 91 can interact with the server to exchange information. The first processor 92 is connected with the first communication interface 91 to realize information interaction with the server, and is used to run the computer program to execute the method provided by one or more technical solutions of the above client side, and the computer program is stored on the first memory 93.

[0094] It should be noted that the specific processing process of the first communication interface 91 and the first processor 92 can be understood with reference to the above client-side directory quota management method.

[0095] Of course, in actual application, each component in the client 90 is coupled together through the first bus system 94. It can be understood that the first bus system 94 is used to realize the connection and communication between the components. The first bus system 94 includes not only a data bus, but also a power bus, a control bus and a state signal bus. However, in order to clearly illustrate, all kinds of buses are marked as the first bus system 94 in Figure 9 .

[0096] The first memory 93 in the embodiments of the present application is used to store various types of data to support the operation of the client 90. Examples of these data include: any computer program used for operation on the client 90.

[0097] The client-side directory quota management method disclosed in the embodiments of the present application can be applied to or implemented by the first processor 92. The first processor 92 can be an integrated circuit chip having a signal processing capability. In implementation, each step of the client-side directory quota management method can be completed by integrated logic circuits of hardware or instructions in the form of software in the first processor 92. The first processor 92 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 92 can implement or execute each client-side directory quota management method, step and logic block diagram disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the client-side directory quota management method disclosed in the embodiments of the present application, the steps can be directly embodied as hardware decoding processor execution completion or execution completion by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the first memory 93, and the first processor 92 reads information in the first memory 93 to complete the steps of the aforementioned client-side directory quota management method in combination with hardware.

[0098] In exemplary embodiments, the client 90 can be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, micro controllers (MCUs), microprocessors (Microprocessors), or other electronic elements, for executing the aforementioned client-side directory quota management method.

[0099] Based on the hardware implementation of the program module and in order to implement the server-side directory quota management method of the embodiments of the present application, the embodiments of the present application further provide a server, Figure 10 A schematic diagram of a hardware structure of the server of the embodiments of the present application is shown in FIG. 10, which shows that the server 1000 includes: Figure 10 a second communication interface 1001 capable of information interaction with a client; ​The second processor 1002 is connected with the second communication interface 1001 to realize information interaction with the client, and is used to run a computer program to execute the method provided by one or more technical solutions on the server side, and the computer program is stored in the second memory 1003.

[0100] It should be noted that the specific processing process of the second communication interface 1001 and the second processor 1002 can be understood with reference to the directory quota management method on the server side.

[0101] Of course, in actual application, various components in the server 1000 are coupled together through the second bus system 1004. It can be understood that the second bus system 1004 is used to realize the connection and communication between the components. The second bus system 1004 includes not only a data bus, but also a power bus, a control bus and a state signal bus. However, in order to clearly illustrate, all kinds of buses are marked as the second bus system 1004 in the Figure 10

[0102] The second memory 1003 in the embodiment of the present application is used to store various types of data to support the operation of the server 1000. Examples of the data include any computer program used to operate on the server 1000.

[0103] The server-side directory quota management method disclosed in the above embodiment of the present application can be applied to the second processor 1002 or implemented by the second processor 1002. The second processor 1002 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned server-side directory quota management method can be completed by integrated logic circuits or instructions in the form of software in the second processor 1002. The second processor 1002 mentioned above can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The second processor 1002 can implement or execute the server-side directory quota management method, steps and logic block diagram disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the server-side directory quota management method disclosed in the embodiment of the present application, the hardware decoding processor can be directly embodied to execute the completion, or the hardware and software modules in the decoding processor are combined to execute the completion. The software module can be located in the storage medium, which is located in the second memory 1003, and the second processor 1002 reads the information in the second memory 1003, and combines the hardware to complete the steps of the above-mentioned server-side directory quota management method.

[0104] ​In an exemplary embodiment, the server 1000 can be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic elements for performing the aforementioned server-side catalog quota management method.

[0105] It can be understood that the memory (including the first memory 93 and the second memory 1003) of the embodiments of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM). The magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0106] To implement the directory quota management method of the embodiments of the present application, the embodiments of the present application further provide a directory quota management system, Figure 11 A schematic diagram of the component structure of the directory quota management system of the embodiments of the present application is shown in Figure 11 The system comprises a client 90 and a server 1000; wherein, The client 90 is configured to generate quota update information corresponding to a target directory of a to-be-processed file meeting a first condition; and send a first request to the server 1000; the first request represents a quota update request, and the target directory is a directory included between a parent directory of the to-be-processed file and a root directory. The server 1000 is configured to receive the first request sent by the client 90; and update directory information with a set quota based on the quota update information carried in the first request.

[0107] It should be noted that the specific processing process of the client 90 and the server 1000 for the directory quota management has been described in detail above, and will not be described here.

[0108] In exemplary embodiments, the embodiments of the present application further provide a storage medium, i.e., a computer storage medium, specifically a computer readable storage medium, for example, including a first memory 93 storing a computer program, the computer program being executable by a first processor 92 in the client 90 to complete the steps of the aforementioned client-side directory quota management method, or including a second memory 1003 storing a computer program, the computer program being executable by a second processor 1002 in the server 1000 to complete the steps of the aforementioned server-side directory quota management method. The computer readable storage medium can be FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0109] In exemplary embodiments, the embodiments of the present application further provide a computer program product comprising a computer program, the computer program being executable by a first processor 92 in the client 90 to complete the steps of the aforementioned client-side directory quota management method, or the computer program being executable by a second processor 1002 in the server 1000 to complete the steps of the aforementioned server-side directory quota management method.

[0110] It should be noted that "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0111] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0112] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A directory quota management method, characterized by, The method applied to a client comprises: Generating quota update information corresponding to a target directory of a to-be-processed file meeting a first condition; the target directory is a directory included between a parent directory of the to-be-processed file and a root directory; Sending a first request to a server to enable the server to update directory information with a quota based on quota update information carried in the first request; the first request represents a quota update request.

2. The method of claim 1, wherein, The method further comprises: When there is a directory with a quota set between the parent directory of the to-be-processed file and the root directory, determining the directory with the quota set as the target directory meeting the first condition.

3. The method of claim 1, wherein, The generating of the quota update information corresponding to each target directory comprises: In response to a second request, querying the second request to generate quota update information corresponding to each target directory; the second request represents a client operation request. The method further comprises: Storing the quota update information corresponding to each target directory into an information structure.

4. The method of claim 1, wherein, The number of the first requests is multiple; the method further comprises: Merging multiple concurrent first requests to obtain one or more aggregated requests; The sending of the first request to the server to enable the server to update directory information with a quota based on quota update information carried in the first request comprises: Sending the one or more aggregated requests to the server to enable the server to update directory information with a quota based on quota update information carried in the one or more aggregated requests.

5. The method of claim 4, wherein, The merging of the multiple concurrent first requests to obtain one or more aggregated requests comprises: Sending the multiple concurrent first requests to a request pool; In the request pool, merging the multiple concurrent first requests by using a request merging strategy to obtain the one or more aggregated requests.

6. The method of claim 5, wherein, The merging of the multiple concurrent first requests by using a request merging strategy to obtain the one or more aggregated requests comprises one of the following: When the multiple concurrent first requests meet a second condition, merging the multiple concurrent first requests by using a target directory or a file name of a request to obtain the one or more aggregated requests; Classifying the multiple concurrent first requests by using a request type to obtain requests of each type for updating the same quota directory; Merging the requests of each type for updating the same quota directory respectively to obtain the corresponding one or more aggregated requests.

7. A directory quota management method characterized by comprising: The method applied to a server comprises: Receiving a first request sent by a client; the first request represents a quota update request; Updating directory information with a quota based on quota update information carried in the first request; The quota update information is generated by the client for a target directory of a to-be-processed file meeting a first condition; the target directory is a directory included between a parent directory of the to-be-processed file and a root directory.

8. The method of claim 7, wherein, The method further comprises: Generating update flag information based on the first request; Verifying accuracy of the update flag information to obtain a verification result; The updating of the directory information with the quota based on the quota update information carried in the first request comprises: When the check result indicates that the accuracy of the update flag information is verified to be passed, the directory information with the quota is updated based on the quota update information carried in the first request.

9. A storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 8.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 8.

Images