Method and apparatus for counting quota information of directory in distributed file system

Abstract

The present disclosure relates to a method and device for quota information statistics of a directory in a distributed file system, the method comprising: in response to an operation record executed by the distributed file system, generating a target path of quota information to be changed based on a first-level directory in the operation record, the target path comprising all directories from the first-level directory affected by the execution of the operation record to the root directory in sequence, the operation record comprising the first-level directory, an information change amount, and an information change object, the operation record being generated based on a received operation request; obtaining quota information corresponding to each directory on the target path, the quota information comprising the number and size of subdirectories, the number and size of subfiles under the corresponding directory; and updating the quota information corresponding to each directory on the target path based on the operation record to obtain updated quota information corresponding to each directory on the target path. The present scheme ensures the accuracy of the quota information in each hierarchical directory.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of distributed storage, and particularly relates to a quota information statistical method and device for directories in a distributed file system. BACKGROUND

[0002] In a distributed file system, disk directory capacity quota can help administrators achieve the purpose of managing directory size. At present, in the related art, each time a client of a distributed file system performs an input / output (I / O) operation on the distributed file system, the client needs to send a capacity statistical request to the distributed file system. After receiving the request, the distributed file system traverses the capacity under the subordinate directories to obtain the current used capacity size and returns the capacity size to the client. Since the distributed file system needs to scan the capacity each time an I / O operation is performed, this has a great impact on the performance of the distributed file system. SUMMARY

[0003] In a first aspect, the present disclosure provides a quota information statistical method for directories in a distributed file system, comprising:

[0004] In response to an operation record executed by the distributed file system, a target path to be changed for quota information is generated based on a first-level directory in the operation record, the target path comprising all directories from the first-level directory affected by the execution of the operation record to the root directory in sequence, the operation record comprising the first-level directory, an information change amount, and an information change object, the operation record being generated based on a received operation request;

[0005] Quota information corresponding to each directory on the target path is obtained, the quota information comprising the total number and total size of subdirectories and subfiles under the corresponding directory;

[0006] Based on the operation record, the quota information corresponding to each directory on the target path is updated to obtain updated quota information corresponding to each directory on the target path.

[0007] In a second aspect, the present disclosure provides a quota information statistical device for directories in a distributed file system, comprising:

[0008] A scanning module is configured to, in response to an operation record executed by the distributed file system, generate a target path to be changed for quota information based on a first-level directory in the operation record, the target path comprising all directories from the first-level directory affected by the execution of the operation record to the root directory in sequence, the operation record comprising the first-level directory, an information change amount, and an information change object, the operation record being generated based on a received operation request;

[0009] obtain quota information corresponding to each directory on the target path, the quota information including total quantity and total size of subdirectories and subfiles under the corresponding directory;

[0010] update the quota information corresponding to each directory on the target path based on the operation record, to obtain updated quota information corresponding to each directory on the target path.

[0011] In a third aspect, the present disclosure provides a distributed file system, comprising: a processor and a memory; the processor is configured to execute any of the quota information statistical methods for directories in a distributed file system provided by the embodiments of the present disclosure by invoking programs or instructions stored in the memory.

[0012] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing programs or instructions, which cause a computer to execute any of the quota information statistical methods for directories in a distributed file system provided by the embodiments of the present disclosure.

[0013] In a fifth aspect, the present disclosure provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the quota information statistical method for directories in a distributed file system according to the first aspect.

[0014] The technical solutions provided by the embodiments of the present disclosure have at least the following advantages compared with the prior art:

[0015] In the embodiments of the present disclosure, in response to an operation record executed by a distributed file system, a target path of quota information to be changed is generated based on a first-level directory in the operation record, the target path including all directories from the first-level directory affected by the executed operation record to the root directory, the operation record including the first-level directory, an information change amount, and an information change object, the operation record being generated based on a received operation request, then quota information corresponding to each directory on the target path is obtained, the quota information including total quantity and total size of subdirectories and subfiles under the corresponding directory, and further, the quota information corresponding to each directory on the target path is updated based on the operation record, to obtain updated quota information corresponding to each directory on the target path. By using the above technical solutions, when the distributed file system is operated, the quota information of the first-level directory affected by the operation and all directories above the first-level directory is updated in real time from bottom to top, and the accuracy of the quota information in each level directory is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a method for calculating quota information of a directory in a distributed file system according to an embodiment of this disclosure;

[0019] Figure 2 A flowchart illustrating a method for calculating quota information of a directory in a distributed file system, as provided in another embodiment of this disclosure;

[0020] Figure 3 A flowchart illustrating a method for calculating quota information of a directory in a distributed file system, as provided in another embodiment of this disclosure;

[0021] Figure 4 This is a schematic diagram of the structure of a quota information statistics device for a directory in a distributed file system provided in an embodiment of the present disclosure. Detailed Implementation

[0022] To better understand the above-described objectives, features, and advantages of this disclosure, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the described embodiments are only some, not all, of the embodiments of this disclosure. The specific embodiments described herein are merely for explaining this disclosure and not for limiting it. Unless otherwise specified, the embodiments of this disclosure and the features within them can be combined with each other. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure are within the scope of protection of this disclosure.

[0023] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0024] The following detailed explanation, with reference to the accompanying drawings, illustrates the method and apparatus for calculating quota information for directories in the distributed file system provided in this disclosure.

[0025] Figure 1This is a flowchart illustrating a method for calculating quota information of directories in a distributed file system according to an embodiment of the present disclosure. This method can be executed by a quota information calculation device for directories in a distributed file system provided in this embodiment of the present disclosure. The quota information calculation device for directories in a distributed file system can be implemented in software and / or hardware and can be integrated into the distributed file system.

[0026] like Figure 1 As shown in the embodiments of this disclosure, the method for calculating quota information of directories in a distributed file system may include the following steps:

[0027] Step 101: In response to the distributed file system execution operation record, generate a target path for quota information to be changed based on the first-level directory in the operation record. The target path includes all directories from the first-level directory affected by the execution of the operation record upwards to the root directory. The operation record includes the first-level directory, the amount of information change, and the information change object. The operation record is generated based on the received operation request.

[0028] For example, an operation request could be a request to create a new directory (the operation command is mkdir, which is equivalent to makedirectories), a request to delete a file or directory (the operation command is remove), a request to move a file or directory (the operation command is mv), and so on.

[0029] In this embodiment of the disclosure, after receiving an operation request, the distributed file system generates an operation record corresponding to the operation request. The operation record may include the first-level directory affected by the execution of the operation record, the information change object, and the amount of information change. When the operation record is executed in the distributed file system, a target path for quota information to be changed is generated based on the first-level directory in the operation record. The target path includes all directories from the first-level directory affected by the execution of the operation request upwards to the root directory.

[0030] The first-level directory refers to the directory directly affected by the execution of the operation request. The first-level directory varies depending on the operation instructions in the operation request.

[0031] For example, suppose a path in a distributed file system is " / a / b / c / d", where 'a' is the root directory and 'd' is the last-level directory on that path. If the received operation request is to create a new directory under directory 'c', or to create a new file under directory 'c', then the first-level directory affected by the operation record generated based on this operation request is directory 'c', and the corresponding target path generated in response to this operation is " / a / b / c". If the received operation request is to delete directory 'c', then the first-level directory affected by the operation record generated based on this operation request is directory 'b', and the corresponding target path generated in response to this operation is " / a / b".

[0032] In this embodiment of the disclosure, the information change objects in the operation record include quantity and / or size, and the amount of information change refers to the number of changes in the information change objects involved in the above-mentioned operation request.

[0033] For example, if the operation request is to create a new directory, the information change object is the quantity, and the information change amount is "+1"; if the operation request is to add a file with a size of 1M, the information change objects are the quantity and size, where the information change amount for the quantity is "+1" and the information change amount for the size is "+1M"; if the operation request is to delete a directory, and the directory size information is 2M and the quantity information is 10, the information change objects are the quantity and size, where the information change amount for the quantity is "-10" and the information change amount for the size is "-2M".

[0034] For example, the distributed file system includes a scanning module and a consistency module. The scanning module interfaces with the business module. When the distributed file system receives an operation request, such as an `mkdir` or `mv` operation request, it generates a statistics task. Simultaneously, it calls the consistency module to generate a corresponding operation record.

[0035] Step 102: Obtain quota information for each directory on the target path. The quota information includes the total number and total size of subdirectories and subfiles under the corresponding directory.

[0036] In this embodiment of the disclosure, for a determined target path, quota information corresponding to each directory on the target path can be obtained. The quota information for a directory includes quantity information and size information. The quantity information refers to the total number of all subfiles and subdirectories contained in the directory, and the size information refers to the total size of all subfiles and subdirectories contained in the directory. It should be noted that in this embodiment of the disclosure, all subfiles and subdirectories contained in the directory refer to all files and directories contained sequentially from the current directory down to the last level directory.

[0037] For example, the distributed file system includes a statistics module and a caching module. The statistics module obtains the first-level directory of the target path and the configuration information data to be updated from the scanning module, and obtains the complete target path and its quota information from the caching module and the lock module, thereby performing the statistics task.

[0038] Step 103: Based on the operation record, update the quota information corresponding to each directory on the target path to obtain the updated quota information corresponding to each directory on the target path.

[0039] In this embodiment of the disclosure, after obtaining the quota information corresponding to each directory on the target path, the quota information corresponding to each directory on the target path can be updated based on the generated operation records to obtain the updated quota information corresponding to each directory on the target path.

[0040] For example, suppose the operation request is to create a new directory under directory dir1. The generated operation record will contain dir1, +1, and quantity information. Here, dir1 represents the first-level directory affected by the operation request, that is, the parent directory of the directory created based on the operation request. After obtaining the quota information corresponding to each directory on the target path, starting from the first-level directory (dir1) on the target path, the total quantity of the quota information corresponding to each directory on the target path is incremented by 1, while the total size remains unchanged, to obtain the new quota information corresponding to each directory on the target path.

[0041] For example, the distributed file system may also include a statistics module. After the aforementioned scanning module generates a statistics task, the statistics module can be called to execute the statistics task. Specifically, the statistics module obtains the last-level directory (i.e., the first-level directory) and its corresponding quota information on the target path from the scanning module. It then updates the quota information sequentially upwards from the first-level directory until the quota information of the file system root directory is updated, thus obtaining the new quota information corresponding to each directory on the target path.

[0042] Furthermore, after the statistics module updates the quota information of each directory on the target path, the consistency module persists the results and deletes the previously generated operation records to ensure the persistence of the latest quota information and the atomicity of deleting the operation records. This ensures the accuracy of the quota information of each level of directory even in special scenarios such as service restart.

[0043] The quota information statistics method for directories in a distributed file system according to this embodiment of the present disclosure generates a target path for quota information to be changed based on the first-level directories in the distributed file system execution operation record in response to the operation record. The target path includes all directories from the first-level directory affected by the execution operation record upwards to the root directory. The operation record includes the first-level directory, the amount of information change, and the information change object. The operation record is generated based on the received operation request. Then, the quota information corresponding to each directory on the target path is obtained. The quota information includes the number and size of subdirectories and the number and size of subfiles under the corresponding directory. Then, based on the operation record, the quota information corresponding to each directory on the target path is updated to obtain the updated quota information corresponding to each directory on the target path. By adopting the above technical solution, the quota information of the first-level directory affected by the operation and all its parent directories is updated in real time from bottom to top when operating on the distributed file system, ensuring the accuracy of the quota information in each level of directory.

[0044] In practical applications, updating quota information in directories may involve multiple levels, and directories may also update quota information concurrently. For example, a directory may need to process multiple quota-related operations simultaneously, or a bottom-up scanning task and a normal operation request may simultaneously operate on the same inode. Therefore, it is necessary to lock the target path of the information to be updated to handle some conflict scenarios. Thus, in one optional embodiment of this disclosure, the target path can be locked before updating the quota information corresponding to each directory on the target path to prevent the target path from being corrupted.

[0045] For example, a locking module can be set in the distributed file system. When the target path is generated from the first-level directory, the locking module can lock the target path to prevent concurrent changes to quota information and protect the target path from being corrupted by other distributed file system operations.

[0046] In this embodiment of the disclosure, the target path is locked before updating the quota information corresponding to each directory on the target path to prevent the target path from being damaged. This protects the target path and ensures the accuracy of the updated quota information.

[0047] In one optional embodiment of this disclosure, the operation request may be to create a first subdirectory under a first directory. In this case, the quota information statistics method for directories in the distributed file system provided by this disclosure may further include: generating first quota information corresponding to the first subdirectory, wherein the quantity information and size information in the first quota information are both 0.

[0048] For example, suppose the operation request is to create a subdirectory b under directory a. Directory a is the first directory mentioned earlier, and subdirectory b is the first subdirectory mentioned earlier. Since subdirectory b is newly created, it lacks corresponding quota information. Therefore, after creating subdirectory b, corresponding quota information, i.e., the first quota information, is generated for it. Because subdirectory b has no subfiles or subdirectories, the quantity information and size information in the first quota information are both 0.

[0049] The operation of generating the first quota information corresponding to the first subdirectory can be performed after updating the quota information corresponding to each directory on the target path, or before, or simultaneously. This disclosure does not limit the execution order of the operation of generating the first quota information corresponding to the first subdirectory.

[0050] In this embodiment of the disclosure, when the operation request is to create a first subdirectory under the first directory, the first quota information corresponding to the first subdirectory is also generated. This ensures that the newly created directory also has corresponding quota information, laying the foundation for accurate updating of the quota information of each directory in subsequent operations.

[0051] In one alternative embodiment of this disclosure, such as Figure 2 As shown, based on the foregoing embodiments, step 102 may include the following sub-steps:

[0052] Step 201: Obtain quota information corresponding to each directory on the target path from the cache area, wherein the cache area is used to store quota information corresponding to each directory in the distributed file system.

[0053] In this embodiment, the cache can cache quota information corresponding to each directory in the distributed file system. Therefore, after determining the target path, the quota information corresponding to each directory on the target path can be obtained from the cache. Compared to directly retrieving directory quota information from the hard drive, caching quota information in the cache and then retrieving the required quota information from the cache when needed improves the speed of quota information retrieval and thus improves the efficiency of quota information updates. Since the target path generated based on the received operation request in this embodiment will only be a directory, the quota information stored in the cache only includes the quota information corresponding to each directory, and not the quota information corresponding to files. This will greatly improve the cache hit rate.

[0054] Step 202: If the second quota information corresponding to the second directory on the target path does not exist in the cache area, load the second quota information from the hard disk, wherein the second directory is any directory on the target path.

[0055] The hard drive is used to persistently store all data in the distributed file system, including quota information for each directory in the distributed file system. The data on the hard drive will not be lost when the distributed file system restarts.

[0056] Because the data stored in the cache has a time limit, data that reaches its storage time limit will be deleted. Therefore, the cache may not store quota information for all directories. Furthermore, data in the cache may be lost due to a system restart. To address these issues, in this embodiment, when obtaining quota information for each directory on the target path, it first retrieves it from the cache. If the cache does not contain quota information for the second directory on the target path (referred to as second quota information for ease of distinction), the second quota information is loaded from the hard disk to ensure that quota information for each directory on the target path can be obtained.

[0057] It is understandable that the second directory can be any directory on the target path, and the number of second directories can be one or more. Correspondingly, the number of second quota information items obtained from the hard drive can be one or more.

[0058] For example, a caching module can be set up in a distributed file system to provide a cache area. The caching module provides the required quota information to the statistics module of the distributed file system. The caching module ensures that the quota information of all directories from the starting directory on the target path to the root directory of the file system exists. When the quota information required by the statistics module is not in the cache area, it can be loaded from the hard drive. Since one of the characteristics of this scheme is that the operation path will only be a directory, the quota information in the cache area will naturally only contain the quota information of the directories. This will greatly improve the cache hit rate, thereby quickly calculating the quota information of each level of directories on the target path.

[0059] The quota information statistics method for directories in the distributed file system of this disclosure obtains the quota information corresponding to each directory on the target path from the cache, and loads the second quota information from the hard disk when the second quota information corresponding to the second directory on the target path does not exist in the cache. Thus, the quota information of each directory on the target path can be obtained while maximizing the acquisition speed of quota information.

[0060] Furthermore, in one optional embodiment of this disclosure, after updating the quota information corresponding to each directory on the target path, the updated quota information can be stored in the hard disk, and the operation record can be deleted at the same time.

[0061] In this embodiment, the bottom-up scanning task needs to simultaneously modify the statistical information (i.e., quota information) in multiple inodes. The modification of quota information in each directory on the target path must either succeed simultaneously or fail simultaneously; otherwise, the quota information will be inaccurate. Therefore, this disclosure proposes a consistency mechanism: after the quota information is updated, the operation of sending the updated quota information to the hard disk for persistent storage and deleting the operation record are performed simultaneously. This ensures the atomicity of the persistence of the latest quota information and the deletion of the operation record, thus guaranteeing the accuracy of the quota information even in special scenarios such as system restarts.

[0062] In the application scenario of moving directories, the solution disclosed herein locks the target path for protection. When the path for updating quota information conflicts with the source path of the moving operation, the target path must be unlocked only after the operation is completed. The moving operation can then be performed after the resource of the target path is obtained.

[0063] In one optional embodiment of this disclosure, the operation request is to move the third directory on the first path to the fourth directory on the second path, wherein the first path and the second path are two different paths, such as... Figure 3 As shown, based on the foregoing embodiments, the method for calculating quota information of directories in a distributed file system provided in this disclosure may further include the following steps:

[0064] Step 301: In response to the operation request, move the third directory to the fourth directory to generate a third path.

[0065] In this embodiment of the disclosure, when the received operation request is to move the third directory on the first path to the fourth directory on the second path, in response to the operation request, the distributed file system moves the third directory to the fourth directory, generating a third path. It is understood that the third path includes all directories on the original second path, the third directory itself, and all subdirectories under the third directory.

[0066] Step 302: Obtain the third quota information corresponding to the third directory.

[0067] The quota information corresponding to the third directory (referred to as the third quota information for easy distinction) includes the total number and total size of all subdirectories and all subfiles contained in the third directory.

[0068] Step 303: Based on the third quota information, update the quota information of the fourth directory and all directories above it on the second path, and update the quota information of each directory on the remaining path after the removal of the third directory on the first path.

[0069] In this embodiment of the disclosure, after obtaining the third quota information corresponding to the third directory, the quota information of all directories on the second path, including the fourth directory and all directories above it (in ascending to the root directory), can be updated based on the third quota information. The total quantity and total size from the third quota information are added to the original quota information to obtain the updated quota information. It is understood that since moving the third directory to the fourth directory only affects the quota information of the fourth directory and all directories above it, and has no impact on the quota information of the subdirectories under the fourth directory, it is sufficient to update only the quota information of the fourth directory and all directories above it on the second path.

[0070] In addition, in this embodiment of the disclosure, the quota information of each directory on the remaining path after the removal of the third directory on the first path can be updated based on the third quota information. The total quantity and total size in the third quota information are subtracted from the original quota information corresponding to each directory to obtain the updated quota information.

[0071] It is understood that in this embodiment of the disclosure, the changes in the quota information of the directories on the first path and the second path occur simultaneously, that is, either both changes are successful or both changes fail.

[0072] The quota information statistics method for directories in the distributed file system of this disclosure responds to an operation request by moving a third directory to a fourth directory to generate a third path and obtaining the third quota information corresponding to the third directory. Then, based on the third quota information, the quota information of the fourth directory and all directories above it on the second path is updated, as well as the quota information of each directory on the remaining path after the removal of the third directory on the first path is updated. Thus, the quota information of the new path and the relevant directories on the remaining path after the directory is moved is updated in a timely manner based on the received operation request, ensuring the accuracy of quota information in each level of target.

[0073] Furthermore, in one alternative embodiment of this disclosure, in such... Figure 3Based on the illustrated embodiment, the quota information statistics method for directories in the distributed file system provided in this disclosure may further include: if there is an operation task in a subdirectory of the third directory, and the operation task has not been executed, then after moving the third directory to the fourth directory, the operation task is executed; based on the operation task, the quota information corresponding to all directories on the third path from the first-level directory affected by the operation task upwards to the root directory is updated. If there is an operation task in a subdirectory of the third directory, and the operation task is being executed, then the third directory is moved to the fourth directory after the operation task is completed.

[0074] The subdirectories of the third directory refer to any directory among all directories involved in the path from the third directory down to the last level. The operation can be any of the following: delete, move, create, etc.

[0075] In this embodiment, when there is an operation task in a subdirectory of the third directory, an appropriate execution strategy can be selected for subsequent actions based on whether the operation task has been executed. Specifically, if the operation task has not been executed, it is executed after the third directory is moved to the fourth directory. Furthermore, based on the operation task, the quota information corresponding to all directories on the third path, from the first-level directory affected by the operation task (which is understood to be the fourth directory here) upwards to the root directory of the third path, is updated. If the operation task is being executed, the third directory is moved to the fourth directory after the operation task is completed. In this case, the execution of the operation task affects the quota information of all directories from the first-level directory corresponding to the operation task upwards to the root directory.

[0076] It is understood that before updating the quota information corresponding to all directories on the third path from the first-level directory affected by the operation task upwards to the root directory of the third path, a corresponding operation record can be generated based on the operation task. Then, a target path for quota information to be changed can be generated based on the first-level directory in the operation record. The target path includes all directories on the third path from the first-level directory affected by the operation task upwards to the root directory of the third path. The specific process of generating the directory path and operation record, as well as updating the quota information, can be found in the relevant description of the foregoing embodiments. The implementation principle is similar and will not be repeated here.

[0077] In other words, in this embodiment of the disclosure, for the moved third directory, if there are no operation tasks in the subdirectories under the third directory, the third directory can be directly moved to the fourth directory on the second path; if there are operation tasks in the subdirectories under the third directory, and the operation tasks have not been executed, the third directory can also be directly moved to the fourth directory on the second path. The move operation affects the quota information of each directory remaining on the first path after the third directory is removed, as well as the quota information of the fourth directory and all directories above it on the third path. When the operation task is executed on the subdirectories under the third directory, the change in quota information caused by the operation task affects the quota information of the relevant directories on the newly generated third path after the third directory is moved.

[0078] In this embodiment of the disclosure, when there is an operation task in a subdirectory of the third directory and the operation task has not been executed, after moving the third directory to the fourth directory, the operation task is executed, and based on the operation task, the quota information corresponding to all directories on the third path from the first-level directory affected by the operation task upwards to the root directory is updated. When there is an operation task in a subdirectory of the third directory and the operation task is being executed, the third directory is moved to the fourth directory after the operation task is completed. Thus, the accuracy of the quota information of each directory can be guaranteed when the entire file system is performing a move operation.

[0079] To implement the above embodiments, this disclosure also provides a quota information statistics device for directories in a distributed file system.

[0080] Figure 4 This is a schematic diagram of a quota information statistics device for a directory in a distributed file system provided in an embodiment of the present disclosure. The device can be implemented in software and / or hardware and can be integrated into a distributed file system.

[0081] like Figure 4 As shown, the quota information statistics device 40 for directories in the distributed file system provided in this embodiment may include: a scanning module 410, an acquisition module 420, and a statistics module 430.

[0082] The scanning module 410 is used to respond to the distributed file system execution operation record, and generate a target path for quota information to be changed based on the first-level directory in the operation record. The target path includes all directories from the first-level directory affected by the execution of the operation record to the root directory. The operation record includes the first-level directory, the amount of information change, and the information change object. The operation record is generated based on the received operation request.

[0083] The acquisition module 420 is used to acquire quota information corresponding to each directory on the target path. The quota information includes the total number and total size of subdirectories and subfiles under the corresponding directory.

[0084] The statistics module 430 is used to update the quota information corresponding to each directory on the target path based on the operation record, so as to obtain the updated quota information corresponding to each directory on the target path.

[0085] In one possible implementation of this disclosure, the quota information statistics device 40 for directories in the distributed file system further includes:

[0086] A locking module is used to lock the target path to prevent the target path from being destroyed.

[0087] In one possible implementation of this disclosure, the operation request is to create a first subdirectory under a first directory, and the quota information statistics device 40 for directories in the distributed file system further includes:

[0088] The information generation module is used to generate the first quota information corresponding to the first subdirectory, wherein the quantity information and size information in the first quota information are both 0.

[0089] In one possible implementation of this disclosure, the quota information statistics device 40 for directories in the distributed file system further includes:

[0090] A caching module is used to provide a cache area, which is used to store quota information corresponding to each directory in the distributed file system.

[0091] The acquisition module 420 is further configured to:

[0092] Obtain the quota information corresponding to each directory on the target path from the cache area;

[0093] If the second quota information corresponding to the second directory on the target path does not exist in the cache area, the second quota information is loaded from the hard disk, wherein the second directory is any directory on the target path.

[0094] In one possible implementation of this disclosure, the quota information statistics device 40 for directories in the distributed file system further includes:

[0095] The consistency module is used to store the updated quota information to the hard disk and delete the operation record.

[0096] In one possible implementation of this disclosure, the operation request is to move the third directory on the first path to the fourth directory on the second path. The quota information statistics device 40 for directories in the distributed file system further includes:

[0097] The execution module is used to respond to the operation request by moving the third directory to the fourth directory and generating a third path;

[0098] The acquisition module 420 is also used to acquire the third quota information corresponding to the third directory;

[0099] The statistics module 430 is further configured to update the quota information of the fourth directory and all directories above it on the second path based on the third quota information, and to update the quota information of each directory on the remaining path after the third directory is removed from the first path.

[0100] In one possible implementation of this disclosure, the execution module is further configured to: if there is an operation task in the subdirectory of the third directory and the operation task has not been executed, then execute the operation task after moving the third directory to the fourth directory; if there is an operation task in the subdirectory of the third directory and the operation task is being executed, then move the third directory to the fourth directory after the operation task has been executed.

[0101] The statistics module 430 is further configured to update the quota information corresponding to all directories on the third path from the first-level directory affected by the operation task upwards to the root directory, based on the operation task.

[0102] The quota information statistics device for directories in a distributed file system provided in this disclosure can execute the quota information statistics method for any distributed file system provided in this disclosure, and has the corresponding functional modules and beneficial effects of the method execution. Content not described in detail in the device embodiments of this disclosure can be referred to the description in any method embodiment of this disclosure.

[0103] This disclosure also provides a distributed file system, including a processor and a memory; the processor executes the steps of the quota information statistics method for directories in the distributed file system as described in the foregoing embodiments by calling programs or instructions stored in the memory. To avoid repetition, these steps will not be repeated here.

[0104] This disclosure also provides a non-transitory computer-readable storage medium that stores a program or instructions that cause a computer to execute the steps of the quota information statistics method for directories in a distributed file system as described in the foregoing embodiments. To avoid repetition, these steps will not be repeated here.

[0105] This disclosure also provides a computer program product, including a computer program, wherein, when executed by a processor, the computer program implements the steps of the various embodiments of the quota information statistics method for directories in a distributed file system as described in the foregoing embodiments.

[0106] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0107] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for statistically analyzing quota information of directories in a distributed file system, characterized in that, include: In response to the distributed file system execution operation record, a target path for quota information to be changed is generated based on the first-level directory in the operation record. The target path includes all directories from the first-level directory affected by the execution of the operation record upwards to the root directory. The operation record includes the first-level directory, the amount of information change, and the information change object. The operation record is generated based on the received operation request. The information change object includes quantity and / or size. The amount of information change refers to the number of changes in the information change objects involved in the operation request. Obtain quota information for each directory on the target path, the quota information including the total number and total size of subdirectories and subfiles under the corresponding directory; Based on the operation record, the quota information corresponding to each directory on the target path is updated to obtain the updated quota information corresponding to each directory on the target path. Before updating the quota information corresponding to each directory on the target path based on the operation record, the method further includes: The target path is locked to prevent it from being corrupted, wherein a locking module is set in the distributed file system, the locking module being used to lock the target path when it is generated.

2. The method for calculating quota information of directories in a distributed file system according to claim 1, characterized in that, The operation request is to create a first subdirectory under the first directory, and the method further includes: Generate the first quota information corresponding to the first subdirectory, wherein the quantity and size information in the first quota information are both 0.

3. The method for calculating quota information of directories in a distributed file system according to claim 1, characterized in that, Obtain quota information for each directory on the target path, including: Obtain quota information corresponding to each directory on the target path from the cache area, wherein the cache area is used to store quota information corresponding to each directory in the distributed file system; If the second quota information corresponding to the second directory on the target path does not exist in the cache area, the second quota information is loaded from the hard disk, wherein the second directory is any directory on the target path.

4. The method for calculating quota information of directories in a distributed file system according to claim 3, characterized in that, The method further includes: The updated quota information is stored on the hard disk, and the operation record is deleted.

5. The method for calculating quota information of directories in a distributed file system according to claim 1, characterized in that, The operation request is to move the third directory on the first path to the fourth directory on the second path, and the method further includes: In response to the operation request, the third directory is moved to the fourth directory, generating a third path; Obtain the third quota information corresponding to the third directory; Based on the third quota information, the quota information of the fourth directory and all directories above it on the second path is updated, and the quota information of each directory on the remaining path after the third directory is removed on the first path is updated.

6. The method for calculating quota information of directories in a distributed file system according to claim 5, characterized in that, The method further includes: If there are operation tasks in the subdirectories of the third directory, and the operation tasks have not been executed, then the operation tasks will be executed after the third directory is moved to the fourth directory. Based on the operation task, update the quota information corresponding to all directories on the third path from the first-level directory affected by the operation task upwards to the root directory; If there are operation tasks in the subdirectories of the third directory, and the operation tasks are being executed, then the third directory will be moved to the fourth directory after the operation tasks are completed.

7. A quota information statistics device for directories in a distributed file system, characterized in that, include: The scanning module is used to respond to the operation record of the distributed file system, and generate a target path for quota information to be changed based on the first-level directory in the operation record. The target path includes all directories from the first-level directory affected by the operation record to the root directory. The operation record includes the first-level directory, the amount of information change, and the information change object. The operation record is generated based on the received operation request. The information change object includes quantity and / or size. The amount of information change refers to the number of changes in the information change object involved in the operation request. The acquisition module is used to acquire quota information corresponding to each directory on the target path. The quota information includes the total number and total size of subdirectories and subfiles under the corresponding directory. The statistics module is used to update the quota information corresponding to each directory on the target path based on the operation records, so as to obtain the updated quota information corresponding to each directory on the target path. The device further includes: A locking module is used to lock the target path to prevent it from being corrupted. The locking module is set in the distributed file system and is used to lock the target path when it is generated.

8. A distributed file system, characterized in that, include: Processor and memory; The processor executes the quota information statistics method for directories in the distributed file system as described in any one of claims 1 to 6 by calling the program or instructions stored in the memory.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that cause a computer to execute a quota information statistics method for a directory in a distributed file system as described in any one of claims 1 to 6.

Images