Storage system quota statistical information management method and device, equipment and medium
By creating key-value pairs in the storage system and aggregating quota statistics for users and user groups using a target data structure, the problem of excessive storage resource consumption is solved, and efficient quota statistics management is achieved.
Patent Information
- Application Number
- CN202511239942.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-01
- Publication Date
- 2025-12-02
AI Technical Summary
Existing storage systems require maintaining two separate sets of statistical metadata when setting user-level and user group-level quotas, resulting in a large amount of metadata and excessive storage resource consumption.
By creating key-value pairs, the system stores quota statistics for existing users and user groups, uses type flags to distinguish between users and user groups, and aggregates all key-value pairs based on the target data structure to achieve simultaneous storage and updating of quota statistics.
This significantly saves storage space for quota statistics, improves update speed and stability, and reduces resource consumption.
Smart Images
Figure CN121050656A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data storage technology, and in particular to a method, apparatus, device, and medium for managing quota statistics information of a storage system. Background Technology
[0002] In existing storage systems, when setting user-level / user group-level quotas for directories or object buckets, two separate sets of statistical metadata are typically required: user-level quota statistical metadata and user group-level quota statistical metadata. This results in a large amount of metadata used for quota information, consuming excessive storage resources. Summary of the Invention
[0003] In view of this, the present invention proposes a method, apparatus, device and medium for managing quota statistics information in a storage system, which solves the problem of excessive storage resource consumption when storing quota statistics information in traditional storage systems.
[0004] On one hand, embodiments of the present invention provide a method for managing storage system quota statistics, the method comprising: Create key-value pairs to store quota statistics for existing users and quota statistics for existing user groups in the storage system, wherein the key in the key-value pair includes a type flag, and the type of the type flag includes either user or user group; Aggregate all key-value pairs based on the target data structure to simultaneously store the quota statistics of existing users and quota statistics of existing user groups in the storage system; In response to receiving a first target operation request sent by the existing user, the quota statistics information of the existing user and the quota statistics information of the user group to which the existing user belongs are updated in the target data structure based on the first target operation request.
[0005] In some implementations, the key has 2 bits. n ,2 n The high m bits are the type flag bits, 2 n Low 2 n -m is the identity identifier; the value in the key-value pair is used to store quota statistics information.
[0006] In some embodiments, the storage system quota statistics information management method provided by the present invention further includes: In response to receiving a file creation request from a new user, quota statistics are allocated to the new user, a new key-value pair is created to store the quota statistics of the new user, and the key-value pair storing the quota statistics of the new user is added to the target data structure.
[0007] In some implementations, in response to receiving a file creation request from a new user, the method further includes allocating quota statistics information to the new user, while simultaneously: Assign a user group to the new user and update the quota information of the user group to which the new user belongs in the target data structure.
[0008] In some implementations, aggregating all key-value pairs based on a target data structure to simultaneously store quota statistics for existing users and quota statistics for existing user groups in the storage system includes storing quota statistics for existing users and quota statistics for existing user groups in the memory of an electronic device based on the target data structure.
[0009] In some embodiments, the storage system quota statistics information management method provided by the present invention further includes: reading the quota statistics information of existing users and quota statistics information of existing user groups stored in the target data structure from the memory at a preset period, and writing them to the disk.
[0010] In some embodiments, the storage system quota statistics information management method provided by the present invention further includes: In response to the electronic device being powered off and then powered on again, the quota statistics of existing users and quota statistics of existing user groups stored in the disk are read and written into the newly created target data structure in the memory.
[0011] On the other hand, embodiments of the present invention also provide a storage system quota statistics management device, comprising: A creation module is configured to create key-value pairs to store quota statistics of existing users and quota statistics of existing user groups in the storage system. The key in the key-value pair includes a type flag, and the type of the type flag includes either user or user group. The storage module is configured to aggregate all key-value pairs based on the target data structure to simultaneously store quota statistics of existing users and quota statistics of existing user groups in the storage system. The update module is configured to respond to receiving a first target operation request sent by the existing user, and update the quota statistics information of the existing user and the quota statistics information of the user group to which the existing user belongs in the target data structure based on the first target operation request.
[0012] On the other hand, embodiments of the present invention also provide an electronic device, including: at least one processor; and a memory storing a computer program executable on the processor, characterized in that the processor executes the steps of the method as described in any of the preceding embodiments when executing the program.
[0013] On the other hand, embodiments of the present invention also provide a computer storage medium, which stores a computer program that, when executed by a processor, performs the steps of the method described in any of the above embodiments.
[0014] The present invention has at least the following beneficial effects: This invention provides a method, apparatus, device, and medium for managing quota statistics in a storage system. The method for managing quota statistics in a storage system provided by this invention creates key-value pairs to store quota statistics for existing users and existing user groups in the storage system. The key in each key-value pair includes a type flag, the type of which can be either user or user group. All key-value pairs are aggregated based on a target data structure to simultaneously store quota statistics for existing users and existing user groups. Upon receiving a first target operation request from an existing user, the method updates the quota statistics for both the existing user and the user group in the target data structure. This significantly saves storage space occupied by quota statistics and improves the update speed of quota statistics. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.
[0016] Figure 1 A flowchart illustrating a method for managing storage system quota statistics information provided in an embodiment of the present invention; Figure 2 A flowchart illustrating another storage system quota statistics management method provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the structure of a storage system quota statistics information management device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a computer storage medium provided in an embodiment of the present invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.
[0018] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.
[0019] The present invention will now be described in detail with reference to the embodiments and accompanying drawings.
[0020] The first aspect of this invention provides a method for managing storage system quota statistics. This method can be applied to electronic devices that have storage systems deployed on them. These electronic devices can be desktop computers, laptops, servers, etc., and are not specifically limited thereto.
[0021] like Figure 1 As shown, the storage system quota statistics information management method provided in this embodiment of the invention may include steps S10 to S30.
[0022] S10. Create key-value pairs to store quota statistics for existing users and quota statistics for existing user groups in the storage system.
[0023] In this key-value pair, the key includes a type flag, and the type of the type flag can be either user or user group.
[0024] In this embodiment of the invention, an existing user refers to a user who has already registered in the storage system and created files or directories. An existing user group refers to a user group that has already been created in the storage system. A key-value pair stores the quota statistics of an existing user or an existing user group. Users and user groups are distinguished by the type flag bit in the key of the key-value pair.
[0025] S20. Aggregate all key-value pairs based on the target data structure to simultaneously store quota statistics for existing users and quota statistics for existing user groups in the storage system. In this embodiment of the invention, the target data structure can be constructed in memory, and the quota statistics for existing users and quota statistics for existing user groups in the storage system can be stored based on this target data structure.
[0026] The target data structure can be a key-value index structure (e.g., std::map (an associative container in the C++ Standard Template Library)), an equivalent skip list, or a tree data structure (e.g., a B+ tree), etc., without specific limitations. Quota statistics can include information such as the number of files, the number of directories, and the usage capacity, without specific limitations.
[0027] In this embodiment of the invention, users and user groups can be distinguished by the type flag bit in the key of the key-value pair, and all key-value pairs can be aggregated by a target data structure to simultaneously store user-level quota statistics and user group-level quota statistics, thereby greatly saving the storage space occupied by quota statistics storage.
[0028] S30. In response to receiving a first target operation request sent by an existing user, update the quota statistics information of the existing user and the quota statistics information of the user group to which the existing user belongs in the target data structure based on the first target operation request.
[0029] The first target operation can be an operation such as creating a file, writing data, or modifying data, and is not specifically limited here. In response to receiving a first target operation request from any of the existing users, the quota statistics information of that user (i.e., the user who sent the first target operation request) and the quota statistics information of the user's user group in the target data structure can be updated. In this embodiment of the invention, during the update, the quota statistics information of the user in the target data structure can be updated first, and then the quota statistics information of the user's user group in the target data structure can be updated. Thus, the update of both the user's quota statistics information and the user's user group's quota information is completed within a single data structure, thereby improving the update speed of quota statistics information.
[0030] In this embodiment of the invention, key-value pairs are created to store quota statistics information of existing users and user groups in the storage system. All key-value pairs are aggregated based on the target data structure to store quota statistics information of existing users and user groups in the storage system. When a first target operation request is received from an existing user, the quota statistics information of the existing user and the quota statistics information of the user group to which the existing user belongs in the target data structure are updated. This not only greatly saves the storage space occupied by quota statistics information storage, but also improves the update speed of quota statistics information.
[0031] In some embodiments of the present invention, the storage system quota statistics information management method provided by the present invention may include the following steps in addition to steps S10 to S30: in response to receiving a file creation request sent by a new user, allocating quota statistics information to the new user, creating a new key-value pair to store the quota statistics information of the new user, and adding the key-value pair storing the quota statistics information of the new user to the target data structure.
[0032] In this embodiment of the invention, a new user refers to a user logging into the storage system for the first time. Creating a file is the process of establishing a new file, which requires basic attributes such as configuration file naming, type selection, and storage path. It also requires allocating storage space through the file system and creating an entry record in the directory. Therefore, when a file creation request is received from a new user, quota statistics information needs to be allocated to that user.
[0033] In this embodiment of the invention, in response to allocating quota statistics to a new user, a new key-value pair can be created to store the quota statistics of the new user, and the quota statistics of the new user can be added to the target data structure to update the quota information of existing users and quota information of existing user groups stored in the target data structure.
[0034] The embodiments of the present invention improve the update speed of quota statistics information through the above-described solution.
[0035] In some embodiments of the present invention, in response to receiving a file creation request sent by a new user, while allocating quota statistics information to the new user, the method further includes: assigning a user group to the new user and updating the quota information of the user group to which the new user belongs in the target data structure.
[0036] In this embodiment of the invention, both user quota statistics and user group quota statistics are stored in key-value pair format in the target data structure. User quota statistics include user type, user identifier, and specific quota statistics. User group quota statistics include user group type, user group identifier, and specific quota statistics. Therefore, in this embodiment of the invention, key-value pairs can be constructed to store user quota statistics and user group quota statistics.
[0037] In some embodiments of the present invention, user information or user group information can be stored based on the key in the key-value pair, and specific quota statistics for users or specific quota statistics for user groups can be stored based on the value in the key-value pair.
[0038] In some embodiments of the present invention, the number of bits in the key-value pair is 2. n ,2 n The high m bits are the type flag bits, 2 n Low 2 n -m is the identification bit; the value in the key-value pair is used to store quota statistics.
[0039] In this embodiment of the invention, the number of bits in the key of a key-value pair can be configured to distinguish between user types and user group types. In some examples, the number of bits in the key can be configured to be 2. n ,2 n The high m bits are the type flag bits, 2n Low 2 n -m bits are for identity identification. For example, a 64-bit integer key can be defined, where the high m bits are type flags used to distinguish users from user groups, and the low (64-m) bits are identity identification bits used to store identity identifiers (e.g., uid (user identity) or gid (group identity)). In some examples, a uniform value structure can also be defined, such as user_rstat_t. In some examples, any of the following can be used in memory: a single key-value index structure (e.g., std::map), an equivalent skip list, and a tree data structure (e.g., B+ tree) to aggregate user quota statistics and user group quota statistics stored in key-value pairs. Taking a key-value index structure as an example, this key-value index structure has multiple nodes, and each node can be used to store a key-value pair.
[0040] In this embodiment of the invention, the number of bits in the key, and the number of bits in the type flag and identity flag bits, can be determined based on usage requirements. For example, the key can be 64 bits, the corresponding type flag bits can be the high 3 bits, and the identity flag bits can be the low 61 bits. It should be understood that the above is only an example of the number of bits in the key, and the number of bits in the type flag and identity flag bits. In this embodiment of the invention, the number of bits in the key, and the number of bits in the type flag and identity flag bits, can also be 64 bits, high 4 bits, low 60 bits, or 64 bits, high 5 bits, low 59 bits, or 128 bits, high 8 bits, low 120 bits, etc., and no specific limitation is made here.
[0041] In this embodiment of the invention, the number of bits in the defined key, as well as the number of bits in the type flag and identity flag bits in the key, can be modified to expand the number of bits in the type flag or identity flag bits online when the number of bits in the type flag or identity flag bits is insufficient, thereby improving the flexibility of quota statistics information management.
[0042] In some embodiments of the present invention, such as Figure 1 and 2 As shown, the storage system quota statistics information management method provided in this embodiment of the invention may include step S40 in addition to steps S10 to S30.
[0043] S40. According to a preset cycle, read the quota statistics of existing users and quota statistics of existing user groups stored in the target data structure from memory and write them to disk.
[0044] In step S40, the preset period can be 100ms, 10s, 30s, 1min, 5min or 10min, etc., and no specific limitation is made here.
[0045] On the one hand, in this embodiment of the invention, by reading the quota statistics information of existing users and quota statistics information of existing user groups stored in the target data structure from memory according to a preset period and writing them to the disk, the quota statistics information can be persisted, ensuring that the quota statistics information will not be lost after a power outage, thus improving the stability of quota statistics management.
[0046] In another aspect, in this embodiment of the invention, since the quota statistics of users and the quota statistics of user groups are stored based on the same target data structure, when writing the quota statistics of users and the quota statistics of user groups stored based on the target data structure to the disk, the entire target data structure is written into a data block of the disk. This greatly reduces write amplification and simplifies the data storage path.
[0047] In some embodiments of the present invention, such as Figure 1 and 2 As shown, the storage system quota statistics information management method provided in this embodiment of the invention may include step S50 in addition to steps S10 to S40.
[0048] S50, in response to the electronic device being powered off and then powered on again, read the quota statistics of existing users and quota statistics of existing user groups stored on the disk and write them into the newly created target data structure in memory.
[0049] In this embodiment of the invention, on the one hand, after the electronic device is powered off normally or abnormally, the quota statistics information of existing users and quota statistics information of existing user groups stored on the disk can be read and written into the newly created target data structure in memory. Thus, it can be ensured that the quota statistics information will not be lost after power failure, thereby improving the stability of quota statistics management.
[0050] In another aspect of this invention, since the user quota statistics and user group quota statistics are stored based on the same target data structure, when writing the user quota statistics and user group quota statistics stored based on the target data structure to the disk, the entire target data structure is written into a single data block on the disk. Therefore, when reading quota statistics from the disk and writing them into memory, only one parsing of the disk data block is needed to complete the process of loading quota statistics into memory, greatly improving the loading efficiency of quota statistics.
[0051] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a storage system quota statistics information management device. For example... Figure 3As shown, the storage system quota statistics management device 3 includes a creation module 30, a storage module 31, and an update module 32. The creation module 30 is configured to create key-value pairs to store quota statistics information for existing users and existing user groups in the storage system. The key in each key-value pair includes a type flag, and the type of the type flag includes either user or user group. The storage module 31 is configured to aggregate all key-value pairs in the memory of the electronic device where the storage system is deployed, based on a target data structure, to store the quota statistics information for existing users and existing user groups in the storage system. The update module 32 is configured to, in response to receiving a first target operation request sent by an existing user, update the quota statistics information for the existing user and the quota statistics information for the user group to which the existing user belongs in the target data structure based on the first target operation request.
[0052] In this embodiment of the invention, quota statistics information of existing users and quota statistics information of existing user groups are stored in the target data structure. When a first target operation request is received from an existing user, the quota statistics information of the existing user and the quota statistics information of the user group to which the existing user belongs is updated in the target data structure. This not only greatly saves the storage space occupied by quota statistics information storage, but also improves the update speed of quota statistics information.
[0053] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 4 As shown, an embodiment of the present invention also provides an electronic device 40, which includes a processor 410 and a memory 420. The memory 420 stores a computer program 421 that can be run on the processor. When the processor 410 executes the program, it performs the steps of the method described above.
[0054] The memory, as a non-volatile storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the compression method described in the embodiments of this application. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory.
[0055] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0056] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 5 As shown, an embodiment of the present invention also provides a computer storage medium 500, which stores a computer program 510 that executes the above method when executed by a processor.
[0057] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.
[0058] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.
[0059] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. The sequence numbers of the disclosed embodiments of this invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.
[0060] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.
[0061] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.
Claims
1. A method for managing storage system quota statistics, characterized in that, The method includes: Create key-value pairs to store quota statistics for existing users and quota statistics for existing user groups in the storage system, wherein the key in the key-value pair includes a type flag, and the type of the type flag includes either user or user group; Aggregate all key-value pairs based on the target data structure to simultaneously store the quota statistics of existing users and quota statistics of existing user groups in the storage system; In response to receiving a first target operation request sent by the existing user, the quota statistics information of the existing user and the quota statistics information of the user group to which the existing user belongs are updated in the target data structure based on the first target operation request.
2. The method according to claim 1, characterized in that, The key has 2 bits. n ,2 n The high m bits are the type flag bits, 2 n Low 2 n -m is the identity identifier; the value in the key-value pair is used to store quota statistics information.
3. The method according to claim 1, characterized in that, Also includes: In response to receiving a file creation request from a new user, quota statistics are allocated to the new user, a new key-value pair is created to store the quota statistics of the new user, and the key-value pair storing the quota statistics of the new user is added to the target data structure.
4. The method according to claim 3, characterized in that, In response to receiving a file creation request from a new user, the method further includes allocating quota statistics information to the new user while simultaneously: Assign a user group to the new user and update the quota information of the user group to which the new user belongs in the target data structure.
5. The method according to claim 1, characterized in that, Aggregating all key-value pairs based on the target data structure to simultaneously store quota statistics of existing users and quota statistics of existing user groups in the storage system includes: storing quota statistics of existing users and quota statistics of existing user groups in the memory of the electronic device based on the target data structure.
6. The method according to claim 5, characterized in that, Also includes: According to a preset cycle, the quota statistics information of existing users and quota statistics information of existing user groups stored in the target data structure are read from the memory and written to the disk.
7. The method according to claim 6, characterized in that, Also includes: In response to the electronic device being powered off and then powered on again, the quota statistics of existing users and quota statistics of existing user groups stored in the disk are read and written into the newly created target data structure in the memory.
8. A storage system quota statistics information management device, characterized in that, include: A creation module is configured to create key-value pairs to store quota statistics of existing users and quota statistics of existing user groups in the storage system. The key in the key-value pair includes a type flag, and the type of the type flag includes either user or user group. The storage module is configured to aggregate all key-value pairs based on the target data structure to simultaneously store quota statistics of existing users and quota statistics of existing user groups in the storage system. The update module is configured to respond to receiving a first target operation request sent by the existing user, and update the quota statistics information of the existing user and the quota statistics information of the user group to which the existing user belongs in the target data structure based on the first target operation request.
9. An electronic device, comprising: At least one processor; as well as A memory storing a computer program executable on the processor, characterized in that the processor executes the program by performing the steps of the method as described in any one of claims 1 to 7.
10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1 to 7.