A method, system, device and storage medium for optimizing hard disk space management
By creating a blob with the same size as writing large file data blocks in BlueStore, and using a data alignment algorithm to merge metadata in small file scenarios, the problems of increasing metadata amount and discontinuous data distribution in large file scenarios, and excessive metadata amount in small file scenarios are solved, and performance improvement and storage cost reduction are achieved.
Patent Information
- Application Number
- CN202310080923.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-03
- Publication Date
- 2025-06-24
- Estimated Expiration
- 2043-02-03
AI Technical Summary
BlueStore increases the amount of metadata and discontinuous data distribution in large file scenarios due to data segmentation, which affects performance; in small file scenarios, excessive amount of metadata caused by small data blocks written, which affects performance and increases storage costs.
In a large file scenario, create a blob with the same size as writing to a large file data block to map to the target disk storage space to avoid data splitting; in a small file scenario, a data alignment algorithm is used to align small file data blocks to merge the amount of metadata entries.
The problem of increasing metadata volume and discontinuous data distribution in large file scenarios is solved, which improves performance and reduces storage costs; in small file scenarios, metadata merging is significantly reduced, which improves performance and reduces storage costs.
Smart Images

Figure CN116166193B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of hard disk space management optimization method design, and particularly to a hard disk space management optimization method, system, device and storage medium. Background Art
[0002] BlueStore is the storage engine of the distributed storage system ceph. The birth of BlueStore is to replace the traditional FileStore as a new generation of high-performance object storage backend. The original design intention of FileStore is to give full play to the performance of ordinary mechanical hard disks. Today, with the accelerating pace of hardware update and iteration and the full popularity of SSDs, the traditional FileStore can no longer meet the development of storage in terms of performance and application scenarios. Therefore, BlueStore with higher performance and a wider range of application scenarios has emerged as the times require.
[0003] BlueStore has a wide range of applications and supports multiple scenarios, including both large file scenarios and small file scenarios. It is precisely because of this general characteristic that BlueStore has certain drawbacks when applied to various scenarios.
[0004] There are two problems when applied to large file scenarios: In large file scenarios, if the size of the written data block is too large, a splitting operation will be triggered, and a large data block will be split into several small data blocks. The reason why a large data block triggers the splitting operation is that the blob has a range. When the data exceeds the carrying limit of the blob, that is, exceeds the maximum value of the blob, the data will be split, and the split data will be mapped to a new blob. The blob corresponds to the pextent, and the data mapped to the new blob will be correspondingly mapped to a new pextent, that is, a new space on the disk. This will cause a large piece of data to be divided into several small pieces and stored at different positions on the disk, resulting in a decrease in data continuity. The decrease in data continuity affects performance. On the other hand, originally only one piece of metadata was needed to store a large data block. After being split into several small data blocks, the same number of metadata needs to be stored, which will lead to an increase in the amount of metadata. The increase in the amount of metadata first affects performance. Secondly, the increase in the storage space required for the increased amount of metadata will lead to an increase in storage costs. Moreover, because metadata is relatively special and needs to be stored on a faster hard disk, the price of the hard disk that meets this requirement is higher, and the cost of storing redundant metadata is also greater.
[0005] When applied to the small file scenario, there are also certain problems: in the small file scenario, the written data blocks are generally small. The smaller the data block, the larger the metadata volume of a 4MB object. For example, in the small file scenario, the written data block size is 4KB. Since the size of an object is 4MB, the number of metadata entries for one object is 4MB / 4KB = 1024 entries. If the number of objects reaches several million, tens of millions, or even hundreds of millions, the metadata volume of the objects is three orders of magnitude higher than the number of objects. The cost of storing this metadata is a huge number. At the same time, such a large amount of metadata also has a great impact on performance. Summary of the Invention
[0006] To solve the above technical problems or at least partially solve the above technical problems, the present invention provides an optimized method, system, device, and storage medium for hard disk space management.
[0007] In a first aspect, the present invention provides an optimized method for hard disk space management, including:
[0008] Configure each blob to map to a continuous target disk storage space on the disk;
[0009] In the large file scenario, create the blob with the same size as the written large file data block and map it to the target disk storage space, and map the large file data block to the target disk storage space;
[0010] In the small file scenario, create blobs of a set size. Each blob supports multiple lextents, and each blob maps to the target disk storage space. Map at least one small file data block to the target disk storage space; use a data alignment algorithm to align the small file data blocks to merge the metadata entry volume; the data alignment algorithm aligns a set number of small file data blocks to an alignment scale, and each group of aligned data uses one piece of metadata.
[0011] Furthermore, in the large file scenario, the BlueStore metadata mapping structure is as follows: Onode is mapped to several lextents through a basic data management unit mapping map. Each lextent corresponds to one blob, and one blob corresponds to a continuous pextent in the disk space, realizing a one-to-one correspondence relationship among lextent, blob, and pextent.
[0012] Furthermore, in the small file scenario, the BlueStore metadata mapping structure is as follows: The Onode is mapped to a number of lextents through the basic data management unit mapping map, each lextent is mapped to one of the blobs, each blob allows multiple lextent mappings, and each blob corresponds to a pextent that is continuous in disk space.
[0013] Furthermore, the alignment scale of the data alignment algorithm is greater than the small file data block size, and the set number is equal to the integer result of the ratio of the alignment scale to the small file data block.
[0014] Furthermore, a plurality of data alignment algorithms with a plurality of alignment scales are preset; the target data alignment algorithm is matched according to the size of the small file data block to be written and the desired integration coefficient. The alignment scale of the target data alignment algorithm is greater than the product of the size of the small file data block to be written and the integration coefficient, and the difference between the alignment scale and the product of the size of the small file data block and the integration coefficient is the smallest.
[0015] Furthermore, obtain the size of the data block to be written, compare whether the size of the data block to be written is greater than the set size. If so, determine that the data block to be written is a large file data block, otherwise determine that the data block to be written is a small file data block.
[0016] Furthermore, when storing the merged metadata, overwrite the metadata at the time of writing with the merged metadata.
[0017] In a second aspect, the present invention provides an optimized system for managing hard disk storage space, including:
[0018] A scene differentiation module, which determines whether the current is a large file scene or a small file scene according to the size relationship between the data block to be written and the set size of the blob;
[0019] A metadata mapping structure creation module, which creates a blob with the same size as the large file data block to be written and maps it to the target disk storage space in the large file scene, and maps the large file data block to the target disk storage space; the metadata mapping structure creation module creates blobs with a set size in the small file scene, each blob supports multiple lextents, each blob is mapped to the target disk storage space, and at least one small file data block is mapped to the target disk storage space;
[0020] A data alignment module, which in the small file scene, uses a data alignment algorithm to align small file data blocks to merge the amount of metadata entries;
[0021] A metadata merging module, which merges the metadata of each small file data block before data alignment into one piece of metadata according to the alignment and grouping after alignment.
[0022] In a third aspect, the present invention provides an electronic device for implementing an optimized method for hard disk space management, including: at least one processing unit, a storage unit, and a bus unit. The processing unit is connected to the storage unit through the bus unit. The storage unit includes a metadata storage area, a data storage area, and a computer program storage area. The computer program storage area stores a computer program, and when the computer program is executed by the processing unit, the optimized method for hard disk space management is implemented.
[0023] In a fourth aspect, the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed, the optimized method for hard disk space management is implemented.
[0024] The above technical solutions provided by the embodiments of the present invention have the following advantages compared with the prior art:
[0025] The present invention determines whether to apply to the large file scenario or the small file scenario according to the size of the data block to be written. In the large file scenario, a blob with the same size as the large file data block to be written is created and mapped to the target disk storage space, and the large file data block is mapped to the target disk storage space without data segmentation, solving the problems of increased metadata volume and discontinuous data distribution caused by data segmentation in the large file scenario.
[0026] In the small file scenario, blobs of a set size are created, each blob supports multiple lextents, each blob is mapped to the target disk storage space, and at least one small file data block is mapped to the target disk storage space; a data alignment algorithm is used to align the small file data blocks to merge the metadata entries; the data alignment algorithm aligns a set number of small file data blocks to an alignment scale, and each group of aligned data uses one piece of metadata. Solve the problem of a large amount of metadata caused by the small size of the written data blocks in the small file scenario. First, metadata merging can achieve the effect of exponentially reducing the metadata volume and improving performance; on the other hand, the exponential reduction of the metadata volume can reduce the cost of metadata storage. Description of the Drawings
[0027] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.
[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0029] Figure 1 It is a schematic diagram of the existing BlueStore metadata mapping structure;
[0030] Figure 2 It is a schematic diagram of the BlueStore metadata mapping structure in the large file scenario provided by the embodiment of the present invention;
[0031] Figure 3 It is a schematic diagram of the BlueStore metadata mapping structure in the small file scenario provided by the embodiment of the present invention;
[0032] Figure 4 It is a schematic diagram of an optimized system for hard disk storage space management provided by the embodiment of the present invention;
[0033] Figure 5 It is a schematic diagram of an electronic device for implementing an optimized method for hard disk space management provided by the embodiment of the present invention. Detailed implementation manners
[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.
[0035] It should be noted that in this text, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without further limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including the said element.
[0036] To make the embodiments of the present invention clearer, the English terms involved in this application and their meanings are as follows:
[0037] Ceph, a distributed storage system. BlueStore, the storage engine of Ceph, is a new generation of high-performance object storage backend. Onode represents an object with a size of 4MB. Extent_map is the basic data management unit mapping map, which records the lextents contained in the Onode. A lextent is the basic data management unit within the Onode object. Data compression, data verification, and data sharing are all implemented based on the lextent granularity; it defines the logical address range of the Onode object and records the offsets and lengths of each basic data management unit of the object on the blob. A blob corresponds to a piece of disk physical space that is not necessarily continuous for a lextent and contains multiple pexents. A pextent is a continuous physical disk space, and the offsets and lengths of pexents are aligned according to the disk data block size.
[0038] Each storage object in BlueStore corresponds to an Onode. Each Onode contains a basic data management unit mapping map that records the basic data management units not contained in the object. The basic data management unit mapping map orderly maps a number of lextents. Each lextent is responsible for managing a logical segment of data within the object and is associated with a blob. The blob contains a number of pexents, and finally maps the data of the storage object to the disk.
[0039] See Figure 1 As shown, in the BlueStore metadata mapping structure, the blob is configured to have a fixed size, regardless of the sizes of large file data blocks and small file data blocks. In the large file scenario, each blob has a limit for carrying data, that is, the maximum value of the blob. Once the data block exceeds the limit of the data carried by the blob, the data block will be split, and the split data will be carried by two or more blobs. Each blob corresponds to one or more pexents, and the data mapped to the new blob will be correspondingly mapped to the new pexents, that is, the data is mapped to the new physical disk space. This will cause a large piece of data to be divided into several small pieces of data and stored in different positions on the disk, resulting in a decrease in data continuity. The decrease in data continuity affects performance. Originally, a large data block only needed to store one piece of metadata. After being split into several small data blocks, the same number of metadata needs to be stored, which will lead to an increase in the amount of metadata. The increase in the amount of metadata first affects performance, and secondly, the increased storage space required for the increased amount of metadata will lead to an increase in storage costs.
[0040] In the small file scenario, the small file data blocks written are generally small, and multiple lextents are associated with one blob. The smaller the small file data block, the more lextents of small file data blocks that can be associated with one blob, and the more metadata required.
[0041] Example 1
[0042] The embodiment of the present invention provides a hard disk space management optimization method, which is applied to BlueStore. On the one hand, it improves the data fragmentation caused by data segmentation in BlueStore in large file scenarios, and reduces the metadata entries in large file scenarios; on the other hand, it improves the problem of excessively large metadata entries in BlueStore small file scenarios because the small file data block is smaller than the blob. It optimizes the amount of metadata and data continuity in different scenarios, reduces the amount of metadata, and increases data continuity, which has a significant effect on improving read and write performance and reducing storage costs.
[0043] An embodiment of the present invention provides a hard disk space management optimization method, comprising:
[0044] Improvement on the mapping method of blob and lextent: Compared with the prior art, blob is a piece of disk physical space corresponding to lextent which is not necessarily continuous and contains multiple segments of pextent. In this application, the blob corresponding to lextent is limited to contain only a continuous target disk storage space on the disk.
[0045] The size of the data block to be written is obtained, and the size of the data block to be written is compared to see whether it is larger than the set size of the blob. If so, the data block to be written is determined to be a large file data block; otherwise, the data block to be written is determined to be a small file data block.
[0046] In the large file scenario, the business IO-oriented disk space adaptive allocation algorithm is used to control the writing of large file data blocks in the disk, namely:
[0047] See also Figure 2 As shown, in the process of writing file data blocks to the disk, when creating a blob for mapping, compared with the fixed size solution of the blob in the prior art, the present application creates a blob with the same size as the large file data block being written.
[0048] The BlueStore metadata mapping structure in the large file scenario is: one lextent corresponds to one blob, and one blob corresponds to one pextent that is continuous on the disk space, realizing a one-to-one correspondence between lextent, blob, and pextent.
[0049] The blob that adapts to large file data blocks by size undertakes large file data blocks, and in cooperation with the improvement that the blob is restricted to only contain a continuous target disk storage space on one disk, it is ensured that the large file data blocks do not need to be segmented during the process of being written into the target disk storage space.
[0050] Doing so can solve the problems of increased metadata volume and discontinuous data distribution caused by data segmentation in the large file scenario. First of all, no matter how large the large file data block is, it is not segmented, so there is only one configuration entry for the corresponding metadata, thus solving the problem of increased metadata volume caused by data segmentation. On the one hand, it can improve performance, and on the other hand, it can reduce the cost required for metadata storage; secondly, not segmenting the data means that one data corresponds to one blob. When there is enough space on the disk, creating one blob can only correspond to one pextent, as Figure 2 shown. In this way, it can be achieved that one lextent corresponds to one blob, and one blob corresponds to one pextent, realizing the one-to-one correspondence relationship among lextent, blob, and pextent. In this way, when the data is written to the disk, it is written into a continuous disk storage space (target disk storage space), thus solving the problem of discontinuous data distribution caused by data segmentation and improving performance.
[0051] In the small file scenario, the data of small file data blocks is aligned through a data alignment algorithm for metadata merging, that is:
[0052] In the small file scenario, blobs of a set size are adopted, each blob maps the target disk storage space, and each of the blobs supports multiple lextents, so that the target disk storage space can be used as the storage space for at least one small file data block.
[0053] During the specific implementation process, referring to Figure 3 shown, the metadata mapping structure of BlueStore in the small file scenario is: Onode is mapped to several lextents through the basic data management unit mapping map, each lextent is mapped to a blob of a fixed size, the blob allows multiple lextents to be mapped, and each blob corresponds to a pextent that is continuous in the disk space. The pextent is the target disk storage space.
[0054] A data alignment algorithm with an alignment scale larger than the size of small file data blocks is used to merge the data of small file data blocks, so as to reduce the number of metadata entries of small file storage objects; the data alignment algorithm merges the metadata of a set number of small file data blocks into one metadata entry, and the set number is the ratio of the alignment scale to the small file data block. Since the data alignment algorithm needs to align a set number of small file data blocks, the alignment scale of the data alignment algorithm is larger than the size of the small file data block, and the set number is equal to the integer result of the ratio of the alignment scale to the small file data block. During the alignment process, the unaligned positions need to be padded with zeros.
[0055] In the specific implementation process, data alignment algorithms with alignment scales of 4KB, 64KB, and 128KB are used to illustrate the merging effect of the data alignment algorithm on metadata.
[0056] Comparison of the metadata entries of a 4MB-sized Onode object when using each data alignment algorithm:
[0057] If the data alignment algorithm with an alignment scale of 4KB is used, the metadata entries of a 4MB-sized Onode object are 4MB / 4KB = 1024;
[0058] If the data alignment algorithm with an alignment scale of 64KB is used, the metadata entries of a 4MB-sized Onode object are 4MB / 64KB = 64;
[0059] If the data alignment algorithm with an alignment scale of 128KB is used, the metadata entries of a 4MB-sized Onode object are 4MB / 128KB = 32.
[0060] It can be found through comparison that as the alignment scale doubles, the amount of metadata entries merged by the corresponding data alignment algorithm shows an exponential decrease. The reason for this is that regardless of the data alignment algorithm used, the size of the alignment scale is the size of the data after the data alignment algorithm completes the alignment. For example, when writing data with a size of 4KB and using a 64KB alignment algorithm for alignment, the 4KB data will first be merged with the data in front or behind to make up 64KB and then written down. That is, this 64KB contains 16 4KB data, and these 16 4KB data share one metadata entry, which is equivalent to merging the 16 metadata entries of 16 4KB data into one metadata entry for a 64KB data; if the 64KB alignment algorithm is not used for alignment, these 16 4KB data each correspond to one metadata entry, that is, 16 metadata entries.
[0061] Doing so can solve the problem of a large amount of metadata caused by small written data blocks in the small file scenario. First, metadata merging can achieve an exponential reduction in the amount of metadata and improve performance; on the other hand, the exponential reduction in the amount of metadata can greatly reduce the cost of metadata storage.
[0062] In one implementation, in the small file scenario, when storing metadata, the merged metadata overwrites the write-time metadata.
[0063] In a preferred implementation, a plurality of data alignment algorithms with various alignment scales are preset. For example, data alignment algorithms with alignment scales of 4KB, 32KB, 64KB, and 128KB are set.
[0064] Match the target data alignment algorithm according to the size of the small file data block to be written and the desired integration coefficient. Among them, the integration coefficient represents the number of small file data blocks included in each group of aligned data; when merging metadata, the metadata of the integration coefficient small file data blocks is merged into one piece of metadata.
[0065] The matching of the target data alignment algorithm according to the size of the small file data block to be written and the desired integration coefficient includes two conditions:
[0066] The alignment scale of the target data alignment algorithm is greater than the product of the size of the small file data block to be written and the integration coefficient;
[0067] The difference between the alignment scale minus the product of the size of the small file data block and the integration coefficient is the smallest.
[0068] Embodiment 2
[0069] Refer to Figure 4 As shown, the embodiment of the present invention provides an optimized system for hard disk storage space management, including:
[0070] A scenario differentiation module, which determines whether the current is a large file scenario or a small file scenario according to the size of the data block to be written and the set size relationship of the blob.
[0071] A metadata mapping structure creation module, which creates a blob with the same size as the written large file data block and maps it to the target disk storage space in the large file scenario, and maps the large file data block to the target disk storage space; the metadata mapping structure creation module creates a blob with a set size in the small file scenario, each blob supports multiple lextents, each blob is mapped to the target disk storage space, and at least one small file data block is mapped to the target disk storage space.
[0072] A data alignment module, which, in a small file scenario, uses a data alignment algorithm to align small file data blocks so as to merge the metadata of each group of aligned data.
[0073] A metadata merging module, which merges the metadata of each small file data block before data alignment into one piece of metadata according to the grouping after alignment.
[0074] Embodiment 3
[0075] Refer to Figure 5 As shown, an embodiment of the present invention provides an electronic device for implementing a hard disk space management optimization method, including: at least one processing unit, a storage unit, and a bus unit. The processing unit is connected to the storage unit through the bus unit. The storage unit includes a metadata storage area, a data storage area, and a computer program storage area. The computer program storage area stores a computer program, and when the computer program is executed by the processing unit, the hard disk space management optimization method as described is implemented.
[0076] Embodiment 4
[0077] An embodiment of the present invention provides a computer-readable storage medium that stores a computer program, and when the computer program is executed, the hard disk space management optimization method as described is implemented.
[0078] In the embodiments provided by the present invention, it should be understood that the disclosed structure can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of structures or units can be in an electrical, mechanical or other form.
[0079] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0080] In addition, in each embodiment of the present invention, the functional units can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
[0081] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features claimed herein.
Claims
1. An optimization method for hard disk space management, characterized in that, including: Configure each blob to map to a continuous target disk storage space on the disk; In the large file scenario, create a blob with the same size as the large file data block being written and map it to the target disk storage space, and map the large file data block to the target disk storage space; In the large file scenario, the BlueStore metadata mapping structure is as follows: Onode is mapped to several lextents through the basic data management unit mapping map, each lextent corresponds to one blob, and one blob corresponds to a continuous pextent in the disk space, realizing a one-to-one correspondence between lextent, blob, and pextent; In the small file scenario, create blobs of a set size, each blob supports multiple lextents, each blob is mapped to the target disk storage space, and at least one small file data block is mapped to the target disk storage space; The BlueStore metadata mapping structure in the small file scenario is as follows: Onode is mapped to several lextents through the basic data management unit mapping map, each lextent is mapped to one blob, the blob allows multiple lextents to be mapped, and each blob corresponds to a continuous pextent in the disk space; Adopt a data alignment algorithm to align small file data blocks to merge the amount of metadata entries; the data alignment algorithm aligns a set number of small file data blocks to an alignment scale, and each group of aligned data uses one piece of metadata.
2. The hard disk space management optimization method according to claim 1, wherein The alignment scale of the data alignment algorithm is greater than the size of the small file data block, and the set number is equal to the integer result of the ratio of the alignment scale to the small file data block.
3. The hard disk space management optimization method according to claim 1, characterized in that Preset multiple data alignment algorithms with multiple alignment scales; match the target data alignment algorithm according to the size of the small file data block to be written and the desired integration coefficient. The alignment scale of the target data alignment algorithm is greater than the product of the size of the small file data block to be written and the integration coefficient, and the difference between the alignment scale and the product of the size of the small file data block and the integration coefficient is the smallest.
4. The hard disk space management optimization method according to claim 1, wherein Obtain the size of the data block to be written, compare whether the size of the data block to be written is greater than the set size. If so, determine that the data block to be written is a large file data block, otherwise determine that the data block to be written is a small file data block.
5. The hard disk space management optimization method according to claim 1, characterized in that When storing metadata, overwrite the metadata during writing with the merged metadata.
6. An optimized system for hard disk storage space management, characterized in that, including: A scene differentiation module, which determines whether the current is a large file scenario or a small file scenario according to the relationship between the size of the data block to be written and the set size of the blob; A metadata mapping structure creation module, which in a large file scenario, creates a mapping of the blob with the same size as the large file data block written to the target disk storage space, and maps the large file data block to the target disk storage space; wherein, in the large file scenario, the BlueStore metadata mapping structure is: an Onode is mapped to a number of lextents through a basic data management unit mapping map, each lextent corresponds to one blob, and one blob corresponds to a continuous pextent on the disk space, realizing a one-to-one correspondence relationship among lextent, blob, and pextent; The metadata mapping structure creation module creates blobs of a set size in a small file scenario, each blob supports multiple lextents, each blob is mapped to the target disk storage space, and at least one small file data block is mapped to the target disk storage space; the BlueStore metadata mapping structure in the small file scenario is: an Onode is mapped to a number of lextents through a basic data management unit mapping map, each lextent is mapped to one blob, the blob allows multiple lextents to be mapped, and each blob corresponds to a continuous pextent on the disk space; A data alignment module, which in a small file scenario, uses a data alignment algorithm to align small file data blocks to merge the amount of metadata entries; A metadata merging module, which merges the metadata of each small file data block before data alignment into one piece of metadata according to the alignment grouping.
7. An electronic device for implementing an optimized method for hard disk space management, characterized in that, Including: At least one processing unit, a storage unit, and a bus unit, the processing unit is connected to the storage unit through the bus unit, the storage unit includes a metadata storage area, a data storage area, and a computer program storage area, the computer program storage area stores a computer program, and when the computer program is executed by the processing unit, it implements the hard disk space management optimization method according to any one of claims 1-5.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the hard disk space management optimization method according to any one of claims 1-5.
Citation Information
Patent Citations
Small file positioning method and system
CN104965845A
Data processing method and system, electronic equipment and storage medium
CN111880734A