Method, electronic device and computer program product for managing a file system

Abstract

According to an example embodiment of the present disclosure, a method, an electronic device and a computer program product for managing a file system are provided. The method comprises: causing a plurality of data blocks to be written into a storage space of the file system, each data block in the plurality of data blocks having a size no less than a threshold size; and creating at least one space fragment in the storage space, each space fragment in the at least one space fragment having a size no more than the threshold size. Thus, the present solution can age the file system quickly and efficiently.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure generally relate to a method, device and computer program product for managing a file system. BACKGROUND

[0002] In the process of user operations in a file system, a large amount of data unavailability and data loss time is caused by timing, lock, thread and content conflicts, which is particularly evident in a file system after long-term use, in which problems such as out-of-memory (OOM), emergency, file system offline, thread blocking, etc. frequently occur. Long-term use of a file system will age it and reduce its performance, and the performance loss of an aged file system on a solid state drive (SSD) can be more severe than that on a hard disk drive (HDD).

[0003] In a conventional solution, in order to discover these problems, a large number of test operations such as unit testing, function testing, integration testing, stress testing and endurance testing are usually performed. However, since a user's file system is usually aged after long-term use, a storage array recovers from many failures, uninterrupted upgrades are rarely performed and reinitialization is rarely performed, while most current system tests are run on newly created storage arrays, there is a significant difference between the system test environment and the user's file system. However, it is difficult to discover the above-mentioned problems in the general system environment test. SUMMARY

[0004] Embodiments of the present disclosure provide a method, device and computer program product for managing a file system.

[0005] In a first aspect of the present disclosure, a method for managing a file system is provided. The method comprises causing a plurality of data blocks to be written to a storage space of the file system, each data block of the plurality of data blocks having a size no less than a threshold size; and creating at least one space fragment in the storage space, each space fragment of the at least one space fragment having a size no more than the threshold size.

[0006] In a second aspect of the present disclosure, an electronic device is provided. The device comprises at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform actions comprising: causing a plurality of data blocks to be written to a storage space of the file system, each data block of the plurality of data blocks having a size no less than a threshold size; and creating at least one space fragment in the storage space, each space fragment of the at least one space fragment having a size no more than the threshold size.

[0007] In a third aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer readable medium and comprises machine executable instructions that, when executed, cause a machine to implement any of the steps of the method described according to the first aspect of the present disclosure.

[0008] The summary is provided to introduce a selection of concepts, in a simplified form, that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the disclosure, and is not intended to limit the scope of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures. In the drawings:

[0010] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart illustrating a method of managing a file system according to an embodiment of the present disclosure is shown;

[0012] Figure 3 A schematic diagram illustrating a method of managing a file system according to an embodiment of the present disclosure is shown;

[0013] Figure 4 A schematic diagram illustrating an environment of managing a file system according to an embodiment of the present disclosure is shown;

[0014] Figure 5 A schematic diagram illustrating a method of managing a file system according to an embodiment of the present disclosure is shown;

[0015] Figure 6 A schematic diagram illustrating a method of managing a file system according to an embodiment of the present disclosure is shown;

[0016] Figure 7 A schematic diagram illustrating a method of managing a file system according to an embodiment of the present disclosure is shown;

[0017] Figure 8 A schematic diagram illustrating a method of testing on an aged file system according to an embodiment of the present disclosure is shown;

[0018] Figure 9 A schematic diagram illustrating a device for recording a file system management process according to an embodiment of the present disclosure is shown;

[0019] Figure 10 A schematic diagram illustrating a file system performance according to an embodiment of the present disclosure is shown;

[0020] Figure 11 a diagram illustrating file system performance according to embodiments of the present disclosure is shown;

[0021] Figure 12 a diagram illustrating file system performance according to embodiments of the present disclosure is shown;

[0022] Figure 13 a diagram illustrating file system performance according to embodiments of the present disclosure is shown; and

[0023] Figure 14 a block diagram illustrating an example device that can be used to implement embodiments of the present disclosure is shown.

[0024] In the various drawings, like or corresponding elements are denoted by like or corresponding reference numerals. DETAILED DESCRIPTION

[0025] Preferred embodiments of the present disclosure will be described in greater detail below, with reference made to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art.

[0026] The term "including" and variations thereof, as used in this document, mean "including, but not limited to," unless expressly specified otherwise. The term "or" means "and / or" unless expressly specified otherwise. The term "based on" means "based, at least in part, on" unless expressly specified otherwise. The terms "one example embodiment" and "an example embodiment" mean "at least one example embodiment." The term "another embodiment" means "at least one additional embodiment." The terms "a first," "a second," etc. can refer to different or identical objects. Other explicitly and implicitly recited definitions can also be possible.

[0027] Generally, a file system has its own life cycle. End users create, read, write, truncate, delete, and copy files, directories, and links in the file system. The file system can be migrated or copied from one place to another. A snapshot can be taken of the file system, resulting in block sharing and write splitting. Sometimes the file system can be taken offline or corrupted. An FSCK (FSCK is used to check and maintain an inconsistent file system, if the system is powered off or the disk has a problem, the fsck command can be used to check the file system) can be performed to recover the file system. From the end user and the storage implementation of the file system perspective, there are many factors that age the file system, including but not limited to the following: fragmentation, cumulative operations, reuse of data structures (inodes, indirect blocks, etc.), resource reuse (index node numbers), re-allocating data blocks, recovering from failures (offline) FSCK, etc.

[0028] Conventionally, stress tests, durability tests, data migration tests, customer upgrade tests, etc. are performed in newly created file systems, but cannot reveal potential problems and performance bottlenecks of the file system, especially after the storage space in the file system is aged.

[0029] To at least partially address one or more of the issues described above and other potential issues, example embodiments of the present disclosure propose a scheme to manage a file system. In the scheme, data is first written in a plurality of sub-storage spaces of a storage space of the file system. The written data is then operated on to generate space fragmentation, thereby aging the file system. In this way, the scheme can simulate a customer file system that has been used for a long time in a short time, helping to reveal potential problems in a newly designed file system and can identify performance bottlenecks of the file system, improving the testing efficiency and effectiveness of subsequent tests.

[0030] In the following, specific examples of the scheme will be described in more detail. Figures 1 to 14 Specific examples of the scheme will be described in more detail. Figure 1 A schematic diagram showing an example of a backup system 100 according to an embodiment of the present disclosure is shown. Figure 1 A block diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. As shown in Figure 1 As shown, the environment 100 includes a host 110, a storage manager 120, and a file system 130. It should be understood that the structure and functionality of the environment 100 are described for illustrative purposes only and are not intended to imply any limitation on the scope of the present disclosure. For example, embodiments of the present disclosure can also be applied to environments different from the environment 100.

[0031] The file system 130 can include one or more storage spaces, such as disks, optical disks, hard disk drives (HDDs), solid state drives (SSDs), and the like. Each storage space can be divided into a plurality of sub-storage spaces. For example, each sub-storage space can have the same size. The file system 130 can include various types of storage spaces, such as a storage space for storing user data (also referred to as a "user data storage space"), a storage space for storing metadata related to the storage system (also referred to as a "metadata storage space"), and the like, according to different types of data stored or according to logical tiers divided. The metadata storage space can store mapping information, index information, status information, and the like related to the storage system, such as mapping information of storage spaces to physical disks, status of storage spaces (such as normal status or failure status), and the like. Data stored in the plurality of sub-storage spaces in the storage space can be inter-associated.

[0032] The storage manager 120 can include a processor 121 and a memory 122. The memory 122 can be any volatile memory media, non-volatile memory media, or a combination of the two now known or later developed. The storage manager 120 can be configured to manage the file system 130 and handle input / output (I / O) requests from the host 110. The host 110 can be any physical computer, virtual machine, server, or the like running a user application.

[0033] The host 110 can send I / O requests to the storage manager 120, such as for removing data from and / or writing data to a storage space in the file system 130, and the like. Metadata of a target storage space can be stored in the metadata storage space. In response to receiving an I / O request from the host 110, the storage manager 120 can first obtain metadata of the target storage space from the metadata storage space, which can indicate mapping information of the target storage space to physical disks, status of the target storage space, and the like. In response to the I / O request being a read request, the storage manager 120 can forward the I / O request to the target storage space based on the obtained metadata to read data from the target storage space, and return the read data to the host 110. In response to the I / O request being a write request, the storage manager 120 can forward the I / O request to the target storage space based on the obtained metadata to write data to the target storage space.

[0034] Figure 2 A flowchart of an example method 200 for managing storage disks according to embodiments of the present disclosure is shown. The method 200 can be performed, for example, by the storage manager 120 as shown. It should be understood that the method 200 can also include additional actions not shown and / or can omit actions shown, without limitation to the scope of the present disclosure. The following describes the method 200 in conjunction with the Figure 1 Figure 1 ​Let me describe method 200 in detail.

[0035] like Figure 2 As shown in block 210, storage manager 120 writes multiple data blocks to the storage space of file system 130, with each data block being at least a threshold size. For example, storage manager 120 may respond to a write request from host 110 by writing data blocks of the same or different sizes to multiple sub-storage spaces within the storage space of file system 130. Here, the threshold size may be the minimum sub-storage space size of file system 130, for example, 8KB. Note that the minimum storage space size may differ for different file systems, and therefore different threshold sizes can be set; this disclosure does not impose any limitations on this.

[0036] At box 220, storage manager 120 creates at least one space fragment in storage space 130, where the size of each space fragment does not exceed the threshold size. For example, storage manager 120 may, in response to an erase or rewrite request initiated by host 110, perform operations such as removing, changing compression ratios, or rewriting multiple data blocks in multiple sub-storage spaces within the storage space of file system 130 written in 210, to create multiple space fragments in the file system. Here, a space fragment refers to a non-contiguous space within multiple sub-storage spaces of the storage space, such space being smaller than the minimum storage space size and unable to be directly written to.

[0037] The following will be combined Figures 3 to 7 Please describe steps 210 and 220 above in detail.

[0038] Figure 3 A schematic diagram 300 of a managed file system according to an embodiment of the present disclosure is shown. Storage manager 120 writes multiple data blocks into subspaces 301, 302, 303, 304, 305, 306, and 307 of the storage space of file system 130. These multiple subspaces can be new, empty storage spaces, and the present disclosure does not impose limitations thereon. As shown in FIG130', the multiple data blocks, after being written, each occupy a subspace in the storage space. Storage manager 120 then removes at least two of the written data blocks from the storage space. For example, as shown in 130”, the blank rectangles represent the subspaces occupied by the removed data blocks, i.e., space fragments. The subspaces occupied by the data blocks before removal are not adjacent. Thus, space fragments, as shown by the blanks in 130”, are created, where the space occupied by the space fragments is no larger than the size of the written data blocks and they are not adjacent.

[0039] In one embodiment, storage manager 120 writes data blocks of the same size (e.g., 8KB, where 8KB is the minimum sub-storage space size) to storage space 310 of file system 130 via I / O tools, then removes the written data blocks at intervals, and disables space reclamation functions such as FSR during removal. This space reclamation function prevents the recombination of empty sub-storage spaces, thereby achieving a higher fragmentation rate. Note that the 8KB data blocks of the same size are merely exemplary.

[0040] Large 8KB data blocks can also help configure complex file structures such as 3-level DUCH.

[0041] In one example, the above 8KB data block can be written using the FIO tool. FIO is an I / O testing tool that supports multi-engine and multi-system testing. This is merely an example; internal tools developed by internal teams or external tools such as LDX can also be used to write or remove data. This disclosure does not impose any limitations. The code for writing an 8KB data block using the FIO tool can be found below, where the file can be the data block from the above process:

[0042] [global]

[0043] do_verify = 0

[0044] ioengine = libaio

[0045] iodepth=32

[0046] direct=1

[0047] readwrite = write

[0048] openfiles=40

[0049] refill_buffers = 0

[0050] numjobs=1

[0051] create_on_open = 1

[0052] bs = 8k

[0053] [_mnt_testAged_2]\Create file name\

[0054] directory = / mnt / obd2151 / afp01 / \Load Pointer\

[0055] filesize = 8k\File size created\

[0056] nrfiles = 65536 \ number of files to create, if the number is too large, the host will run out of memory

[0057] create_serialize = 1 \ if set, will have a high fragmentation rate, but will take more time to create files

[0058] The above code is merely exemplary and is not intended to limit the present disclosure.

[0059] The above-described case is the case where there is no data block compression and data block duplication storage. Data compression and deduplication are common functions to improve storage efficiency, which are widely used in file systems. The following further combines Figures 4 to 7 to describe the case of a file system supporting data compression and deduplication.

[0060] Figure 4 A schematic diagram 400 of an environment for managing a file system according to an embodiment of the present disclosure is shown. In response to receiving an I / O request from the host 110, the storage manager 120 can first obtain metadata Leaf IB-A, Leaf IB-B, Leaf IB-C and Leaf IB-D / E from the metadata storage space, wherein Leaf IB-B, Leaf IB-C and Leaf IB-D / E correspond to compressed data block B 470, compressed data block C (for the convenience of subsequent description, referred to as first compressed data block 480 in the figure and hereinafter) 470, and compressed data block D respectively, and the compressed data block A corresponding to Leaf IB-A is removed. The plurality of metadata can indicate mapping information of the storage space to the physical disk, the state of the target storage space, etc., and the plurality of metadata can be further indexed to the metadata ILC-VBM-i via offset A, offset B, offset C, offset D and offset E, which is a metadata of VBM type, and ILC represents that it can store compressed data, which is only an example, other types of data or metadata can also be applied, and the data can also be named in other ways, which is not limited by the present disclosure.

[0061] For example, the index 1 in the metadata ILC-VBM-i, where w: 10 indicates that the weight of the metadata Leaf IB-B in all the metadata 40 stored in the metadata ILC-VBM-i is 10, and the data length B indicates the data length of the compressed data block B corresponding to the metadata Leaf IB-B in the storage space. The metadata Leaf IB-D / E stores two identical compressed data, and the data weight corresponding to the metadata Leaf IB-D / E is 20, and the data corresponding to the metadata Leaf IB-D / E only occupies one word of storage space in the data, i.e., the compressed data block D storage space. Through the different weights of different data, it can be quickly determined which data is problematic. Zipheader-B, Zipheader-C, and Zipheader-D represent the address information of the stored compressed data block B 470, the first compressed data block 480, and the compressed data block D 490. The compressed data block A (not shown) corresponding to the metadata Leaf IB-A is removed from the storage space of the file system 130, Figure 4 The fork shape on Leaf IB-A in the above figure indicates that it is removed, so its weight, length, and offset in index 0 in the metadata ILC-VBM-i are 0, and there is free storage space 460 in the storage space because it is removed. The compressed data block A is removed only for the purpose of describing the embodiments of the present disclosure, and is not intended to be limiting. In the following description, the metadata will not be described again, and the operation of the storage manager 120 on the data blocks in the storage space will be directly described.

[0062] Figure 5 A schematic diagram 500 of managing a file system according to an embodiment of the present disclosure is shown. First, refer to the above description of Figure 4 As described above, the storage manager 120 can compress a first data block in the plurality of data blocks by a first compression rate to obtain a first compressed data block 480 in response to a write request of the host 110, and write it into a first sub-space in the storage space of the file system 130. The write process can refer to the description in Figure 4 The first sub-space, the second sub-space, etc. described later are a plurality of sub-storage spaces in the storage space of the file system 130, which can be storage spaces of the same size or different sizes, and the present disclosure does not limit this.

[0063] For example, the storage manager 120 can compress the first data block of 8KB by the first compression rate of 80% to obtain the first compressed data block 480 of 6.4KB, and then write it into the first sub-space, where the first sub-space correspondingly occupies 6.4KB of storage space in the storage space. The size of the data block and the compression rate described above are only exemplary, and other data sizes and compression rate sizes can also be applied according to different storage and data structures.

[0064] Then, the storage manager 120 can cause the first compressed data block to be removed from the first sub-space, for example, the storage manager 120 can remove the Offset-C corresponding to the first compressed data block 480, then the first compressed data block 480 is removed from the first sub-space and the metadata ILC-VBM-i for Offset-C is removed and the first sub-space is released. The storage manager 120 then re-compresses the first data block by a second compression rate that is greater than the first compression rate to obtain a second compressed data block 580. Since the first compression rate is less than the second compression rate, the size of the second compressed data block 580 is less than the first compressed data block 480, and thus the storage manager 120 can re-write the second compressed data block 580 to the first sub-space, thus creating a space fragment 510.

[0065] For example, continuing the example described above, the storage manager 120 can compress the 8KB first data block by a second compression rate of 70% to obtain a second compressed data block 580 of size 5.6KB, and then the storage manager 120 can write the second compressed data block 580 of size 5.6KB to the first sub-space in the storage space, and the data length C is updated to 5.6KB, thus creating a space fragment 510 of size 6.4KB-5.6KB=0.8KB, where the space fragment refers to a separate storage space that cannot store data without an operation of space recycling.

[0066] In an alternative embodiment, the first compression rate can be 100%, i.e., the first data block is not compressed, and the second compression rate can be any compression rate less than 100%, thus creating free storage space.

[0067] Similar operations can also be performed on the compressed data block B and the compressed data block D described above, which will not be described again here.

[0068] By repeatedly compressing the same data block or different data blocks with different compression rates, space fragments such as 10% of the minimum storage space or even smaller can be created, which cannot be further written into data, thus aging the file system. In addition, the re-write IO with different compression data rates can trigger the write split of the data block, and in the case where the original metadata is not associated with the storage compressed data block, it can also cause the increase and deletion of metadata related to the implementation of data block sharing, thus further aging the file system.

[0069] Figure 6 A schematic diagram 600 of managing a file system according to an embodiment of the present disclosure is shown. Figure 6 With Figure 5The difference lies in the fact that the size of the rewritten compressed data block is larger than the size of the originally written compressed data block, so it cannot be written into the storage space it originally occupied. Storage manager 120 first responds to multiple write requests from host 110 by performing multiple write operations. Storage manager 120 writes the second data block from the multiple data blocks into the second subspace of the storage space, then compresses the third data block from the multiple data blocks at a third compression ratio to obtain a third compressed data block, and writes the third compressed data block into the third subspace of the storage space. The size of the second data block is larger than the size of the third data block, and the second and third subspaces are not adjacent.

[0070] For example, first refer to Figure 5 The storage manager 120 can first write an 8KB compressed data block A520 to the second subspace of the storage space of the file system 130. Then, it can compress the 8KB third data block to obtain a third compressed data block of size 5.6KB (shown as the second compressed data block in the figure) through a third compression ratio of 70%. The storage manager 120 can then write the 5.6KB third compressed data block to the third subspace of the storage space. As shown in the figure, there is also a compressed data block B between compressed data block A and the third subspace.

[0071] Then, the storage manager 120 removes the second data from the second subspace, removes the third compressed data block from the third subspace, compresses the third data block at a fourth compression rate less than the third compression rate to obtain a fourth compressed data block, and finally writes the fourth compressed data block into the second subspace.

[0072] For example, storage manager 120 can remove an 8KB compressed data block A520 from the second subspace of the storage space of file system 130, such as... Figure 4 As shown, after removal, all data in the VBM metadata is updated to 0, and then the 5.6KB third compressed data block is removed from the third subspace. Storage manager 120 obtains a 7.2KB fourth compressed data block 610 with a fourth compression ratio of 90%. Since the fourth data block 610 is larger than the size of the third subspace (i.e., the size of the third compressed data block is 5.6KB), it cannot be written to the third subspace. Because the size of the second subspace (i.e., the size of compressed data block A520 is 8KB) is larger than the size of the fourth data block 610, storage manager 120 can write the 7.2KB fourth compressed data block 610 to the second subspace in the storage space, and the data length C is updated to 7.2KB. Thus, as... Figure 6 As shown, space fragment 620 with a size of 8KB - 7.2KB = 0.8KB and space fragment 630 with a size of 5.6KB were created.

[0073] In an alternative embodiment, since the metadata ILC-VBM-i can store up to 12 compressed data entries (the number 12 is exemplary and only represents the attributes of different types of metadata), if the metadata ILC-VBM-i does not have a suitable block to store the newly written compressed data, a new metadata VBM should be created to store the compressed data.

[0074] The above describes how various spatial fragments are created by writing and removing compressed data. The following describes the case of duplicate data.

[0075] Figure 7 A schematic diagram 700 of a managed file system according to an embodiment of the present disclosure is shown. As described above regarding... Figure 4 The data discussed, Offset-D and Offset-E, are duplicate data that share a single data block. Storage manager 120 rewrites Offset-E at different compression / deduplication rates, causing Offset-D and Offset-E to no longer share the same data block. Offset-E must undergo a write split, i.e., rewriting the recompressed data block, the write process as described above. Figures 5 to 7 The rewriting of Offset-C is similar and will not be repeated here. Storage Manager 120 also adds new records related to Offset-E in ILC-VBM-i, such as its weight, length, etc.

[0076] like Figure 7 As shown, compressed data block E710, rewritten at a different compression ratio, occupies the storage space previously occupied by compressed data block A, thus creating space fragment 720.

[0077] In one embodiment, the storage manager 120 may also perform the operations described above, such as compression rewriting, on compressed data block B470 and compressed data block C730 to create more space fragments.

[0078] In an alternative embodiment, performing the write and remove operations of the aforementioned data blocks simultaneously can result in snapshot creation / deletion operations with different file size I / O and the creation of more space fragments. These operations can test more metadata areas, such as inodes (inode cache, inode IDs with generation numbers for reuse) and accelerate file system aging.

[0079] Figure 8 A schematic diagram 800 illustrates a test performed on an aged file system according to an embodiment of this disclosure. The test can be conducted via the above-mentioned... Figures 2 to 7The described space fragmentation creation operation is used to age file systems 1 (810), 2 (820), and so on, up to 30 (830). Test cases 1 (840), 2 (850), and 30 (860) can include, for example, unit tests, functional tests, integration tests, stress tests, and endurance tests. There are two ways to age and test the file systems. In one approach (as shown by solid lines), the storage manager 120 can first age multiple file systems and then test the aged file systems via multiple test cases. In another approach (as shown by dashed lines), the storage manager 120 can test the file systems simultaneously with aging them.

[0080] Figure 9 A schematic diagram 900 of an apparatus for detecting the aging process of a file system according to an embodiment of this disclosure is shown. This disclosure proposes using an Elastic Stack system to detect and log traces and events during the aging process of a file system 130. Filebeat 920, a tool mounted on the storage space of the file system, is used to read the target trace and send the data to Logstash 930 using a backpressure-sensitive protocol. Logstash 930 can extract and transform the data, then load it into Elasticsearch 940, the core software of Elastic Core, for logging the transformed log traces and events. Finally, Kibana 950 will display detailed log trace information in a GUI, telling the user when, where, and how the event occurred, as well as feedback on the aging operation. The aging profile (workload and operations) will be modified based on the results obtained from the Elastic Stack.

[0081] For example, the storage manager 120 can use the aforementioned system to obtain data fragments, metadata fragments, free space fragments, and memory fragments of the file system 130 to determine the aging rate of the file system 130. The storage manager 120 can also use the aforementioned system to obtain the frequency of conflict operations (block sharing, block reallocation, space shrinking / expansion, etc.) and interrupt operations to determine the aging rate of the file system 130.

[0082] In one embodiment, the storage manager 120 may age the file system 130 by the following steps: (1) first configuring the file system 130 using client data on memory and disk. The client data is pre-defined, for example, based on the data structure frequently used by the client (hash table, DB, DUCH, dentry, etc.), whether it is a local file system or a global file system; (2) aging the file system 130 using internal and external (FIO, LDX) tools via the defined workload profile (IO workload and file system operations). Specific aging methods can be found in [reference needed]. Figures 2 to 7 The method for creating fragments is described; (3) the aging rate of file system 130 is monitored and determined by the Elastic Stack system; (4) the operation of step (2) is continuously performed based on different test purposes to obtain the predetermined aging rate.

[0083] Figures 10 to 13 Schematic diagrams 1000, 1100, 1200, and 1300 illustrate the performance of a file system according to embodiments of this disclosure. Curves 1030, 1130, 1230, and 1330 represent a 50GB file system filled with 8KB data blocks, then intermittently deleted, requiring 15 hours to configure. Curves 1020, 1120, 1220, and 1320 represent a 50GB file system filled with 16KB data blocks, then intermittently deleted, requiring 6 hours to configure. Curves 1010, 1110, 1210, and 1310 represent a newly created 50GB file system. It can be seen that the aging method using 8KB data blocks with intermittent deletion, i.e., curves 1030, 1130, 1230, and 1330, has the longest file system response time, the lowest I / O, the lowest bandwidth, and the highest I / O per second. Therefore, for this type of file system, aging with 8KB data blocks and intermittent deletion is the most effective. This example is merely illustrative; different sizes of small data blocks can be used for aging operations depending on the file system and data structure.

[0084] The performance graphs above are merely exemplary. In one embodiment, the storage manager 120 may also determine the performance of the file system 130 by at least one of the following: the response time of the file system, the average bandwidth of the file system reading / writing data, the number of read / write operations per second of the file system, and the failure rate of the file system.

[0085] Figure 14 A schematic block diagram of an example device 1400 that can be used to implement embodiments of the present disclosure is shown. For example, such as Figure 1The storage manager 120 shown can be implemented by device 1400. As shown, device 1400 includes a central processing unit (CPU) 1401, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 1402 or loaded from storage space 1408 into random access memory (RAM) 1403. Various programs and data required for the operation of device 1400 can also be stored in RAM 1403. CPU 1401, ROM 1402, and RAM 1403 are interconnected via bus 1404. Input / output (I / O) interface 1407 is also connected to bus 1404.

[0086] Multiple components in device 1400 are connected to I / O interface 1470, including: input unit 1406, such as a keyboard, mouse, etc.; output unit 1407, such as various types of displays, speakers, etc.; storage space 1408, such as a disk, optical disk, etc.; and communication unit 1409, such as a network card, modem, wireless transceiver, etc. Communication unit 1409 allows device 1400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] The various processes and handling described above, such as method 200, can be executed by processing unit 1401. For example, in some embodiments, method 200 can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage space 1408. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1400 via ROM 1402 and / or communication unit 1409. When the computer program is loaded into RAM 1403 and executed by CPU 1401, one or more actions of method 200 described above can be performed.

[0088] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0089] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0090] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0091] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0092] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0094] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0096] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for managing a file system, comprising: Multiple data blocks are written to the storage space of the file system, wherein the size of each data block is greater than or equal to a threshold size; as well as This causes the creation of space fragments in the storage space, each of which is no larger than the threshold size, and the space fragments include discontinuous spaces in the storage space; The space debris mentioned above is created based on the following: The given multiple write data blocks are repeatedly compressed to a target size not exceeding a threshold size, so that data cannot be further written to the multiple space fragments created, thereby aging the file system for testing the aged file system through multiple test cases, wherein each compression in the repeated compression is performed at a different compression ratio. The multiple space fragments created do not include compressed data blocks; and The repeated compression of the given write data blocks includes: The first data block among the plurality of data blocks is compressed at a first compression ratio to obtain a first compressed data block; Write the first compressed data block into the first subspace of the storage space; Remove the first compressed data block from the first subspace; The first compressed data block is compressed at a second compression ratio to obtain a second compressed data block, wherein the first compression ratio is less than the second compression ratio; and The second compressed data block is written into the first subspace to create a first space fragment.

2. The method according to claim 1, further comprising: Write the second data block from the plurality of data blocks into the second subspace of the storage space; The third data block among the plurality of data blocks is compressed using a third compression ratio to obtain a third compressed data block; as well as The third compressed data block is written into the third subspace of the storage space. The size of the second data block is larger than the size of the third data block, and the second subspace and the third subspace are not adjacent.

3. The method according to claim 2, further comprising: Remove the second data block from the second subspace; The third compressed data block is compressed using a fourth compression ratio to obtain a fourth compressed data block, wherein the fourth compression ratio is less than the third compression ratio; as well as Write the fourth compressed data block into the second subspace.

4. The method according to claim 3, further comprising: The third compressed data block is removed from the third subspace.

5. The method according to claim 1, further comprising: After the space fragments are created, the performance of the file system is determined, and the performance includes at least one of the following: the response time of the file system, the average bandwidth of the file system for reading / writing data, the number of read / write operations per second of the file system, and the failure rate of the file system.

6. An electronic device, comprising: At least one processing unit; At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform an action, the action including: Multiple data blocks are written to the file system's storage space, wherein the size of each data block is greater than or equal to a threshold size; and This causes the creation of space fragments in the storage space, each of which is no larger than the threshold size, and the space fragments include discontinuous spaces in the storage space; The space debris mentioned above is created based on the following: The given multiple write data blocks are repeatedly compressed to a target size not exceeding a threshold size, so that data cannot be further written to the multiple space fragments created, thereby aging the file system for testing the aged file system through multiple test cases, wherein each compression in the repeated compression is performed at a different compression ratio. The multiple space fragments created do not include compressed data blocks; and The repeated compression of the given write data blocks includes: The first data block among the plurality of data blocks is compressed at a first compression ratio to obtain a first compressed data block; Write the first compressed data block into the first subspace of the storage space; Remove the first compressed data block from the first subspace; The first compressed data block is compressed at a second compression ratio to obtain a second compressed data block, wherein the first compression ratio is less than the second compression ratio; and The second compressed data block is written into the first subspace to create a first space fragment.

7. The electronic device according to claim 6, wherein the size of each of the plurality of data blocks is a preset size.

8. The electronic device according to claim 6, wherein the action further includes: Write the second data block from the plurality of data blocks into the second subspace of the storage space; The third data block among the plurality of data blocks is compressed using a third compression ratio to obtain a third compressed data block; as well as The third compressed data block is written into the third subspace of the storage space. The size of the second data block is larger than the size of the third data block, and the second subspace and the third subspace are not adjacent.

9. The electronic device according to claim 8, wherein the action further includes: Remove the second data block from the second subspace; The third compressed data block is compressed using a fourth compression ratio to obtain a fourth compressed data block, wherein the fourth compression ratio is less than the third compression ratio; as well as Write the fourth compressed data block into the second subspace.

10. The electronic device of claim 9, wherein the action further comprises: The third compressed data block is removed from the third subspace.

11. The electronic device of claim 6, wherein the action further comprises: After the space fragments are created, the performance of the file system is determined, and the performance includes at least one of the following: the response time of the file system, the average bandwidth of the file system for reading / writing data, the number of read / write operations per second of the file system, and the failure rate of the file system.

12. A computer program product tangibly stored on a non-transient computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the steps of the method according to any one of claims 1 to 5.

Images