Cache file management method and device, storage medium and computer program product

Abstract

The invention provides a cache file management method and device, a storage medium and a computer program product, and relates to the technical field of computers. The cache file management method comprises the following steps: determining the file size of a to-be-created file according to a file creation request; a target file set corresponding to the file size of the to-be-created file in a cache file pool is determined, the cache file pool comprises a plurality of file sets, and the plurality of file sets are provided with cache file size ranges which are not overlapped and different hierarchy numbers; the cache file size range of the file set with the smaller hierarchy number is smaller than the cache file size range of the file set with the larger hierarchy number; according to the target file set, a cache file with the file size closest to the file size of the to-be-created file is retrieved to serve as the to-be-created file; and renaming the to-be-created file to serve as the created business file.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a cache file management method and device, a storage medium, and a computer program product. Background Art

[0002] In existing file systems, when an application needs to create or delete a file, it needs to call the create (creat) / delete (unlink or remove) interface provided by the file system to trigger a disk IO (input / output) operation to complete the file creation / deletion operation. Summary of the invention

[0003] The inventors noticed that in the related art, since the application will trigger a large number of parallel create / delete file operations, the device where the file system is located will be in an IO busy state, which will significantly affect the business performance of the file system and reduce the overall availability and stability of the file system.

[0004] Accordingly, the present disclosure provides a cache file management method, which effectively reduces IO pressure and improves system availability and stability by directly calling cache files instead of creating new files.

[0005] In a first aspect of the present disclosure, a cache file management method is provided, comprising: determining a file size of a file to be created according to a file creation request; determining a target file set in a cache file pool corresponding to the file size of the file to be created, the cache file pool comprising a plurality of file sets, the plurality of file sets being provided with non-overlapping cache file size ranges and different numbers of levels, the cache file size range of a file set having a smaller number of levels being smaller than the cache file size range of a file set having a larger number of levels; retrieving a cache file whose file size is closest to the file size of the file to be created according to the target file set as the file to be created; and renaming the file to be created as the created business file.

[0006] In some embodiments, retrieving the cache file includes: determining whether the target file set is empty; and retrieving the target cache file from the target file set if the target file set is not empty.

[0007] In some embodiments, when the target file set is empty, determine whether the number of levels of the target file set is greater than the minimum number of levels; when the number of levels of the target file set is greater than the minimum number of levels, use the file set in the cache file pool whose number of levels is less than the number of levels of the target file set as the file set to be detected; and search each file set in the file set to be detected in order of the number of levels from large to small until the target cache file is retrieved.

[0008] In some embodiments, when the target cache file is not retrieved in the to-be-detected file set, a file creation request is sent to a file system so that the file system creates the business file.

[0009] In some embodiments, when the number of levels of the target file set is equal to the minimum number of levels, a file creation request is sent to a file system so that the file system creates the business file.

[0010] In some embodiments, after the business file is used up, the current file size of the business file is detected; and the business file is stored in a destination file set corresponding to the current file size in the cache file pool.

[0011] In some embodiments, when the number of levels of the target file set is not the minimum number of levels, or when the number of levels of the target file set is the minimum number of levels and the total number of cached files in the target file set does not reach the upper limit of the total number of files of the target file set, the business file is stored in the file set; when the number of levels of the target file set is the minimum number of levels and the total number of cached files in the target file set reaches the upper limit of the total number of files, the business file is deleted.

[0012] In some embodiments, determining the file size of the file to be created includes: determining the type of the file to be created according to the file creation request; and determining the file size of the file to be created according to the type of the file to be created.

[0013] In some embodiments, the total number of cached files in each of the multiple file sets is detected; when the total number of cached files in the i-th file set is greater than the upper limit of the total number of files in the i-th file set, it is determined whether the number of levels of the i-th file set is the minimum number of levels, where i is a natural number not greater than the total number of file sets; when the number of levels of the i-th file set is not the minimum number of levels, a specified number of cached files in the i-th file set is moved into the j-th file set, where the specified number is not less than the difference between the total number of cached files in the i-th file set and the upper limit of the total number of files, the number of levels of the i-th file set is greater than the number of levels of the j-th file set, where j is a natural number not greater than the total number of file sets; and the file sizes of the specified number of cached files are adjusted to the target file size of the j-th file set.

[0014] In some embodiments, when the number of levels of the i-th file set is the minimum number of levels, a predetermined number of cache files in the i-th file set is deleted, wherein the predetermined number is the difference between the total number of cache files in the i-th file set and the upper limit of the total number of files in the i-th file set.

[0015] In some embodiments, the total number of cached files in each file set of the multiple file sets is detected; when the total number of cached files in the i-th file set is less than the lower limit of the total number of files in the i-th file set, it is determined whether the number of levels of the i-th file set is the maximum number of levels, and i is a natural number not greater than the total number of file sets; when the number of levels of the i-th file set is not the maximum number of levels, the file set in the cache file pool whose number of levels is greater than the number of levels of the i-th file set is used as the file set to be processed; in descending order of the number of levels, the cached files in each file set of the file set to be processed are moved into the i-th file set in sequence until the total number of cached files in the i-th file set is not less than the lower limit of the total number of files in the i-th file set; the file size of the cached files moved into the i-th file set is adjusted to the target file size of the i-th file set.

[0016] In some embodiments, the file size of each cache file in each file set of the plurality of file sets is detected; if the file size of the mth cache file in the ith file set is larger than the target file size of the ith file set, the file size of the mth cache file is adjusted to the target file size of the ith file set, wherein , , N is the total number of file sets, and Mi is the total number of cached files in the i-th file set.

[0017] In some embodiments, the total number of cache files in the cache file pool is detected; when the total number of cache files is greater than a predetermined total number of files threshold, the cache files in the cache file pool are deleted in order of file size from large to small until the total number of cache files in the cache file pool is no greater than the total number of files threshold.

[0018] In some embodiments, the total size of cache files in the cache file pool is detected; when the total size of cache files is greater than a predetermined total file size threshold, the cache files in the cache file pool are deleted in order of file size from large to small until the total size of cache files in the cache file pool is no greater than the total file size threshold.

[0019] In a second aspect of the present disclosure, a cache file management device is provided, comprising: a memory; a processor, coupled to the memory, the processor being configured to execute a cache file management method as described in any of the above embodiments based on instructions stored in the memory.

[0020] In a third aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the cache file management method as described in any of the above embodiments is implemented.

[0021] In a fourth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the cache file management method as described in any of the above embodiments is implemented.

[0022] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 The present invention is a flowchart of a cache file management method according to an embodiment of the present invention.

[0025] Figure 2 A schematic diagram of the structure of a cache file pool according to an embodiment of the present disclosure.

[0026] Figure 3 A schematic diagram of cache file recovery according to an embodiment of the present disclosure.

[0027] Figure 4 The present invention is a schematic diagram of cache file processing according to an embodiment of the present invention.

[0028] Figure 5 A schematic diagram of cache file movement according to an embodiment of the present disclosure.

[0029] Figure 6 This is a schematic diagram of cache file movement according to another embodiment of the present disclosure.

[0030] Figure 7 This is a schematic diagram of cache file movement according to another embodiment of the present disclosure.

[0031] Figure 8 The present invention is a schematic diagram of the structure of a cache file management device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0033] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0034] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0035] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0036] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0037] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0038] The inventors have noticed that in file systems such as MySQL, some business logics involve operations to create / delete tables. For example, some business logics are implemented based on temporary tables. A temporary table created in a session (client connection) is only visible to the current session. When the session ends (i.e., the client connection is disconnected), the temporary table is automatically deleted. Each time a temporary table is created, a table structure file for the temporary table is created in a temporary directory. When the session ends, the table structure file is deleted. When the above-mentioned types of business are executed in parallel on the same database, a large number of files will be created / deleted in parallel. In this scenario, the business performance of the database will experience severe jitters. In addition, due to the frequent creation / deletion of files, the device where the database is located is in an IO busy state, and other applications or services running on the device will also experience IO jams.

[0039] The inventor also noted that MySQL uses binlog (binary log) to record data change operations, in which binlog file writing always adopts append writing mode (i.e., append only). In the case of long-term operation, the binlog file will continue to expand. In order to shrink the storage space, it is necessary to release the space occupied by the historical binlog log (for example, recycle it once a week). In order to facilitate the recycling operation, MySQL splits the binlog data into several binlog files, that is, limits the size of a single binlog file. When the file reaches the specified size, a new file is created and the subsequent data is stored in this new file. In this way, when recycling historical data, it is only necessary to delete the binlog files that are earlier than a certain time point in history. The operation of splitting the binlog file is called "rotate". When the binlog file write triggers the rotate operation, business performance jitter will occur. In addition, since the binlog file is a global log (i.e., the changes generated by different sessions of the entire MySQL instance will be written to the binlog file), a global lock is required when writing the binlog file, and thus, the above-mentioned rotate operation is also locked. When the binlog file split is triggered (that is, a new binlog file is created), the parallel write performance will be further degraded.

[0040] Based on this, the present disclosure provides a cache file management method, which can effectively avoid affecting the business performance of the system and effectively improve the overall availability and stability of the system.

[0041] Figure 1The following is a flow chart of a cache file management method according to an embodiment of the present disclosure. In some embodiments, the following cache file management method is executed by a cache file management device, including steps 11-14.

[0042] In step 11, the file size of the file to be created is determined according to the file creation request.

[0043] It should be noted here that the file size of the file to be created is the size of the storage space occupied by the file to be created.

[0044] In some embodiments, the type of the file to be created is determined based on the file creation request, and then the file size of the file to be created is determined based on the type of the file to be created.

[0045] For example, if the file to be created is a log file, the file size of the file to be created is relatively large. If the file to be created is a metadata file or a configuration file, the file size of the file to be created is relatively small.

[0046] In step 12, a target file set corresponding to the file size of the file to be created in the cache file pool is determined.

[0047] It should be noted here that the cache file pool includes multiple file sets, and the multiple file sets have non-overlapping cache file size ranges and different numbers of levels. The cache file size range of the file set with a smaller number of levels is smaller than the cache file size range of the file set with a larger number of levels.

[0048] For example, Figure 2 As shown, the cache file pool includes 3 file sets. The level number of file set 1 is Level0, and file set 1 includes 5 cache files. The size range of cache files in file set 1 is less than 1M, that is, the file sizes of cache files included in file set 1 are all less than 1M. The level number of file set 2 is Level1, and file set 2 includes 3 cache files. The size range of cache files in file set 2 is 1M to 128M, that is, the file sizes of cache files included in file set 1 are all between 1M and 128M. The level number of file set 3 is Level2, and file set 3 includes 2 cache files. The size range of cache files in file set 3 is greater than 128M, that is, the file sizes of cache files included in file set 3 are all greater than 128M.

[0049] It should also be noted that a target file size can be set for each file set so that the file size of the cache file stored in each file set is adjusted to the target file size of the file set. For example, the target file size of file set 1 is 64K, the target file size of file set 2 is 64M, and the target file size of file set 3 is 256M.

[0050] In step 13, according to the target file set, a cache file whose file size is closest to the file size of the file to be created is retrieved as the file to be created.

[0051] In some embodiments, it is determined whether the target file set is empty, and if the target file set is not empty, the target cache file is retrieved from the target file set.

[0052] For example, if the file to be created is 100M, and the cache file size range of file set 2 in the cache file pool is 1M to 128M, the file to be created corresponds to file set 2 in the cache file pool. If file set 2 includes 3 cache files, the cache file with the file size closest to 100M is retrieved from the 3 cache files as the file to be created.

[0053] It should be noted here that by retrieving the cache file whose file size is closest to the file size of the file to be created from the target file set, the frequency of subsequent file expansion operations can be effectively avoided, further improving resource utilization efficiency.

[0054] In some embodiments, when the target file set is empty, it is determined whether the number of levels of the target file set is greater than the minimum number of levels. When the number of levels of the target file set is greater than the minimum number of levels, a file set in the cache file pool whose number of levels is less than that of the target file set is used as the file set to be detected. Next, each file set in the file set to be detected is retrieved in order of the number of levels from large to small until the target cache file is retrieved.

[0055] For example, if the file to be created is 100M, and the cache file size range of file set 2 in the cache file pool is 1M to 128M, the file to be created corresponds to file set 2 in the cache file pool. Since file set 2 is empty, and the level number Level1 of file set 2 is greater than the level number Level0 of file set 1, a search is performed in file set 1 to retrieve the cache file with a file size closest to 100M.

[0056] For another example, if the file to be created is 500M, and the cache file size range of file set 3 in the cache file pool is greater than 128M, that is, the file to be created corresponds to file set 3 in the cache file pool. Since file set 3 is empty, and the level number Level2 of file set 3 is greater than the level number Level1 of file set 2 and the level number Level0 of file set 1, a search is first performed in file set 2 to retrieve the cache file with a file size closest to 500M. If file set 2 is also empty, the search continues in file set 1 to retrieve the cache file with a file size closest to 500M.

[0057] It should be noted here that if the file to be created is 100M, the file to be created corresponds to file set 2 in the cache file pool. At this time, file set 2 is empty, while there are cache files in file set 3 and file set 1. Since the cache file size range of file set 3 is greater than 256M, and the cache file size range of file set 1 is less than 1M, the file size of the cache file retrieved from file set 3 will be significantly larger than the file size of the cache file retrieved from file set 1. For example, the cache file retrieved from file set 3 is 256M, while the cache file retrieved from file set 1 is 64k. If the storage space occupied by the created file is about 100M, it will lead to a waste of remaining storage resources. In order to improve the efficiency of storage resource utilization, only the file sets in the cache file pool whose number of levels is less than the number of levels of the target file set are used as the file sets to be detected.

[0058] In some embodiments, when the target cache file is not retrieved in the file set to be detected, a file creation request is sent to the file system so that the file system creates a business file.

[0059] For example, if the file to be created is 100M, the cache file size range of file set 2 in the cache file pool is 1M to 128M, that is, the file to be created corresponds to file set 2 in the cache file pool. Since file set 2 is empty, the search continues in file set 1. If file set 1 is also empty, it means that the current cache file pool cannot provide a suitable cache file. In this case, the file system is requested to create a business file.

[0060] For another example, if the file to be created is 500M, and the cache file size range of file set 3 in the cache file pool is greater than 128M, the file to be created corresponds to file set 3 in the cache file pool. Since file set 3 is empty, the search is performed in file set 2 and file set 1 in sequence. If file set 2 and file set 1 are also empty, it indicates that the current cache file pool cannot provide a suitable cache file. In this case, the file system is requested to create a business file.

[0061] In some embodiments, when the number of levels of the target file set is equal to the minimum number of levels, a file creation request is sent to the file system so that the file system creates a business file.

[0062] For example, if the file to be created is 80k, and the cache file size range of file set 1 in the cache file pool is less than 1M, the file to be created corresponds to file set 1 in the cache file pool. Since file set 1 is empty and the level number Level0 of file set 1 is the minimum level number, it means that the current cache file pool cannot provide a suitable cache file. In this case, the file system is requested to create a business file.

[0063] In step 14, the file to be created is renamed to serve as the created business file.

[0064] For example, the file name of the file to be created extracted from the cache file pool is A, and the file name of the file to be created is renamed to B to facilitate calling of the file.

[0065] It should be noted that when moving a business file into the cache file pool, the business file also needs to be renamed. "Rename" here means modifying the absolute path of the file, including only modifying the file name, or moving the business file to another directory on the same physical storage device, and multiple cache files at the same level must not have the same name. This process mainly includes the following reasons.

[0066] 1) The business system may create / delete files with the same name multiple times, which may cause duplicate name errors when there are multiple files with the same name.

[0067] 2) Prevent files that the business system considers to be deleted from being checked out again by the business system.

[0068] It should be noted here that the resource overhead of the file renaming operation is much smaller than the resource overhead of creating a new file, thereby effectively avoiding the impact on the business performance of the system.

[0069] In some embodiments, after the use of the business file is completed, the current file size of the business file is detected, and the business file is stored in a destination file set corresponding to the current file size in the cache file pool.

[0070] For example, Figure 3 As shown in the figure, 3 files are deleted by the delete (unlink) operation, among which the file size (file_size) of file 1 is 8k, file 2 is 100M, and file 3 is 500M. Since the cache file size range in file set 1 is less than 1M, the cache file size range in file set 2 is 1M to 128M, and the cache file size range in file set 3 is greater than 128M, that is, file 1 corresponds to file set 1, file 2 corresponds to file set 2, and file 3 corresponds to file set 3. In this case, file 1 with a file size of 8k is moved into file set 1, file 2 with a file size of 100M is moved into file set 2, and file 3 with a file size of 500M is moved into file set 3.

[0071] Through the above processing, cache files are moved into the destination file set corresponding to their file sizes so that cache files can be classified according to their file sizes, thereby facilitating rapid extraction of cache files with desired file sizes from the cache file pool.

[0072] In some embodiments, when the number of levels of the target file set is not the minimum number of levels, or when the number of levels of the target file set is the minimum number of levels and the total number of cached files in the target file set does not reach the upper limit of the total number of files in the target file set, the business file is stored in the file set. When the number of levels of the target file set is the minimum number of levels and the total number of cached files in the target file set reaches the upper limit of the total number of files, the business file is deleted.

[0073] For example, Figure 3 As shown, file 1 corresponds to file set 1, file 2 corresponds to file set 2, and file 3 corresponds to file set 3. If the total number of cached files in file set 1 currently does not reach the upper limit of the total number of files in file set 1, then file 1 with a file size of 8k is moved into file set 1. In addition, since among file sets 1, 2, and 3, file set 3 has the largest number of levels and file set 1 has the smallest number of levels, according to subsequent embodiments, even if the total number of cached files in file sets 2 and 3 exceeds the corresponding upper limit of the total number of files, the redundant cached files can still be moved to a file set with a smaller number of levels. Therefore, in this case, file 1 is moved into file set 1, file 2 is moved into file set 2, and file 3 is moved into file set 3.

[0074] For example, Figure 3 As shown in the figure, if the total number of cached files in file set 1 has reached the upper limit of the total number of files in file set 1, and the number of levels of file set 1 is the smallest, it is impossible to move the redundant cached files in file set 1 to other file sets. Therefore, file 1 cannot be moved into file set 1. In this case, file 1 is deleted, and file 2 is moved into file set 2, and file 3 is moved into file set 3.

[0075] In the cache file management method provided in the above embodiment of the present disclosure, by directly calling the cache file instead of creating a new file, the IO pressure is effectively reduced, the performance jitter problem of the application is alleviated, and the availability and stability of the system are improved.

[0076] It should be noted here that the present disclosure does not require any modification to the existing file system, and only implements the solution described in the above embodiment by taking over the file management interface in the existing file system. That is, the solution provided by the present disclosure can be "transparently" connected to the existing file system without any modification to the application or operating system.

[0077] For example, when the target application deletes a file, the file to be deleted is moved into the cache file pool to simulate the file deletion operation. When the target application creates a file, the cache file is moved out of the cache file pool to simulate the file creation operation. For the target application, the file it deletes is not physically deleted, but moved into the cache file pool. In addition, the file created for the target application is not physically created, but moved out of the cache file pool. Therefore, the cache file pool involved in the present disclosure can be regarded as a "recycle bin". The present disclosure shields the visibility of the files in the "recycle bin" to the target application by taking over the file management interface in the original file system.

[0078] For example, by utilizing hook technology, the file operation API (Application Programming Interface) called by the application is hijacked (usually provided by the C runtime library of the operating system, such as glibc in Linux, etc.). In Linux, the APIs responsible for creating files include creat and open, and the APIs responsible for deleting files include unlink and remove. The API implemented in the C runtime library glibc generally uses a "weak symbol" method to declare API interface functions. The present disclosure provides a dynamic link library and defines a function with the same name as the glibc API in a "strong symbol" method in the dynamic library, thereby overwriting the API function. When starting the application, by specifying an environment variable named "LD_PRELOAD", the dynamic library can be injected into the application in a preloaded manner at runtime.

[0079] In some embodiments, Figure 4 As shown, a cache file pool with two file sets is provided for MySQL. In the cache file pool, the level number of file set 1 is Level0, and the target file size is 16k. The level number of file set 2 is Level1, and the target file size is 128M. The threshold between the two file sets is 1M. Here, it is agreed that MySQL's tablespace files (data files that store table data, usually with the extension ibd), binlog files (usually named mysql-bin.xxxxxx, where xxxxxx is the ID of the binlog file), and storage engine's undo-log (rollback log) and redo-log (redo log) are large-size files. Table metadata files (usually with the extension frm) and other files (mainly metadata files and configuration files, etc.) are small-size files.

[0080] exist Figure 2In the example, the quit operation exits the current client and triggers the deletion of the temporary table. The drop table operation deletes a table and triggers the deletion of the table's meta files and data files. The purge operation triggers the deletion of all binlog files before the specified binlog file. In addition, the create temporary table operation creates a temporary table and triggers the creation of the temporary table metadata file. The flush logs operation triggers binlog rotate, which will create a new binlog file.

[0081] Scenario 1, creating / deleting temporary tables in parallel. Each time a temporary table is created, a temporary metadata file is created for it and deleted at the end of the session. When deleting a temporary table file, the present disclosure will hijack the API for deleting the file and put the file into file set 1 for caching. When creating a new temporary table, the system will hijack the API for creating a file and search for files of suitable size from file set 1 (usually the size of such files is between 8KB and 10KB). Therefore, by using the present disclosure, when deleting a temporary table, the file is not actually deleted (the file to be deleted is just moved to file set 1); when creating a new temporary table, the file is not actually created, but the cached file in file set 1 is renamed to the temporary table file to be created. In this way, the problem of severe performance jitter in this scenario will be significantly alleviated.

[0082] Scenario 2: binlog rotate. Each time a binlog rotate operation is performed, the present disclosure first searches from file set 2. If the file cannot be found, it searches from file set 1. In this way, the creation of a new binlog file will be replaced by a file rename operation, thereby significantly alleviating the performance jitter caused by creating files with a global binlog lock.

[0083] It should also be noted here that in order to manage each file set in the cache file pool, a corresponding upper limit value and lower limit value of the total number of files can be set for each file set. If the total number of cached files in a file set is greater than the upper limit value of the total number of files in the file set, or less than the lower limit value of the total number of files in the file set, the total number of cached files in the file set will be adjusted so that the total number of cached files in each file set is within a reasonable range.

[0084] In some embodiments, the method for adjusting the total number of cache files in each file set includes the following contents.

[0085] 1) When the system is idle, detect the total number of cached files in each of the multiple file sets.

[0086] 2) When the total number of cached files in the i-th file set is greater than the upper limit of the total number of files in the i-th file set, determine whether the number of levels of the i-th file set is the minimum number of levels, where i is a natural number not greater than the total number of file sets.

[0087] 3) When the number of levels of the i-th file set is not the minimum number of levels, move the specified number of cache files in the i-th file set into the j-th file set, where the specified number is not less than the difference between the total number of cache files in the i-th file set and the upper limit of the total number of files, the number of levels of the i-th file set is greater than the number of levels of the j-th file set, and j is a natural number not greater than the total number of file sets.

[0088] 4) Adjust the file sizes of the specified number of cache files to the target file size of the jth file set.

[0089] For example, in Figure 5 In the cache file pool shown in the figure, the total number of cache files in file set 3 exceeds the upper limit of the total number of files in file set 3. Since among the three file sets in the cache file pool, file set 3 has the largest number of levels, Level 2, some cache files in file set 3 can be moved to file sets with smaller levels so that the total number of cache files in file set 3 is within a reasonable range. Figure 5 As shown, one file in file set 3 is moved to file set 2, and the file size of the cache file moved to file set 2 is adjusted to the target file size of file set 2.

[0090] 5) When the number of levels of the i-th file set is the minimum number of levels, a predetermined number of cache files in the i-th file set is deleted, where the predetermined number is the difference between the total number of cache files in the i-th file set and the upper limit of the total number of files in the i-th file set.

[0091] For example, in Figure 5 In the cache file pool shown, the total number of cache files in file set 1 exceeds the upper limit of the total number of files in file set 1. Since among the three file sets in the cache file pool, the level number Level0 of file set 1 is the smallest, that is, there is no file set with a smaller level number than Level0 in the cache file pool. In this case, some cache files in file set 1 can be directly deleted so that the total number of cache files in file set 1 is within a reasonable range. If the current total number of cache files in file set 1 is 8, and the upper limit of the total number of files in file set 1 is 5, since 8-5=3, 3 cache files in file set 1 need to be deleted.

[0092] In some embodiments, the method for adjusting the total number of cache files in each file set includes the following contents.

[0093] 1) When the system is idle, detect the total number of cached files in each of the multiple file sets.

[0094] 2) When the total number of cached files in the i-th file set is less than the lower limit of the total number of files in the i-th file set, determine whether the number of levels of the i-th file set is the maximum number of levels, where i is a natural number not greater than the total number of file sets.

[0095] 3) When the number of levels of the i-th file set is not the maximum number of levels, the file set in the cache file pool whose number of levels is greater than the number of levels of the i-th file set is used as the file set to be processed.

[0096] 4) In descending order of the number of levels, move the cached files in each file set in the to-be-processed file set into the i-th file set in sequence until the total number of cached files in the i-th file set is not less than the lower limit of the total number of files in the i-th file set.

[0097] 5) The file size of the cache file moved into the i-th file set is adjusted to the target file size of the i-th file set.

[0098] For example, in Figure 6 In the cache file pool shown in the figure, the total number of cache files in file set 1 is less than the upper limit of the total number of files in file set 1. Since among the three file sets in the cache file pool, file set 1 has the smallest level 0, some cache files in the file sets with larger levels can be moved to file set 1 so that the total number of cache files in file set 1 is within a reasonable range. Figure 6 As shown, one file in file set 3 is moved to file set 1, and the file size of the cache file moved to file set 1 is adjusted to the target file size of file set 1.

[0099] For example, in Figure 7 In the cache file pool shown in the figure, the total number of cache files in file set 1 is less than the upper limit of the total number of files in file set 1. Since among the three file sets in the cache file pool, file set 1 has the smallest level 0, some cache files in the file sets with larger levels can be moved to file set 1 so that the total number of cache files in file set 1 is within a reasonable range. Figure 7 As shown, since file set 3 is currently empty, one file in file set 2 is moved to file set 1, and the file size of the cache file moved to file set 1 is adjusted to the target file size of file set 1.

[0100] Since the probability of calling a small file is much greater than the probability of calling a large file, the above processing can ensure that there are a sufficient number of cache files in the file set with a low number of levels.

[0101] It should be noted here that since each file set is set with a target file size, the file size of the cache files in each file set can be adjusted so that the file size of the cache files in each file set will not exceed the target file size of the file set, thereby effectively saving storage resources.

[0102] In some embodiments, the file size of each cache file in each file set of multiple file sets is detected, and if the file size of the mth cache file in the ith file set is larger than the target file size of the ith file set, the file size of the mth cache file is adjusted to the target file size of the ith file set, wherein , , N is the total number of file sets, Mi is the total number of cached files in the i-th file set.

[0103] For example, Figure 2 As shown, the target file size of file set 2 is 64M. If the size of a cache file in file set 2 is 80M, the file size of the cache file is adjusted to 64M, thereby ensuring that the file size of each cache file in file set 2 does not exceed the target file size of file set 2.

[0104] It should also be noted that at least one of the total number of cache files and the total size of cache files can be set for the cache file pool, and the total number of cache files can be set to avoid too many cache files in the cache file pool. The total size of cache files can be set to avoid too large a total size of cache files in the cache file pool.

[0105] In some embodiments, the total number of cache files in the cache file pool is detected. When the total number of cache files is greater than a predetermined total file threshold, the cache files in the cache file pool are deleted in order of file size from large to small until the total number of cache files in the cache file pool is no greater than the total file threshold.

[0106] For example, the total file count threshold of the cache file pool is 256. If the current total number of cache files in the cache file pool has reached 260, the cache files in the cache file pool are deleted in descending order of file size until the total number of cache files in the cache file pool is within a predetermined range.

[0107] Since the probability of calling a small file is much greater than that of calling a large file, when the total number of cache files in the cache file pool is too large, the cache files in the cache file pool are deleted in descending order of file size.

[0108] In some embodiments, the total size of cache files in the cache file pool is detected. When the total size of cache files is greater than a predetermined total file size threshold, the cache files in the cache file pool are deleted in order of file size from large to small until the total size of cache files in the cache file pool is no greater than the total file size threshold.

[0109] For example, the total file size threshold of the cache file pool is 2G. If the current total cache file size in the cache file pool has reached 2.1G, the cache files in the cache file pool are deleted in descending order of file size until the total number of cache files in the cache file pool is within the predetermined range.

[0110] Since the probability of calling a small file is much greater than the probability of calling a large file, when the total size of the cached files is too large, the cached files in the cache file pool are deleted in descending order of file size.

[0111] Figure 8 The present invention is a schematic diagram of the structure of a cache file management device according to an embodiment of the present invention.

[0112] like Figure 8 As shown, the cache file management device 80 can be in the form of a general-purpose computing device. The cache file management device 80 includes a memory 81, a processor 82, and a bus 83 connecting different system components.

[0113] The memory 81 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions of a corresponding embodiment of at least one cache file management method being executed. The non-volatile storage medium includes, but is not limited to, a disk memory, an optical memory, a flash memory, etc.

[0114] The processor 82 may be implemented by a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, and other discrete hardware components. Accordingly, each module such as an acquisition module, a calculation module, and an adjustment module may be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or may be implemented by a dedicated circuit that executes corresponding steps.

[0115] For example, the processor 82 is configured to execute instructions stored in the memory to implement the following Figure 1 The method of any one of the embodiments.

[0116] The bus 83 may use any of a variety of bus structures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0117] The interfaces 84, 85, 86 of the cache file management device 80 and the memory 81 and the processor 82 can be connected via a bus 83. The input and output interface 84 can provide a connection interface for input and output devices such as a display, a mouse, and a keyboard. The network interface 85 provides a connection interface for various networked devices. The storage interface 86 provides a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0118] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0119] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0120] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0121] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0122] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figure 1 The method of any one of the embodiments.

[0123] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figure 1 The method of any one of the embodiments.

[0124] By implementing the above-mentioned embodiments of the present disclosure, the following beneficial effects can be obtained.

[0125] 1) The disclosure can be "transparently" adapted to various applications and services to improve their stability and availability.

[0126] 2) The present disclosure introduces a delayed recovery and step-by-step shrinking mechanism when deleting files, which reduces the number of file deletion operations, reduces the IO pressure brought to the system by file deletion operations, and alleviates the performance jitter of the application.

[0127] 3) The present disclosure converts the operations of creating and deleting files into very lightweight file renaming operations, which also reduces IO pressure and alleviates performance jitter.

[0128] 4) This disclosure introduces a step-by-step adaptation solution to match the appropriate file size when the file is created, reducing the frequency of subsequent file expansion operations and further improving efficiency.

[0129] In some embodiments, the functional unit described above may be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof, for performing the functions described in the present disclosure.

[0130] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0131] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A cache file management method, comprising: Determine the file size of the file to be created according to the file creation request; Determine a target file set in a cache file pool corresponding to the file size of the file to be created, the cache file pool comprising a plurality of file sets, the plurality of file sets being provided with non-overlapping cache file size ranges and different numbers of levels, the cache file size range of a file set having a smaller number of levels being smaller than the cache file size range of a file set having a larger number of levels; According to the target file set, a cache file whose file size is closest to the file size of the file to be created is retrieved as the file to be created; The file to be created is renamed to serve as the created business file.

2. The cache file management method according to claim 1, wherein: The retrieving the cache file comprises: Determine whether the target file set is empty; When the target file set is not empty, the target cache file is retrieved from the target file set.

3. The cache file management method according to claim 2, wherein: The retrieving the cache file comprises: When the target file set is empty, determining whether the number of levels of the target file set is greater than a minimum number of levels; In the case that the number of levels of the target file set is greater than the minimum number of levels, a file set in the cache file pool whose number of levels is less than the number of levels of the target file set is used as a file set to be detected; Each file set in the to-be-detected file set is searched in sequence in descending order of the number of levels until the target cache file is retrieved.

4. The cache file management method according to claim 3, wherein: The retrieving the cache file comprises: When the target cache file is not found in the to-be-detected file set, a file creation request is sent to the file system so that the file system creates the service file.

5. The cache file management method according to claim 3, wherein: The retrieving the cache file comprises: When the number of levels of the target file set is equal to the minimum number of levels, a file creation request is sent to the file system so that the file system creates the business file.

6. The cache file management method according to claim 1, further comprising: After the business file is used up, detecting the current file size of the business file; The business file is stored in a destination file set corresponding to the current file size in the cache file pool.

7. The cache file management method according to claim 6, wherein: The storing of the business file into a target file set corresponding to the current file size in the cache file pool comprises: When the number of levels of the target file set is not the minimum number of levels, or when the number of levels of the target file set is the minimum number of levels and the total number of cache files in the target file set does not reach the upper limit of the total number of files in the target file set, storing the business file in the file set; When the number of levels of the target file set is the minimum number of levels and the total number of cache files in the target file set reaches the upper limit of the total number of files, the business file is deleted.

8. The cache file management method according to claim 1, wherein: Determining the file size of the file to be created includes: Determining the type of the file to be created according to the file creation request; The file size of the file to be created is determined according to the type of the file to be created.

9. The cache file management method according to any one of claims 1 to 8, further comprising: Detecting the total number of cached files in each of the multiple file sets; When the total number of cached files in the i-th file set is greater than the upper limit of the total number of files in the i-th file set, determining whether the number of levels of the i-th file set is the minimum number of levels, where i is a natural number not greater than the total number of file sets; If the number of levels of the i-th file set is not the minimum number of levels, a specified number of cache files in the i-th file set is moved into the j-th file set, wherein the specified number is not less than the difference between the total number of cache files in the i-th file set and the upper limit of the total number of files, the number of levels of the i-th file set is greater than the number of levels of the j-th file set, and j is a natural number not greater than the total number of file sets; The file sizes of the specified number of cache files are adjusted to the target file size of the j-th file set.

10. The cache file management method according to claim 9, further comprising: When the number of levels of the i-th file set is the minimum number of levels, a predetermined number of cache files in the i-th file set is deleted, where the predetermined number is the difference between the total number of cache files in the i-th file set and the upper limit of the total number of files in the i-th file set.

11. The cache file management method according to any one of claims 1 to 8, further comprising: Detecting the total number of cached files in each of the multiple file sets; When the total number of cached files in the i-th file set is less than the lower limit of the total number of files in the i-th file set, determine whether the number of levels of the i-th file set is the maximum number of levels, where i is a natural number not greater than the total number of file sets; In the case that the number of levels of the i-th file set is not the maximum number of levels, a file set in the cache file pool whose number of levels is greater than the number of levels of the i-th file set is used as a file set to be processed; In descending order of the number of levels, the cached files in each file set in the to-be-processed file set are moved into the i-th file set in sequence until the total number of cached files in the i-th file set is not less than the lower limit of the total number of files in the i-th file set; The file size of the cache file moved into the i-th file set is adjusted to the target file size of the i-th file set.

12. The cache file management method according to any one of claims 1 to 8, further comprising: Detecting the file size of each cache file in each file set of the plurality of file sets; If the file size of the mth cache file in the i-th file set is larger than the target file size of the i-th file set, the file size of the mth cache file is adjusted to the target file size of the i-th file set, wherein , , N is the total number of file sets, and Mi is the total number of cached files in the i-th file set.

13. The cache file management method according to any one of claims 1 to 8, further comprising: Detecting the total number of cache files in the cache file pool; When the total number of cache files is greater than a predetermined total number of files threshold, the cache files in the cache file pool are deleted in descending order of file size until the total number of cache files in the cache file pool is no greater than the total number of files threshold.

14. The cache file management method according to any one of claims 1 to 8, further comprising: Detecting the total size of cache files in the cache file pool; When the total size of the cache files is greater than a predetermined total file size threshold, the cache files in the cache file pool are deleted in descending order of file size until the total size of the cache files in the cache file pool is no greater than the total file size threshold.

15. A cache file management device, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the cache file management method according to any one of claims 1 to 14 based on instructions stored in the memory.

16. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the cache file management method according to any one of claims 1 to 14 is implemented.

17. A computer program product, comprising computer instructions, wherein when the computer instructions are executed by a processor, the cache file management method according to any one of claims 1 to 14 is implemented.

Images