A file pre-reading cache allocation method and device based on access heat

Abstract

The application discloses a file pre-reading cache allocation method and device based on access heat, and the method comprises the following steps: calculating the cache upper limit of each file through the access heat; judging whether the sum of the pre-reading cache usage and the preset pre-reading step length of the corresponding file is greater than the cache upper limit of the corresponding file, and adjusting the pre-reading step length; performing data pre-reading according to the pre-reading step length, judging whether the pre-reading cache reaches the cache overall capacity, marking and selecting replacement; judging whether the replaced old data block has been used, if yes, ending the operation, otherwise, updating the pre-reading step length of the file corresponding to the old data block, and ending the operation. The device comprises a statistical module, a first judging module, a second judging module, a cache allocation module, a pre-reading step length calculation module, a hit updating module and a third judging module, which are used for executing corresponding programs. The application can adapt to different load conditions, improve the hit rate and use efficiency of the cache, and further improve the access performance of the file system.

Description

Technical Field

[0001] This invention belongs to the field of storage technology, specifically relating to a method and apparatus for allocating file pre-read cache based on access popularity. Background Technology

[0002] In the file system domain, read-ahead refers to reading more data than expected at once and caching it in memory. By setting up a read-ahead mechanism, the data reading efficiency of the storage system can be improved. Specifically, each time an I / O request is received, in addition to obtaining the data required for that I / O request, extra data is also obtained and placed in memory. This way, when the next read request arrives, the data can be directly retrieved from memory. Through read-ahead, upper-layer applications can achieve lower latency and higher bandwidth. However, the data read from the underlying file system into memory may not be accessed by upper-layer applications in a timely manner. When the read-ahead data is not accessed in time, it will cause a significant waste of storage system resources, and it will also affect normal I / O performance.

[0003] Current prefetching algorithms and mechanisms generally lack management of cache allocation, relying solely on system configuration parameters for prefetching. Consequently, they fail to fully leverage the effectiveness of the prefetch cache and achieve optimal system performance under both high and low system loads. A few prefetching algorithms and mechanisms simply adjust prefetching parameters based on the number of files; similarly, they fail to fully utilize the prefetch cache's effectiveness when the load on different files varies significantly.

[0004] For example, Chinese patent application number 202110998403.X discloses a file pre-read cache allocation method and apparatus. The method includes: determining whether the existing pre-read cache can meet the concurrency requirements based on the total pre-read cache size, the number of concurrent files, and a maximum file cache size threshold; in response to determining that the existing pre-read cache cannot meet the concurrency requirements, reducing the maximum file cache size threshold to the quotient of the total pre-read cache size and the number of concurrent files, and further determining whether the maximum file cache size threshold can meet the pre-read requirements based on the minimum cache size; in response to determining that the maximum file cache size threshold can meet the pre-read requirements, providing a pre-read cache that meets the reduced maximum file cache size threshold for each file to concurrently buffer file reading. This scheme only allocates cache based on the number of file accesses, without considering the differences in access patterns of different files, thus making it difficult to reasonably utilize the caching efficiency.

[0005] In view of the above, this application proposes a file pre-read cache allocation method and apparatus based on access popularity. Summary of the Invention

[0006] The purpose of this invention is to address the above-mentioned problems by proposing a file pre-read cache allocation method and apparatus based on access popularity, which can adapt to different load conditions, improve cache hit rate and utilization efficiency, and thus improve the access performance of the file system.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] This invention proposes a file pre-read cache allocation method based on access popularity, comprising the following steps:

[0009] S1. Collect global read request information and read request information for each file within the time window. The read request information includes the number of reads and the amount of data read.

[0010] S2. Determine whether the read request hits the pre-read cache. If yes, mark the corresponding data block in the pre-read cache as used and fill the reply message with the data specified by the read request in the data block to respond to the read request. Otherwise, read the data block and add it to the pre-read cache, mark the data block as used, and fill the reply message with the data specified by the read request in the data block to respond to the read request.

[0011] S3. Determine whether the read request meets the pre-read condition. The pre-read condition is continuous read operations. If yes, proceed to step S4; otherwise, end the operation.

[0012] S4. Calculate the global access popularity and the access popularity of each file based on the statistical read request information;

[0013] S5. Calculate the cache limit for each file. The cache limit is the ratio of the access popularity of the corresponding file to the global access popularity multiplied by the total cache capacity.

[0014] S6. Determine whether the sum of the pre-read cache usage and the preset pre-read step size of the corresponding file is greater than the cache limit of the corresponding file. The preset pre-read step size tmp = c * step. If yes, set the current pre-read step size to the previous pre-read step size. Otherwise, update the current pre-read step size to the preset pre-read step size. Here, c is the inflation factor, step is the previous pre-read step size, i.e. the pre-read step size of the previous time window, and the pre-read step size is the number of data blocks.

[0015] S7. Based on the current pre-read step size, pre-read data blocks are formed by pre-reading data blocks. The pre-read data blocks are marked as unused. It is determined whether the current usage of the pre-read cache and the sum of the pre-read data blocks reach the total cache capacity. If so, the pre-read data blocks are put into the pre-read cache and the old data blocks in the pre-read cache are replaced with redundant pre-read data blocks. Then, step S8 is executed. Otherwise, the data blocks are directly put into the pre-read cache and the operation ends.

[0016] S8. Determine whether the old data block to be replaced has been used. If yes, end the operation. Otherwise, update the pre-read step size of the file corresponding to the old data block to step = d * step, where d is the decay factor, and end the operation.

[0017] Preferably, in step S1, the global read request information and the read request information of each file within the time window are statistically analyzed using a sliding time window method, and the statistical information is updated as the time window slides.

[0018] Preferably, the global access popularity (total_weight) is calculated as follows:

[0019] total_weight = a + b;

[0020] The access popularity (file_weght) of each file is calculated as follows:

[0021] file_weght=a*file_num / total_num+b*file_size / total_size

[0022] Where a and b are weighting coefficients, total_num is the total number of global reads, total_size is the total amount of data read globally, file_num is the number of file reads, and file_size is the total amount of data read from a file.

[0023] Preferably, in step S7, the LRU algorithm is used to replace the old data blocks in the prefetch cache with the redundant prefetch data blocks.

[0024] A file prefetching cache allocation device based on access popularity includes:

[0025] The statistics module is used to collect global read request information and read request information for each file within a time window. The read request information includes the number of reads and the amount of data read.

[0026] The first judgment module is used to determine whether the read request hits the pre-read cache. If so, the corresponding data block in the pre-read cache is marked as used and the data specified by the read request in the data block is filled into the reply message to respond to the read request. Otherwise, the data block is read and added to the pre-read cache, the data block is marked as used and the data specified by the read request in the data block is filled into the reply message to respond to the read request.

[0027] The second judgment module is used to determine whether the read request meets the pre-read condition. The pre-read condition is continuous read operations. If so, the cache allocation module is triggered; otherwise, the operation ends.

[0028] The cache allocation module is used to calculate the global access popularity and the access popularity of each file based on the statistical read request information, and to calculate the cache limit for each file. The cache limit is the ratio of the access popularity of the corresponding file to the global access popularity multiplied by the total cache capacity.

[0029] The pre-read step calculation module is used to determine whether the sum of the pre-read cache usage and the preset pre-read step of the corresponding file is greater than the cache limit of the corresponding file. The preset pre-read step tmp = c * step. If yes, the current pre-read step is set to the previous pre-read step; otherwise, the current pre-read step is updated to the preset pre-read step. Here, c is the inflation factor, step is the previous pre-read step, i.e. the pre-read step of the previous time window, and the pre-read step is the number of data blocks.

[0030] The hit update module is used to pre-read data blocks according to the current pre-read step size to form pre-read data blocks, mark the pre-read data blocks as unused, and determine whether the current usage of the pre-read cache and the sum of the pre-read data blocks reach the total cache capacity. If so, the pre-read data blocks are put into the pre-read cache and the old data blocks in the pre-read cache are replaced with redundant pre-read data blocks, triggering the third judgment module. Otherwise, they are directly put into the pre-read cache and the operation ends.

[0031] The third judgment module is used to determine whether the old data block to be replaced has been used. If so, the operation ends; otherwise, the pre-read step size of the file corresponding to the old data block is updated to step = d * step, where d is the decay factor, and the operation ends.

[0032] Preferably, the statistics module uses a sliding time window method for statistics, and the statistical information is updated as the time window slides.

[0033] Preferably, the global access popularity (total_weight) is calculated as follows:

[0034] total_weight = a + b;

[0035] The access popularity (file_weght) of each file is calculated as follows:

[0036] file_weght=a*file_num / total_num+b*file_size / total_size

[0037] Where a and b are weighting coefficients, total_num is the total number of global reads, total_size is the total amount of data read globally, file_num is the number of file reads, and file_size is the total amount of data read from a file.

[0038] Preferably, the hit update module uses the LRU algorithm to replace old data blocks in the prefetch cache with redundant prefetch data blocks.

[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0040] This application adjusts the upper limit of the file pre-read cache based on the access frequency of the file, allocates the cache, and adjusts the pre-read step size according to the usage status flag of the replaced data block. It is more responsive and achieves on-demand cache allocation compared with the existing technology. It has strong adaptability and high utilization efficiency. It can adaptively adjust the pre-read parameters under different loads to improve the cache hit rate and utilization efficiency, thereby improving the access performance of the file system. It is suitable for scenarios with high file concurrency, low file concurrency, and large differences in file load, and has a wide range of applications. Attached Figure Description

[0041] Figure 1 This is a flowchart of the file pre-read cache allocation method based on access popularity according to the present invention;

[0042] Figure 2 This is a schematic diagram of the file pre-read cache allocation device based on access popularity according to the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0045] Example 1:

[0046] like Figure 1 As shown, a file pre-read cache allocation method based on access popularity includes the following steps:

[0047] S1. Collect global read request information and read request information for each file within the time window. The read request information includes the number of reads and the amount of data read.

[0048] S2. Determine whether the read request hits the pre-read cache. If yes, mark the corresponding data block in the pre-read cache as used and fill the response message with the data specified by the read request in the data block to respond to the read request. Otherwise, read the data block and add it to the pre-read cache, mark the data block as used, and fill the response message with the data specified by the read request in the data block to respond to the read request.

[0049] S3. Determine whether the read request meets the pre-read condition. The pre-read condition is continuous read operations. If yes, proceed to step S4; otherwise, end the operation.

[0050] S4. Calculate the global access popularity and the access popularity of each file based on the statistical read request information;

[0051] S5. Calculate the cache limit for each file. The cache limit is the ratio of the access popularity of the corresponding file to the global access popularity multiplied by the total cache capacity.

[0052] S6. Determine whether the sum of the pre-read cache usage and the preset pre-read step size of the corresponding file is greater than the cache limit of the corresponding file. The preset pre-read step size tmp = c * step. If yes, set the current pre-read step size to the previous pre-read step size. Otherwise, update the current pre-read step size to the preset pre-read step size. Here, c is the inflation factor, step is the previous pre-read step size, i.e. the pre-read step size of the previous time window, and the pre-read step size is the number of data blocks.

[0053] S7. Based on the current pre-read step size, pre-read data blocks are formed by pre-reading data blocks. The pre-read data blocks are marked as unused. It is determined whether the current usage of the pre-read cache and the sum of the pre-read data blocks reach the total cache capacity. If so, the pre-read data blocks are put into the pre-read cache and the old data blocks in the pre-read cache are replaced with redundant pre-read data blocks. Then, step S8 is executed. Otherwise, the data blocks are directly put into the pre-read cache and the operation ends.

[0054] S8. Determine whether the old data block to be replaced has been used. If yes, end the operation. Otherwise, update the pre-read step size of the file corresponding to the old data block to step = d * step, where d is the decay factor, and end the operation.

[0055] In one embodiment, in step S1, the global read request information and the read request information of each file within the time window are statistically analyzed using a sliding time window method, and the statistical information is updated as the time window slides.

[0056] In one embodiment, the global access popularity (total_weight) is calculated as follows:

[0057] total_weight = a + b;

[0058] The access popularity (file_weght) of each file is calculated as follows:

[0059] file_weght=a*file_num / total_num+b*file_size / total_size

[0060] Where a and b are weighting coefficients, total_num is the total number of global reads, total_size is the total amount of data read globally, file_num is the number of file reads, and file_size is the total amount of data read from a file.

[0061] In one embodiment, in step S7, replacing the old data block in the prefetch cache with the redundant prefetch data block uses the LRU algorithm.

[0062] Specifically, the file pre-read cache allocation method based on access popularity in this application performs the following operations:

[0063] 1) Extract information from read requests, and compile statistics on global read request information and read request information for each file. Read request information includes the number of reads and the amount of data read. Specifically, the main statistics include the total number of reads (total_num) and the total amount of data read (total_size) globally within the sliding time window, and the number of reads (file_num) and the amount of data read (file_size) for each file. These statistics are updated as the time window slides.

[0064] 2) Read data according to the read request. If the read request hits the read-ahead cache, mark the corresponding data block in the read-ahead cache as used and fill the response message with the data specified in the read request in the data block. If the read-ahead cache is not hit, read the data block and add it to the read-ahead cache, mark the data block as used, and fill the response message with the data specified in the read request in the data block.

[0065] 3) Determine whether the pre-read condition is met based on historical read requests and the current read request. If the pre-read condition is met, pre-read subsequent data blocks. The pre-read condition is continuous read operations (the most recent read requests for this file can be continuous or at fixed intervals). If the pre-read condition is not met, the operation ends. If the pre-read condition is met, proceed to step 4.

[0066] 4) Calculate the total weight and file weight of the overall access popularity based on the statistical information using a weighted summation. total_weight = a + b, file_weight = a * file_num / total_num + b * file_size / total_size, where a and b are weighting coefficients (e.g., when a is 0 and b is 1, the access popularity only refers to the amount of statistical data). The values ​​of a and b can be adjusted according to actual needs.

[0067] 5) Calculate the maximum cache limit for a given file, file_max, based on the total cache capacity (cache), the file access frequency (file_weight), and the global access frequency (total_weight). file_max = cache * file_weight / total_weight.

[0068] 6) Update the current pre-read step size based on the file's cache limit, used cache capacity (pre-read cache usage), and the previous pre-read step size. If the used cache capacity and the preset pre-read step size exceed the file's cache limit, do not update and retain the previous pre-read step size. Otherwise, update the current pre-read step size to the preset pre-read step size, where the preset pre-read step size tmp = c * step, c is the inflation factor (e.g., c = 2, the pre-read step size doubles each time), and step is the previous pre-read step size, i.e., the pre-read step size of the previous time window. The pre-read step size increases by a certain ratio, representing the number of data blocks.

[0069] 7) Prefetch data blocks according to the prefetch step size to form prefetch data blocks. Mark the prefetch data blocks as unused. If the sum of the current prefetch cache usage and the prefetch data blocks reaches the total cache capacity, the LRU algorithm needs to be used to replace the old data blocks in the prefetch cache with the redundant prefetch data blocks, and proceed to step 8). The LRU algorithm can also be replaced by other algorithms well known to those skilled in the art. For example, if there is only one free slot in the current prefetch cache, but the prefetch step size is 2, and there are two prefetch data blocks, the first prefetch data block is placed directly into the free slot. When placing the second prefetch data block, an old data block in the prefetch cache needs to be replaced. If the sum of the current prefetch cache usage and the prefetch data blocks does not reach the total cache capacity, the prefetch data block is placed directly into the prefetch cache, and the operation ends.

[0070] 8) If the old data block to be replaced is marked as unused, then update the pre-read step size of the file corresponding to the old data block to step = d * step, where d is the decay factor (e.g., d is 0.5, and each decay is half of the previous value). If the old data block to be replaced is marked as used, then end the operation.

[0071] This application adjusts the upper limit of the file pre-read cache based on the access frequency of the file, allocates the cache, and adjusts the pre-read step size according to the usage status flag of the replaced data block. It is more responsive and achieves on-demand cache allocation compared with the existing technology. It has strong adaptability and high utilization efficiency. It can adaptively adjust the pre-read parameters under different loads to improve the cache hit rate and utilization efficiency, thereby improving the access performance of the file system. It is suitable for scenarios with high file concurrency, low file concurrency, and large differences in file load, and has a wide range of applications.

[0072] Example 2:

[0073] like Figure 2 As shown, a file pre-read cache allocation device based on access popularity includes:

[0074] The statistics module is used to collect global read request information and read request information for each file within a time window. The read request information includes the number of reads and the amount of data read.

[0075] The first judgment module is used to determine whether the read request hits the pre-read cache. If so, the corresponding data block in the pre-read cache is marked as used and the data specified by the read request in the data block is filled into the reply message to respond to the read request. Otherwise, the data block is read and added to the pre-read cache, the data block is marked as used and the data specified by the read request in the data block is filled into the reply message to respond to the read request.

[0076] The second judgment module is used to determine whether the read request meets the pre-read condition. The pre-read condition is continuous read operations. If so, the cache allocation module is triggered; otherwise, the operation ends.

[0077] The cache allocation module is used to calculate the global access popularity and the access popularity of each file based on the statistical read request information, and to calculate the cache limit for each file. The cache limit is the ratio of the access popularity of the corresponding file to the global access popularity multiplied by the total cache capacity.

[0078] The pre-read step calculation module is used to determine whether the sum of the pre-read cache usage and the preset pre-read step of the corresponding file is greater than the cache limit of the corresponding file. The preset pre-read step tmp = c * step. If yes, the current pre-read step is set to the previous pre-read step; otherwise, the current pre-read step is updated to the preset pre-read step. Here, c is the inflation factor, step is the previous pre-read step, i.e. the pre-read step of the previous time window, and the pre-read step is the number of data blocks.

[0079] The hit update module is used to pre-read data blocks according to the current pre-read step size to form pre-read data blocks, mark the pre-read data blocks as unused, and determine whether the current usage of the pre-read cache and the sum of the pre-read data blocks reach the total cache capacity. If so, the pre-read data blocks are put into the pre-read cache and the old data blocks in the pre-read cache are replaced with redundant pre-read data blocks, triggering the third judgment module. Otherwise, they are directly put into the pre-read cache and the operation ends.

[0080] The third judgment module is used to determine whether the old data block to be replaced has been used. If so, the operation ends; otherwise, the pre-read step size of the file corresponding to the old data block is updated to step = d * step, where d is the decay factor, and the operation ends.

[0081] In one embodiment, the statistics module uses a sliding time window method to perform statistics, and the statistical information is updated as the time window slides.

[0082] In one embodiment, the global access popularity (total_weight) is calculated as follows:

[0083] total_weight = a + b;

[0084] The access popularity (file_weght) of each file is calculated as follows:

[0085] file_weght=a*file_num / total_num+b*file_size / total_size

[0086] Where a and b are weighting coefficients, total_num is the total number of global reads, total_size is the total amount of data read globally, file_num is the number of file reads, and file_size is the total amount of data read from a file.

[0087] In one embodiment, the hit update module uses the LRU algorithm to replace old data blocks in the prefetch cache with redundant prefetch data blocks.

[0088] Specifically, the file pre-read cache allocation device based on access popularity in this application includes a statistics module, a first judgment module, a second judgment module, a cache allocation module, a pre-read step size calculation module, a hit update module, and a third judgment module. Among them:

[0089] The statistics module tracks read request information. It uses a sliding time window to collect statistics on read operations within that window, primarily including the amount of data read and the number of reads. The statistics are updated as the time window slides. The statistics include global statistics and file-specific statistics, specifically the global read count (total_num), global read data volume (total_size), file read count (file_num), and file read data volume (file_size) within the sliding time window.

[0090] The first judgment module determines whether the read request hits the pre-read cache. If the read request hits the pre-read cache, it marks the corresponding data block in the pre-read cache as used and fills the data specified by the read request in the data block into the reply message to respond to the read request. If the pre-read cache is not hit, it reads the data block and adds it to the pre-read cache, marks the data block as used, and fills the data specified by the read request in the data block into the reply message to respond to the read request.

[0091] The second judgment module is used to determine whether the read request meets the pre-read condition. It judges whether the pre-read condition is met based on the historical read requests and the current read request. When the pre-read condition is met, the subsequent data blocks are pre-read. The pre-read condition is continuous read operation (the most recent read requests for this file can be continuous or at fixed intervals). If the pre-read condition is met, the cache allocation module is triggered; otherwise, the operation ends.

[0092] The cache allocation module uses a weighted sum of the read data volume and read count from the statistics to calculate the global access popularity and the access popularity of each file. The global access popularity `total_weight` = a + b, and the access popularity of each file `file_weight` = a * file_num / total_num + b * file_size / total_size, where a and b are weighting coefficients (e.g., when a is 0 and b is 1, the access popularity only considers the statistical data volume). The values ​​of a and b can be adjusted according to actual needs. Based on the total cache capacity `cache`, the file access popularity `file_weight`, and the global access popularity `total_weight`, the upper limit of the cache that can be used for the corresponding file, `file_max`, is calculated. `file_max` = cache * file_weight / total_weight`.

[0093] The pre-read step calculation module calculates the pre-read step based on popularity and caching information. It compares the sum of the file's pre-read step and its used cache capacity (pre-read cache usage) with the file's cache limit. If the cache limit is exceeded after the update, the pre-read step remains unchanged as the previous one. Otherwise, the pre-read step is updated to a preset pre-read step, tmp = c * step, where c is the expansion factor (e.g., c = 2, doubling the pre-read step each time), and step is the previous pre-read step, i.e., the pre-read step of the previous time window. The pre-read step increases proportionally, representing the number of data blocks.

[0094] The hit update module updates the prefetch step size based on the status (whether used) of data blocks in the prefetch cache. During the prefetch process, prefetch data is loaded and placed into the prefetch cache. Prefetch data blocks are marked as unused. If the sum of the current prefetch cache usage and the prefetch data blocks reaches the total cache capacity, the LRU algorithm is used to replace the old data blocks in the prefetch cache with the redundant prefetch data blocks, triggering the third judgment module. The LRU algorithm can also be replaced by other algorithms well-known to those skilled in the art. For example, if there is only one free slot in the current prefetch cache, but the prefetch step size is 2 and there are two prefetch data blocks, the first prefetch data block is placed directly into the free slot. When placing the second prefetch data block, an old data block in the prefetch cache needs to be replaced. If the sum of the current prefetch cache usage and the prefetch data blocks does not reach the total cache capacity, the prefetch data block is placed directly into the prefetch cache, and the operation ends.

[0095] The third judgment module determines whether the old data block being replaced from the pre-read cache has been used (missed). If it has not been used (missed), the pre-read step size of the file corresponding to the old data block is adjusted to step = d * step, where d is the decay factor (e.g., d is 0.5, and it decays to half of the previous value each time). The operation ends. If it is a hit, no adjustment is made and the operation ends.

[0096] This application adjusts the upper limit of the file pre-read cache based on the access frequency of the file, allocates the cache, and adjusts the pre-read step size according to the usage status flag of the replaced data block. It is more responsive and achieves on-demand cache allocation compared with the existing technology. It has strong adaptability and high utilization efficiency. It can adaptively adjust the pre-read parameters under different loads to improve the cache hit rate and utilization efficiency, thereby improving the access performance of the file system. It is suitable for scenarios with high file concurrency, low file concurrency, and large differences in file load, and has a wide range of applications.

[0097] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] The embodiments described above are merely specific and detailed examples of the embodiments described in this application, and should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the appended claims.

Claims

1. A file pre-read cache allocation method based on access popularity, characterized in that: The file pre-read cache allocation method based on access popularity includes the following steps: S1. Collect global read request information and read request information for each file within the time window, respectively. The read request information includes the number of reads and the amount of data read. S2. Determine whether the read request hits the pre-read cache. If yes, mark the corresponding data block in the pre-read cache as used and fill the reply message with the data specified by the read request in the data block to respond to the read request. Otherwise, read the data block and add it to the pre-read cache, mark the data block as used, and fill the reply message with the data specified by the read request in the data block to respond to the read request. S3. Determine whether the read request meets the pre-read condition, where the pre-read condition is a continuous read operation. If yes, proceed to step S4; otherwise, end the operation. S4. Calculate the global access popularity and the access popularity of each file based on the statistical read request information; S5. Calculate the cache limit for each file, where the cache limit is the ratio of the access popularity of the corresponding file to the global access popularity multiplied by the total cache capacity. S6. Determine whether the sum of the pre-read cache usage and the preset pre-read step size of the corresponding file is greater than the cache limit of the corresponding file. The preset pre-read step size tmp = c * step. If yes, set the current pre-read step size to the previous pre-read step size. Otherwise, update the current pre-read step size to the preset pre-read step size. Here, c is the expansion factor, step is the previous pre-read step size, i.e. the pre-read step size of the previous time window, and the pre-read step size is the number of data blocks. S7. Based on the current pre-read step size, pre-read data blocks are formed by pre-reading data blocks. The pre-read data blocks are marked as unused. It is determined whether the current usage of the pre-read cache and the sum of the pre-read data blocks reach the total cache capacity. If so, the pre-read data blocks are put into the pre-read cache and the old data blocks in the pre-read cache are replaced with redundant pre-read data blocks. Then, step S8 is executed. Otherwise, the data blocks are directly put into the pre-read cache and the operation ends. S8. Determine whether the old data block to be replaced has been used. If yes, end the operation. Otherwise, update the pre-read step size of the file corresponding to the old data block to step = d * step, where d is the decay factor, and end the operation.

2. The file pre-read cache allocation method based on access popularity as described in claim 1, characterized in that: In step S1, the global read request information and the read request information of each file within the time window are statistically analyzed using a sliding time window method, and the statistical information is updated as the time window slides.

3. The file pre-read cache allocation method based on access popularity as described in claim 1, characterized in that: The global access popularity (total_weight) is calculated as follows: total_weight = a + b; The access popularity (file_weght) of each file is calculated as follows: file_weght=a*file_num / total_num+b*file_size / total_size Where a and b are weighting coefficients, total_num is the total number of global reads, total_size is the total amount of data read globally, file_num is the number of file reads, and file_size is the total amount of data read from a file.

4. The file pre-read cache allocation method based on access popularity as described in claim 1, characterized in that: In step S7, the replacement of old data blocks in the prefetch cache with redundant prefetch data blocks adopts the LRU algorithm.

5. A file pre-read cache allocation device based on access popularity, characterized in that: The file pre-read cache allocation device based on access popularity includes: The statistics module is used to separately count the global read request information and the read request information for each file within a time window. The read request information includes the number of reads and the amount of data read. The first judgment module is used to determine whether the read request hits the pre-read cache. If so, the corresponding data block in the pre-read cache is marked as used and the data specified by the read request in the data block is filled into the reply message to respond to the read request. Otherwise, the data block is read and added to the pre-read cache, the data block is marked as used and the data specified by the read request in the data block is filled into the reply message to respond to the read request. The second judgment module is used to determine whether the read request meets the pre-read condition, which is a continuous read operation. If so, the cache allocation module is triggered; otherwise, the operation is terminated. The cache allocation module is used to calculate the global access popularity and the access popularity of each file based on the statistical read request information, and to calculate the cache limit of each file. The cache limit is the ratio of the access popularity of the corresponding file to the global access popularity multiplied by the total cache capacity. The pre-read step size calculation module is used to determine whether the sum of the pre-read cache usage and the preset pre-read step size of the corresponding file is greater than the cache limit of the corresponding file. The preset pre-read step size tmp = c * step. If yes, the current pre-read step size is set to the previous pre-read step size; otherwise, the current pre-read step size is updated to the preset pre-read step size. Here, c is the inflation factor, step is the previous pre-read step size, i.e. the pre-read step size of the previous time window, and the pre-read step size is the number of data blocks. The hit update module is used to pre-read data blocks according to the current pre-read step size to form pre-read data blocks, mark the pre-read data blocks as unused, and determine whether the current usage of the pre-read cache and the sum of the pre-read data blocks reach the total cache capacity. If so, the pre-read data blocks are put into the pre-read cache and the old data blocks in the pre-read cache are replaced with redundant pre-read data blocks, triggering the third judgment module. Otherwise, they are directly put into the pre-read cache and the operation ends. The third judgment module is used to determine whether the old data block to be replaced has been used. If so, the operation ends; otherwise, the pre-read step size of the file corresponding to the old data block is updated to step = d * step, where d is the decay factor, and the operation ends.

6. The file pre-read cache allocation device based on access popularity as described in claim 5, characterized in that: The statistics module uses a sliding time window method for statistics, and the statistical information is updated as the time window slides.

7. The file pre-read cache allocation device based on access popularity as described in claim 5, characterized in that: The global access popularity (total_weight) is calculated as follows: total_weight = a + b; The access popularity (file_weght) of each file is calculated as follows: file_weght=a*file_num / total_num+b*file_size / total_size Where a and b are weighting coefficients, total_num is the total number of global reads, total_size is the total amount of data read globally, file_num is the number of file reads, and file_size is the total amount of data read from a file.

8. The file pre-read cache allocation device based on access popularity as described in claim 5, characterized in that: The hit update module uses the LRU algorithm to replace old data blocks in the prefetch cache with redundant prefetch data blocks.

Images