Data caching methods and related products

Abstract

The present application provides a data caching method and related products. The data caching method is applied to a terminal device, the terminal device includes a memory and a storage disk, the storage disk is used to cache data blocks stored in a remote device, and the method includes: the memory determines M first cached data blocks with continuous storage address sequence from N cached data blocks of the storage disk according to the metadata linked list of the storage disk, wherein the metadata linked list records the storage address sequence of the N cached data blocks, and the metadata linked list also records the data size and storage address of each cached data block in the N cached data blocks, and the sum of the data sizes corresponding to the M first cached data blocks is greater than or equal to the data size of the data to be cached, wherein M and N are positive integers, and N≥M; the memory writes the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk. In this way, the memory can continuously write the data to be cached into the storage disk, thereby improving the read and write efficiency.

Description

Technical Field

[0001] The present application relates to the field of caching, and in particular to a data caching method and related products. Background Art

[0002] In the related art, the memory can write part of the data into the cache, and when the data is needed, the memory reads the data from the cache. However, the cache space is limited, and when there is no remaining space in the cache, some data needs to be eliminated before new data can be written.

[0003] In the prior art, the principle of eliminating data in the cache is that when the data requested to be read by the memory does not exist in the cache, the data is considered to be a miss. The memory obtains data from the data source and writes it to the cache. When the cache is full, each cache miss (cache miss) is returned to the source, and the corresponding size of data needs to be eliminated in the cache so that there is enough space in the cache to write the data obtained from the metadata. The key to ensuring the cache hit rate is whether the cache elimination is scientific. For this purpose, the industry has proposed a large number of cache elimination algorithms, such as LRU (Least Recently Used). Such an algorithm uses a priority queue to represent the popularity of data. As the data in the cache hits, the position of the data in the queue is continuously adjusted, and the data at the end of the queue is replaced when new data is written. However, such an algorithm writes data to a random location in the cache each time, and cannot perform merge writes. In this way, when the data at the end of the queue is distributed in different storage locations, the newly written data may be divided into multiple blocks and cached in different locations, resulting in poor read and write performance. Summary of the invention

[0004] The present application provides a data caching method and related products, which can improve the read and write performance of a storage disk.

[0005] In a first aspect, the present application provides a data caching method, which is applied to a terminal device, wherein the terminal device includes a memory and a storage disk, and the storage disk is used to cache data blocks stored in a remote device. The method includes:

[0006] The memory determines, from the N cached data blocks of the storage disk, M first cached data blocks with continuous storage address sequence according to the metadata linked list of the storage disk, wherein the metadata linked list records the storage address sequence of the N cached data blocks, and the metadata linked list also records the data size and storage address of each cached data block in the N cached data blocks, and the sum of the data sizes corresponding to the M first cached data blocks is greater than or equal to the data size of the data to be cached, wherein M and N are positive integers, and N≥M;

[0007] The memory writes the data to be cached into a storage space corresponding to the storage address of the first cached data block in the storage disk.

[0008] In the data caching method of the embodiment of the present application, the memory determines M first cached data blocks with continuous storage address sequence from the N cached data blocks of the storage disk according to the metadata linked list corresponding to the storage disk, and then writes the data to be cached into the storage space corresponding to the storage address of the M first cached data blocks. In this way, since the storage address sequence of the M first cached data blocks is continuous, the memory can write the data to be cached continuously into the storage disk, avoiding low read and write efficiency caused by splitting the data to be written into different locations. In this way, the read and write performance of the storage disk is better, which helps to reduce the delay when the user device obtains data.

[0009] Moreover, since the metadata linked list records the storage address sequence of the N cached data blocks, the memory can directly determine the M first cached data blocks from the N cached data blocks more flexibly according to the storage address sequence recorded in the metadata linked list, rather than being limited to continuing to write data at the storage address written last time, thereby enabling more flexible selection of cache addresses for caching data. For example, when selecting the M first cached data blocks, the memory can skip data with higher popularity or data that is defined as not eligible for elimination, thus enabling more flexible determination of data to be eliminated, making data elimination in the storage disk more reasonable.

[0010] In some embodiments, the metadata linked list also records the heat parameters of each cached data block; the memory determines M first cached data blocks with consecutive storage addresses from the N cached data blocks of the storage disk according to the metadata linked list of the storage disk, including: the memory determines the M first cached data blocks that meet the heat requirements of cold data from the N stored data blocks according to the metadata linked list. In this way, each first cached data block meets the heat requirements corresponding to the cold data, which can avoid eliminating hot data when writing to the cached data, making the eliminated data more reasonable, and improving the hit rate of hot data. It can also avoid the need for the memory to frequently obtain hot data from the source database of the remote device, so that it can provide data to users more quickly according to user needs, which helps to reduce the delay when the user device obtains data.

[0011] In some embodiments, the heat requirement of the cold data is: the heat parameter of each of the first cached data blocks is within the first heat parameter range corresponding to the cold data. In this way, the heat of each of the M first cached data blocks is relatively low, which effectively avoids the elimination of hot data and helps to improve the hit rate of the cached data in the storage disk.

[0012] In some embodiments, the heat requirement of the cold data is: the average heat parameter of the M first cached data blocks is within the heat parameter mean value range corresponding to the cold data. This ensures that the average heat of the M first cached data blocks is low, which can avoid the elimination of hot data and enable the memory to determine the M first cached data blocks more quickly, thereby helping to reduce the delay when the user device obtains data.

[0013] In some embodiments, the heat requirement of the cold data is: the M first cached data blocks are cached data blocks of the first cached data block group among multiple cached data block groups, each cached data block group includes one or more cached data blocks with consecutive storage addresses, and the sum of the data sizes of one or more cached data blocks in each cached data block group is greater than or equal to the data size of the data to be cached; the average value of the heat parameters of the M first cached data blocks is lower than the average value of the heat parameters of each second cached data block group except the first cached data block group among the multiple cached data block groups, and the average value of the heat parameters of each second cached data block group is the average value of the heat parameters of one or more cached data blocks in the second cached data block group. In this way, the M first cached data blocks with relatively low heat can be determined, and the elimination of hot data can be effectively avoided, which helps to improve the hit rate of cached data in the storage disk.

[0014] In some implementations, the metadata linked list also records the priority of each cached data block, and the priority of each first cached data block meets the priority requirement. In this way, by manually setting the priority condition and the priority of the cached data block, some cached data blocks can be set to be retained, avoiding the elimination of some cached data blocks that cannot be eliminated.

[0015] In some embodiments, before the memory writes the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk, the method further includes: the memory determines, from the M first cached data blocks, a second cached data block whose heat parameter is within the second heat parameter range corresponding to the relatively hot data; and the memory writes the second cached data block into a preset storage location of the memory. In this way, the second cached data block with relatively high heat among the M first cached data blocks can be written to the preset address storage location, which can avoid the need for the memory to obtain the data again from the source database when providing the second cached data block to the user device, thereby reducing the delay when the user device obtains data.

[0016] In some embodiments, the metadata linked list also records the heat parameter and priority of each cached data block in the N cached data blocks, and the comprehensive parameters of the M first cached data blocks meet the preset comprehensive parameter requirements, and the comprehensive parameters are obtained based on the heat parameter and priority of the first cached data block. In this way, the user can determine the priority of the cached data block according to actual needs, so that the memory will use the comprehensive parameters determined by the heat parameter and priority as the basis for screening the first cached data block, and the first cached data block screened out for replacement is determined after comprehensively considering the data heat and data priority of each cached data block, so that the first cached data block to be eliminated can be more reasonable and more in line with actual needs.

[0017] In some embodiments, the comprehensive parameter requirement is that the comprehensive parameter of each of the first cached data blocks is within a preset comprehensive parameter range. This enables the comprehensive parameter of each of the M first cached data blocks to be optimal, so that the M first cached data blocks obtained in this way better meet actual needs.

[0018] In some embodiments, the comprehensive parameter requirement is that the average value of the heat parameters of the M first cached data blocks is within a preset comprehensive parameter average value range. This ensures that the average value of the M first cached data blocks is optimal, which can ensure that the obtained M first cached data blocks meet actual needs and enable the memory to determine the M first cached data blocks more quickly, thereby helping to reduce the delay when the user device obtains data.

[0019] In some embodiments, the comprehensive parameter requirement is: the M first cached data blocks are cached data blocks of the first data block group to be screened among multiple data block groups to be screened, each of the data block groups to be screened includes one or more cached data blocks, and the sum of the data sizes of the one or more cached data blocks in each of the data block groups to be screened is greater than or equal to the data size of the data to be cached; the average value of the comprehensive parameters of the M first cached data blocks in the first data block group to be screened is better than the average value of the comprehensive parameters of each second data block group to be screened except the first data block group to be screened among the multiple data block groups to be screened, and the average value of the comprehensive parameters of each second cached data block group is the average value of the comprehensive parameters of one or more cached data blocks in the second data block group to be screened. In this way, the M first cached data blocks with relatively better comprehensive parameters can be determined.

[0020] In some embodiments, the sum of the data sizes corresponding to the M-1 first cached data blocks with sequentially continuous storage addresses is smaller than the data size of the data to be cached. In this way, the sum of the data sizes of the M first cached data blocks can be more closely matched with the data size of the data to be cached. This avoids the situation where the data size of the first cached data block is too much larger than the data size of the data to be cached, thereby causing the excessive cached data to be erased when the data to be cached is written into the storage space corresponding to the storage address of the first cached packet.

[0021] In some embodiments, the N cached data blocks are N consecutive cached data blocks after the data block last written to the storage disk according to the storage address sequence, and the sum of the data sizes of the N cached data blocks is less than the data size threshold. In this way, the memory searches for the first cached data block to be eliminated in a storage interval after the last write position according to the storage address sequence. Instead of traversing all storage locations of the storage disk to find the first cached data block to be eliminated, the first cached data block to be eliminated can be found more quickly, thereby improving the efficiency of cache data writing.

[0022] In a second aspect, an embodiment of the present application provides a data caching device, wherein the data caching device is used to be deployed in a memory of a terminal device, wherein the terminal device includes the memory and a storage disk, wherein the storage disk is used to cache data blocks stored in a remote device, and wherein the data caching device includes:

[0023] a processing unit, configured to determine, according to a metadata linked list of the storage disk, M first cached data blocks having a continuous storage address sequence from the N cached data blocks of the storage disk, wherein the metadata linked list records the storage address sequence of the N cached data blocks, and the metadata linked list also records the data size and storage address of each cached data block in the N cached data blocks, and the sum of the data sizes corresponding to the M first cached data blocks is greater than or equal to the data size of the data to be cached, wherein M and N are positive integers, and N≥M;

[0024] The cache unit is used to write the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk.

[0025] In the data cache device of the embodiment of the present application, the memory can determine the M first cached data blocks with continuous storage address sequence from the N cached data blocks of the storage disk according to the metadata linked list corresponding to the storage disk, and then write the data to be cached into the storage space corresponding to the storage address of the M first cached data blocks. In this way, since the storage address sequence of the M first cached data blocks is continuous, the memory can write the data to be cached continuously into the storage disk, avoiding the low read and write efficiency caused by splitting the data to be written into different locations. In this way, the read and write performance of the storage disk is better, which helps to reduce the delay when the user device obtains data.

[0026] Moreover, since the metadata linked list records the storage address sequence of the N cached data blocks, the memory can directly determine the M first cached data blocks from the N cached data blocks more flexibly according to the storage address sequence recorded in the metadata linked list, rather than being limited to continuing to write data at the storage address written last time, thereby enabling more flexible selection of cache addresses for caching data. For example, when selecting the M first cached data blocks, the memory can skip data with higher popularity or data that is defined as not eligible for elimination, thus enabling more flexible determination of data to be eliminated, making data elimination in the storage disk more reasonable.

[0027] In some embodiments, the metadata linked list also records the heat parameters of each cached data block; the processing unit is specifically used for: the memory determines the M first cached data blocks that meet the heat requirements of cold data from the N stored data blocks according to the metadata linked list. In this way, each first cached data block meets the heat requirements corresponding to the cold data, which can avoid eliminating hot data when writing the data to be cached, making the eliminated data more reasonable, and can improve the hit rate of hot data. It can also avoid the need for the memory to frequently obtain hot data from the source database of the remote device, so that it can provide data to users more quickly according to user needs, which helps to reduce the delay when the user device obtains data.

[0028] In some embodiments, the heat requirement of the cold data is: the heat parameter of each of the first cached data blocks is within the first heat parameter range corresponding to the cold data. In this way, the heat of each of the M first cached data blocks is relatively low, which effectively avoids the elimination of hot data and helps to improve the hit rate of the cached data in the storage disk.

[0029] In some embodiments, the heat requirement of the cold data is: the average heat parameter of the M first cached data blocks is within the heat parameter mean range corresponding to the cold data. This ensures that the average heat of the M first cached data blocks is low, which can avoid the elimination of hot data and enable the memory to determine the M first cached data blocks faster, thereby helping to reduce the delay when the user device obtains data.

[0030] In some embodiments, the heat requirement of the cold data is: the M first cached data blocks are cached data blocks of the first cached data block group among multiple cached data block groups, each cached data block group includes one or more cached data blocks with consecutive storage addresses, and the sum of the data sizes of one or more cached data blocks in each cached data block group is greater than or equal to the data size of the data to be cached; the average value of the heat parameters of the M first cached data blocks is lower than the average value of the heat parameters of each second cached data block group except the first cached data block group among the multiple cached data block groups, and the average value of the heat parameters of each second cached data block group is the average value of the heat parameters of one or more cached data blocks in the second cached data block group. In this way, the M first cached data blocks with relatively low heat can be determined, and the elimination of hot data can be effectively avoided, which helps to improve the hit rate of cached data in the storage disk.

[0031] In some implementations, the metadata linked list also records the priority of each cached data block, and the priority of each first cached data block meets the priority requirement. In this way, by manually setting the priority condition and the priority of the cached data block, some cached data blocks can be set to be retained, avoiding the elimination of some cached data blocks that cannot be eliminated.

[0032] In some embodiments, the processing unit is further used to: determine, from the M first cached data blocks, a second cached data block whose heat parameter is within the second heat parameter range corresponding to the relatively hot data; the cache unit is further used to write the second cached data block to a preset storage location of the memory. In this way, the second cached data block with relatively high heat among the M first cached data blocks can be written to the preset address storage location, which can avoid the need for the memory to obtain the data again from the source database when providing the second cached data block to the user device, thereby reducing the delay when the user device obtains data.

[0033] In some embodiments, the metadata linked list also records the heat parameter and priority of each cached data block in the N cached data blocks, and the comprehensive parameters of the M first cached data blocks meet the preset comprehensive parameter requirements, and the comprehensive parameters are obtained based on the heat parameter and priority of the first cached data block. In this way, the user can determine the priority of the cached data block according to actual needs, so that the memory will use the comprehensive parameters determined by the heat parameter and priority as the basis for screening the first cached data block, and the first cached data block screened out for replacement is determined after comprehensively considering the data heat and data priority of each cached data block, so that the first cached data block to be eliminated can be more reasonable and more in line with actual needs.

[0034] In some embodiments, the comprehensive parameter requirement is that the comprehensive parameter of each of the first cached data blocks is within a preset comprehensive parameter range. This enables the comprehensive parameter of each of the M first cached data blocks to be optimal, so that the M first cached data blocks obtained in this way better meet actual needs.

[0035] In some embodiments, the comprehensive parameter requirement is that the average value of the heat parameters of the M first cached data blocks is within a preset comprehensive parameter average value range. This ensures that the average value of the M first cached data blocks is optimal, which can ensure that the obtained M first cached data blocks meet actual needs and enable the memory to determine the M first cached data blocks more quickly, thereby helping to reduce the delay when the user device obtains data.

[0036] In some embodiments, the comprehensive parameter requirement is: the M first cached data blocks are cached data blocks of the first data block group to be screened among multiple data block groups to be screened, each of the data block groups to be screened includes one or more cached data blocks, and the sum of the data sizes of the one or more cached data blocks in each of the data block groups to be screened is greater than or equal to the data size of the data to be cached; the average value of the comprehensive parameters of the M first cached data blocks in the first data block group to be screened is better than the average value of the comprehensive parameters of each second data block group to be screened except the first data block group to be screened among the multiple data block groups to be screened, and the average value of the comprehensive parameters of each second cached data block group is the average value of the comprehensive parameters of one or more cached data blocks in the second data block group to be screened. In this way, the M first cached data blocks with relatively better comprehensive parameters can be determined.

[0037] In some embodiments, the sum of the data sizes corresponding to the M-1 first cached data blocks with sequentially continuous storage addresses is smaller than the data size of the data to be cached. In this way, the sum of the data sizes of the M first cached data blocks can be more closely matched with the data size of the data to be cached. This avoids the situation where the data size of the first cached data block is too much larger than the data size of the data to be cached, thereby causing the excessive cached data to be erased when the data to be cached is written into the storage space corresponding to the storage address of the first cached packet.

[0038] In some embodiments, the N cached data blocks are N consecutive cached data blocks after the data block last written to the storage disk according to the storage address sequence, and the sum of the data sizes of the N cached data blocks is less than the data size threshold. In this way, the memory searches for the first cached data block to be eliminated in a storage interval after the last write position according to the storage address sequence. Instead of traversing all storage locations of the storage disk to find the first cached data block to be eliminated, the first cached data block to be eliminated can be found more quickly, thereby improving the efficiency of cache data writing.

[0039] In a third aspect, the present application provides a terminal device comprising a memory and a storage disk, wherein the storage disk is used to cache data blocks stored in a remote device, and the memory is used to implement the data caching method described in any embodiment of the first party above, or the memory is deployed with the data caching device described in any embodiment of the second aspect above.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions instruct the terminal device to execute the method provided by the above-mentioned first aspect or various possible implementations of the first aspect, or the computer instructions instruct the terminal device to deploy the data caching device provided by the above-mentioned second aspect or various possible implementations of the second aspect.

[0041] In a fifth aspect, the present application provides a computer program product, the computer program product comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. The computer instructions instruct a terminal device to execute the data caching method described in any one of the embodiments of the first aspect, or the computer instructions instruct the terminal device to deploy the data caching device described in any one of the embodiments of the second aspect.

[0042] In a sixth aspect, the present application provides a chip, which may include a programmable logic circuit and / or program instructions, and when the chip is running, it is used to implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A framework diagram of a storage system according to an embodiment of the present application;

[0044] Figure 2A A schematic diagram of a data caching process in the prior art;

[0045] Figure 2B Another schematic diagram of a data caching process in the prior art;

[0046] Figure 3 A schematic diagram of a data caching method according to an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the structure of the metadata linked list of an embodiment of the present application;

[0048] Figure 5 A schematic diagram of a scenario of a data caching method according to an embodiment of the present application;

[0049] Figure 6 Another schematic diagram of the structure of the metadata linked list of the embodiment of the present application;

[0050] Figure 7 A schematic diagram of the structure of a cache controller according to an embodiment of the present application;

[0051] Figure 8 Another schematic diagram of a data caching method according to an embodiment of the present application;

[0052] Fig. 9 It is another structural schematic diagram of the metadata linked list of the embodiment of the present application;

[0053] Fig.10 It is another flowchart of the data caching method according to an embodiment of the present application;

[0054] Fig.11 Another schematic diagram of the structure of the cache controller according to the embodiment of the present application;

[0055] Fig.12 This is another flow chart of the data caching method according to an embodiment of the present application;

[0056] Fig.13 This is yet another flow chart of the data caching method according to an embodiment of the present application;

[0057] Fig.14 A module diagram of a data cache device according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0059] See also Figure 1 , Figure 1 The storage system 1000 of the embodiment of the present application is a framework diagram. The storage system 1000 includes a terminal device 100 and a remote device 200. The remote device 200 is deployed with an active database. The terminal device 100 can obtain data from the source database.

[0060] Specifically, the terminal device 100 includes a memory 10 and a storage disk 20. The memory 10 may include a memory cache 11 and a cache controller 12. The cache controller 12 may obtain data from a source database and store it in the memory cache 11. When the storage space available for storing new data in the memory cache 11 is insufficient, or the storage space occupied by the data cached in the memory cache 11 is too large, the cache controller 12 controls to cache part or all of the data in the memory cache 11 in the storage disk 20. When the memory 10 needs to obtain data, the cache controller 12 first searches for the required data from the memory cache 11. If the required data is not in the memory cache 11, the cache controller 12 continues to search for the required data from the storage disk 20. If the required data is not in the storage disk 20, the cache controller 12 will obtain the required data from the source database of the remote device 200.

[0061] The terminal device 100 of the embodiment of the present application may be, for example, but not limited to, a server, a mobile phone, a personal computer, a smart bracelet, etc. The storage disk 20 of the embodiment of the present application is a non-volatile storage device, such as a hard disk drive (HHD), a solid state disk (SSD), etc.

[0062] The data cached by the memory 10 to the storage disk 20 each time can be regarded as a cached data block. The size of the data block cached by the memory 10 to the storage disk 20 each time may be different or the same. Each time the memory 10 needs to cache the data to be cached to the storage disk 20, the cached data block of the corresponding size to the data to be cached needs to be eliminated from the storage disk 20.

[0063] If the data blocks to be eliminated are selected according to the LRU algorithm, the cached data blocks to be eliminated are the cached data blocks with the lowest temperature. However, since the data size of the cached data block with the lowest temperature may be smaller than the data size of the data to be cached, it may be necessary to eliminate the cached data block with the lowest temperature and the second-to-last cached data block, or eliminate the cached data block with the lowest temperature, the second-to-last cached data block, and the third-to-last cached data block. It can be seen that when the memory 10 caches data to the storage disk, it may be necessary to eliminate multiple cached data blocks that are sorted from high to low in temperature and are at the end. However, the multiple cached data blocks at the end may be stored in different locations in the storage disk, so that the data to be cached will also be cached in different locations, and the data writing efficiency is low.

[0064] See also Figure 2A , Figure 2A This is a schematic diagram of a scenario where data is written according to the LRU algorithm. Figure 2A , the storage disk includes m consecutive storage blocks (Block), such as Figure 2ABlock 1 to Block m in the cache. The storage disk stores n cached data blocks, and the memory selects the data to be eliminated according to the cache queue. The cache queue includes cached data 1, cached data 2, cached data 3, ..., cached data n-2, cached data n-1, and cached data n arranged according to the heat of the cached data blocks. The data size of the data to be cached is 3M, so 3M of cached data needs to be eliminated. The sum of the data sizes of cached data block n, cached data block n-1, and cached data block n-2 at the end of the cache queue is 3MB. If cached data block n is stored in Block x to block x+15, cached data block n-1 is stored in Block x+100 to block x+115, and cached data block n-2 is stored in Block x+200 to block x+215. Then according to the LRU algorithm, the memory will divide the data to be cached into three parts, namely, data to be cached 1, data to be cached 2, and data to be cached 3, which are written to Block x to block x+15, Block x+100 to block x+115, and Block x+200 to block x+215, respectively. Among them, 0<x<x+215<m. In this way, when the memory needs to read the data from the storage disk, it needs to read from multiple locations, resulting in poor data reading performance. For example, in the Content Delivery Network (CDN) scenario, the memory reads data from the storage disk according to user needs and sends it to the user. The storage disk capacity of the edge node of the CDN is large. If the memory needs to read data from multiple addresses in the storage disk separately, the data reading efficiency will be low, resulting in a large delay when the user obtains the data.

[0065] If data is written into the storage space corresponding to the continuous storage addresses and data is eliminated continuously according to the storage addresses, the data to be cached can be written into the storage space corresponding to the continuous storage addresses. However, in this way, data can only be written after the storage address last written in the order of the storage addresses, and other data whose storage addresses are located after the data eliminated last time will be eliminated accordingly. Figure 2B As shown, Figure 2B The following is a schematic diagram of a data cache scenario when writing data continuously. The storage disk includes m consecutive storage blocks, such as Figure 2B If the last written data A is stored in block q, the subsequent cached data B can only be written to block q+1, block q+2, block q+3, etc. In this way, there is almost no room for choice of storage space for caching data, and it is easy to eliminate some hot data or data that is defined as not to be eliminated.

[0066] To this end, this application provides a caching method. Figure 3 , Figure 3 The following is a flow chart of a caching method according to an embodiment of the present application. The caching method includes:

[0067] S32. The memory determines M first cached data blocks with consecutive storage addresses from the N cached data blocks of the storage disk according to the metadata linked list corresponding to the storage disk, where M and N are positive integers and N≥M.

[0068] The metadata linked list records the storage address sequence of the N cached data blocks of the storage disk, and the metadata linked list also records the data size and storage address of each cached data block in the M cached data blocks.

[0069] See also Figure 4 , Figure 4 A schematic diagram of the structure of a metadata linked list provided for an embodiment of the present application. The metadata linked list includes multiple metadata, each of which includes a data identifier (key), data size (size), storage address information (offset) of a cached data block, a data identifier (previous key) of a previous cached data block located before the cached data block in storage address order, and a data identifier (next key) of a next cached data block located after the cached data block. In this way, the memory can directly obtain the storage address order of multiple stored data blocks in the storage disk through the metadata linked list.

[0070] It can be understood that in this embodiment, the memory determines M first cached data blocks from N cached data blocks according to the screening conditions. The screening conditions include: the storage address sequence of the M first cached data blocks is continuous, and the sum of the data sizes corresponding to the M first cached data blocks is greater than or equal to the data size of the data to be cached. In this way, it can be ensured that there is enough storage space to store the data to be cached.

[0071] Optionally, when the cache demand of the data to be cached is obtained, the memory can determine the M first cached data blocks with consecutive storage addresses from the N cached data blocks of the storage disk according to the metadata linked list corresponding to the storage disk. For example, when the data in the memory cache of the memory exceeds 50% of the capacity of the memory cache, the memory uses the data in the memory cache as the data to be cached and confirms that the cache demand of the data to be cached is obtained.

[0072] S34. The memory writes the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk.

[0073] See also Figure 5 , Figure 5Schematic diagram of a cache method according to an embodiment of the present application. The storage disk includes m storage blocks, where m is a positive integer. Figure 5 Block 1 to Block m in the memory. N cached data blocks are stored in Block 1 to Block m. The N cached data blocks are respectively stored data block 1, stored data block 2, ..., stored data block N, where N is a positive integer. When it is necessary to cache the data to be cached to the storage disk, determine the M first cached data blocks with continuous storage address sequence from the N cached data blocks. Since in the present application, the storage address sequence of the N cached data blocks can be recorded according to the metadata linked list, the memory can directly determine the M first cached data blocks with continuous storage address sequence and the sum of data sizes greater than or equal to the data size of the data to be cached from the N cached data blocks according to the metadata linked list.

[0074] For example, the size of the data to be cached is 3MB. The memory analyzes the metadata linked list to obtain that the storage address sequence of the stored data block i, the cached data block j, and the cached data block q is continuous. Figure 5 As shown, the stored data block i is stored in block y to block y+15, and the cached data block j is stored in block y+16 to block y+31. The cached data block q is stored in block y+32 to block y+47. The data sizes of the stored data block i, the cached data block j, and the cached data block q are all 1MB. Then the sum of the data sizes of the stored data block i, the cached data block j, and the cached data block q is 3MB, which is equal to the data size of the data to be cached. Then, the memory can store the data to be cached in the storage space corresponding to the storage address of the stored data block i, the cached data block j, and the cached data block q. That is, the data to be cached is stored in block y to block y+47. In this way, the data to be cached can be merged and written, and there is no need to store the data to be cached in different locations, thereby improving the reading and writing efficiency.

[0075] It can be understood that when the data to be cached is written into the storage space corresponding to the storage address of the first cached data block in the storage disk, the first cached data block will be erased.

[0076] It can be seen that in the technical solution of the embodiment of the present application, the memory determines the M first cached data blocks with continuous storage address sequence from the N cached data blocks of the storage disk according to the metadata linked list corresponding to the storage disk, and then writes the data to be cached into the storage space corresponding to the storage address of the M first cached data blocks. In this way, since the storage address sequence of the M first cached data blocks is continuous, the memory can write the data to be cached continuously into the storage disk, avoiding low read and write efficiency caused by splitting the data to be written into different locations. In this way, the read and write performance of the storage disk is better, which helps to reduce the delay when the user device obtains data.

[0077] Moreover, since the metadata linked list records the storage address sequence of the N cached data blocks, the memory can directly determine the M first cached data blocks from the N cached data blocks more flexibly according to the storage address sequence recorded in the metadata linked list, rather than being limited to continuing to write data at the storage address written last time, thereby enabling more flexible selection of cache addresses for caching data. For example, when selecting the M first cached data blocks, the memory can skip data with higher popularity or data that is defined as not eligible for elimination, thus enabling more flexible determination of data to be eliminated, making data elimination in the storage disk more reasonable.

[0078] It can be understood that step S32 and step S34 of the embodiment of the present application can be implemented by the cache controller 12 of the memory 10. That is, the cache controller 12 determines M first cached data blocks with consecutive storage addresses from the N cached data blocks of the storage disk according to the metadata linked list corresponding to the storage disk, and writes the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk.

[0079] Furthermore, the sum of the data sizes corresponding to the M-1 first cached data blocks with consecutive storage addresses is smaller than the data size of the data to be cached. Figure 5 In the example shown, the sum of the data sizes of the stored data block i and the cached data block j with sequentially continuous storage addresses is less than 3MB, and the sum of the data sizes of the cached data block j and the cached data block q with sequentially continuous storage addresses is also less than 3MB. In this way, the sum of the data sizes of the M first cached data blocks can be more closely matched with the data size of the data to be cached. This avoids the data size of the first cached data block being too much larger than the data size of the data to be cached, thereby erasing excessive cached data blocks when writing the data to be cached into the storage space corresponding to the storage address of the first cached packet.

[0080] Optionally, the N cached data blocks may be N consecutive cached data blocks after the data block last written to the storage disk according to the storage address sequence, and the sum of the data sizes of the N cached data blocks is less than the data size threshold. In this way, the memory searches for the first cached data block to be eliminated in a storage interval after the last write position according to the storage address sequence. Instead of traversing all the storage locations of the storage disk to find the first cached data block to be eliminated, the first cached data block to be eliminated can be found more quickly, thereby improving the efficiency of cache data writing.

[0081] The data size threshold can be set according to the total capacity of the storage disk, for example, it can be set to 20% of the total capacity of the storage disk. The data size threshold can also be set according to the application scenario. For example, in a certain application scenario, when the data block stored in the memory to the cache is usually about 10MB, the data size threshold can be set to 80MB.

[0082] See also Figure 6 , Figure 6 Another structural diagram of the metadata linked list provided for the present embodiment. Based on the above embodiment, in an optional embodiment, the metadata linked list also records the heat parameters of each cached data block. In this way, the memory can determine the M first cached data blocks with consecutive storage address order from the N cached data blocks of the storage disk according to the heat parameters of each cached data block in the metadata linked list. In this way, the memory can skip the hotter data when selecting the M first cached data blocks, which can improve the hit rate of the hotter data and avoid the need for the memory to frequently obtain the hotter data from the source database, so that data can be provided to users more quickly according to user needs, which helps to reduce the delay when the user device obtains data.

[0083] Specifically, step S32 includes: the memory determines, from the N stored data blocks, M first cached data blocks that meet the heat requirement of the cold data according to the metadata linked list. The heat requirement may be preset. In this way, the memory selects the M first cached data blocks according to the heat requirement, which can avoid eliminating data with higher heat, making the eliminated data more reasonable, thereby helping to improve the hit rate of the data and the read and write efficiency of the storage disk.

[0084] The heat parameter of each cached data block can be determined based on the number of times or frequency that the cached data block is read. The more times the cached data block is read, the larger the heat parameter; or the higher the frequency that the cached data block is read, the larger the heat parameter. The embodiment of the present application does not limit the calculation rules of the heat parameters. There is no limitation on the calculation rules of the heat parameters. In this way, the technical solution of the present application is highly adaptable and can be applied to more scenarios.

[0085] Specifically, the heat requirement of the cold data includes a first heat requirement or a second heat requirement.

[0086] The first heat requirement may be: the heat parameter of each first cached data block is within the first heat parameter range corresponding to the cold data. For example, if the heat of the cached data block is higher, the heat parameter is larger, then the first heat parameter range corresponding to the cold data may be a heat parameter less than the first heat parameter threshold. The heat parameter threshold may be obtained based on the heat parameter corresponding to the critical point of the cold data and the hot data determined in the actual application. In this way, the heat of each of the M first cached data blocks is relatively low, which effectively avoids the elimination of hot data and helps to improve the hit rate of cached data in the storage disk.

[0087] The first heat requirement may also be: the average heat parameter of the M first cached data blocks is within the heat parameter average range corresponding to the cold data. This ensures that the average heat of the M first cached data blocks is low, which can avoid eliminating hot data and enable the memory to determine the M first cached data blocks faster, thereby helping to reduce the delay when the user device obtains data.

[0088] The second heat requirement may be: the M first cached data blocks are cached data blocks of the first cached data block group among multiple cached data block groups, each cached data block group includes one or more cached data blocks with consecutive storage addresses, and the sum of the data sizes of one or more cached data blocks in each cached data block group is greater than or equal to the data size of the data to be cached; the average value of the heat parameters of the M first cached data blocks is lower than the average value of the heat parameters of each second cached data block group except the first cached data block group among multiple cached data block groups, and the average value of the heat parameters of each second cached data block group is the average value of the heat parameters of one or more cached data blocks in the second cached data block group. In this way, the M first cached data blocks with relatively low heat can be determined, and the elimination of hot data can be effectively avoided, which helps to improve the hit rate of cached data in the storage disk.

[0089] Optionally, if the number of cached data blocks in any cached data block group is h, the sum of the data sizes of h-1 cached data blocks with consecutive storage addresses in the cached data block group is less than or equal to the data size of the data to be cached. h is a positive integer. Among them, the number of cached data blocks in the first cached data block group is M. That is, each cached data block in the first cached data block group is a first cached data block. This avoids eliminating too many cached data blocks and can also reduce the data processing capacity of the cache controller.

[0090] Figure 71 is a schematic diagram of the structure of the cache controller 12 according to an embodiment of the present application. Figure 7 Combined with Figure 1 , the cache controller 12 includes a metadata module 121, a storage disk control module 122 and a cache policy module 123. The storage disk control module 122 is used to cache the data in the memory cache 11 to the storage disk 20 according to the metadata linked list. The metadata module 121 is used to cache the data in the memory cache 11 to the storage disk 20 according to the records in the storage disk control module 122, and adjust the information of each metadata in the metadata linked list. The cache policy module 123 is used to obtain the calculation rules of the set heat parameters, so that the storage disk control module 122 can determine the heat parameters of the data blocks to be cached in the storage disk 20 according to the calculation rules of the heat parameters, and use the heat parameters as the basis for determining the M first cached data blocks.

[0091] In the embodiment of the present application, step S32 and step S34 may be implemented by the storage disk control module 122. That is, the storage disk control module 122 determines, according to the metadata linked list corresponding to the storage disk, M first cached data blocks with consecutive storage addresses from the N cached data blocks of the storage disk, and a storage space corresponding to the storage address of the first cached data block in the storage disk for writing the data to be cached.

[0092] Specifically, the heat parameter of each cached data block in the metadata linked list is determined by the storage disk control module 122 according to a heat parameter calculation rule. The heat parameter calculation rule is obtained by the storage disk control module 122 from the cache strategy module 123.

[0093] In the embodiment of the present application, the cache strategy module 123 may provide a calculation rule interface 1231 for setting the calculation rule of the heat parameter, so that the user can flexibly set the calculation rule of the heat parameter according to actual needs through the calculation rule interface 1231 of the cache strategy module 123. The cache strategy module 123 uses the calculation rule interface 1231 to obtain the calculation rule of the heat parameter, and sends the calculation rule of the heat parameter to the storage disk control module 122.

[0094] The storage disk control module 122 obtains the calculation rules of the heat parameters from the cache strategy module 123, and the storage disk control module 122 calculates and determines the heat parameters of each cached data block according to the heat calculation rules. The storage disk control module 122 can provide a heat requirement interface 1221 for setting the heat requirement of cold data, so that the user can flexibly set the first heat requirement according to actual needs through the heat requirement interface 1221. The storage disk control module 122 can use the heat requirement interface 1221 to obtain the first heat requirement, and use the first heat requirement as a basis for screening the M first cached data blocks. Then, the storage disk control module 122 determines the M first cached data blocks from the N cached data blocks according to the metadata linked list.

[0095] It can be seen that in the embodiment of the present application, the storage disk control module 122 does not need to interact with the cache strategy module 123 in the process of determining the M first cached data blocks and writing the data to be cached into the storage space corresponding to the storage address of the M first cached data blocks in the storage disk, and is isolated from the cache strategy module 123, so that the storage disk control module 123 is completely decoupled from the cache strategy module 122. In this way, the storage disk control module 122 in the embodiment of the present application determines the M first cached data blocks and writes the data to be cached into the storage disk, and adapts to the calculation rules of various heat parameters, so that the data caching method of the present application is more adaptable.

[0096] like Figure 8 , Figure 8 Another flow chart of the data caching method according to an embodiment of the present application is shown in FIG. Figure 3 In the embodiment shown, in some optional embodiments, step S32 includes:

[0097] S3201. The memory uses the first cached data block among N cached data blocks as the starting data block according to the storage address sequence.

[0098] The N cached data blocks may be N consecutive cached data blocks after the data block last written to the storage disk according to the storage address sequence, and the sum of the data sizes of the N cached data blocks is less than the data size threshold.

[0099] S3202: The memory determines a candidate cached data block group corresponding to the starting point data block.

[0100] The candidate cached data block group is a cached data block group among the multiple cached data block groups, and the cached data block that ranks first in the storage position order among the multiple cached data blocks in the candidate cached data block group is the starting data block.

[0101] S3203: The memory determines whether the cached data blocks in the candidate cached data block group meet the first heat requirement.

[0102] Specifically, the memory determines whether the heat parameters of the cached data blocks in the candidate cached data block group meet the first heat requirement based on the heat parameters of each cached data block in the candidate cached data block group in the metadata linked list.

[0103] For explanations of the first heat requirement, please refer to the above embodiment, which will not be described here again to avoid redundancy.

[0104] S3204: If yes, the memory uses each cached data block in the candidate cached data block group as the first cached data block;

[0105] S3205, if not, the memory determines whether there is a next cached data block according to the storage address sequence;

[0106] S3206: If yes, the memory uses the next cached data block as the starting data block and returns to step S3202;

[0107] S3207: If not, the memory uses each cached data block in the first cached data block group as a first cached data block.

[0108] It can be understood that the heat parameters of the cached data blocks in the first cached data block group meet the above second heat requirement.

[0109] In this way, first, according to the metadata linked list, in a storage interval after the last write position, M first cached data blocks with continuous storage address sequence and heat parameters that meet the first heat requirement are searched. If M first cached data blocks are not found in the storage interval, the cached data blocks that meet the second heat requirement in the storage interval are used as the first cached data blocks. In this way, it is possible to avoid traversing all storage locations of the storage disk to find the first cached data blocks to be eliminated, which helps to improve the data cache efficiency. In addition, in this way, cached data blocks with heat parameters within the first heat parameter range or cached data blocks with an average value of heat parameters within the heat parameter mean range are preferentially used as the first cached data blocks, which can effectively avoid the elimination of hot data. Moreover, even if cached data blocks with heat parameters within the first heat parameter range or cached data blocks with an average value of heat parameters within the heat parameter mean range are not found, the memory will find M first cached data blocks with relatively low heat parameters and continuous storage address sequence from these N cached data blocks, which can also effectively avoid the elimination of hot data. The M first cached data blocks determined in this way are more reasonable.

[0110] See also Fig. 9 , Fig. 9 Another structural diagram of the metadata linked list provided in the embodiment of the present application. Fig. 9 As shown, in some optional embodiments, the metadata linked list also records the priority of each stored data block, and the priority of each first cached data block meets the priority requirement. The priority requirement can be set manually. In this way, by manually setting the priority conditions and the priority of the cached data blocks, it is possible to set to retain certain cached data blocks and avoid eliminating some cached data blocks that cannot be eliminated. For example, the priority requirement is that the priority is lower than the priority threshold. If for a certain cached data block s, regardless of the heat parameter, the cached data block s needs to be retained in the storage disk, the priority of the cached data block s can be set to a level higher than or equal to the priority threshold. In this way, it is possible to always retain the cached data block s in the storage disk and avoid the cached data block s from being eliminated.

[0111] See also Fig.10 , Fig.10 This is another flow chart of the data exchange cache method according to the embodiment of the present application, based on Figure 3 In the embodiment shown, in some optional embodiments, step S32 includes:

[0112] S3211. The memory uses the first cached data block among the N cached data blocks as the starting data block according to the storage address sequence.

[0113] The N cached data blocks may be N consecutive cached data blocks after the data block last written to the storage disk according to the storage address sequence, and the sum of the data sizes of the N cached data blocks is less than the data size threshold.

[0114] S3212: The memory determines a candidate cached data block group corresponding to the starting point data block.

[0115] The candidate cached data block group is a cached data block group among the multiple cached data block groups, and the cached data block that ranks first in the storage position order among the multiple cached data blocks in the candidate cached data block group is the starting data block.

[0116] S3213. The memory determines whether the cached data blocks in the candidate cached data block group meet the first heat requirement, and whether the priority of each cached data block in the cached data block group meets the preset priority requirement.

[0117] S3214: If yes, the memory uses each cached data block in the candidate cached data block group as the first cached data block;

[0118] S3215, if not, the memory determines whether there is a next cached data block according to the storage address sequence;

[0119] S3216: If yes, use the next cached data block in the memory as the starting data block, and return to step S3212;

[0120] S3217. If not, the memory uses each cached data block in a third cached data block group among the multiple cached data block groups as a first cached data block, and the third cached data block group is a cached data block with the lowest average value of a heat parameter among one or more cached data blocks whose priority level meets the priority requirements.

[0121] In this way, first, according to the metadata linked list, in a storage interval after the last write position, search for M first cached data blocks with continuous storage address sequence, heat parameters that meet the first heat requirement, and priorities that meet the priority requirements. If M first cached data blocks are not found in the storage interval, then the cached data blocks with continuous storage addresses, meeting the second heat requirement, and meeting the priority requirements in the storage interval are used as the first cached data blocks. This can avoid traversing all storage locations of the storage disk to find the first cached data block to be eliminated, which helps to improve data caching efficiency. Moreover, in this way, the cached data blocks whose heat parameters are within the first heat parameter range or the cached data blocks whose heat parameter average values ​​are within the heat parameter average value range are given priority as the first cached data blocks, which can effectively avoid the elimination of hot data. Moreover, even if the cached data blocks whose heat parameters are within the first heat parameter range or the cached data blocks whose heat parameter average values ​​are within the heat parameter average value range are not found, the memory will find M first cached data blocks with relatively low heat parameters and continuous storage address order from the N cached data blocks, which can also effectively avoid the elimination of hot data. The M first cached data blocks determined in this way are more reasonable.

[0122] See also Fig.11 , Fig.11 Another schematic diagram of the structure of the cache controller according to the embodiment of the present application. Figure 7 In the illustrated embodiment, the memory further includes an application program interface (API) module 124. The API module 124 is used to obtain the priority of the cached data block and send the priority of the cached data block to the storage disk control module 122. The storage disk control module 122 obtains the priority of the cached data block from the API module 124 and uses the priority of the cached data block as a basis for screening the first cached data block.

[0123] The API module 124 may provide a priority interface 1241 for setting the priority of the cached data block. Thus, the user may flexibly set the priority of the cached data block according to actual needs through the priority interface 1241.

[0124] In some optional embodiments, comprehensive parameters of the M first cached data blocks meet preset comprehensive parameter requirements, and the comprehensive parameters are obtained according to heat parameters and priorities of the first cached data blocks.

[0125] In this way, the user can determine the priority of the cached data block according to actual needs, so that the memory will use the comprehensive parameters determined by the heat parameter and the priority as the basis for screening the first cached data block. The first cached data block screened out for replacement is determined after comprehensively considering the data heat and data priority of each cached data block, thereby making the first cached data block to be eliminated more reasonable and more in line with actual needs.

[0126] Specifically, the comprehensive parameter requirement includes a first comprehensive parameter requirement or a second comprehensive parameter requirement.

[0127] The first comprehensive parameter requirement may be that the comprehensive parameter of each first cached data block is within a preset comprehensive parameter range, so that the comprehensive parameter of each first cached data block in the M first cached data blocks is relatively optimal, and the M first cached data blocks obtained in this way better meet actual needs.

[0128] The first comprehensive parameter requirement may also be that the average value of the heat parameters of the M first cached data blocks is within a preset comprehensive parameter mean range. This can ensure that the average value of the M first cached data blocks is optimal, which can not only ensure that the obtained M first cached data blocks meet actual needs, but also enable the memory to determine the M first cached data blocks more quickly, thereby helping to reduce the delay when the user device obtains data.

[0129] The second comprehensive parameter requirement is: the M first cached data blocks are cached data blocks of the first data block group to be screened among the multiple data block groups to be screened, each data block group to be screened includes one or more cached data blocks, and the sum of the data sizes of the one or more cached data blocks in each data block group to be screened is greater than or equal to the data size of the data to be cached; the average value of the comprehensive parameters of the M first cached data blocks in the first data block group to be screened is better than the average value of the comprehensive parameters of each second data block group to be screened except the first data block group to be screened among the multiple data block groups to be screened, and the average value of the comprehensive parameters of each second cached data block group is the average value of the comprehensive parameters of one or more cached data blocks in the second data block group to be screened. In this way, the M first cached data blocks with relatively better comprehensive parameters can be determined.

[0130] Among them, the average value of the comprehensive parameters of the M first cached data blocks is better than the average value of the comprehensive parameters of each second cached data block group except the first cached data block group in the multiple data block groups to be screened. It can be understood that if the comprehensive parameter is larger, the comprehensive parameter is better, then the average value of the comprehensive parameters of the M first cached data blocks is greater than or equal to the average value of the comprehensive parameters of each second cached data block group; if the comprehensive parameter is smaller, the comprehensive parameter is better, then the average value of the comprehensive parameters of the M first cached data blocks is less than or equal to the average value of the comprehensive parameters of each second cached data block group.

[0131] Optionally, if the number of cached data blocks in any data block group to be screened is h, the sum of the data sizes of h-1 cached data blocks with consecutive storage addresses in the data block group to be screened is less than or equal to the data size of the data to be cached. h is a positive integer. Among them, the number of cached data blocks in the first data block group to be screened is M. That is, each cached data block in the first data block group to be screened is the first cached data block. This avoids eliminating too many cached data blocks and can also reduce the data processing capacity of the cache controller.

[0132] like Fig.12 , Fig.12 Another flow chart of the data caching method according to an embodiment of the present application. Figure 3 In the embodiment shown, in some optional embodiments, step S32 includes:

[0133] S3221. The memory uses the first cached data block among the N cached data blocks as the starting data block according to the storage address sequence.

[0134] The N cached data blocks may be N consecutive cached data blocks after the data block last written to the storage disk according to the storage address sequence, and the sum of the data sizes of the N cached data blocks is less than the data size threshold.

[0135] S3222. The memory determines the candidate data block group corresponding to the starting data block.

[0136] The candidate data block group is a data block group to be screened among the multiple data block groups to be screened, and among the multiple cached data blocks of the candidate data block group, the cached data block that ranks first in the storage position order is the starting data block.

[0137] S3223. The memory determines whether the comprehensive parameters of the cached data blocks in the candidate data block group meet the first comprehensive parameter requirements.

[0138] Specifically, the memory obtains the comprehensive parameters of each cached data block according to the heat parameters and priority of each cached data block in the candidate data block group in the metadata linked list, and then determines whether the cached data blocks in the candidate data block group meet the first comprehensive parameter requirement according to the comprehensive parameters of each cached data block in the candidate data block group.

[0139] For explanations of the first comprehensive parameter requirements, please refer to the above embodiment, which will not be repeated here to avoid redundancy.

[0140] S3224: If yes, the memory uses each cached data block in the candidate data block group as a first cached data block;

[0141] S3225, if not, the memory determines whether there is a next cached data block according to the storage address sequence;

[0142] S3226: If yes, the memory uses the next cached data block as the starting data block and returns to step S3222;

[0143] S3227: If not, the memory uses each cached data block in the first to-be-screened data block group as a first cached data block.

[0144] It can be understood that the comprehensive parameters of the cached data blocks in the first to-be-screened data block group meet the above-mentioned second comprehensive parameter requirements.

[0145] In this way, the memory first searches for M first cached data blocks whose storage addresses are sequentially continuous and whose comprehensive parameters meet the requirements of the first comprehensive parameter in a storage interval after the last write position according to the metadata linked list. If the M first cached data blocks are not found in the storage interval, the cached data blocks that meet the requirements of the second comprehensive parameter in the storage interval are used as the first cached data blocks. This can avoid traversing all storage locations of the storage disk to find the first cached data blocks to be eliminated, which helps to improve the efficiency of data caching. Moreover, in this way, the cached data blocks whose comprehensive parameters are within the comprehensive parameter range or the cached data blocks whose average values ​​of the comprehensive parameters are within the comprehensive parameter mean range are preferentially used as the first cached data blocks, which can effectively avoid the data with better comprehensive evaluation. Moreover, even if the cached data blocks whose comprehensive parameters are within the comprehensive parameter range or the cached data blocks whose average values ​​of the comprehensive parameters are within the comprehensive parameter mean range are not found, the memory will find the M first cached data blocks whose comprehensive parameters are relatively better and whose storage addresses are sequentially continuous from the N cached data blocks, which can also effectively avoid the elimination of data with better comprehensive parameters. The M first cached data blocks determined in this way are more reasonable.

[0146] See also Fig.13 , Fig.13This is another flow chart of the data caching method of the embodiment of the present application. Based on the above embodiment, in some optional embodiments, before step S34, the data caching method further includes the steps of:

[0147] S36, the memory determines, from the M first cached data blocks, a second cached data block whose heat parameter is within a second heat parameter range corresponding to the relatively hot data;

[0148] For example, if the cached data block has a higher temperature, the cached data block has a larger temperature parameter, and the second temperature parameter range may be a temperature parameter greater than or equal to the second temperature parameter threshold. In this way, a second cached data block with relatively high temperature can be determined from the M first cached data.

[0149] S38. The memory writes the second cached data block into a preset storage location of the memory.

[0150] Step S36 and step S38 may be implemented by the storage disk control module 122. The storage disk control module 122 may obtain the second thermal parameter range through the thermal requirement interface 1221.

[0151] In this way, the second cached data block with relatively high popularity among the M first cached data blocks can be written to the preset address storage location, which can avoid the need to obtain the second cached data block again from the source database when the memory needs to provide the second cached data block to the user device, thereby reducing the delay when the user device obtains data.

[0152] Furthermore, the memory can write the second cached data block into the storage disk as the cached data to be written into the storage disk next time, so as to avoid eliminating the second cached data block with relatively high popularity among the M plurality of first cached data blocks.

[0153] See also Fig.14 , Fig.14 1 is a structural diagram of a data cache device 1400 of an embodiment of the present application. The embodiment of the present application also provides a data cache device 1400. The data cache device 1400 is used to be deployed in the memory of a terminal device. The terminal device includes a memory and a storage disk. The storage disk is used to cache data blocks stored in a remote device. The data cache device includes:

[0154] The processing unit 1401 is used to determine, from the N cached data blocks of the storage disk according to the metadata linked list of the storage disk, M first cached data blocks with continuous storage address sequence, wherein the metadata linked list records the storage address sequence of the N cached data blocks, and the metadata linked list also records the data size and storage address of each cached data block in the N cached data blocks, and the sum of the data sizes corresponding to the M first cached data blocks is greater than or equal to the data size of the data to be cached, wherein M and N are positive integers, and N≥M;

[0155] The cache unit 1402 is used to write the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk.

[0156] In the data cache device 1400 of the embodiment of the present application, the memory can determine the M first cached data blocks with continuous storage address sequence from the N cached data blocks of the storage disk according to the metadata linked list corresponding to the storage disk, and then write the data to be cached into the storage space corresponding to the storage address of the M first cached data blocks. In this way, since the storage address sequence of the M first cached data blocks is continuous, the memory can write the data to be cached continuously into the storage disk, avoiding low read and write efficiency caused by splitting the data to be written into different locations. In this way, the read and write performance of the storage disk is better, which helps to reduce the delay when the user device obtains data.

[0157] Moreover, since the metadata linked list records the storage address sequence of the N cached data blocks, the memory can directly determine the M first cached data blocks from the N cached data blocks more flexibly according to the storage address sequence recorded in the metadata linked list, rather than being limited to continuing to write data at the storage address written last time, thereby enabling more flexible selection of cache addresses for caching data. For example, when selecting the M first cached data blocks, the memory can skip data with higher popularity or data that is defined as not eligible for elimination, thus enabling more flexible determination of data to be eliminated, making data elimination in the storage disk more reasonable.

[0158] The data cache device 1400 of this embodiment can be deployed in the memory cache controller 12. The processing unit 1401 of this embodiment can be understood as a part of the storage disk control module 122 of the above embodiment; the cache unit 1402 can be understood as a part of the storage disk control module 122 of the above embodiment.

[0159] In some embodiments, the metadata linked list also records the heat parameters of each cached data block; the processing unit 1401 is specifically used for: the memory determines M first cached data blocks that meet the heat requirements of cold data from N stored data blocks according to the metadata linked list. In this way, each first cached data block meets the heat requirements corresponding to the cold data, which can avoid eliminating hot data when writing to the cached data, making the eliminated data more reasonable, and can improve the hit rate of hot data. It can also avoid the need for the memory to frequently obtain hot data from the source database of the remote device, so that it can provide data to the user more quickly according to the user's needs, which helps to reduce the delay when the user device obtains data.

[0160] In some embodiments, the heat requirement of cold data is: the heat parameter of each first cached data block is within the first heat parameter range corresponding to the cold data. In this way, the heat of each of the M first cached data blocks is relatively low, which effectively avoids the elimination of hot data and helps to improve the hit rate of cached data in the storage disk.

[0161] In some embodiments, the heat requirement of cold data is: the average heat parameter of the M first cached data blocks is within the heat parameter mean range corresponding to the cold data. This ensures that the average heat of the M first cached data blocks is low, which can avoid the elimination of hot data and enable the memory to determine the M first cached data blocks more quickly, thereby helping to reduce the delay when the user device obtains data.

[0162] In some embodiments, the heat requirement of cold data is: the M first cached data blocks are cached data blocks of the first cached data block group in multiple cached data block groups, each cached data block group includes one or more cached data blocks with consecutive storage addresses, and the sum of the data sizes of one or more cached data blocks in each cached data block group is greater than or equal to the data size of the data to be cached; the average value of the heat parameters of the M first cached data blocks is lower than the average value of the heat parameters of each second cached data block group except the first cached data block group in multiple cached data block groups, and the average value of the heat parameters of each second cached data block group is the average value of the heat parameters of one or more cached data blocks in the second cached data block group. In this way, the M first cached data blocks with relatively low heat can be determined, and the elimination of hot data can be effectively avoided, which helps to improve the hit rate of cached data in the storage disk.

[0163] In some embodiments, the metadata linked list also records the priority of each cached data block, and the priority of each first cached data block meets the priority requirement. In this way, by manually setting the priority conditions and the priority of the cached data blocks, some cached data blocks can be set to be retained, avoiding the elimination of some cached data blocks that cannot be eliminated.

[0164] In some embodiments, the processing unit 1401 is further used to: determine, from the M first cached data blocks, a second cached data block whose heat parameter is within the second heat parameter range corresponding to the relatively hot data; the cache unit 1402 is further used to write the second cached data block to a preset storage location of the memory. In this way, the second cached data block with relatively high heat among the M first cached data blocks can be written to the preset address storage location, which can avoid the need for the memory to obtain the data again from the source database when providing the second cached data block to the user device, thereby reducing the delay when the user device obtains data.

[0165] In some implementations, the metadata linked list also records the heat parameter and priority of each cached data block in the N cached data blocks, and the comprehensive parameters of the M first cached data blocks meet the preset comprehensive parameter requirements, and the comprehensive parameters are obtained based on the heat parameter and priority of the first cached data block. In this way, the user can determine the priority of the cached data block according to actual needs, so that the memory will use the comprehensive parameters determined according to the heat parameter and priority as the basis for screening the first cached data block, and the first cached data block screened out for replacement is determined after comprehensively considering the data heat and data priority of each cached data block, so that the first cached data block to be eliminated can be more reasonable and more in line with actual needs.

[0166] In some embodiments, the comprehensive parameter requirement is that the comprehensive parameter of each first cached data block is within a preset comprehensive parameter range. This enables the comprehensive parameter of each first cached data block in the M first cached data blocks to be optimal, so that the M first cached data blocks obtained in this way better meet actual needs.

[0167] In some embodiments, the comprehensive parameter requirement is that the average value of the heat parameters of the M first cached data blocks is within a preset comprehensive parameter average value range. This ensures that the average value of the M first cached data blocks is optimal, which can ensure that the obtained M first cached data blocks meet actual needs and enable the memory to determine the M first cached data blocks more quickly, thereby helping to reduce the delay when the user device obtains data.

[0168] In some embodiments, the comprehensive parameter requirements are as follows: the M first cached data blocks are cached data blocks of the first data block group to be screened among the multiple data block groups to be screened, each data block group to be screened includes one or more cached data blocks, and the sum of the data sizes of the one or more cached data blocks in each data block group to be screened is greater than or equal to the data size of the data to be cached; the average value of the comprehensive parameters of the M first cached data blocks in the first data block group to be screened is better than the average value of the comprehensive parameters of each second data block group to be screened except the first data block group to be screened among the multiple data block groups to be screened, and the average value of the comprehensive parameters of each second cached data block group is the average value of the comprehensive parameters of one or more cached data blocks in the second data block group to be screened. In this way, the M first cached data blocks with relatively better comprehensive parameters can be determined.

[0169] In some embodiments, the sum of the data sizes corresponding to the M-1 first cached data blocks with sequentially continuous storage addresses is smaller than the data size of the data to be cached. In this way, the sum of the data sizes of the M first cached data blocks can be more closely matched with the data size of the data to be cached. This avoids the situation where the data size of the first cached data block is too much larger than the data size of the data to be cached, thereby erasing the excessive cached data when writing the data to be cached into the storage space corresponding to the storage address of the first cached packet.

[0170] In some embodiments, the N cached data blocks are N consecutive cached data blocks after the data block last written to the storage disk according to the storage address sequence, and the sum of the data sizes of the N cached data blocks is less than the data size threshold. In this way, the memory searches for the first cached data block to be eliminated in a storage interval after the last write position according to the storage address sequence. Instead of traversing all storage locations of the storage disk to find the first cached data block to be eliminated, the first cached data block to be eliminated can be found more quickly, thereby improving the efficiency of cache data writing.

[0171] It should be noted that the explanation and technical effects of the data caching method of the above embodiment are also applicable to the data caching device 1400 of this embodiment, and will not be described again here to avoid redundancy.

[0172] The present application provides a computer-readable storage medium, which stores computer instructions. The computer instructions instruct the terminal device to execute the data caching method of the above-mentioned embodiments, or the computer instructions instruct the terminal device to deploy the data caching device provided by the above-mentioned embodiments.

[0173] The present application provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. The computer instructions instruct a terminal device to execute the data caching method of any of the above embodiments, or the computer instructions instruct a terminal device to deploy a data caching device of any of the above embodiments.

[0174] An embodiment of the present application provides a chip, which may include a programmable logic circuit and / or program instructions, and is used to implement the data caching method of any of the above embodiments when the chip is running.

[0175] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0176] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0177] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) is integrated in the processor.

[0178] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0179] It should also be understood that the first, second, third, fourth and various numerical numbers involved in this document are only distinctions made for the convenience of description and are not intended to limit the scope of the present application.

[0180] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0181] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0182] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0183] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0184] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0185] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0187] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0188] The steps in the data caching method in the embodiment of the present application can be adjusted in order, merged and deleted according to actual needs.

[0189] The modules in the data cache device in the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0190] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data caching method, characterized in that: Applied to a terminal device, the terminal device includes a memory and a storage disk, the storage disk is used to cache data blocks stored in a remote device, the method includes: The memory determines, from the N cached data blocks of the storage disk, M first cached data blocks with continuous storage address sequence according to the metadata linked list of the storage disk, wherein the metadata linked list records the storage address sequence of the N cached data blocks, and the metadata linked list also records the data size and storage address of each cached data block in the N cached data blocks, and the sum of the data sizes corresponding to the M first cached data blocks is greater than or equal to the data size of the data to be cached, wherein M and N are positive integers, and N≥M; The memory writes the data to be cached into a storage space corresponding to the storage address of the first cached data block in the storage disk.

2. The method according to claim 1, characterized in that The metadata linked list also records the heat parameter of each cached data block; the memory determines, according to the metadata linked list of the storage disk, M first cached data blocks with consecutive storage addresses from the N cached data blocks of the storage disk, including: The memory determines, according to the metadata linked list, the M first cached data blocks that meet the heat requirement of the cold data from the N cached data blocks.

3. The method according to claim 2, characterized in that The heat requirement of the cold data is: the heat parameter of each of the first cached data blocks is within the first heat parameter range corresponding to the cold data, or the average heat parameter of the M first cached data blocks is within the heat parameter mean range corresponding to the cold data.

4. The method according to claim 2, characterized in that: The heat requirement of the cold data is: the M first cached data blocks are cached data blocks of a first cached data block group among multiple cached data block groups, each cached data block group includes one or more cached data blocks with consecutive storage addresses, and the sum of the data sizes of the one or more cached data blocks in each cached data block group is greater than or equal to the data size of the data to be cached; The average value of the heat parameters of the M first cached data blocks is lower than the average value of the heat parameters of each second cached data block group except the first cached data block group among the multiple cached data block groups, and the average value of the heat parameter of each second cached data block group is the average value of the heat parameters of one or more cached data blocks in the second cached data block group.

5. The method according to any one of claims 1 to 4, characterized in that: The metadata linked list also records the priority of each cached data block, and the priority of each of the first cached data blocks meets the priority requirement.

6. The method according to claim 5, characterized in that Before the memory writes the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk, the method further includes: The memory determines, from the M first cached data blocks, a second cached data block whose heat parameter is within a second heat parameter range corresponding to the relatively hot data; The memory writes the second cached data block into a preset storage location of the memory.

7. A data cache device, characterized in that: The data caching device is used to be deployed in the memory of the terminal device, the terminal device includes the memory and a storage disk, the storage disk is used to cache the data blocks stored in the remote device, and the data caching device includes: a processing unit, configured to determine, according to a metadata linked list of the storage disk, M first cached data blocks having a continuous storage address sequence from the N cached data blocks of the storage disk, wherein the metadata linked list records the storage address sequence of the N cached data blocks, and the metadata linked list also records the data size and storage address of each cached data block in the N cached data blocks, and the sum of the data sizes corresponding to the M first cached data blocks is greater than or equal to the data size of the data to be cached, wherein M and N are positive integers, and N≥M; The cache unit is used to write the data to be cached into the storage space corresponding to the storage address of the first cached data block in the storage disk.

8. The data cache device according to claim 7, characterized in that: The metadata linked list also records the heat parameter of each cached data block; the processing unit is specifically used for: The memory determines, according to the metadata linked list, the M first cached data blocks that meet the heat requirement of the cold data from the N cached data blocks.

9. The data cache device according to claim 8, characterized in that: The heat requirement of the cold data is: the heat parameter of each of the first cached data blocks is within the first heat parameter range corresponding to the cold data, or the average heat parameter of the M first cached data blocks is within the heat parameter mean range corresponding to the cold data.

10. The data cache device according to claim 8, characterized in that: The heat requirement of the cold data is: the M first cached data blocks are cached data blocks of a first cached data block group among multiple cached data block groups, each cached data block group includes one or more cached data blocks with consecutive storage addresses, and the sum of the data sizes of the one or more cached data blocks in each cached data block group is greater than or equal to the data size of the data to be cached; The average value of the heat parameters of the M first cached data blocks is lower than the average value of the heat parameters of each second cached data block group except the first cached data block group among the multiple cached data block groups, and the average value of the heat parameter of each second cached data block group is the average value of the heat parameters of one or more cached data blocks in the second cached data block group.

11. The data cache device according to any one of claims 7 to 10, characterized in that: The metadata linked list also records the priority of each cached data block, and the priority of each of the first cached data blocks meets the priority requirement.

12. The data cache device according to claim 11, characterized in that: The processing unit is further configured to: determine, from the M first cached data blocks, a second cached data block whose heat parameter is within a second heat parameter range corresponding to the relatively hot data; The cache unit is further configured to write the second cached data block into a preset storage location of the memory.

13. A terminal device, characterized in that: It includes a memory and a storage disk, the storage disk is used to cache data blocks stored in a remote device, the memory is used to implement the data caching method described in any one of claims 1-6, or the memory is deployed with the data caching device described in any one of claims 7-12.

14. A computer program product, characterized in that The computer program product includes computer instructions, which instruct the terminal device to execute the data caching method described in any one of claims 1-6, or the computer instructions instruct the terminal device to deploy the data caching device described in any one of claims 7-12.

Images