Cache management method and apparatus

Abstract

The present application belongs to the technical field of computers. Disclosed are a cache management method and apparatus. The method is applied to a cache of a computer system, and the computer system further comprises a processor in communication with the cache. The method comprises: grouping memory access requests from a processor; applying different allocate policies to two groups of memory access requests; comparing performance costs consumed by processing the two groups of memory access requests after the corresponding allocate policies are applied to the two groups of memory access requests; and on this basis, determining a target allocate policy for the remaining groups of memory access requests other than the two groups of memory access requests. By means of the method, a better allocate policy can be determined for the remaining groups of memory access requests, and the better allocate policy can achieve caching of data that is highly likely to be accessed again by the processor in the future, thereby improving subsequent cache hit rates of memory access requests in the remaining groups, that is, improving the performance of the processor.

Description

Cache management method and device

[0001] The present application claims priority to the Chinese patent application No. 202411324632.3, filed on September 20, 2024, and entitled "Cache management method and device", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of computer, in particular to a cache management method and device. BACKGROUND

[0003] In the field of high-performance computing, the processor usually requires high bandwidth when accessing data, which makes the access bandwidth of the traditional memory (such as double data rate synchronous dynamic random access memory (DDR)) unable to match the demand of the processor for bandwidth.

[0004] Currently, the general practice is to use the technical solution of DDR+cache to meet the demand of the processor for high access bandwidth and large storage capacity of the memory. In this case, in order to improve the performance of the processor, it is necessary to improve the hit rate of the cache, and therefore, effective management of the data cached in the cache becomes a problem to be solved. SUMMARY

[0005] The present application provides a cache management method and device, which can determine a more optimal allocation strategy for the access request initiated by the processor, and the more optimal allocation strategy can realize caching of data that is more likely to be accessed by the processor in the future, thereby improving the cache hit rate of the subsequent access request in the remaining group, that is, improving the performance of the processor.

[0006] The technical solution provided by the present application is as follows:

[0007] In a first aspect, the present application provides a cache management method, which is applied to a cache of a computer system, the computer system further comprising a processor in communication with the cache, and a size of a cacheline in the cache is greater than a size of a data block requested to be accessed by a memory access request initiated by the processor. The method comprises: dividing the memory access request from the processor into a first group, a second group, and a remaining group other than the first group and the second group; storing a first data set in the cache by using a first allocate strategy, and storing a second data set in the cache by using a second allocate strategy, the first data set comprising data requested to be accessed by a cache-miss memory access request in the first group, and the second data set comprising data requested to be accessed by a cache-miss memory access request in the second group; determining a target allocate strategy from the first allocate strategy and the second allocate strategy according to sizes of the first data set and the second data set; and storing a remaining data set in the cache by using the target allocate strategy, the remaining data set comprising data requested to be accessed by a cache-miss memory access request in the remaining group.

[0008] The method provided by the present application divides the memory access request from the processor into groups, applies different allocate strategies to two groups of the memory access requests, and compares performance costs of processing the two groups of the memory access requests after the two groups of the memory access requests are processed by using the corresponding allocate strategies, and determines a target allocate strategy for the memory access request in the remaining group based on the comparison. By the method, a better allocate strategy can be determined for the memory access request in the remaining group, and the better allocate strategy can cache data that is more likely to be accessed by the processor in the future, thereby improving cache hit rates of subsequent memory access requests in the remaining group, i.e., improving performance of the processor.

[0009] It should be noted that the method provided by the present application is applicable to a large-capacity cache, or is understood to be applicable to a cache whose management granularity (i.e., a size of a cacheline) is greater (or much greater) than a size of a data block requested to be accessed by a memory access request initiated by the processor. In one example, when the computer system is implemented as a computing device, the large-capacity cache can be implemented by a dynamic random access memory (DRAM) in the computing device. In another example, when the computer system is implemented as a computing device cluster, the large-capacity cache can be implemented by a DDR of a computing device in the computing device cluster. In yet another example, the large-capacity cache has a capacity greater than 1 gigabyte (GB). Thus, a size of a cacheline of the large-capacity cache is much greater than a size of a data block requested to be accessed by a memory access request initiated by the processor.

[0010] In a possible design, the computer system further includes a memory in communication with the cache, and the determining the target allocate strategy from the first allocate strategy and the second allocate strategy according to the size of the first data set and the second data set includes: calculating a first cost, a second cost, a third cost, and a fourth cost; comparing a first difference value and a second difference value to obtain a first comparison result, wherein the first difference value is a difference between the first cost and the second cost, and the second difference value is a difference between the third cost and the fourth cost; and determining the target allocate strategy from the first allocate strategy and the second allocate strategy according to the first comparison result. The first cost is a cost consumed by the processor in processing the memory access requests in the first group, and the second cost is a cost consumed in obtaining the first data set from the memory. The third cost is a cost consumed by the processor in processing the memory access requests in the second group, and the fourth cost is a cost consumed in obtaining the second data set from the memory.

[0011] In this possible design, the greater the first difference value, the greater the amount of data requested by the memory access requests in the first group that are cache hits, and since the amount of data requested by each memory access request is generally a preset size, the greater the first difference value, the greater the number of memory access requests in the first group that are cache hits, that is, the higher the cache hit rate of the memory access requests in the first group. Conversely, the smaller the first difference value, the smaller the amount of data requested by the memory access requests in the first group that are cache hits, and thus the smaller the number of memory access requests in the first group that are cache hits, that is, the lower the cache hit rate of the memory access requests in the first group. Similarly, the greater the second difference value, the higher the cache hit rate of the memory access requests in the second group, and conversely, the smaller the second difference value, the lower the cache hit rate of the memory access requests in the second group.

[0012] Therefore, when the first comparison result indicates that the first difference value is greater than the second difference value, it means that the cache hit rate benefit (corresponding to the first difference value) brought by caching the memory access requests in the first group by using the first allocate strategy is higher than the cache hit rate benefit (corresponding to the second difference value) brought by caching the memory access requests in the second group by using the second allocate strategy, and thus the cache management apparatus determines the first allocate strategy as the target allocate strategy. Conversely, when the first comparison result indicates that the first difference value is less than the second difference value, it means that the cache hit rate benefit (corresponding to the second difference value) brought by caching the memory access requests in the second group by using the second allocate strategy is higher than the cache hit rate benefit (corresponding to the first difference value) brought by caching the memory access requests in the first group by using the first allocate strategy, and thus the cache management apparatus determines the second allocate strategy as the target allocate strategy. That is, the present application determines a more optimal allocate strategy for the remaining groups from the first allocate strategy and the second allocate strategy through the possible design.

[0013] In another possible design, the first allocate strategy indicates storing all data in the first data set in the cache, and the second allocate strategy indicates not storing all data in the second data set in the cache. The possible design provides two exemplary allocate strategies.

[0014] In yet another possible design, for a first memory access request from the processor and a cache miss, before storing the data accessed by the first memory access request in the cache according to the first allocate strategy or the second allocate strategy, the method further includes: determining that the first memory access request does not carry a cache flag, the cache flag being used to indicate whether to store the data accessed by the first memory access request in the cache.

[0015] In yet another possible design, the method further includes: when the first memory access request carries the cache flag, determining whether to store the data accessed by the first memory access request in the cache according to an indication of the cache flag.

[0016] Through the above two possible designs, the cache flag can be used to indicate that the data identified as hot data by the processor is cached in the cache, and the data identified as cold data by the processor does not need to be cached in the cache, and these data are not affected by the allocate strategy. In this way, not only the cache hit rate of the processor can be improved, but also the cold data can be removed from the cache in time, so as to release the space of the cache for other data that can be accessed by the processor.

[0017] In yet another possible design, the method further includes: dividing the memory access requests from the processor into a third group, a fourth group, and a remaining group other than the third group and the fourth group; updating, according to a frequency of the memory access requests in the third group hitting the cacheline, the re-reference prediction values (RRPV) of the cacheline using the first replacement algorithm, and updating, according to a frequency of the memory access requests in the fourth group hitting the cacheline, the RRPV of the cacheline using the second replacement algorithm, where the RRPV is used to select the cacheline of the data to be replaced in the cache when the cache is full and the memory access request from the processor is cache miss; comparing sizes of a third data set and a fourth data set to obtain a second comparison result, where the third data set includes data accessed by the memory access requests in the third group that are cache miss, and the fourth data set includes data accessed by the memory access requests in the fourth group that are cache miss; determining a target replacement algorithm from the first replacement algorithm and the second replacement algorithm according to the second comparison result; and updating, according to a frequency of the memory access requests in the remaining group hitting the cacheline, the RRPV of the cacheline using the target replacement algorithm.

[0018] With this possible design, the memory access requests from the processor can be grouped, and different replacement algorithms can be applied to two groups of the memory access requests, and sizes of data that are not cache hit in the two groups of the memory access requests after the two groups of the memory access requests are applied with the corresponding replacement algorithms are compared, so as to obtain advantages and disadvantages of cache hit conditions of the two groups of the memory access requests, and a target replacement algorithm is determined for the memory access requests in the remaining group other than the two groups of the memory access requests. With this method, a better replacement algorithm can be determined for the memory access requests in the remaining group, and the better replacement algorithm can keep the data frequently accessed by the processor in the cache, so that the cache hit rate of the memory access requests in the remaining group can be improved, and thus the processing performance of the processor can be improved.

[0019] In yet another possible design, the first replacement algorithm is a static re-reference interval prediction (SRRIP) algorithm, and the second replacement algorithm is a bimodal RRIP (BRRIP) algorithm. This possible design provides two exemplary replacement algorithms.

[0020] In yet another possible design, when the RRPV value of a first cacheline in the cache is updated according to a frequency of hits of the first cacheline, the first cacheline is any cacheline in the cache, and the method further includes: determining that the first cacheline is hit once in a case that a memory access request from the processor hits data stored in the first location of the first cacheline and the number of hits exceeds a threshold.

[0021] As the scheme of the present application is applicable to a scenario where the management granularity of the cache, i.e., the size of a cacheline, is greater than the size of a memory access data block requested by a memory access request, in this case, a memory access request hits a cacheline, and actually hits part of the data in the cacheline. Thus, the hit of the cacheline by a memory access request does not indicate that the heat of all the data in the cacheline is high, or that the heat of the cacheline is high, which can be referred to as a false hit of the cacheline. Based on this possible design, the present application can determine that a cacheline is hit truly only when the number of hits of a fixed size of data in the cacheline exceeds a threshold, so as to more accurately evaluate the heat of the cacheline.

[0022] In yet another possible design, the first data is data requested to be accessed by a memory access request from the processor and not cached each time the first data is used to replace the second data, and the second data is data stored in a cacheline of the cache to be replaced, and the method further includes: if there is no second cacheline with the RRPV value being the first value in the cache, determining to keep the RRPV value of a third cacheline with the RRPV value being the second value in the cache unchanged, and updating the RRPV value of the third cacheline according to a frequency of hits of the third cacheline when the number of times of keeping the RRPV value of the third cacheline unchanged exceeds a threshold. The first value represents the longest access interval defined in the RRPV, and the second value represents the shortest access interval defined in the RRPV.

[0023] Through this possible design, the duration of the data cached by the cacheline with the RRPV value being the second value (i.e., the highest heat) in the cache can be prolonged, i.e., the hot data can stay in the cache for a longer time. In this way, not only can the performance overhead generated by the computer system when loading and / or removing data in the cacheline be reduced, but also the cache hit rate of the memory access request initiated by the processor can be improved, thereby improving the processing performance of the processor.

[0024] In yet another possible design, for a second memory access request from the processor and a cache miss, after the RRPV value of the cacheline in the cache selects a fourth cacheline for the second memory access request to replace data in the fourth cacheline, the method further includes: when the fourth cacheline is not marked with the maintenance flag, replacing the data in the fourth cacheline with the data accessed by the second memory access request. The maintenance flag is used to indicate that the data in the cacheline is to be maintained.

[0025] In yet another possible design, the method further includes: when the fourth cacheline is marked with the maintenance flag, selecting a fifth cacheline for the second memory access request to replace data in the fifth cacheline based on the RRPV value of the cacheline in the cache.

[0026] With the above two possible designs, the indication from the processor can be responded to by responding to the maintenance flag, so that the hot data identified by the processor can be kept in the cacheline, and the cacheline including the hot data can be prevented from being determined as the cacheline to replace data based on the RRPV value of the cacheline, so that the cache hit rate of the processor in the future period of time can be improved.

[0027] In yet another possible design, the method further includes: when a third memory access request from the processor carries the maintenance flag and the data accessed by the third memory access request is stored in the fourth cacheline, marking the fourth cacheline with the maintenance flag; and when a fourth memory access request from the processor carries the release flag and the fourth memory access request hits the data stored in the fourth cacheline, releasing the maintenance flag marked for the fourth cacheline, or removing the data stored in the fourth cacheline. With this possible design, the present application can set the maintenance flag for the cacheline including the hot data identified by the processor in the cache in response to the indication from the processor, and release / remove the maintenance flag for the cacheline in response to the indication from the processor.

[0028] In a second aspect, the present application provides a cache management apparatus. The cache management apparatus is configured to perform any of the methods provided in the first aspect. The cache management apparatus can be divided into functional modules according to any of the methods provided in the first aspect. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. For example, the cache management apparatus can be divided into a grouping unit, a cache unit, and a determining unit, etc. according to functions. The possible technical solutions and beneficial effects of each functional module are described above in the first aspect and any of the possible design manners in the first aspect, which will not be repeated here.

[0029] In a third aspect, the present application provides a cache. The computer system comprising the cache further comprises a processor in communication with the cache. The cache is configured to cache data accessed by the processor. The size of a cacheline in the cache is greater than the size of a data block requested to be accessed by one access request from the processor. The cache is managed by the method provided in the first aspect and any of the possible design manners in the first aspect.

[0030] In a fourth aspect, the present application provides a computer system. The computer system comprises a processor and a cache. The cache is configured to cache data accessed by the processor. The size of a cacheline in the cache is greater than the size of a data block requested to be accessed by one access request from the processor. The cache is managed by the method provided in the first aspect and any of the possible design manners in the first aspect.

[0031] In a fifth aspect, the present application provides a chip. The chip comprises a processor and a cache. The cache is configured to cache data accessed by the processor. The size of a cacheline in the cache is greater than the size of a data block requested to be accessed by one access request from the processor. The cache is managed by the method provided in the first aspect and any of the possible design manners in the first aspect.

[0032] It can be understood that any of the cache management apparatus, the cache, the computer system, or the chip provided above can be applied to the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method, which will not be repeated here.

[0033] In the present application, the names of the cache management apparatus and the like do not constitute a limitation on the devices or functional modules themselves. In actual implementation, these devices or functional modules can appear with other names. As long as the functions of each device or functional module are similar to those of the present application, they are within the protection scope of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0034] Fig. 1 is a schematic diagram of an implementation environment of the method provided by the embodiments of the present application;

[0035] Fig. 2 is a schematic diagram of the structure of a computer system provided by the embodiments of the present application;

[0036] Fig. 3 is a schematic diagram of the flow of a cache management method provided by the embodiments of the present application;

[0037] Fig. 4 is a schematic diagram of the process of determining a target allocate policy in a first allocate policy and a second allocate policy provided by the embodiments of the present application;

[0038] Fig. 5 is a schematic diagram of the flow of another cache management method provided by the embodiments of the present application;

[0039] Fig. 6 is a schematic diagram of the flow of yet another cache management method provided by the embodiments of the present application;

[0040] Fig. 7 is a schematic diagram of determining that a first cacheline is hit provided by the embodiments of the present application;

[0041] Fig. 8 is a schematic diagram of the flow of a method of updating the RRPV value of a cacheline provided by the embodiments of the present application;

[0042] Fig. 9 is a schematic diagram of an exemplary process of updating the RRPV value of a cacheline provided by the embodiments of the present application;

[0043] Fig. 10 is a schematic diagram of the flow of yet another cache management method provided by the embodiments of the present application;

[0044] Fig. 11 is a schematic diagram of the structure of a cache management apparatus provided by the embodiments of the present application. DETAILED DESCRIPTION

[0045] To make the objects, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0046] To facilitate understanding, the technologies and background involved in the embodiments of the present application will be explained first.

[0047] 1), cache

[0048] Cache is a random access memory (RAM) with relatively fast access speed, and the data access speed is close to that of a processor, such as a central processing unit (CPU). Cache is generally present between a processor and a memory (or main memory). In the conventional technology, cache is generally implemented by a static RAM (SRAM) with small capacity but fast access speed, and is used to cache data frequently accessed by the processor in a recent period of time. Among them, "caching" means "temporary storage".

[0049] Cache manages the data cached by itself in units of cachelines. For example, cache loads data from the memory in units of cachelines. For another example, cache removes / deletes data in units of cachelines.

[0050] In a computer system configured with cache, when the processor initiates a memory access request, it first determines whether the data requested by the memory access request exists in the cache. When the data accessed by the processor exists in the cache, it is called cache hit. At this time, the processor accesses data from the cache with high access speed, without the need to access data from the memory or other storage with slow access speed, which can improve the performance of the processor. Among them, memory access includes access and storage. When the data accessed by the processor does not exist in the cache, it is called cache miss. At this time, the processor needs to access data from the memory or other storage.

[0051] Therefore, keeping the data cached in the cache as the data recently accessed or frequently accessed by the processor can improve the hit rate of the memory access request initiated by the processor in the cache, thereby improving the speed of the processor accessing data and further improving the processing performance of the processor.

[0052] 2), re-reference interval prediction (RRIP), BRRIP, and SRRIP

[0053] Since the capacity of the cache is limited, when the used capacity of the cache approaches or reaches the maximum capacity of the cache, if new data needs to be cached in the cache at this time, one or more data in the cachelines which have cached data need to be removed so that the new data can be cached in the cacheline which is emptied after the data is removed. In other words, when the used capacity of the cache approaches or reaches the maximum capacity of the cache, if new data needs to be cached in the cache at this time, the data originally cached in the cacheline which has cached data needs to be replaced by the new data. In this case, the cache needs to decide which cacheline to remove / replace the data in.

[0054] Generally, the cache uses a cache replacement algorithm / strategy to decide which cacheline to remove / replace the data in. Among them, the cache replacement algorithm / strategy mainly functions to exploit the temporal locality of processor access behavior, and tries to keep the data that is most likely to be frequently accessed by the processor in the future in the cache, so as to improve the cache hit rate, reduce system delay, and thus improve the performance of the processor.

[0055] RRIP is an extension of the cache replacement strategy, which is an improved strategy based on the not recently used (NRU) strategy. RRIP introduces an RRPV value, and can predict whether the cacheline in the cache will be accessed / hit in the near future by updating the RRPV value.

[0056] In RRIP, each cacheline in the cache is assigned an RRPV value. The RRPV value of the cacheline is used to represent the interval of the cacheline from the next access / hit. In the related art, the larger the RRPV value of the cacheline, the farther the interval of the cacheline from the next access / hit. Conversely, the smaller the RRPV value of the cacheline, the closer the interval of the cacheline from the next access / hit. In other words, the RRPV value of the cacheline can indicate the hotness of the data cached in the cacheline. In this way, when the cache needs to replace the data in the cacheline with new data, the cacheline where the low-heat data is located can be determined by checking the RRPV values of all cachelines, and the low-heat data cached in the cacheline is replaced by the new data. In this way, the high-heat data can be kept in the cache.

[0057] BRRIP and SRRIP are further extensions of RRIP. In BRRIP and SRRIP, the update of the RRPV value of a cacheline is generally related to the access pattern of the data, i.e. BRRIP and SRRIP adjust the RRPV value of a cacheline according to the frequency and / or interval at which the cacheline is accessed.

[0058] The difference between BRRIP and SRRIP is that after a cacheline whose data needs to be replaced is selected according to the RRPV value and new data is cached in the cacheline, the latest RRPV value set for the cacheline is different. For example, SRRIP fixes the largest RRPV value for the cacheline of the cached new data, while BRRIP sets the second largest RRPV value for the cacheline of the cached new data with a certain probability. In an example, when the RRPV value is recorded by 2 bits, the largest RRPV value is 3 and the second largest RRPV value is 2.

[0059] 3), allocate policy / mechanism

[0060] The allocate policy is used to decide which data of the data loaded by the processor from the memory or other storage can be cached in the cache.

[0061] It should be understood that when the memory access request initiated by the processor is a cache miss, the data needs to be loaded from the memory or other storage. At this time, whether the newly loaded data of the processor can be cached in the cache can be decided based on the allocate policy. For example, the allocate policy indicates that the data accessed by the memory access request initiated by the processor in period 1 is cached in the cache. In this way, when the processor subsequently accesses the data again, the data can be directly read from the cache, without repeatedly loading from the memory or other storage.

[0062] Embodiments of the present application provide a cache management method. The method is applied to a cache, and a size of a cache line in the cache is greater than a size of a data block requested to be accessed by a memory access request. The method groups the memory access requests from a processor, applies different allocate strategies to two groups of the memory access requests, and compares performance costs of processing the two groups of the memory access requests after the two groups of the memory access requests apply the corresponding allocate strategies. Based on the comparison, a target allocate strategy is determined for a memory access request in a group other than the two groups. By the method, a better allocate strategy can be determined for the memory access request in the group other than the two groups, and the better allocate strategy can cache data that is likely to be accessed by the processor in the future, thereby improving a cache hit rate of the memory access request in the group other than the two groups, i.e., improving performance of the processor.

[0063] Optionally, the method provided by the embodiments of the present application supports a scenario in which a capacity of the cache is greater (e.g., much greater) than a capacity of a conventional cache. In one example, the capacity of the cache is greater than 1 GB.

[0064] Because the cache has a large capacity, if a conventional small-capacity cache management granularity is still used to manage the large-capacity cache, i.e., a small-capacity cache line corresponding to the conventional cache is still used to manage the large-capacity cache, the large-capacity cache needs to maintain a large number of address and memory address corresponding relationships of the small-capacity cache lines, which increases address management overhead of the cache line. Therefore, in the embodiments of the present application, when the capacity of the cache is large, the capacity of the cache line used to manage the cache is also large, and the capacity of the cache line is usually greater than the size of the data block requested to be accessed by the memory access request.

[0065] For example, in a high bandwidth memory (HBM) cache, a data block accessible by a memory access request has a size of 1 kilobyte (KB), and a cacheline in the cache has a capacity of 4 KB. Compared with cache management based on a 1 KB size cacheline in a conventional cache, when a 4 KB size cacheline is used to manage the cache, it is equivalent to that 4 1 KB size cachelines share an address of a 4 KB size cacheline. That is, the correspondence between the addresses of the 4 cachelines and the memory addresses originally needs to be stored, and by this approach, only the correspondence between the address of one cacheline and the memory address needs to be stored, so that the address management overhead of the cacheline can be greatly reduced. The HBM cache is a representative application scenario of a cache using DRAM (referred to as a DRAM cache).

[0066] Referring to FIG. 1, FIG. 1 shows a schematic diagram of an implementation environment of the method provided by the embodiments of the present application. As shown in FIG. 1, the implementation environment can be implemented as a computer system, which includes a processor, a cache, and a memory. For example, the computer system can be implemented as a server, such as various application servers, or as a general-purpose computer, a notebook computer, a tablet computer, a mobile phone, a wearable device, a vehicle-mounted terminal device, and the like, which includes a processor, a cache, and a memory, but is not limited thereto.

[0067] The processor can include a general processor and / or a special hardware chip. The general processor can include a CPU, a microprocessor, or a graphics processing unit (GPU). The CPU is a single-CPU, for example, or a multi-CPU, for example. The special hardware chip is a hardware module with high performance. The special hardware chip includes at least one of a digital signal processor (DSP), a data processing unit (DPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, a neural processing unit (NPU), a tensor processing unit (TPU), an artificial intelligent (AI) chip, a network processer (NP), or an intelligent driving processor. The processor can also be an integrated circuit chip with a signal processing capability.

[0068] The cache is used to cache data frequently accessed by the processor. The process of determining whether the data requested by the processor initiated memory access request is cached in the cache can refer to related technologies, which will not be described here.

[0069] Optionally, the cache can be located inside the processor or outside the processor, which is not limited. In one example, when the processor is a CPU, the cache described in the embodiments of the present application can be a level 3 cache (L3 cache) of the CPU. It should be understood that the cache of the processor is usually divided into three levels with different sizes, namely, a level 1 cache (L1 cache), a level 2 cache (L2 cache), and an L3 cache. The data access speed of the L1 cache > the data access speed of the L2 cache > the data access speed of the L3 cache. Generally, the L1 cache and the L2 cache are exclusive caches of the CPU core, and the L3 cache can be shared by multiple CPU cores.

[0070] In one example, when the computer system is implemented as a computing device, the cache can be implemented by a DRAM in the computing device. In another example, when the computer system is implemented as a cluster of computing devices, the cache can be implemented by a DDR of a computing device in the cluster of computing devices.

[0071] The memory is also referred to as a main memory or a main storage, and is used to temporarily store operation data in the processor, and exchange data with an external storage such as a hard disk.

[0072] For example, the memory is used to store a computer program, and the computer program includes an operating system and executable code (i.e., program instructions). The memory is, for example, a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), a flash memory, or other types of static storage devices that can store static information and instructions, or is, for example, a static random access memory (SRAM), a DRAM, a synchronous DRAM (SDRAM), a DDR, a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synchlink DRAM (SLDRAM), or other types of dynamic storage devices that can store information and instructions, or is, for example, a read-only optical disc or other optical disc storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or is any other medium that can be used to carry or store desired executable code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0073] The above-mentioned processor, cache, and memory can be connected through a bus. The bus is any type of communication bus used to interconnect the internal devices (e.g., processor, cache, memory, etc.) of the computer system. For example, a system bus. Alternatively, the above-mentioned devices inside the computer system can also be connected to each other in communication through other connection modes other than the bus, for example, the above-mentioned devices inside the computer system are interconnected through internal logic interfaces.

[0074] Optionally, the computer system can further include a communication interface, which uses a transceiving module such as but not limited to a transceiver to realize communication with other devices or communication networks. For example, the communication interface can be any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card, and the like devices having network access functions. The communication interface includes a receiving unit for receiving data / packets, and a sending unit for sending data / packets.

[0075] Referring to FIG. 2, FIG. 2 exemplarily shows a structural schematic diagram of a computer system provided by an embodiment of the present application.

[0076] As shown in FIG. 2, the computer system 200 includes CPUs 1-4. Any of the CPUs 1-4 can be a multi-core processor or a single-core processor, which is not limited. Inside each CPU, an L3 cache tag is arranged, which is used to record the correspondence between cache addresses and memory addresses. Thus, when initiating a memory access request, the CPU compares the memory address carried in the memory access request with the corresponding relationship recorded in the L3 cache tag, to determine whether the data requested by the memory access request exists in the L3 cache, so as to decide whether to read the data from the L3 cache or from the memory.

[0077] The computer system 200 further includes an L3 cache shown in FIG. 2, which is used to cache data frequently accessed by the CPUs 1-4.

[0078] The computer system 200 further includes a DDR as a memory.

[0079] In addition, the computer system 200 further includes a root agent (hydra home agent, HHA) for accessing the memory DDR. Here, the HHA is a module for processing cache coherency protocols in a multi-chip system, which is not limited by the embodiments of the present application.

[0080] The computer system 200 further includes a system bus for communication between the CPUs, the cache, the HHA, and the DDR. In some examples, the communication between the devices in the computer system 200 can also be realized through a network chip (network on chip, NOC). The NOC is a network-based communication interconnection module, which is located inside a system on chip (SoC) and can be used to connect CPU cores, memory units, and various functional modules, so as to realize high-speed, efficient, low-delay, and low-power data communication.

[0081] It should be understood that the above is an exemplary description of the implementation environment of the cache management method provided by the embodiments of the present application, and does not constitute a limitation on the implementation environment of the method. Those skilled in the art can know that the implementation environment can be adjusted according to application requirements as the business needs change, and the embodiments of the present application do not enumerate them one by one.

[0082] The embodiments of the present application also provide a cache management device, which implements effective management of the cache in the computer system by executing the cache management method provided by the embodiments of the present application, so as to improve the cache hit rate of the memory access request initiated by the processor in the computer system, thereby improving the processing performance of the processor. In one example, the cache management device is implemented by a hardware logic circuit that manages the cache.

[0083] The cache management method provided by the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0084] Referring to FIG. 3, FIG. 3 shows a flowchart of a cache management method provided by the embodiments of the present application. Optionally, the method is applied to the cache in the computer system shown in FIG. 1 or FIG. 2, and is executed by the cache management device. As shown in FIG. 3, the method includes the following steps 101-105.

[0085] Step 101, the cache management device divides the memory access requests from the processor in the computer system into a first group, a second group, and a remaining group other than the first group and the second group.

[0086] Among them, the memory access request from the processor includes a read request for requesting to read data and a write request for requesting to write data.

[0087] Optionally, the cache management device can divide the memory access requests initiated by the processor into at least three groups approximately uniformly based on the memory address carried in the memory access request. Here, the memory address refers to the address of the data requested to be accessed in the memory access request. The memory address generally also includes the storage address of the data stored in the memory address in the cache.

[0088] In one example, the cache management device divides the memory access requests into at least three groups approximately uniformly according to different values of at least two preset bits in the memory address carried by the memory access request. Among them, the first group can be any one of the at least three groups, the second group is any one of the at least three groups other than the first group, and the third group is the remaining group of the at least three groups other than the first group and the second group.

[0089] In this example, embodiments of the present application do not limit the positions of the at least two preset bits in the memory address. For example, taking the at least two preset bits as the first two bits in the memory address as an example, since the values of the two bits include 0, 1, 2, and 3, a total of 4 values, based on the first two bits in the memory address carried by the memory access request, the cache management device can approximately uniformly divide the memory access request into 4 groups. For another example, taking the at least two preset bits as the last three bits in the memory address, since the values of the three bits include 0, 1, 2, 3, 4, 5, 6, and 7, a total of 8 values, based on the last three bits in the memory address carried by the memory access request, the cache management device can approximately uniformly divide the memory access request into 8 groups. In this way, when the cache management device receives a memory access request, it checks the value of the preset bits in the memory address carried by the memory access request, and determines the group to which the memory access request belongs according to the value of the preset bits, so as to process the memory access request according to the processing mode corresponding to the group.

[0090] In another example, the cache management device approximately uniformly divides the memory access request into three groups according to the preset value of the at least two preset bits in the memory address carried by the memory access request. In this example, embodiments of the present application still do not limit the positions of the preset bits in the memory address, and do not limit the specific values of the preset values. Specifically, the cache management device determines the memory access request whose value of the preset bits in the memory address carried by the memory access request is a first value as the memory access request of a first group, determines the memory access request whose value of the preset bits in the memory address carried by the memory access request is a second value as the memory access request of a second group, and determines the memory access request whose value of the preset bits in the memory address carried by the memory access request is a third value as the memory access request of the remaining group.

[0091] The first value and the second value are different values of the preset bit, and the third value is a value of the preset bit other than the first value and the second value. For example, taking the preset bit as the first two bits of the memory address, the first value as “01”, the second value as “10”, and the third value as “00” or “11” as an example, the cache management device determines, after receiving each memory access request, the value of the first two bits of the memory address carried by the memory access request. When the value of the first two bits is “01”, it is determined that the memory access request belongs to the first group. When the value of the first two bits is “10”, it is determined that the memory access request belongs to the second group. When the value of the first two bits is “00” or “11”, it is determined that the memory access request belongs to the third group. For another example, taking the preset bit as the last three bits of the memory address, the first value as “000”, the second value as “001”, and the third value as “010”, “011”, “100”, “110”, “101”, or “111” as an example, the cache management device determines, after receiving each memory access request, the value of the last three bits of the memory address carried by the memory access request. When the value of the last three bits is “000”, it is determined that the memory access request belongs to the first group. When the value of the last three bits is “001”, it is determined that the memory access request belongs to the second group. When the value of the last three bits is “010”, “011”, “100”, “110”, “101”, or “111”, it is determined that the memory access request belongs to the third group.

[0092] Optionally, the cache management device can determine the group of the memory access request according to the group identifier carried in the memory access request initiated by the processor. In this embodiment, the processor needs to add a group identifier to the memory access request when initiating the memory access request. In this way, the cache management device can determine the group of the memory access request by checking the group identifier of the memory access request after receiving each memory access request.

[0093] Step 102, the cache management device stores a first data set in the cache according to a first allocate strategy, and the first data set includes data accessed by a cache-miss memory access request in the first group.

[0094] In one example, the first allocate strategy indicates that all data in the first data set is stored (or cached) in the cache.

[0095] In this case, for each memory access request in the first group that is cache-miss, when the processor reads the data accessed by the memory access request (i.e., the data in the first data set) from the memory and loads the read data into the cache, the cache management device determines to cache all the read data, i.e., to keep all the read data in the cache.

[0096] Step 103, the cache management device stores the second data set in the cache according to the second allocate strategy, the second data set comprising data accessed by the cache-missed access requests in the second group.

[0097] In an example embodiment, the second allocate strategy indicates that all data in the second data set is not stored (i.e., cached) in the cache.

[0098] In an example embodiment, the second allocate strategy indicates that data accessed by the cache-missed access requests in the second group is selected to be cached in the cache with a preset probability. The preset probability can be 90%, or 95%, or 98%, etc., without limitation.

[0099] In this case, for each access request in the second group that is cache-missed, when the processor reads data (i.e., data in the second data set) accessed by the each access request from the memory and loads the read data into the cache, the cache management device determines whether to cache the data according to the preset probability.

[0100] Taking the example of steps 101-104 in each clock cycle when the preset probability is 90%, and the cache management device, for 10 access requests in the second group that are cache-missed in a clock cycle, when the processor reads data accessed by the 10 access requests from the memory and loads the read data into the cache, the cache management device determines to remove data accessed by 9 access requests from the cache (i.e., not cache) and determines to cache data accessed by the remaining one access request in the cache.

[0101] Step 104, the cache management device determines a target allocate strategy from the first allocate strategy and the second allocate strategy according to the sizes of the first data set and the second data set.

[0102] In an example embodiment, the target allocate strategy is a strategy for storing (i.e., caching) the remaining data set in the cache, the remaining data set comprising data accessed by cache-missed access requests in the remaining group.

[0103] In an example embodiment, the process of determining the target allocate strategy from the first allocate strategy and the second allocate strategy according to the sizes of the first data set and the second data set by the cache management device can be implemented by steps 1041-1043 shown in FIG. 4.

[0104] Step 1041, the cache management device counts the first cost, the second cost, the third cost, and the fourth cost.

[0105] The first cost is a cost consumed by the processor in processing the memory access requests in the first group, and the second cost is a cost consumed in obtaining the first data set from the memory. The third cost is a cost consumed by the processor in processing the memory access requests in the second group, and the fourth cost is a cost consumed in obtaining the second data set from the memory.

[0106] The cost refers to a performance cost consumed by the computer system in processing the memory access requests, such as a time or power consumed in processing the memory access requests, without limitation. It should be understood that a size of data accessed by the memory access requests affects the time or power consumed in processing the memory access requests. Generally, the larger the size of data accessed by the memory access requests, the longer the time consumed in processing the memory access requests, and the higher the power consumed. Therefore, the size of the cost can also be directly indicated by the size of data.

[0107] In this way, in an example, the first cost can be indicated by a size of data accessed by the memory access requests in the first group, and the second cost can be indicated by a size of the first data set obtained from the memory, i.e., the second cost is indicated by a size of data accessed by the cache-miss memory access requests in the first group. Similarly, the third cost can be indicated by a size of data accessed by the memory access requests in the second group, and the fourth cost can be indicated by a size of the second data set obtained from the memory, i.e., the fourth cost is indicated by a size of data accessed by the cache-miss memory access requests in the second group.

[0108] In step 1042, the cache management apparatus compares the first difference value and the second difference value to obtain a first comparison result. The first difference value is a difference between the first cost and the second cost, and the second difference value is a difference between the third cost and the fourth cost.

[0109] It can be seen that the first difference value is a difference between a cost consumed by the computer system in processing the memory access requests in the first group (i.e., the first cost) and a cost consumed in reading data accessed by the cache-miss memory access requests in the first group from the memory (i.e., the second cost). Thus, the larger the first difference value, the larger the amount of data accessed by the cache-hit memory access requests in the first group. Since the amount of data accessed by each memory access request is generally a preset size, the larger the first difference value, the more the number of cache-hit memory access requests in the first group, i.e., the higher the cache-hit rate of the memory access requests in the first group. Conversely, the smaller the first difference value, the smaller the amount of data accessed by the cache-hit memory access requests in the first group, i.e., the fewer the number of cache-hit memory access requests in the first group, i.e., the lower the cache-hit rate of the memory access requests in the first group.

[0110] Similarly, the second difference value is the difference between the cost (i.e., the third cost) of the computer system processing the access requests in the second group and the cost (i.e., the fourth cost) of reading the requested access data from the memory for the access requests in the second group that miss the cache. Thus, the greater the second difference value, the greater the amount of data requested by the access requests in the second group that hit the cache, and thus the greater the number of access requests in the second group that hit the cache, i.e., the higher the cache hit rate of the access requests in the second group. Conversely, the smaller the second difference value, the smaller the amount of data requested by the access requests in the second group that hit the cache, and thus the smaller the number of access requests in the second group that hit the cache, i.e., the lower the cache hit rate of the access requests in the second group.

[0111] At step 1043, the cache management device determines the target allocate policy from the first allocate policy and the second allocate policy according to the first comparison result.

[0112] Since the greater the first difference value, the higher the cache hit rate of the access requests in the first group, and the greater the second difference value, the higher the cache hit rate of the access requests in the second group, when the first comparison result indicates that the first difference value is greater than the second difference value, it means that the cache hit rate gain (corresponding to the first difference value) caused by caching the access requests in the first group using the first allocate policy is higher than the cache hit rate gain (corresponding to the second difference value) caused by caching the access requests in the second group using the second allocate policy, and thus the cache management device determines the first allocate policy as the target allocate policy.

[0113] Conversely, when the first comparison result indicates that the first difference value is smaller than the second difference value, it means that the cache hit rate gain (corresponding to the second difference value) caused by caching the access requests in the second group using the second allocate policy is higher than the cache hit rate gain (corresponding to the first difference value) caused by caching the access requests in the first group using the first allocate policy, and thus the cache management device determines the second allocate policy as the target allocate policy.

[0114] When the first comparison result indicates that the first difference value is equal to the second difference value, the cache management device can randomly determine the first allocate policy or the second allocate policy as the target allocate policy, which is not limited.

[0115] At step 105, the cache management device stores the remaining data set in the cache according to the target allocate policy, the remaining data set including data requested by the access requests in the remaining group that miss the cache.

[0116] Since the target allocate policy is the better one of the first allocate policy and the second allocate policy, the cache management device caches the memory access requests in the rest groups by using the target allocate policy, so that the cache hit rate of the memory access requests in the rest groups can be improved, and thus the processor performance of the computer system can be improved.

[0117] In the embodiments of the present application, the cache management device periodically performs steps 101-104, and when the cache management device determines the target allocate policy by performing steps 101-104 on the memory access requests initiated by the processor in a certain execution period (e.g., the first period), the cache management device stores the rest data sets in the cache by using the target allocate policy determined in the first execution period in the next execution period of the first period. For example, the period length of the execution period is the period length of the clock cycle of the computer system, or the period length of the execution period includes the period length of multiple clock cycles of the computer system, which is not limited.

[0118] In this way, by the method described in steps 101-105, the memory access requests from the processor in each execution period can be grouped, and different allocate policies can be applied to two groups of memory access requests, and the performance cost of processing the two groups of memory access requests after applying the corresponding allocate policies is compared, so as to determine the target allocate policy used to cache data in the next execution period for the memory access requests in the rest groups. By this method, a better allocate policy can be determined for the memory access requests in the rest groups, and the better allocate policy can realize caching of data that is more likely to be accessed by the processor in the future, so that the subsequent cache hit rate of the memory access requests in the rest groups can be improved, and thus the processing performance of the processor can be improved.

[0119] In some embodiments, for a memory access request from the processor and a cache miss, such as the first memory access request, before the cache management device caches the data accessed by the first memory access request according to an allocate policy, such as the first allocate policy or the second allocate policy, such as before the cache management device caches the data accessed by the first memory access request according to the first allocate policy in step 102, or before the cache management device caches the data accessed by the first memory access request according to the second allocate policy in step 103, or before the cache management device caches the data accessed by the first memory access request according to the target allocate policy in step 105, the cache management device further performs the method described in FIG. 5, so that the data determined as hot data by the processor can be cached in the cache, and the data determined as cold data by the processor does not need to be cached in the cache, and these data are not affected by the allocate policy. In this way, not only the cache hit rate of the processor can be improved, but also the cold data can be removed from the cache in time, so as to release the space of the cache for other data that can be accessed by the processor.

[0120] Referring to FIG. 5, FIG. 5 shows a flowchart of another cache management method provided by the embodiments of the present application. Optionally, the method is applied to the cache in the computer system shown in FIG. 1 or FIG. 2, and is performed by the cache management device. As shown in FIG. 5, the method comprises the following steps 201-203.

[0121] In step 201, the cache management device determines whether the first memory access request carries a cache flag.

[0122] The cache flag is used to indicate whether the data accessed by the first memory access request needs to be cached in the cache. When the cache flag indicates that the data accessed by the first memory access request needs to be cached in the cache, it means that the processor has determined that the data accessed by the first memory access request is hot data. Conversely, when the cache flag indicates that the data accessed by the first memory access request does not need to be cached in the cache, it means that the processor has determined that the data accessed by the first memory access request is cold data.

[0123] For example, the cache flag can be represented by "hot", in which case it means that the processor has determined that the data accessed by the first memory access request is hot data. For another example, the cache flag can be represented by "cold", in which case it means that the processor has determined that the data accessed by the first memory access request is cold data.

[0124] It can be understood that the cache flag is added in the first memory access request by the processor. For example, the processor adds the cache flag in the first memory access request according to the indication of the upper application.

[0125] Thus, when the cache management device determines that the first access request does not carry the cache flag, step 202 is performed.

[0126] When the cache management device determines that the first access request carries the cache flag, step 203 is performed.

[0127] Step 202: When the cache management device determines that the first access request does not carry the cache flag, the data accessed by the first access request is cached according to the allocate policy.

[0128] For example, when the cache management device determines that the first access request does not carry the cache flag, the data accessed by the first access request is cached according to the first allocate policy or the second allocate policy.

[0129] Step 203: When the cache management device determines that the first access request carries the cache flag, it is determined whether the data accessed by the first access request is cached in the cache according to the indication of the cache flag.

[0130] For example, when the cache flag is "hot", the cache management device determines to cache the data accessed by the first access request in the cache.

[0131] For another example, when the cache flag is "cold", the cache management device determines not to cache the data accessed by the first access request in the cache.

[0132] Through the method described in steps 201-203, the data that is determined to be hot data by the processor can be cached in the cache, and the data that is determined to be cold data by the processor does not need to be cached in the cache, and these data are not affected by the allocate policy. Thus, not only the cache hit rate of the processor can be improved, but also the cold data can be removed from the cache in time, so as to release the space of the cache for other data that can be accessed by the processor.

[0133] In some other embodiments, when the cache is full or will be full, and new data needs to be cached in the cache, the present embodiment further provides another cache management method, through which the cacheline with the lowest hot degree (the lowest probability of being accessed) can be selected as the cacheline that needs to be replaced in the full or full cache.

[0134] Referring to FIG. 6, FIG. 6 shows a flowchart of another cache management method provided by the present embodiment, which is applied to the cache in the computer system shown in FIG. 1 or FIG. 2 and is performed by the cache management device. As shown in FIG. 6, the method includes steps 301-306.

[0135] Step 301, the cache management device divides the memory access requests from the processor in the computer system into a third group, a fourth group, and a remaining group other than the third group and the fourth group.

[0136] Here, the detailed description of the cache management device dividing the memory access requests from the processor in the computer system into a third group, a fourth group, and a remaining group other than the third group and the fourth group can refer to the description of the cache management device dividing the memory access requests from the processor in the computer system into a first group, a second group, and a remaining group other than the first group and the second group in step 101, and will not be repeated here.

[0137] In one example, the third group and the first group described above are the same group, the fourth group and the second group described above are the same group, and the remaining group and the remaining group described above are the same group.

[0138] Step 302, the cache management device updates the RRPV value of the cacheline according to the frequency of the memory access request in the third group hitting the cacheline.

[0139] The first replacement algorithm is used to update the RRPV value of the cacheline in the cache, and the RRPV value of the cacheline is used to select the cacheline of the data to be replaced in the cache when the cache is full or will be full and the memory access request from the processor is cache miss. The detailed description of the RRPV value can refer to the description in the term, and will not be repeated here.

[0140] In one example, the first replacement algorithm is SRRIP algorithm.

[0141] In RRP (including SRRIP or BRRIP), the RRPV value of the cacheline needs to be updated according to the frequency of the cacheline being hit. Therefore, the cache management device needs to determine the hit rate of the cacheline in the cache. However, when the capacity of the cacheline in the cache is greater than or much greater than the size of the data block requested by a memory access request, the cacheline is hit by a memory access request, and in fact, the data hit is part of the data in the cacheline. Therefore, when the cacheline is hit by a memory access request, it does not mean that the heat of all the data in the cacheline is high, or that the heat of the cacheline is high, which can be called false hit at this time. Based on this, the cache management device needs to determine whether the cacheline is hit (real hit), and update the RRPV value of the cacheline according to the frequency of the cacheline being hit.

[0142] For any cacheline in the cache, such as the first cacheline, in the process that the cache management device updates the RRPV value of the first cacheline according to the frequency of the first cacheline being hit (i.e. "true hit"), in the case that the memory access request from the processor hits the data stored in the first position of the first cacheline and the number of hits exceeds a threshold value, the cache management device determines that the first cacheline is hit (i.e. "true hit") once. Thus, the cache management device updates the RRPV value of the first cacheline based on the frequency of the first cacheline being true hit. By determining whether the cacheline in the cache is true hit, and further updating the RRPV value of the cacheline according to the frequency of true hit, the data in the cacheline can be replaced more accurately when the data in the cacheline is replaced, so as to improve the cache hit rate of the cache.

[0143] For example, in combination with FIG. 7, the cache management device is configured to indicate the offset of the fixed size data in the fixed position of the first cacheline by a register, for example, the offset of a 64 byte (B) data in the first cacheline. The cache management device counts the number of times that the 64B data is hit by the memory access request from the processor. When the counted number of times exceeds a threshold value, the cache management device determines that the first cacheline is hit once. In this way, the cache management device can count the frequency of the first cacheline being true hit by the memory access request in the first group, and update the RRPV value of the first cacheline based thereon. For example, when the cache management device counts that the number of times that the first cacheline is true hit by the memory access request in the first group reaches the threshold value within a preset time period, the cache management device updates the RRPV value of the first cacheline based on the replacement algorithm. The value of the threshold value is not limited in the embodiments of the present application.

[0144] In the process that the cache management device updates the RRPV value of the cacheline based on the first replacement algorithm (or the second replacement algorithm described below), since the granularity of the cacheline is large, the cache management device needs to maintain the cacheline where the data with high access frequency is located as long as possible. In this way, not only the performance overhead generated by the computer system when loading and / or removing the data in the cacheline can be reduced, but also the cache hit rate of the memory access request initiated by the processor can be improved, so as to improve the processing performance of the processor.

[0145] To maintain the cacheline where the data with high access frequency is located as long as possible, reference is made to FIG. 8, which shows a flowchart of a method for updating the RRPV value of a cacheline according to an embodiment of the present application. Optionally, the method is applied to the cache in the computer system shown in FIG. 1 or FIG. 2 and is executed by the cache management device. As shown in FIG. 8, the method comprises the following steps S1-S3 each time the first data is used to replace the second data. The first data is the data requested by the access request from the processor and with cache miss, and the second data is the data stored in the cacheline to be replaced in the cache.

[0146] In step S1, the cache management device determines whether there is a second cacheline with the RRPV value being the first value in the cache.

[0147] The second cacheline is any cacheline in the cache.

[0148] The RRPV value being the first value means the longest access interval defined in the RRIP algorithm. When the RRPV value of the cacheline is the first value, it means that the cacheline has the lowest heat, i.e., the data in the cacheline has a very low probability of being accessed by the processor in the future.

[0149] It should be understood that each time the cache management device uses the first data requested by the access request from the processor and with cache miss to replace the second data in the cacheline selected as the data to be replaced according to the RRPV value of each cacheline, the cache management device updates the RRPV value of the cacheline in the cache according to the RRIP algorithm.

[0150] In the embodiment of the present application, to maintain the cacheline where the data with high access frequency is located as long as possible, each time the cache management device uses the first data to replace the second data, the cache management device determines whether there is a second cacheline with the RRPV value being the first value in the cache at this time, i.e., whether there is a second cacheline with the lowest heat in the cache at this time. For example, the cache management device traverses the RRPV value of the cacheline in the cache to determine whether there is a second cacheline with the RRPV value being the first value in the cache.

[0151] When the cache management device determines that there is no second cacheline with the RRPV value being the first value in the cache, i.e., there is no second cacheline with the lowest heat in the cache, the cache management device executes step S2.

[0152] When the cache management device determines that the second cacheline with the lowest hotness exists in the cache, the cache management device normally updates the RRPV values of the cachelines in the cache according to the RRIP algorithm.

[0153] If the second cacheline with the first RRPV value does not exist in the cache, the cache management device determines to keep the RRPV value of the third cacheline with the second RRPV value unchanged in the cache.

[0154] The second RRPV value represents the shortest access interval defined in the RRIP algorithm. When the RRPV value of a cacheline is the second RRPV value, it means that the hotness of the cacheline is the highest, i.e., the probability of the data of the cacheline being accessed by the processor in the future is high.

[0155] When the cache management device determines that the second cacheline with the lowest hotness does not exist in the cache in step S1, the cache management device determines to keep the RRPV value of the third cacheline with the second RRPV value unchanged in the cache, i.e., to keep the RRPV value of the third cacheline with the highest hotness unchanged.

[0156] In other words, after the cache management device replaces the second data with the first data, if the cache management device determines that the second cacheline with the first RRPV value (i.e., the lowest hotness) does not exist in the cache at this time, when the RRPV values of the cachelines in the cache are updated according to the RRIP algorithm after the first data replaces the second data, the cache management device keeps the RRPV value of the third cacheline with the second RRPV value (i.e., the highest hotness) unchanged.

[0157] When the cache management device determines that the number of times of keeping the RRPV value of the third cacheline unchanged exceeds the threshold value, the cache management device updates the RRPV value of the third cacheline according to the frequency of the third cacheline being hit.

[0158] That is, when the cache management device determines that the number of times of keeping the RRPV value of the third cacheline unchanged exceeds the threshold value based on S1-S2, the cache management device starts to normally update the RRPV value of the third cacheline according to the RRIP algorithm.

[0159] In one example, referring to FIG. 9, the cache management apparatus can set a counter for each cacheline. In this way, after each time new data (e.g., the first data requested to be accessed by a memory access request from the processor and which is not cached) replaces the originally cached data (e.g., the second data) in a cacheline, the cache management apparatus traverses the RRPV values of each cacheline to determine whether there is a cacheline with an RRPV value of 3 (indicating the lowest degree of hotness). When the cache management apparatus determines that there is a cacheline with an RRPV value of 3, the cache management apparatus normally updates the RRPV value of the cacheline according to the RRIP algorithm. When the cache management apparatus determines that there is no cacheline with an RRPV value of 3, the cache management apparatus determines whether the value of the counter of the third cacheline with an RRPV value of 0 (indicating the highest degree of hotness) exceeds a threshold value. The value of the third counter represents the number of times the RRPV value of the third cacheline remains unchanged when data in the cache is replaced. When the cache management apparatus determines that the value of the third counter exceeds the threshold value, the cache management apparatus normally updates the RRPV value of the third cacheline according to the RRIP algorithm. When the cache management apparatus determines that the value of the third counter does not exceed the threshold value, the cache management apparatus increments the counter of the third cacheline so that the cache management apparatus can determine again whether the value of the third counter exceeds the threshold value in the future.

[0160] Through the processes of S1-S3, the duration of the data cached in the cacheline with the RRPV value of the second value (i.e., the highest degree of hotness) in the cache can be prolonged, i.e., the hot data can stay in the cache for a longer time. In this way, not only can the performance overhead generated by the computer system when loading and / or removing data in the cacheline be reduced, but also the cache hit rate of the memory access request initiated by the processor can be improved, thereby improving the processing performance of the processor.

[0161] In step 303, the cache management apparatus updates the RRPV value of the cacheline according to a second replacement algorithm based on the frequency of the memory access request in the fourth group hitting the cacheline.

[0162] The second replacement algorithm is another replacement algorithm different from the first replacement algorithm, and the second replacement algorithm has the same function and purpose as the first replacement algorithm, which will not be described herein again.

[0163] In one example, the second replacement algorithm is the BRRIP algorithm.

[0164] Specifically, the cache management apparatus updates the RRPV value of the cacheline according to the frequency of the cacheline hit by the memory access request in the fourth group using the second replacement algorithm. For details, reference can be made to the description of the cache management apparatus updating the RRPV value of the cacheline according to the frequency of the cacheline hit by the memory access request in the third group using the first replacement algorithm in step 302, which will not be repeated here.

[0165] In step 304, the cache management apparatus compares the sizes of the third data set and the fourth data set to obtain a second comparison result. The third data set includes data requested to be accessed by the memory access request with cache miss in the third group, and the fourth data set includes data requested to be accessed by the memory access request with cache miss in the fourth group.

[0166] It should be understood that the larger the third data set is, the higher the cache miss rate of the memory access request in the third group is, i.e., the lower the cache hit rate is, when the RRPV value of the cacheline is updated based on the frequency of the cacheline hit by the memory access request in the third group using the first replacement algorithm. On the contrary, the smaller the third data set is, the lower the cache miss rate of the memory access request in the third group is, i.e., the higher the cache hit rate is, when the RRPV value of the cacheline is updated based on the frequency of the cacheline hit by the memory access request in the third group using the first replacement algorithm.

[0167] Similarly, the larger the fourth data set is, the higher the cache miss rate of the memory access request in the fourth group is, i.e., the lower the cache hit rate is, when the RRPV value of the cacheline is updated based on the frequency of the cacheline hit by the memory access request in the fourth group using the second replacement algorithm. On the contrary, the smaller the fourth data set is, the lower the cache miss rate of the memory access request in the fourth group is, i.e., the higher the cache hit rate is, when the RRPV value of the cacheline is updated based on the frequency of the cacheline hit by the memory access request in the fourth group using the second replacement algorithm.

[0168] In step 305, the cache management apparatus determines a target replacement algorithm from the first replacement algorithm and the second replacement algorithm according to the second comparison result.

[0169] The target replacement algorithm is used to update the RRPV value of the cacheline in the cache based on the frequency of the cacheline hit by the memory access request in the remaining group.

[0170] In the case that the number of access requests included in the third group and the fourth group is equivalent, if the second comparison result indicates that the size of the third data set is greater than the size of the fourth data set, it means that the cache miss rate of the access requests in the third group using the first replacement algorithm is higher than the cache miss rate of the access requests in the fourth group using the second replacement algorithm, that is, the cache hit rate of the access requests in the third group using the first replacement algorithm is lower than the cache hit rate of the access requests in the fourth group using the second replacement algorithm. In other words, in the case that the second comparison result indicates that the size of the third data set is greater than the size of the fourth data set, it means that the cache hit benefit brought by the application of the first replacement algorithm is lower than the cache hit benefit brought by the application of the fourth replacement algorithm. At this time, the cache management device determines the second replacement algorithm as the target replacement algorithm.

[0171] On the contrary, in the case that the number of access requests included in the third group and the fourth group is equivalent, if the second comparison result indicates that the size of the third data set is less than the size of the fourth data set, it means that the cache miss rate of the access requests in the third group using the first replacement algorithm is lower than the cache miss rate of the access requests in the fourth group using the second replacement algorithm, that is, the cache hit rate of the access requests in the third group using the first replacement algorithm is higher than the cache hit rate of the access requests in the fourth group using the second replacement algorithm. In other words, in the case that the second comparison result indicates that the size of the third data set is less than the size of the fourth data set, it means that the cache hit benefit brought by the application of the first replacement algorithm is higher than the cache hit benefit brought by the application of the fourth replacement algorithm. At this time, the cache management device determines the first replacement algorithm as the target replacement algorithm.

[0172] Step 306, the cache management device updates the RRPV value of the cacheline according to the frequency of the access requests in the remaining group hitting the cacheline using the target replacement algorithm.

[0173] The detailed description of the cache management device updating the RRPV value of the cacheline according to the frequency of the access requests in the remaining group hitting the cacheline using the target replacement algorithm can refer to the description of the cache management device updating the RRPV value of the cacheline according to the frequency of the access requests in the third group hitting the cacheline using the first replacement algorithm in step 302, and will not be repeated here.

[0174] In the embodiments of the present application, the cache management apparatus periodically performs steps 301-305, and when the cache management apparatus determines the target replacement algorithm for the memory access requests initiated by the processor in a certain execution period (e.g., the first period) according to steps 301-305, the cache management apparatus updates the RRPV value of the cacheline according to the frequency of the memory access request hitting the cacheline in the remaining group in the next execution period of the first period by using the target replacement algorithm determined in the first period. For example, the execution period has a period length of a clock cycle of the computer system, or the execution period has a period length including a plurality of clock cycles of the computer system, which is not limited.

[0175] In this way, the method described in steps 301-306 can group the memory access requests from the processor in each execution period, and apply different replacement algorithms to two groups of memory access requests, and compare the sizes of the data that are not cached in the two groups of memory access requests after the two groups of memory access requests apply the corresponding replacement algorithms, so as to obtain the advantages and disadvantages of the cache hits of the two groups of memory access requests, and determine the target replacement algorithm for the memory access requests in the remaining group in the next execution period. By this method, a better replacement algorithm can be determined for the memory access requests in the remaining group, and the better replacement algorithm can keep the data frequently accessed by the processor in the cache, so that the subsequent cache hit rate of the memory access requests in the remaining group can be improved, thereby improving the processing performance of the processor.

[0176] In some other embodiments, in response to the indication of the processor, the hot data identified by the processor is kept in the cacheline, and the cacheline including the hot data is prevented from being determined as the cacheline to be replaced based on the RRPV value of the cacheline. In response to the indication of the processor, the present embodiments can set a maintenance flag for the cacheline including the hot data identified by the processor in the cache, and remove the maintenance flag of the cacheline in response to the indication of the processor. The maintenance flag of the cacheline is used to indicate that the data in the cacheline is maintained unchanged.

[0177] In this case, referring to FIG. 10, FIG. 10 shows a flowchart of another cache management method provided by the embodiments of the present application. Optionally, the method is applied to the cache in the computer system shown in FIG. 1 or FIG. 2, and is performed by the cache management apparatus. As shown in FIG. 10, the method includes steps 401-403.

[0178] Step 401, for the second access request from the processor and cache miss, after the cache management device selects the cacheline (denoted as the fourth cacheline) to be replaced data for the data requested by the second access request based on the RRPV value of the cacheline in the cache, the cache management device determines whether the fourth cacheline is marked with a maintenance mark.

[0179] Optionally, for any access request from the processor received by the cache management device, such as the third access request, when the cache management device determines that the third access request carries a maintenance mark (such as a "keep" mark) and determines that the data requested by the third access request is cached in the fourth cacheline, the cache management device marks the fourth cacheline with the maintenance mark.

[0180] In the embodiments of the present application, the priority of maintaining the data in the cacheline based on the maintenance mark of the cacheline is higher than the priority of determining the cacheline to be replaced data based on the RRPV value and replacing the data in the cacheline by the cache management device. The cache management device preferentially performs the operation corresponding to the high priority.

[0181] Therefore, after the cache management device selects the fourth cacheline to be replaced data based on the RRPV value of the cacheline in the cache, it is determined first whether the fourth cacheline is marked with a maintenance mark.

[0182] When the cache management device determines that the fourth cacheline is not marked with a maintenance mark, step 402 is performed.

[0183] When the cache management device determines that the fourth cacheline is marked with a maintenance mark, step 403 is performed.

[0184] Step 402, when the fourth cacheline is not marked with a maintenance mark, the cache management device replaces the data in the fourth cacheline with the data requested by the second access request.

[0185] In this step, the fourth cacheline is not marked with a maintenance mark, and the cache device replaces the originally cached data in the fourth cacheline with the data requested by the second access using the RRPV value updated based on the replacement algorithm (such as the first replacement algorithm or the second replacement algorithm) after the fourth cacheline is selected as the cacheline to be replaced data.

[0186] Step 403, when the fourth cacheline is marked with the maintain flag, the fifth cacheline of the data to be replaced is selected based on the RRPV value of the cacheline in the cache.

[0187] In this step, the fourth cacheline is marked with the maintain flag, and the cache device determines that the data in the fourth cacheline needs to be maintained according to the indication of the maintain flag. Therefore, the cache management device reselects the cacheline of the data to be replaced for the second cache request based on the RRPV value of the cacheline in the cache, which is recorded as the fifth cacheline. Subsequently, the cache management device can execute steps 401-403 again for the fifth cacheline until it is determined that the data accessed by the second cache request can be used to replace the originally cached data in the cacheline of the data to be replaced.

[0188] In this way, based on the process described in steps 401-403, the hot data (i.e., data marked with the maintain flag) recognized by the processor in the cache can be guaranteed to remain in the cacheline, and the cacheline including the hot data can be avoided to be determined as the cacheline of the data to be replaced based on the RRPV value of the cacheline, thereby improving the cache hit rate of the cache request initiated by the processor, and further improving the processing performance of the processor. The hot data recognized by the processor in the cache can be data frequently accessed by the upper application, but is not limited thereto.

[0189] It can be understood that when the hot data recognized by the processor changes to cold data, if the cacheline in the cache that caches the data is marked with the maintain flag, the processor can carry a release flag (such as a "clear" flag) in the cache request initiated for accessing the data to remove the maintain flag for the cacheline that caches the data, or remove the data in the cacheline.

[0190] Still taking the fourth cacheline as an example, when the fourth cacheline is marked with the maintenance mark, if the cache management apparatus receives a fourth access request from the processor carrying the release mark, and the fourth access request hits the data stored in the fourth cacheline, the cache management apparatus releases the maintenance mark marked for the fourth cacheline, so that the cache management apparatus can start to decide whether to replace the data in the fourth cacheline according to the RRPV value of the fourth cacheline. Alternatively, when the cache management apparatus receives a fourth access request from the processor carrying the release mark, and the fourth access request hits the data stored in the fourth cacheline, the cache management apparatus directly removes the data stored in the fourth cacheline, so as to release the storage space of the fourth cacheline which has turned into cold data as soon as possible, to be used to cache other more valuable data.

[0191] It can be understood that the methods described in the various embodiments above can be used alone or in combination, and are not limited in this regard.

[0192] By using the methods described in the various embodiments above in combination, the hit rate of the cache in the computer system can be improved in the scenario where the capacity of the cacheline in the cache is greater (for example, much greater) than the size of the data block that can be accessed by one access request, thereby improving the processor performance of the computer system.

[0193] The above mainly describes the solutions provided by the embodiments of the present application from the perspective of methods.

[0194] To implement the above functions, referring to FIG. 11, FIG. 11 shows a structural schematic diagram of a cache management apparatus provided by an embodiment of the present application. As shown in FIG. 11, the cache management apparatus 1100 is applied to a cache of a computer system, the computer system further includes a processor in communication with the cache, and the size of the cacheline in the cache is greater than the size of the data block requested to be accessed by any access request initiated by the processor. The cache management apparatus 1100 is used to execute the cache management method described above, for example, the methods shown in FIGS. 3 to 10. The cache management apparatus 1100 can include a grouping unit 1101, a cache unit 1102, and a determination unit 1103.

[0195] The grouping unit 1101 is configured to group the access requests from the processor into a first group, a second group, and a remaining group other than the first group and the second group. The caching unit 1102 is configured to store a first data set in the cache according to a first allocate policy, and store a second data set in the cache according to a second allocate policy. The first data set includes data accessed by access requests in the first group that miss in the cache, and the second data set includes data accessed by access requests in the second group that miss in the cache. The determining unit 1103 is configured to determine a target allocate policy from the first allocate policy and the second allocate policy according to sizes of the first data set and the second data set. The caching unit 1102 is further configured to store a remaining data set in the cache according to the target allocate policy, the remaining data set including data accessed by access requests in the remaining group that miss in the cache.

[0196] For example, in combination with FIG. 3, the grouping unit 1101 can be configured to perform step 101, the caching unit 1102 can be configured to perform steps 102, 103, and 105, and the determining unit 1103 can be configured to perform step 104.

[0197] Optionally, the computer system further includes a memory in communication with the cache. The determining unit 1103 is specifically configured to: count a first cost, a second cost, a third cost, and a fourth cost; compare a first difference value and a second difference value to obtain a first comparison result, the first difference value being a difference between the first cost and the second cost, and the second difference value being a difference between the third cost and the fourth cost; and determine the target allocate policy from the first allocate policy and the second allocate policy according to the first comparison result. The first cost is a cost consumed by the processor in processing the access requests in the first group, and the second cost is a cost consumed in obtaining the first data set from the memory. The third cost is a cost consumed by the processor in processing the access requests in the second group, and the fourth cost is a cost consumed in obtaining the second data set from the memory.

[0198] For example, in combination with FIG. 4, the determining unit 1103 can be configured to perform steps 1041-1043.

[0199] Optionally, for a first access request from the processor and missing in the cache, before storing data accessed by the first access request in the cache according to the first allocate policy or the second allocate policy, the determining unit 1103 is further configured to determine that the first access request does not carry a cache flag, the cache flag being used to indicate whether to store the data accessed by the first access request in the cache.

[0200] As an example, with reference to FIG. 5, the determining unit 1103 can be configured to perform step 201.

[0201] Optionally, the determining unit 1103 is further configured to, when the first memory access request carries the cache flag, determine whether to store the data requested by the first memory access request in the cache according to the indication of the cache flag.

[0202] As an example, with reference to FIG. 5, the determining unit 1103 can be configured to perform step 203.

[0203] Optionally, the cache management apparatus 1100 further comprises an updating unit 1104. The grouping unit 1101 is further configured to group the memory access requests from the processor into a third group, a fourth group, and a remaining group other than the third group and the fourth group. The updating unit 1104 is configured to update, according to a frequency of the memory access requests in the third group hitting a cacheline, an RRPV value of the cacheline by using a first replacement algorithm, and update, according to a frequency of the memory access requests in the fourth group hitting the cacheline, the RRPV value of the cacheline by using a second replacement algorithm. The RRPV value is used to select a cacheline of data to be replaced in the cache when the cache is full and a memory access request from the processor is cache-miss. The determining unit 1103 is further configured to compare sizes of a third data set and a fourth data set to obtain a second comparison result, and determine a target replacement algorithm from the first replacement algorithm and the second replacement algorithm according to the second comparison result. The third data set comprises data requested by cache-miss memory access requests in the third group, and the fourth data set comprises data requested by cache-miss memory access requests in the fourth group. The updating unit 1104 is further configured to update, according to a frequency of the memory access requests in the remaining group hitting the cacheline, the RRPV value of the cacheline by using the target replacement algorithm.

[0204] As an example, with reference to FIG. 6, the grouping unit 1101 can be configured to perform step 301, the updating unit 1104 can be configured to perform steps 302, 303, and 306, and the determining unit 1103 can be configured to perform steps 304-305.

[0205] Optionally, when the RRPV value of the first cacheline in the cache is updated according to a frequency of the first cacheline being hit in the cache, the first cacheline is any cacheline in the cache, and the determining unit 1103 is further configured to determine that the first cacheline is hit once in a case that a memory access request from the processor hits data stored at a first position in the first cacheline and the number of hits exceeds a threshold.

[0206] Optionally, the first data is data requested to be accessed by a memory access request from the processor and cache miss, and the second data is data stored in a cacheline of the cache to be replaced, when the first data is replaced by the second data each time, the determining unit 1103 is further configured to determine to keep the RRPV value of a third cacheline with the RRPV value of the second value in the cache unchanged, if the second cacheline with the RRPV value of the first value does not exist in the cache, wherein the first value represents the longest access interval defined in the RRPV, and the second value represents the shortest access interval defined in the RRIP. The updating unit 1104 is further configured to update the RRPV value of the third cacheline according to a frequency at which the third cacheline is hit, when a number of times of keeping the RRPV value of the third cacheline unchanged exceeds a threshold.

[0207] As an example, in combination with FIG. 8, the determining unit 1103 can be configured to perform steps S1-S2, and the updating unit 1104 can be configured to perform step S3.

[0208] Optionally, for a second memory access request from the processor and cache miss, after a fourth cacheline with data to be replaced is selected for the second memory access request based on the RRPV value of the cacheline in the cache, the cache management apparatus 1100 further includes a replacing unit 1105 configured to replace data in the fourth cacheline with data requested to be accessed by the second memory access request, when the fourth cacheline is not marked with a maintenance mark. The maintenance mark is used to indicate that the data in the cacheline is maintained unchanged.

[0209] As an example, in combination with FIG. 10, the replacing unit 1105 can be configured to perform step 402.

[0210] Optionally, the determining unit 1103 is further configured to select a fifth cacheline with data to be replaced for the second memory access request based on the RRPV value of the cacheline in the cache, when the fourth cacheline is marked with the maintenance mark.

[0211] As an example, in combination with FIG. 10, the determining unit 1103 can be configured to perform step 403.

[0212] Optionally, the cache management apparatus 1100 further comprises a marking unit 1106 configured to mark a maintenance flag for a fourth cache line when the third access request from the processor carries the maintenance flag and the data requested to be accessed by the fourth access request is stored in the fourth cache line; and a deleting unit 1107 configured to remove the maintenance flag marked for the fourth cache line or remove the data stored in the fourth cache line when the fourth access request from the processor carries the removal flag and the fourth access request hits the data stored in the fourth cache line.

[0213] The above optional mode can be implemented by referring to the above-mentioned method embodiments, and details are not described herein. In addition, the above-mentioned cache management apparatus 1100 and the beneficial effects can be implemented by referring to the above-mentioned method embodiments, and details are not described herein.

[0214] Those skilled in the art should easily understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented in hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0215] It should be noted that the division of modules / cells in FIG. 11 is illustrative, and is only a logical function division. In actual implementation, another division mode can be used. For example, two or more functions can be integrated in one processing module. The integrated module can be implemented in the form of hardware or software function module.

[0216] The embodiments of the present application also provide a cache, and a computer system including the cache further includes a processor in communication with the cache, the cache is configured to cache data accessed by the processor, the size of a cache line in the cache is greater than the size of a data block requested to be accessed by one access request from the processor, and the cache is managed by the cache management method provided in the above method embodiments.

[0217] The embodiments of the present application provide a computer system, and the computer system includes a processor and a cache, the cache is configured to cache data accessed by the processor, the size of a cache line in the cache is greater than the size of a data block requested to be accessed by one access request from the processor, and the cache is managed by the cache management method provided in the above method embodiments.

[0218] For example, the computer system is the computer system shown in FIG. 1 or FIG. 2, and part or all of the functions of the cache management method provided by the embodiment of the present application are implemented by the logic circuit for managing the cache in the computer system.

[0219] It should be noted that each device in the computer system can be arranged on a chip independent of each other, or at least partially or entirely arranged on the same chip. Whether each device is arranged independently on different chips or integrated on one or more chips often depends on the needs of product design. The embodiment of the present application does not limit the specific implementation form of each device in the computer system. And the description of the flow corresponding to each of the above figures has its own emphasis, and the part not described in detail in a certain flow can be referred to the related description of other flows.

[0220] In the above embodiment, all or part of the cache management method can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the cache management method can be implemented in the form of a computer program product. The computer program product providing a program development platform includes one or more computer instructions, and when the computer program instructions are loaded and executed on a computer system, part or all of the functions of the cache management method provided by the embodiment of the present application are implemented.

[0221] In addition, the computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium stores computer program instructions providing a program development platform.

[0222] The embodiment of the present application also provides a chip, which includes a processor and a cache, the cache is used to cache data accessed by the processor, the size of the cache line in the cache is greater than the size of the data block requested to be accessed by one access request from the processor, and the cache is managed by the cache management method provided in the above method embodiment.

[0223] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing related hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0224] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0225] In the embodiments of the present application, the terms "first", "second" and "third" are only used for descriptive purposes and should not be construed or implied to indicate or imply relative importance. The term "at least one" means one or more, and the term "multiple" means at least two, unless otherwise explicitly limited.

[0226] In the present application, the term "and / or" is only used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.

[0227] It should be understood that the terms used in the description of various described examples herein are only for the purpose of describing specific examples and are not intended to be limiting. As used in the description of various described examples and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0228] It should be understood that determining B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0229] It should be understood that the term "includes" (also referred to as "includes", "including", "comprises" and / or "comprising") when used in the specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0230] It should also be understood that in various embodiments of the present application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0231] The above is only an optional embodiment of the present application and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the concept and principles of the present application shall be included in the protection scope of the present application.

[0232] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A cache management method characterized by, A cache applied to a computer system, the computer system further comprising a processor in communication with the cache, a size of a cacheline in the cache being larger than a size of a data block requested to be accessed by a memory access request; the method comprising: classifying the memory access requests from the processor into a first group, a second group, and a rest group other than the first group and the second group; storing a first data set in the cache with a first allocation strategy, the first data set comprising data requested to be accessed by memory access requests with cache misses in the first group; storing a second data set in the cache with a second allocation strategy, the second data set comprising data requested to be accessed by memory access requests with cache misses in the second group; determining a target allocation strategy from the first allocation strategy and the second allocation strategy according to sizes of the first data set and the second data set; and storing a rest data set in the cache with the target allocation strategy, the rest data set comprising data requested to be accessed by memory access requests with cache misses in the rest group.

2. The method of claim 1, wherein, The computer system further comprises a memory in communication with the cache, and the determining the target allocation strategy from the first allocation strategy and the second allocation strategy according to sizes of the first data set and the second data set comprises: counting a first cost, a second cost, a third cost, and a fourth cost; wherein the first cost is a cost consumed by the processor to process memory access requests in the first group, the second cost is a cost consumed to fetch the first data set from the memory, the third cost is a cost consumed by the processor to process memory access requests in the second group, and the fourth cost is a cost consumed to fetch the second data set from the memory; comparing a first difference value and a second difference value to obtain a first comparison result; the first difference value is a difference between the first cost and the second cost, and the second difference value is a difference between the third cost and the fourth cost; determining the target allocation strategy from the first allocation strategy and the second allocation strategy according to the first comparison result.

3. The method according to claim 1 or 2, characterized in that, The first allocation strategy indicates to store all data in the first data set in the cache, and the second allocation strategy indicates not to store all data in the second data set in the cache.

4. The method according to any one of claims 1 to 3, characterized in that, For a first memory access request from the processor and with a cache miss, before storing data requested to be accessed by the first memory access request in the cache with the first allocation strategy or the second allocation strategy, the method further comprises: determining that the first memory access request does not carry a cache flag, the cache flag being used to indicate whether to store data requested to be accessed by the first memory access request in the cache.

5. The method of claim 4, wherein, The method further comprises: When the first access request carries the cache flag, determining whether to store data requested by the first access request in the cache according to the cache flag.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: Dividing access requests from the processor into a third group, a fourth group and a remaining group other than the third group and the fourth group; According to a frequency of hitting a cacheline by an access request in the third group, updating a re-reference prediction value (RRPV) of the cacheline by using a first replacement algorithm, the RRPV being used to select a cacheline of data to be replaced in the cache when the cache is full and an access request from the processor is cache-missed; According to a frequency of hitting a cacheline by an access request in the fourth group, updating a re-reference prediction value (RRPV) of the cacheline by using a second replacement algorithm; Comparing sizes of a third data set and a fourth data set to obtain a second comparison result, wherein the third data set comprises data requested by a cache-missed access request in the third group, and the fourth data set comprises data requested by a cache-missed access request in the fourth group; According to the second comparison result, determining a target replacement algorithm from the first replacement algorithm and the second replacement algorithm; According to a frequency of hitting a cacheline by an access request in the remaining group, updating a re-reference prediction value (RRPV) of the cacheline by using the target replacement algorithm.

7. The method of claim 6, wherein, The first replacement algorithm is a static re-reference interval prediction (SRRIP) algorithm, and the second replacement algorithm is a dynamic re-reference interval prediction (BRRIP) algorithm.

8. The method according to claim 6 or 7, characterized in that, When updating a re-reference prediction value (RRPV) of a first cacheline in the cache according to a frequency of hitting the first cacheline, the first cacheline is any cacheline in the cache, and the method further comprises: In a case where an access request from the processor hits data stored in a first position in the first cacheline and the number of hits exceeds a threshold, determining that the first cacheline is hit once.

9. The method according to any one of claims 6 to 8, characterized in that, In each time of replacing second data by first data, the first data is data requested by a cache-missed access request from the processor, and the second data is data stored in a cacheline of data to be replaced in the cache, and the method further comprises: If a second cacheline with a RRPV value of a first value does not exist in the cache, determining to keep a re-reference prediction value (RRPV) of a third cacheline with a RRPV value of a second value unchanged, wherein the first value represents a longest access interval defined in a re-reference interval prediction (RRIP), and the second value represents a shortest access interval defined in the RRIP. determining that the number of times of keeping the RRPV value of the third cacheline unchanged exceeds a threshold value, updating the RRPV value of the third cacheline according to the frequency of hits of the third cacheline.

10. The method according to any one of claims 6 to 9, characterized in that, For a second memory access request from the processor and a cache miss, after selecting a fourth cacheline to be replaced with data of the second memory access request based on the RRPV values of the cachelines in the cache, the method further comprises: when the fourth cacheline is not marked with the maintenance mark, replacing the data in the fourth cacheline with the data accessed by the second memory access request; wherein the maintenance mark is used to indicate that the data in the cacheline is maintained unchanged.

11. The method of claim 10, wherein, The method further comprises: when the fourth cacheline is marked with the maintenance mark, selecting a fifth cacheline to be replaced with data of the second memory access request based on the RRPV values of the cachelines in the cache.

12. The method according to claim 10 or 11, characterized in that, The method further comprises: when a third memory access request from the processor carries the maintenance mark and the data accessed by the third memory access request is stored in the fourth cacheline, marking the maintenance mark for the fourth cacheline; when a fourth memory access request from the processor carries an unmarking mark and the fourth memory access request hits the data stored in the fourth cacheline, unmarking the maintenance mark marked for the fourth cacheline, or removing the data stored in the fourth cacheline.

13. A cache management device, characterized by, A cache for a computer system, the computer system further comprising a processor in communication with the cache, and a size of a cacheline in the cache is greater than a size of a data block requested to be accessed by a memory access request; the apparatus comprises: a grouping unit configured to group memory access requests from the processor into a first group, a second group, and a remaining group other than the first group and the second group; a caching unit configured to store a first data set in the cache by using a first allocate strategy, and store a second data set in the cache by using a second allocate strategy; wherein the first data set comprises data requested to be accessed by memory access requests in the first group that are cache misses, and the second data set comprises data requested to be accessed by memory access requests in the second group that are cache misses; a determining unit configured to determine a target allocate strategy from the first allocate strategy and the second allocate strategy according to sizes of the first data set and the second data set; the caching unit is further configured to store a remaining data set in the cache by using the target allocate strategy, the remaining data set comprising data requested to be accessed by memory access requests in the remaining group that are cache misses.

14. The apparatus of claim 13, wherein, The computer system further comprises a memory in communication with the cache, and the determining unit is specifically configured to: count a first cost, a second cost, a third cost and a fourth cost; the first cost is a cost consumed by the processor in processing the memory access requests in the first group, the second cost is a cost consumed by the memory in obtaining the first data set, the third cost is a cost consumed by the processor in processing the memory access requests in the second group, and the fourth cost is a cost consumed by the memory in obtaining the second data set; compare the first difference and the second difference to obtain a first comparison result; the first difference is a difference between the first cost and the second cost, and the second difference is a difference between the third cost and the fourth cost; determine the target allocate strategy from the first allocate strategy and the second allocate strategy according to the first comparison result.

15. The apparatus of claim 13 or 14, wherein, for a first memory access request from the processor and with a cache miss, before storing data accessed by the first memory access request in the cache according to the first allocate strategy or the second allocate strategy, the determining unit is further configured to determine that the first memory access request does not carry a cache flag, and the cache flag is used to indicate whether to store the data accessed by the first memory access request in the cache.

16. The apparatus of claim 15, wherein the determining unit is further configured to determine, when the first memory access request carries the cache flag, whether to store the data accessed by the first memory access request in the cache according to an indication of the cache flag.

17. The apparatus of any one of claims 13-16, wherein, The apparatus further comprises an updating unit, the grouping unit is further configured to group the memory access requests from the processor into a third group, a fourth group and a remaining group other than the third group and the fourth group; the updating unit is configured to update a re-reference prediction value (RRPV) of a cacheline according to a frequency of cache line hits of the memory access requests in the third group by using a first replacement algorithm, and update the RRPV of the cacheline according to a frequency of cache line hits of the memory access requests in the fourth group by using a second replacement algorithm; the RRPV is used to select a cacheline of data to be replaced in the cache when the cache is full and a memory access request from the processor is a cache miss; the determining unit is further configured to compare sizes of a third data set and a fourth data set to obtain a second comparison result, and determine a target replacement algorithm from the first replacement algorithm and the second replacement algorithm according to the second comparison result; the third data set comprises data accessed by the memory access requests with cache misses in the third group, and the fourth data set comprises data accessed by the memory access requests with cache misses in the fourth group. The updating unit is further configured to update the RRPV value of the cacheline according to a frequency of a hit of the cacheline by a memory access request in the remaining group.

18. The apparatus of claim 17, wherein, When the RRPV value of the first cacheline is updated according to a frequency of a hit of the first cacheline in the cache, The determining unit is further configured to determine that the first cacheline is hit once in a case that a memory access request from the processor hits data stored in a first position in the first cacheline and a number of hits exceeds a threshold.

19. The apparatus of claim 17 or 18, wherein, The first data is data requested to be accessed by a memory access request from the processor and cache-missed each time the first data is used to replace the second data, and the second data is data stored in a cacheline of the cache to be replaced, The determining unit is further configured to determine to keep the RRPV value of a third cacheline with a second RRPV value unchanged in a case that a second cacheline with a first RRPV value does not exist in the cache, the first RRPV value representing a longest access interval defined in a reference interval prediction RRPV, and the second RRPV value representing a shortest access interval defined in a reference interval prediction RRPV. The updating unit is further configured to update the RRPV value of the third cacheline according to a frequency of a hit of the third cacheline in a case that a number of times of keeping the RRPV value of the third cacheline unchanged exceeds a threshold.

20. The apparatus of any one of claims 17-19, wherein, For a second memory access request from the processor and cache-missed, the apparatus further comprises: The replacing unit is configured to replace data in the fourth cacheline with data requested to be accessed by the second memory access request in a case that the fourth cacheline is not marked with a maintenance mark, the maintenance mark being used to indicate that data in a cacheline is kept unchanged.

21. The apparatus of claim 20, wherein, The determining unit is further configured to select a fifth cacheline with data to be replaced for the second memory access request based on the RRPV value of a cacheline in the cache in a case that the fourth cacheline is marked with the maintenance mark.

22. The apparatus of claim 20 or 21, wherein, The apparatus further comprises: The marking unit is configured to mark the fourth cacheline with the maintenance mark when a third memory access request from the processor carries the maintenance mark and data requested to be accessed by the third memory access request is stored in the fourth cacheline. A deleting unit is configured to, when a fourth memory access request from the processor carries an unmarking flag and the fourth memory access request hits the data stored in the fourth cache line, unmark the maintaining flag marked for the fourth cache line, or remove the data stored in the fourth cache line.

23. A cache, characterized in that, The computer system comprising the cache further comprises a processor in communication with the cache, the cache being configured to cache data accessed by the processor, a size of a cache line in the cache being greater than a size of a data block requested to be accessed by one memory access request from the processor, and the cache being managed by the cache management method according to any one of claims 1 to 12.

24. A chip, characterized by The chip comprises a processor and a cache, the cache being configured to cache data accessed by the processor, a size of a cache line in the cache being greater than a size of a data block requested to be accessed by one memory access request from the processor, and the cache being managed by the cache management method according to any one of claims 1 to 12.

25. A computer system, characterized by The computer system comprises a processor and a cache, the cache being configured to cache data accessed by the processor, a size of a cache line in the cache being greater than a size of a data block requested to be accessed by one memory access request from the processor, and the cache being managed by the cache management method according to any one of claims 1 to 12.

Images