Cache management method, cache management apparatus, processor, and electronic apparatus

Abstract

Provided in the embodiments of the present disclosure are a cache management method, a cache management apparatus, a processor, and an electronic apparatus. The cache management method comprises: acquiring a cache access preference of a first thread for a first cache of a multi-thread processor, which first thread is among a plurality of threads running in the multi-thread processor; and in response to the cache access preference of the first thread for the first cache meeting a cache replacement condition, preferentially replacing from the first cache a data block, which is loaded into the first cache by the first thread. In the method, by means of acquiring cache access preferences of different threads for at least one cache, during cache replacement, a data block from a thread with a low cache access preference is preferentially replaced, thereby resolving the contention and collision of threads for a cache, improving the utilization efficiency of the cache, and realizing low overheads and low power consumption of hardware storage.

Description

Cache management method, cache management device, processor and electronic device

[0001] This application claims priority to Chinese Patent Application No. 202410384511.1 filed on March 29, 2024, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] Embodiments of the present disclosure relate to a cache management method, a cache management device, a processor, and an electronic device. Background Art

[0003] Simultaneous Multi-Threading (SMT) is a key technology for improving the overall performance of central processing units (CPUs). This technology leverages mechanisms such as multi-issue and out-of-order execution in high-performance CPU cores to enable a single physical CPU core to execute instructions from multiple threads simultaneously. From the software and operating system perspectives, a single physical CPU core can be viewed as multiple virtual CPU cores. Depending on the maximum number of active threads supported, SMT can be referred to as SMT2 (up to two active threads) or SMT4 (up to four active threads).

[0004] Compared to single-threaded execution, SMT technology can improve the resource utilization efficiency of high-performance CPUs. When a modern multi-issue high-performance CPU core executes a single thread, its internal multiple execution units and hardware resources are often underutilized most of the time. When a thread is paused for some reason (such as a second-level cache (L2 Cache) miss), the hardware execution unit can only idle, which leads to a waste of hardware resources and reduces the performance-power ratio. In SMT mode, when one thread is paused, other threads can still run, which improves the utilization of hardware resources, thereby improving the multi-threaded throughput, overall performance, and performance-power ratio of the CPU core.

[0005] Summary of the Invention

[0006] At least one embodiment of the present disclosure provides a cache management method, comprising: obtaining a cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor; and in response to the cache access preference of the first thread for the first cache satisfying a cache replacement condition, preferentially replacing a data block loaded into the first cache by the first thread out of the first cache.

[0007] For example, in the cache management method provided in at least one embodiment of the present disclosure, the cache replacement condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

[0008] For example, in the cache management method provided in at least one embodiment of the present disclosure, obtaining the cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor includes: calculating, for the first cache, a hit rate of the first thread accessing the cache.

[0009] For example, in the cache management method provided in at least one embodiment of the present disclosure, calculating the hit rate of the first thread accessing the cache for the first cache includes: recording a first value of the number of cache hits of the first thread accessing the first cache for a first time period; recording a second value of the number of cache misses of the first thread accessing the first cache; and calculating the hit rate of the first thread accessing the first cache based on the first value and the second value, wherein the hit rate = first value / (first value+second value).

[0010] For example, in the cache management method provided in at least one embodiment of the present disclosure, calculating the hit rate of the first thread accessing the cache for the first cache also includes: providing a first counter and a second counter for the first thread for the first cache; the first counter is configured to record the first value; and the second counter is configured to record the second value.

[0011] For example, in the cache management method provided in at least one embodiment of the present disclosure, the first cache includes multiple cache lines, each of the multiple cache lines includes a data block and a first identification bit; the first identification bit is configured to indicate whether the data block is loaded into the first cache by a thread with low cache access preference.

[0012] For example, in the cache management method provided in at least one embodiment of the present disclosure, in response to the cache access preference of the first thread to the first cache satisfying the cache replacement condition, the data block loaded by the first thread to the first cache is preferentially replaced out of the first cache, including: in response to the cache access preference of the first thread to the first cache satisfying the cache replacement condition, setting the first identification bit corresponding to the data block.

[0013] For example, in the cache management method provided in at least one embodiment of the present disclosure, in response to the cache access preference of the first thread to the first cache satisfying the cache replacement condition, the data block loaded into the first cache by the first thread is preferentially replaced out of the first cache, including: for the first cache, when cache replacement is required, in response to the first identification bit corresponding to the first data block among multiple data blocks indicating that the first data block is loaded into the first cache by a thread with low cache access preference, the first data block is preferentially replaced out of the first cache.

[0014] For example, in the cache management method provided in at least one embodiment of the present disclosure, obtaining the cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor includes: calculating, for the first cache, the probability that an instruction of the first thread accessing the cache will cause a pipeline stall.

[0015] At least one embodiment of the present disclosure provides a cache management device, comprising: an acquisition module configured to acquire a cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor; and a processing module configured to preferentially replace a data block loaded into the first cache by the first thread out of the first cache in response to the cache access preference of the first thread for the first cache satisfying a cache replacement condition.

[0016] For example, in the cache management device provided in at least one embodiment of the present disclosure, the cache replacement condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

[0017] For example, in the cache management device provided in at least one embodiment of the present disclosure, the acquisition module includes: a first calculation module configured to calculate, for the first cache, a hit rate of the first thread accessing the cache.

[0018] At least one embodiment of the present disclosure provides a processor, comprising: a pipeline configured to run multiple threads; a first cache; and a cache controller configured to obtain a cache access preference of a first thread among the multiple threads for the first cache, and in response to the cache access preference of the first thread for the first cache satisfying a cache replacement condition, preferentially replacing a data block loaded into the first cache by the first thread out of the first cache.

[0019] For example, in the processor provided by at least one embodiment of the present disclosure, the first cache includes multiple cache lines, each of the multiple cache lines includes a data block and a first identification bit; the first identification bit is configured to indicate whether the data block is loaded into the first cache by a thread with low cache access preference.

[0020] For example, in the processor provided by at least one embodiment of the present disclosure, a counter group is further included, wherein the counter group includes a first counter and a second counter for the first cache, the first counter is configured to record a first value, and the second counter is configured to record a second value, the first value represents the number of cache hits when the first thread accesses the first cache; the second value represents the number of cache misses when the first thread accesses the first cache.

[0021] For example, the processor provided in at least one embodiment of the present disclosure further includes a multi-level cache, wherein the multi-level cache includes the first cache.

[0022] At least one embodiment of the present disclosure provides an electronic device, including a processor provided according to at least one embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0024] FIG1A is a schematic block diagram of a computer system;

[0025] FIG1B is a schematic diagram of a pipeline of a processor core;

[0026] FIG2A is a flow chart of a cache replacement method RRIP based on reuse distance;

[0027] FIG2B is a flowchart of a cache management method provided by at least one embodiment of the present disclosure;

[0028] FIG3 is a flowchart of a cache management method provided by at least one embodiment of the present disclosure;

[0029] FIG4 is a schematic diagram of a counter group provided by at least one embodiment of the present disclosure;

[0030] FIG5 is a schematic diagram of a cache line provided by at least one embodiment of the present disclosure;

[0031] FIG6 is a schematic block diagram of a cache management device provided by at least one embodiment of the present disclosure;

[0032] FIG7 is a schematic block diagram of a processor provided by at least one embodiment of the present disclosure; and

[0033] FIG8 is a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0035] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by persons of ordinary skill in the field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0036] The present disclosure is described below using several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numeral in each drawing.

[0037] FIG1A is a schematic block diagram of a computer system.

[0038] As shown in Figure 1A, the L1 cache and L2 cache are located inside the processor core and are essential components of the processor, storing frequently accessed data and instructions. The L1 cache is divided into a data cache and an instruction cache, storing data and instructions, respectively. The L2 cache, located after the L1 cache, does not distinguish between data and instructions.

[0039] In a multi-core processor, each processor core has its own L1 cache and L2 cache. These caches are private and serve only the corresponding processor core. In some processor architectures, all processor cores share a last level cache (LLC) (for example, L3 or L4 cache), which is located outside the processor core but still part of the processor chip. The last level cache (LLC) is used to store data and instructions that need to be shared across multiple processor cores.

[0040] In a multi-level cache memory architecture, the first-level cache (L1 Cache) has the fastest access speed but the smallest capacity. The last-level cache (LLC) has the largest capacity but the slowest access speed. The second-level cache (L2 Cache) has an access speed and capacity between the first-level cache (L1 Cache) and the next level (e.g., the last level). Furthermore, data reads from the first-level cache (L1 Cache) have a higher priority than data reads from the second-level cache (L2 Cache), and data reads from the second-level cache (L2 Cache) have a higher priority than data reads from the last-level cache (LLC).

[0041] As shown in Figure 1A, the system agent is the communication center between the processor and external devices (such as input / output devices and memory controllers). It is configured to coordinate and manage data transfers between external devices and ensure that the processor can efficiently access external devices. The system agent typically includes a bus controller, an interrupt controller, and other necessary logic to handle system-level tasks. The processor connects to the system agent, which connects to the memory controller and other devices via the PCIe bus.

[0042] As shown in Figure 1A, the memory controller (MC) connects to the memory via a bus and is configured to send and receive data. When the processor needs to access the memory, the memory controller sends the request to the memory and waits for a response. Once the memory is ready, it returns the data to the memory controller via the bus, which then passes the data to the processor.

[0043] The process of the processor core reading data includes the following steps S01 to S07.

[0044] Step S01: The processor core outputs the virtual address of the target data or instruction of an access request to an address translator (e.g., an address translation pipeline). The address translator translates the virtual address into a physical address. The processor core then determines whether the target data or instruction of the access request is in the L1 cache. The access request can be a read request for data or instructions.

[0045] Step S02: If the target data or instruction of the access request exists in the L1 cache, the target data or instruction of the access request is retrieved from the L1 cache and sent to the processor core through the following step S07.

[0046] Step S03: If the target data or instruction of the access request does not exist in the first-level cache (L1 Cache), a storage item is requested from the miss address cache (MAB) and the storage item is allocated to the access request.

[0047] Step S04: the miss address cache memory requests the target data or instruction of the access request from the cache memory at the next level (eg, the second level cache, L2 Cache) based on the storage item.

[0048] Step S05: The cache memory at the next level obtains the target data or instruction of the access request, and returns the target data or instruction of the access request to the miss address cache memory.

[0049] If the target data or instruction of the access request is stored in the L2 cache (L2 cache), the L1 cache retrieves the target data or instruction from the L2 cache (L2 cache). If the target data or instruction of the access request is not stored in the L2 cache (L2 cache), the L2 cache can retrieve the target data or instruction from the memory located at the level below the L2 cache (L2 cache). For example, the memory located at the level below the L2 cache (L2 cache) can be the L3 cache (L3 cache), the L4 cache (L4 cache), or the main memory.

[0050] Step S06: The miss address cache memory writes the target data or instruction of the access request into the first level cache (L1 Cache).

[0051] Step S07: The first-level cache (L1 Cache) provides the target data or instruction of the access request to the processor core for further processing by the processor core.

[0052] As mentioned above, modern high-performance CPUs often have multiple levels of cache. When processing a memory access request, they first search the highest-level cache (L1 cache). If the data block is in that level of cache (a cache hit), the data is retrieved directly. Otherwise, the second-level cache (L2 cache) is searched, and so on to lower levels of cache. The higher the cache level, the smaller the capacity and the lower the access latency, so the space becomes more valuable.

[0053] FIG1B is a schematic diagram of a pipeline of a processor core, where the dashed line with an arrow represents a redirected instruction flow.

[0054] As shown in Figure 1B, the processor core (e.g., CPU core) of a single-core or multi-core processor uses pipeline technology to improve instruction-level parallelism. The processor core internally includes multiple pipeline stages. For example, after the pipeline inputs program counters from various sources and the next program counter (PC) is selected by a multiplexer (Mux), the instruction corresponding to the program counter undergoes branch prediction, instruction fetch, instruction decoding, instruction dispatch and rename, instruction execution, and instruction retirement. Waiting queues are set between each pipeline stage as needed. These queues are typically first-in, first-out (FIFO) queues. For example, after the branch prediction unit, there's a branch prediction (BP) FIFO queue to store branch prediction results; after the instruction fetch unit, there's an instruction cache (IC) FIFO to cache fetched instructions; after the instruction decode unit, there's a decode (DE) FIFO to cache decoded instructions; and after the instruction dispatch and rename unit, there's a finish (RT) FIFO to cache instructions waiting for completion confirmation. The processor core's pipeline also includes an instruction queue to cache instructions waiting for execution by the instruction execution unit after dispatch and renaming.

[0055] To support high operating frequencies, each pipeline stage may contain multiple pipeline stages (operation cycles), with each pipeline stage performing limited operations to further improve the performance of the processor core. Although each pipeline stage performs limited operations, this allows each clock to be kept to a minimum, thereby improving the performance of the CPU core by increasing the CPU's operating frequency. Each pipeline stage can also further improve the performance of the processor core by accommodating more instructions (i.e., superscalar technology). Superscalar refers to a method of executing multiple instructions in parallel within a cycle. A processor with increased instruction-level parallelism that can process multiple instructions within a cycle is called a superscalar processor. Superscalar processors add additional resources to conventional scalar processors, creating multiple pipelines, each of which executes its own assigned instructions to achieve parallelism.

[0056] Similarly, to improve the parallelism of instruction execution in the processor, the processor can also adopt simultaneous multi-threading (SMT) technology. The processor's pipeline structure for instruction execution (also referred to as "pipeline") can support the simultaneous execution of two or more (hardware) threads, such as SMT2 (supporting up to two concurrent threads), SMT4 (supporting up to four concurrent threads), or SMT8 (supporting up to eight concurrent threads). In the pipeline of a processor that supports simultaneous multi-threading technology, the processor's computing resources are shared by multiple threads. For example, each thread can have its own independent logical register; in the queues of various control functions of the pipeline, they can be shared by multiple threads, such as the instruction scheduling queue, or they can be statically divided among multiple threads, such as the instruction reordering queue. Simultaneous multi-threading technology can use the parallelism between threads to improve pipeline resource utilization.

[0057] Processors that use Simultaneous Multithreading (SMT) can operate in either SMT or single-threaded mode. For example, in SMT mode, when one thread encounters a wait state, other threads can continue executing. This effectively improves hardware resource utilization, thereby enhancing the CPU core's multithreading capabilities, overall performance, and performance-per-watt ratio.

[0058] In SMT mode, different hardware threads share the cache, but their memory access behavior is uneven. Simultaneously running threads may have different cache access preferences: some threads frequently access certain data blocks, while others may only traverse the data once or twice, never accessing them again for an extended period. Data blocks used by the former must remain in the cache, while data blocks used by the latter do not. Some SMT solutions do not distinguish between these two situations, typically allowing all threads to share cache access. This can lead to intense competition for cache space and "cache conflicts," where threads with poor cache access preferences frequently flush data blocks from threads with good cache access preferences, resulting in low cache utilization.

[0059] When a new data block needs to be added to the cache, if the cache is already full, a block must be removed from the existing cache to make room for the new block. To improve cache space utilization, modern processors commonly use cache replacement strategies. For example, data blocks with poor spatial locality are preferentially removed from the cache to reserve space for other data blocks with better spatial locality.

[0060] Some cache replacement strategies mostly rely on identifying the "reuse distance" of data blocks. Figure 2A is a flowchart of a cache replacement method RRIP based on reuse distance. As shown in Figure 2A, for each data block in the cache, the system maintains an aging (age) attribute that reflects its "freshness". This attribute records how many replacement events the data block has experienced since it was inserted into the cache. As the age value increases, it means that the data block stays in the cache longer, and the possibility of being reused in the future is relatively reduced, so it is more likely to become a target for replacement. When the cache needs to determine a data block to be replaced (victim), the RRIP method will give priority to selecting the data block with the largest current age value as a candidate. If the age values ​​of all candidate data blocks do not reach the maximum threshold, the system will uniformly increase the age values ​​of all candidate data blocks and perform screening and judgment again.

[0061] For example, as shown in FIG2A , the cache replacement method RRIP based on reuse distance includes steps S10 to S13 .

[0062] Step S10: Determine whether there is a data block with the largest age value. If so, execute step S11; if not, execute step S13.

[0063] Step S11: remove the data block.

[0064] Step S12: insert a new data block, whose age = maximum age - 1.

[0065] Step S13, all data blocks age+1, return to step S10.

[0066] The inventors of the present disclosure have noticed that the above-mentioned cache replacement method based on reuse distance needs to record the reuse distance of each cache data block (such as the age attribute in RRIP). As the cache capacity increases, the related hardware resource consumption also increases significantly. For example, for an instance using RRIP technology, assuming that the age attribute occupies 3 bits, with a cache design of 32MB capacity and 16-way group associative structure, a space of up to 192KB is required just to store the age attribute. In addition, some solutions do not distinguish between threads and cannot solve the cache conflict problem between threads in the case of concurrent multi-threading (especially SMT4). Therefore, there are problems of high hardware overhead, high power consumption, and low accuracy in the SMT mode.

[0067] At least one embodiment of the present disclosure provides a cache management method, which includes: obtaining a cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor; in response to the cache access preference of the first thread for the first cache satisfying a cache replacement condition, preferentially replacing a data block loaded into the first cache by the first thread out of the first cache.

[0068] The cache management method provided by at least one embodiment of the present disclosure obtains the cache access preferences of different threads for at least one cache, and when performing cache replacement, preferentially replaces data blocks from threads with low cache access preferences, thereby resolving contention and conflicts between threads for the cache, improving the utilization efficiency of the cache, and achieving low overhead and low power consumption for hardware storage.

[0069] The cache management method provided in accordance with the present disclosure is described below in a non-restrictive manner through multiple embodiments and examples. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present disclosure.

[0070] FIG2B is a flow chart of a cache management method provided by at least one embodiment of the present disclosure. As shown in FIG2B , the cache management method provided by the embodiment of the present disclosure includes the following steps S101 to S102.

[0071] Step S101: obtaining a cache access preference of a first thread among a plurality of threads running in a multi-threaded processor to a first cache of the multi-threaded processor.

[0072] For example, the cache management method shown in FIG2B is applied to a processor that supports simultaneous multi-threading technology, which can be a single-core or multi-core central processing unit (CPU), a graphics processing unit (GPU), an AI accelerator, etc., and the present disclosure does not limit this. For example, the cache management method shown in FIG2B can be applied to various multi-threaded processors, such as a multi-threaded processor including 2 threads, a multi-threaded processor including 4, 6, or 8 threads, etc., and the embodiments of the present disclosure do not limit this and can be set according to actual needs.

[0073] It should be noted that the term "first thread" refers to the thread currently being described, and does not necessarily refer to the first thread among multiple threads running in a multi-threaded processor. It can be any of the multiple threads mentioned above. In addition, the term "thread" in this disclosure refers to a hardware thread in a multi-threaded processor, which can be based on a software thread in application software, for example, but is not limited to multiple different software threads. A hardware thread is the actual computing resource provided by the processor core. Each hardware thread has its own register set and execution unit, and can independently execute instruction streams.

[0074] For example, the cache hierarchy of a multi-threaded processor can be configured according to different architectural requirements and performance goals. It can include only a first-level cache (L1 Cache) or further include a second-level cache (L2 Cache). In high-performance processors, it can also further include a third-level cache (L3 Cache) or even a fourth-level cache (L4 Cache) and other more complex multi-level cache structures. The embodiments of the present disclosure are not limited to this.

[0075] It should be noted that "first cache" is used to refer to the cache that is currently being described, and it does not necessarily have to be the first-level cache (L1 Cache) in a multi-threaded processor. It can be the first-level cache (L1 Cache) or any level of cache in a multi-level cache structure, such as the second-level cache (L2 Cache), the third-level cache (L3 Cache), etc.

[0076] For example, if there is only one level of cache in a multi-threaded processor, the cache access preference corresponding to each of the multiple threads running in the multi-threaded processor can be obtained. For example, if there are multiple levels of cache in a multi-threaded processor, the cache access preference of each of the multiple threads for each level of cache can be obtained.

[0077] It should be noted that in the case of multiple cache levels in a multi-threaded processor, it is not necessary to obtain cache access preferences for each cache level, and can be set according to actual needs. For example, for a cache with a large capacity, cache replacement may not be required, which will not be discussed in detail later.

[0078] Step S102: In response to the cache access preference of the first thread to the first cache meeting the cache replacement condition, preferentially replacing the data block loaded into the first cache by the first thread out of the first cache.

[0079] For example, when a thread needs to access a data block in memory, it is not obtained directly from the memory, but is searched from the cache first. If the required data exists in the cache, it is read directly; if not, it needs to be read from the memory and written to the appropriate cache level. For example, if data block M is read from the memory by the first thread and loaded into the first cache, and the cache access preference of the first thread for the first cache meets the cache replacement condition, then when performing cache replacement, data block M will be replaced out of the first cache first, thereby ensuring efficient use of cache space and avoiding unnecessary replacement pressure on other data blocks with good spatial locality and higher cache value.

[0080] For example, the cache replacement condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold. The preset threshold can be flexibly set according to actual application requirements, and the present disclosure does not impose any restrictions on this.

[0081] For example, cache access preference can be reflected based on indicators such as the cache hit rate of threads accessing the cache and the probability that instructions accessing the cache by threads cause pipeline stalls.

[0082] FIG3 is a flowchart of a cache management method provided by at least one embodiment of the present disclosure.

[0083] For example, as shown in FIG3 , the cache management method provided by the embodiment of the present disclosure includes steps S20 to S24 .

[0084] Step S20: Determine whether any data block comes from a thread with low cache access preference. If so, execute step S22; if not, execute step S21.

[0085] Step S21: Determine whether there is a data block with the largest age value. If so, proceed to step S22; if not, proceed to step S24.

[0086] Step S22: remove the data block.

[0087] Step S23: insert a new data block, whose age = maximum age - 1.

[0088] Step S24, all data blocks age+1, return to step S21.

[0089] For example, if a thread's cache access preference for a data block at a certain level of cache is lower than that of other threads and is below a preset threshold, the thread is likely to be executing a "cache flushing" code snippet, such as traversing a large array or performing random memory access. The data blocks accessed by this thread are stored in the cache and will not be accessed again, but will occupy cache storage space. Therefore, once the thread is determined to have low cache access preference, its data blocks are prioritized for eviction, freeing cache space for other threads with high cache access preference.

[0090] For example, the cache access preference of a thread can be reflected by the cache hit rate of the thread. The lower the hit rate, the lower the cache access preference of the thread.

[0091] For example, in step S101 , for the first cache, a hit rate of the first thread accessing the cache is calculated.

[0092] For example, step S101 includes steps S201 to S203.

[0093] Step S201: for a first time period, record a first value of the number of cache hits when a first thread accesses a first cache.

[0094] Step S202: for the first time period, record a second value of the number of cache misses when the first thread accesses the first cache.

[0095] Step S203: Calculate the hit rate of the first thread accessing the first cache based on the first value and the second value, for example, hit rate = first value / (first value + second value).

[0096] For example, the first time period can be flexibly set based on actual application requirements, and this disclosure does not impose any restrictions on this. For example, if two counters are used to record the first and second values, respectively, the setting of the first time period can depend on the maximum count value of the counters included in the counter group. For example, when the maximum count value of the counter is set to 1024, the first time period can be set to the time range from the last time the counter was cleared to the time when 1024 cache replacements occurred. That is, during the first time period, the two counters respectively record the cache hits and misses during the 1024 cache replacements.

[0097] For example, for the first cache, a first counter and a second counter are provided for the first thread, the first counter is configured to record the first value, and the second counter is configured to record the second value.

[0098] For example, the first counter and the second counter may be saturation counters. For example, when the count value of one of the first counter and the second counter reaches the maximum, in order to ensure the accuracy of the counter, the following processing methods may be used:

[0099] 1. The count values ​​of the first counter and the second counter no longer increase and remain unchanged at the current value;

[0100] 2. The count values ​​of the first counter and the second counter are reset to zero;

[0101] 3. The count values ​​of the first counter and the second counter are halved at the same time (for example, reduced by the same value or reduced by the same proportion) to ensure normal counting.

[0102] It should be noted that the embodiment of the present disclosure does not limit the maximum count value of the counter, which can be set according to actual needs.

[0103] For example, in a multi-threaded processor, each thread provides a counter group for recording the hit status of the corresponding thread on each level of cache, thereby calculating the corresponding hit rate.

[0104] FIG4 is a schematic diagram of a counter group provided by at least one embodiment of the present disclosure.

[0105] For example, the number of counters set in the above manner depends on the maximum number of threads and cache levels supported by the processor. For example, as shown in Figure 4, a processor core can support a maximum of K threads (threads 0 to K-1) and is equipped with a three-level cache structure (L1 cache, L2 cache, and last-level cache (LLC)). Therefore, 6K saturation counters are required. For example, taking thread 0 as an example, an L1 hit counter and an L1 miss counter are set to record thread 0's hits to the L1 cache (L1 cache), an L2 hit counter and an L2 miss counter are set to record thread 0's hits to the L2 cache (L2 cache), and an LLC hit counter and an LLC miss counter are set to record thread 0's hits to the last-level cache (LLC). The situation for other threads is similar to that for thread 0 and will not be further described here. For example, when a new thread is started, its corresponding set of counters needs to be cleared.

[0106] For example, the first cache includes a plurality of cache lines (or cache blocks), and each of the plurality of cache lines includes a data block and a first identification bit. Here, the data block corresponds to the cache line.

[0107] FIG5 is a schematic diagram of a cache line provided by at least one embodiment of the present disclosure.

[0108] For example, the first flag is configured to indicate whether the data block is loaded into the first cache by a thread with low cache access preference. For example, as shown in FIG5 , the first flag is a cache access preference flag. For example, when the cache access preference flag is a first value (e.g., 0), it indicates that the data block is not loaded into the first cache by a thread with low cache access preference; when the cache access preference flag is a second value (e.g., 1), it indicates that the data block is loaded into the first cache by a thread with low cache access preference.

[0109] For example, in response to the cache access preference of the first thread to the first cache meeting the cache replacement condition, the first flag bit corresponding to the data block is set.

[0110] For example, assuming that data block A is loaded into the L2 cache by thread 0, if the hit rate of thread 0 to the L2 cache is significantly lower than that of other threads according to the above calculation method and is lower than a preset threshold, it means that data block A is loaded into the L2 cache by thread 0, which has a low cache access preference. In this case, the cache access preference flag corresponding to data block A is set, for example, to the second value (e.g., 1).

[0111] For example, the first flag bit in the cache line can be implemented by extending the flag bit. It should be noted that the position of the first flag bit in the cache line can be set according to actual conditions, and the embodiments of the present disclosure do not limit this.

[0112] For example, as shown in Figure 5, each cache line also includes a tag item (Tag) and other identification bits. For example, other identification bits may include a replacement identification bit (Replacement Bit), a dirty data identification bit (Dirty Bit), a valid identification bit (Valid Bit), etc., and may also include more identification bits, which are not limited in the embodiments of the present disclosure. The replacement identification bit (Replacement Bit) is used to indicate whether the corresponding data block is replaced out of the cache during cache replacement, the dirty data identification bit (Dirty Bit) is used to indicate whether the data in the corresponding data block has been modified, and the valid identification bit (Valid Bit) is used to indicate whether the data in the corresponding data block is valid. For example, a cache line may also include other information, which is not limited in the embodiments of the present disclosure.

[0113] It should be noted that in the embodiment of the present disclosure, in the case of multi-level cache, for the cache lines of each level of cache, reference can be made to the relevant description of the cache lines of the first cache in the above embodiment and the relevant content of the cache lines shown in Figure 5, which will not be repeated here.

[0114] For example, for the first cache, when cache replacement is required, in response to the first identification bit corresponding to the first data block among multiple data blocks indicating that the first data block is loaded into the first cache by a thread with low cache access preference, the first data block is preferentially replaced out of the first cache.

[0115] For example, when cache replacement is required, the cache replacement logic circuit reads the identification bits of several alternative data blocks (including cache access preference identification bits) to check whether the data blocks are loaded into the cache by threads with low cache access preference. If so, they are replaced out of the cache first.

[0116] For example, a thread's cache access preference can be reflected based on the criticality of the instructions accessing the cache. For example, the probability of a thread's cache access instructions causing pipeline stalls (e.g., reorder buffer (ROB) stalls) can be used to reflect the thread's cache access preference. A low percentage of ROB stalls indicates low criticality for the instructions accessing the cache for that thread. Lower criticality indicates lower cache access preference for that thread.

[0117] For example, in step S101 , for the first cache, the probability that an instruction accessing the cache by the first thread causes pipeline stall is calculated.

[0118] For example, step S101 includes steps S301 to S303.

[0119] Step S301: for a first time period, record a third value indicating that the pipeline is stalled due to an instruction accessed by a first thread to a cache.

[0120] Step S302: for the first time period, record a fourth value indicating that the pipeline is stalled due to the instruction accessed by the first thread to the cache.

[0121] Step S303: Calculate the probability that the first thread's access to the cached instruction causes a pipeline stall based on the third and fourth values. For example, the probability that the first thread's access to the cached instruction causes a pipeline stall = the third value / (the third value + the fourth value).

[0122] For example, for the relevant content of the first time period, reference may be made to the relevant description of the first time period in the above embodiment, which will not be repeated here.

[0123] For example, for the first cache, a third counter and a fourth counter are provided for the first thread. The third counter is configured to record the third value, and the fourth counter is configured to record the fourth value. For details about the third and fourth counters, refer to the description of the first and second counters in the above embodiment, and will not be repeated here.

[0124] For example, in a multi-threaded processor, each thread provides a counter group to record situations where instructions accessed by the corresponding thread cause pipeline stalls, thereby calculating the corresponding probability. For example, the relevant content of the counter group can refer to the relevant description of the counter group in the above embodiment, and will not be repeated here.

[0125] FIG6 is a schematic block diagram of a cache management device according to at least one embodiment of the present disclosure. As shown in FIG6 , the cache management device 10 according to the embodiment of the present disclosure includes an acquisition module 11 and a processing module 12 .

[0126] For example, the acquisition module 11 is configured to acquire a cache access preference of a first thread among multiple threads running in the multi-threaded processor to a first cache of the multi-threaded processor.

[0127] For example, the processing module 12 is configured to preferentially replace the data block loaded into the first cache by the first thread out of the first cache in response to the cache access preference of the first thread to the first cache satisfying the cache replacement condition.

[0128] For details about the acquisition module 11 and the processing module 12, please refer to the description of steps S101 and S102 shown in FIG2B in the embodiment of the cache management method, and will not be repeated here. The various modules shown in FIG6 can be implemented by software, hardware, firmware, or any combination thereof.

[0129] For example, in at least one embodiment of the present disclosure, the cache replacement condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

[0130] For example, in at least one embodiment of the present disclosure, the acquisition module 11 includes a first calculation module, and the first calculation module is configured to calculate, for the first cache, a hit rate of the first thread accessing the cache.

[0131] For example, in at least one embodiment of the present disclosure, the first calculation module includes a first recording unit, a second recording unit, and a first calculation unit. The first recording unit is configured to record a first value of the number of cache hits of the first thread accessing the first cache for a first time period; the second recording unit is configured to record a second value of the number of cache misses of the first thread accessing the first cache for the first time period; and the first calculation unit is configured to calculate a hit rate of the first thread accessing the first cache based on the first value and the second value, where the hit rate = the first value / (the first value + the second value).

[0132] For example, in at least one embodiment of the present disclosure, the first computing module also includes a first providing unit, which is configured to provide a first counter and a second counter for the first thread with respect to the first cache; the first counter is configured to record a first value; and the second counter is configured to record a second value.

[0133] For example, in at least one embodiment of the present disclosure, the first cache includes multiple cache lines, each of the multiple cache lines includes a data block and a first identification bit; the first identification bit is configured to indicate whether the data block is loaded into the first cache by a thread with low cache access preference.

[0134] For example, in at least one embodiment of the present disclosure, the processing module 12 is further configured to set a first flag bit corresponding to the data block in response to the cache access preference of the first thread to the first cache meeting a cache replacement condition.

[0135] For example, in at least one embodiment of the present disclosure, the processing module 12 is further configured to, for the first cache, when cache replacement is required, preferentially replace the first data block out of the first cache in response to a first identification bit corresponding to a first data block among multiple data blocks indicating that the first data block is loaded into the first cache by a thread with a low cache access preference.

[0136] For example, in at least one embodiment of the present disclosure, the acquisition module 11 includes a second calculation module, which is configured to calculate, for the first cache, a probability that an instruction of the first thread accessing the cache will cause a pipeline stall.

[0137] FIG7 is a schematic block diagram of a processor provided by at least one embodiment of the present disclosure.

[0138] As shown in Figure 7, the processor 20 provided in the embodiment of the present disclosure includes a pipeline 21, a first cache 22, and a cache controller 23. For example, the processor 20 can be a single-core or multi-core central processing unit (CPU), a graphics processing unit (GPU), an AI accelerator, etc., and the present disclosure does not limit this.

[0139] In at least one embodiment of the present disclosure, the pipeline 21 is configured to run multiple threads; the present application has no limitation on the pipeline of the processor, for example, see the exemplary structure and working method shown in FIG. 1B .

[0140] In at least one embodiment of the present disclosure, the cache controller 23 is configured to obtain a cache access preference of a first thread among multiple threads for the first cache 22, and in response to the cache access preference of the first thread for the first cache 22 satisfying a cache replacement condition, preferentially replace a data block loaded by the first thread into the first cache 22 from the first cache 22. For example, the cache controller 23 may be integrally formed with the first cache 22. For example, when a processor includes multiple levels of cache, each level of cache may have a corresponding cache manager.

[0141] For example, the first cache 22 includes multiple cache lines, each of which includes a data block and a first flag. The first flag is configured to indicate whether the data block is loaded into the first cache 22 by a thread with a low cache access preference. For details about the first flag, please refer to the description of FIG. 5 in the embodiment of the cache management method, and will not be repeated here.

[0142] For example, the processor 20 provided in at least one embodiment of the present disclosure further includes a counter group. For example, the counter group includes a first counter and a second counter for the first cache 22, the first counter being configured to record a first value, and the second counter being configured to record a second value, the first value representing the number of cache hits when the first thread accesses the first cache 22; the second value representing the number of cache misses when the first thread accesses the first cache 22. For relevant content about the counter group, reference can be made to the relevant description of FIG. 4 in the embodiment of the cache management method described above, and no further details will be given here. The counter group can be implemented, for example, by registers or static random access memory (SRAM).

[0143] For example, the processor 20 provided in at least one embodiment of the present disclosure further includes a multi-level cache. For example, the multi-level cache includes a first cache 22. For example, the multi-level cache may include only a first-level cache (L1 Cache), or may further include a second-level cache (L2 Cache). In high-performance processors, a third-level cache (L3 Cache) or even a fourth-level cache (L4 Cache) may further include a more complex multi-level cache structure. The embodiments of the present disclosure do not impose any restrictions on this, and the multi-level cache may be configured according to actual needs.

[0144] At least one embodiment of the present disclosure further provides an electronic device, which includes the processor of any of the above embodiments. Figure 8 is a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure.

[0145] The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device 900 shown in FIG8 is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0146] For example, as shown in Figure 8, in some examples, the electronic device 900 includes a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which may include a processor of any of the above-mentioned embodiments, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the computer system are also stored in the RAM 903. The processor 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0147] For example, the following components may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 908 including, for example, a magnetic tape, hard disk, etc.; and a communication device 909, which may also include, for example, a network interface card such as a LAN card or modem. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or wired to exchange data, performing communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. Removable media 911, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in the drive 910 as needed, so that computer programs read therefrom can be installed into the storage device 908 as needed. Although FIG. 8 shows the electronic device 900 including various devices, it should be understood that implementation or inclusion of all of the devices shown is not required. More or fewer devices may alternatively be implemented or included.

[0148] For example, the electronic device 900 may further include a peripheral interface (not shown in the figure), etc. The peripheral interface may be various types of interfaces, such as a USB interface, a lightning interface, etc. The communication device 909 may communicate with a network and other devices through wireless communication, such as the Internet, an intranet and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communications may use any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0149] For example, the electronic device 900 can be any device such as a mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, navigator, server, etc., or it can be any combination of data processing devices and hardware, and the embodiments of the present disclosure are not limited to this.

[0150] Although the present disclosure has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the embodiments of the present disclosure. Therefore, such modifications or improvements, as long as they do not depart from the spirit of the present disclosure, are within the scope of protection claimed by the present disclosure.

[0151] Regarding this disclosure, the following points need to be explained:

[0152] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0153] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present disclosure, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale.

[0154] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0155] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A cache management method, comprising: Obtaining a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor; as well as In response to the cache access preference of the first thread to the first cache meeting a cache replacement condition, the data block loaded into the first cache by the first thread is preferentially replaced out of the first cache.

2. The cache management method according to claim 1, wherein: The cache replacement condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

3. The cache management method according to claim 1 or 2, wherein: The obtaining of a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor includes: For the first cache, a hit rate of the first thread accessing the cache is calculated.

4. The cache management method according to claim 3, wherein: Calculating, for the first cache, a hit rate of the first thread accessing the cache includes: For the first time period, Recording a first value of a cache hit count when the first thread accesses the first cache; recording a second value of the number of cache misses when the first thread accesses the first cache; and A hit rate of the first thread accessing the first cache is calculated based on the first value and the second value, wherein the hit rate=first value / (first value+second value).

5. The cache management method according to claim 4, wherein: Calculating a hit rate of the first thread accessing the cache for the first cache further includes: For the first cache, a first counter and a second counter are provided for the first thread, wherein the first counter is configured to record the first value, and the second counter is configured to record the second value.

6. The cache management method according to any one of claims 1 to 5, wherein: The first cache includes a plurality of cache lines, each of the plurality of cache lines includes a data block and a first identification bit; The first flag is configured to indicate whether the data block is accessed by a cache with low access preference. The threads are loaded into the first cache.

7. The cache management method according to claim 6, wherein: In response to the cache access preference of the first thread to the first cache meeting a cache replacement condition, preferentially replacing the data block loaded by the first thread into the first cache from the first cache, comprising: In response to the cache access preference of the first thread to the first cache meeting a cache replacement condition, a first flag bit corresponding to the data block is set.

8. The cache management method according to claim 6 or 7, wherein: In response to the cache access preference of the first thread to the first cache meeting a cache replacement condition, preferentially replacing the data block loaded by the first thread into the first cache from the first cache, comprising: For the first cache, when cache replacement is required, in response to the first identification bit corresponding to the first data block among multiple data blocks indicating that the first data block is loaded into the first cache by a thread with low cache access preference, the first data block is preferentially replaced out of the first cache.

9. The cache management method according to claim 1 or 2, wherein: The obtaining of a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor includes: For the first cache, a probability that an instruction accessing the cache by the first thread causes a pipeline stall is calculated.

10. A cache management device, comprising: An acquisition module configured to acquire a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor; as well as The processing module is configured to preferentially replace the data block loaded into the first cache by the first thread out of the first cache in response to the cache access preference of the first thread to the first cache satisfying a cache replacement condition.

11. The cache management device according to claim 10, wherein: The cache replacement condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

12. The cache management device according to claim 10 or 11, wherein: The acquisition module includes: The first calculation module is configured to calculate, for the first cache, a hit rate of the first thread accessing the cache.

13. A processor comprising: Pipelines, are configured to run multiple threads; First cache; as well as The cache controller is configured to obtain a cache access preference of a first thread among the multiple threads for the first cache, and in response to the cache access preference of the first thread for the first cache satisfying a cache replacement condition, preferentially replace the data block loaded into the first cache by the first thread out of the first cache.

14. The processor of claim 13, wherein: The first cache includes a plurality of cache lines, each of the plurality of cache lines includes a data block and a first identification bit; The first flag is configured to indicate whether the data block is loaded into the first cache by a thread with low cache access preference.

15. The processor according to claim 13 or 14, further comprising a counter group, wherein The counter group includes a first counter and a second counter for the first cache, The first counter is configured to record a first value, The second counter is configured to record a second value, The first value represents the number of cache hits when the first thread accesses the first cache; The second value represents the number of cache misses when the first thread accesses the first cache.

16. The processor according to any one of claims 13 to 15, further comprising a multi-level cache, wherein: The multi-level cache includes the first cache.

17. An electronic device comprising the processor according to any one of claims 13 to 16.

Images