Cache management method, cache management apparatus, and processor and electronic apparatus

Abstract

Provided in the embodiments of the present disclosure are a cache management method, a cache management apparatus, and a processor and an electronic apparatus. The cache management method comprises: acquiring a cache access preference of a first thread for a first cache of a multi-thread processor, which first thread is among a plurality of threads running in the multi-thread processor; and in response to the cache access preference of the first thread for the first cache meeting a cache bypass condition, stopping the load of a data block, which is accessed by the first thread, into the first cache. In the method, by means of acquiring cache access preferences of different threads for at least one cache, and making a thread having a low cache access preference bypass a cache to be accessed, the contention and conflict between threads for a cache is resolved, thereby improving the utilization efficiency of the cache, and realizing low overhead and low power consumption of hardware storage.

Description

Cache management method, cache management device, processor and electronic device

[0001] This application claims priority to Chinese patent application No. 202410381320.X filed on March 29, 2024. The contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] Embodiments of the present disclosure relate to a cache management method, a cache management device, a processor, and an electronic device. Background Art

[0003] Simultaneous Multi-Threading (SMT) is a key technology for improving the overall performance of central processing units (CPUs). This technology leverages mechanisms such as multi-issue and out-of-order execution in high-performance CPU cores to enable a single physical CPU core to execute instructions from multiple threads simultaneously. From the software and operating system perspectives, a single physical CPU core can be viewed as multiple virtual CPU cores. Depending on the maximum number of active threads supported, SMT can be referred to as SMT2 (up to two active threads) or SMT4 (up to four active threads).

[0004] Compared to single-threaded processing, SMT technology can improve the resource utilization efficiency of high-performance CPUs. When a modern multi-issue high-performance CPU core executes a single thread, its internal multiple execution units and hardware resources are often underutilized most of the time. When a thread is paused for some reason (such as a L2 cache miss), the hardware execution unit can only idle, which leads to a waste of hardware resources and reduces the performance-power ratio. In SMT mode, when one thread is paused, other threads can still run, which improves the utilization of hardware resources, thereby improving the multi-threaded throughput, overall performance and performance-power ratio of the CPU core.

[0005] Summary of the Invention

[0006] At least one embodiment of the present disclosure provides a cache management method, which includes: obtaining a cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor; and in response to the cache access preference of the first thread for the first cache satisfying a cache bypass condition, preventing a data block accessed by the first thread from being loaded into the first cache.

[0007] For example, in the cache management method provided in at least one embodiment of the present disclosure, the cache bypass condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

[0008] For example, in the cache management method provided in at least one embodiment of the present disclosure, obtaining the cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor includes: calculating a data block reuse rate based on the reuse of a data block loaded by the first thread into the first cache from the time it enters the first cache to the time it is replaced out of the first cache.

[0009] For example, in the cache management method provided in at least one embodiment of the present disclosure, the data block reuse rate is calculated based on the reuse of the data block loaded by the first thread to the first cache from the time it enters the first cache to the time it is replaced out of the first cache, including: for a first time period, recording a first value of the number of times the data block loaded by the first thread to the first cache is re-accessed from the time it enters the first cache to the time it is replaced out of the first cache; recording a second value of the number of times the data block loaded by the first thread to the first cache is not re-accessed from the time it enters the first cache to the time it is replaced out of the first cache; and calculating the data block reuse rate loaded by the first thread to the cache based on the first value and the second value, wherein the data block reuse rate = first value / (first value+second value).

[0010] For example, in the cache management method provided in at least one embodiment of the present disclosure, the data block reuse rate is calculated based on the reuse of the data block loaded by the first thread into the first cache from the time it enters the first cache to the time it is replaced out of the first cache, and also includes: for the first cache, providing a first counter and a second counter for the first thread; the first counter is configured to record the first value; the second counter is configured to record the second value.

[0011] For example, in the cache management method provided in at least one embodiment of the present disclosure, the first cache includes multiple cache lines, each of the multiple cache lines includes a data block, a first domain, and a second domain; the first domain is configured to indicate a thread that loads the data block into the first cache; and the second domain is configured to indicate whether the data block has been accessed again from the time it enters the first cache to the time it is replaced out of the first cache.

[0012] For example, in the cache management method provided by at least one embodiment of the present disclosure, when a data block among multiple data blocks is replaced out of the first cache, in response to the second domain corresponding to the data block indicating that the data block was accessed again from the time it entered the first cache to the time it was replaced out of the first cache, the first value corresponding to the thread indicated by the first domain corresponding to the data block is incremented by 1; in response to the second domain corresponding to the data block indicating that the data block was not accessed again from the time it entered the first cache to the time it was replaced out of the first cache, the second value corresponding to the thread indicated by the first domain corresponding to the data block is incremented by 1.

[0013] For example, in the cache management method provided in at least one embodiment of the present disclosure, obtaining the cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor includes: calculating, for the first cache, a hit rate of the first thread accessing the cache.

[0014] For example, in the cache management method provided in at least one embodiment of the present disclosure, calculating the hit rate of the first thread accessing the cache for the first cache includes: recording a third value of the number of cache hits of the first thread accessing the first cache for a first time period; recording a fourth value of the number of cache misses of the first thread accessing the first cache; and calculating the hit rate of the first thread accessing the first cache based on the third value and the fourth value, wherein the hit rate = third value / (third value+fourth value).

[0015] For example, in the cache management method provided in at least one embodiment of the present disclosure, calculating the hit rate of the first thread accessing the cache for the first cache also includes: providing a third counter and a fourth counter for the first thread for the first cache; the third counter is configured to record the third value; and the fourth counter is configured to record the fourth value.

[0016] For example, in the cache management method provided in at least one embodiment of the present disclosure, obtaining the cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor includes: calculating, for the first cache, the probability that an instruction of the first thread accessing the cache will cause a pipeline stall.

[0017] At least one embodiment of the present disclosure provides a cache management device, comprising: an acquisition module configured to acquire a cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor; and a processing module configured to prevent a data block accessed by the first thread from being loaded into the first cache in response to the cache access preference of the first thread for the first cache satisfying a cache bypass condition.

[0018] For example, in the cache management device provided in at least one embodiment of the present disclosure, the cache bypass condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

[0019] For example, in the cache management device provided in at least one embodiment of the present disclosure, the acquisition module includes: a first calculation module, configured to calculate the data block reuse rate based on the reuse of the data block loaded into the first cache by the first thread from the time it enters the first cache to the time it is replaced out of the first cache.

[0020] For example, in the cache management device provided in at least one embodiment of the present disclosure, the acquisition module includes: a second calculation module configured to calculate, for the first cache, a hit rate of the first thread accessing the cache.

[0021] At least one embodiment of the present disclosure provides a processor, comprising: a pipeline configured to run multiple threads; a first cache; and a cache controller configured to obtain a cache access preference of a first thread among multiple threads running in the multi-threaded processor for the first cache of the multi-threaded processor, and in response to the cache access preference of the first thread for the first cache satisfying a cache bypass condition, prevent a data block accessed by the first thread from being loaded into the first cache.

[0022] For example, in the processor provided by at least one embodiment of the present disclosure, the first cache includes multiple cache lines, each of the multiple cache lines includes a data block, a first domain, and a second domain; the first domain is configured to indicate a thread that loads the data block into the first cache; and the second domain is configured to indicate whether the data block has been accessed again from the time it enters the first cache to the time it is replaced out of the first cache.

[0023] For example, in the processor provided by at least one embodiment of the present disclosure, a counter group is further included, wherein the counter group includes a first counter and a second counter for the first cache, the first counter is configured to record a first value, and the second counter is configured to record a second value, the first value represents the number of times the data block loaded by the first thread to the first cache is accessed again from the time it enters the first cache to the time it is replaced out of the first cache; the second value represents the number of times the data block loaded by the first thread to the first cache is not accessed again from the time it enters the first cache to the time it is replaced out of the first cache.

[0024] For example, the processor provided in at least one embodiment of the present disclosure further includes a multi-level cache, wherein the multi-level cache includes the first cache.

[0025] At least one embodiment of the present disclosure provides an electronic device, including a processor provided according to at least one embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0027] FIG1A is a schematic block diagram of a computer system;

[0028] FIG1B is a schematic diagram of a pipeline of a processor core;

[0029] FIG2 is a flowchart of a cache management method provided by at least one embodiment of the present disclosure;

[0030] FIG3 is a flowchart of a cache management method provided by at least one embodiment of the present disclosure;

[0031] FIG4 is a schematic diagram of a counter group provided by at least one embodiment of the present disclosure;

[0032] FIG5 is a schematic diagram of a cache line provided by at least one embodiment of the present disclosure;

[0033] FIG6 is a schematic diagram of a counter group provided by at least one embodiment of the present disclosure;

[0034] FIG7 is a schematic block diagram of a cache management device provided by at least one embodiment of the present disclosure;

[0035] FIG8 is a schematic block diagram of a processor provided by at least one embodiment of the present disclosure; and

[0036] FIG9 is a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0038] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by persons of ordinary skill in the field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0039] The present disclosure is described below using several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numeral in each drawing.

[0040] FIG1A is a schematic block diagram of a computer system.

[0041] As shown in Figure 1A, the L1 cache and L2 cache are located inside the processor core and are essential components of the processor, storing frequently accessed data and instructions. The L1 cache is divided into a data cache and an instruction cache, storing data and instructions, respectively. The L2 cache, located after the L1 cache, does not distinguish between data and instructions.

[0042] In a multi-core processor, each processor core has its own L1 cache and L2 cache. These caches are private and serve only the corresponding processor core. In some processor architectures, all processor cores share a last level cache (LLC) (for example, L3 or L4 cache), which is located outside the processor core but still part of the processor chip. The last level cache (LLC) is used to store data and instructions that need to be shared across multiple processor cores.

[0043] In a multi-level cache memory architecture, the first-level cache (L1 Cache) has the fastest access speed but the smallest capacity. The last-level cache (LLC) has the largest capacity but the slowest access speed. The second-level cache (L2 Cache) has an access speed and capacity between the first-level cache (L1 Cache) and the next level (e.g., the last level). Furthermore, data reads from the first-level cache (L1 Cache) have a higher priority than data reads from the second-level cache (L2 Cache), and data reads from the second-level cache (L2 Cache) have a higher priority than data reads from the last-level cache (LLC).

[0044] As shown in Figure 1A, the system agent is the communication center between the processor and external devices (such as input / output devices and memory controllers). It is configured to coordinate and manage data transfers between external devices and ensure that the processor can efficiently access external devices. The system agent typically includes a bus controller, an interrupt controller, and other necessary logic to handle system-level tasks. The processor connects to the system agent, which connects to the memory controller and other devices via the PCIe bus.

[0045] As shown in Figure 1A, the memory controller (MC) connects to the memory via a bus and is configured to send and receive data. When the processor needs to access the memory, the memory controller sends the request to the memory and waits for a response. Once the memory is ready, it returns the data to the memory controller via the bus, which then passes the data to the processor.

[0046] The process of the processor core reading data includes the following steps S01 to S07.

[0047] Step S01: The processor core outputs the virtual address of the target data or instruction of an access request to an address translator (e.g., an address translation pipeline). The address translator translates the virtual address into a physical address. The processor core then determines whether the target data or instruction of the access request is in the first-level cache (L1 Cache). The access request can be a read request for data or instructions.

[0048] Step S02: If the target data or instruction of the access request exists in the L1 cache, the target data or instruction of the access request is retrieved from the L1 cache and sent to the processor core through the following step S07.

[0049] Step S03: If the target data or instruction of the access request does not exist in the first-level cache (L1 Cache), a storage item is requested from the miss address cache (MAB) and the storage item is allocated to the access request.

[0050] Step S04: the miss address cache memory requests the target data or instruction of the access request from the cache memory at the next level (eg, the second level cache, L2 Cache) based on the storage item.

[0051] Step S05: The cache memory at the next level obtains the target data or instruction of the access request, and returns the target data or instruction of the access request to the miss address cache memory.

[0052] If the target data or instruction of the access request is stored in the L2 cache (L2 cache), the L1 cache retrieves the target data or instruction from the L2 cache (L2 cache). If the target data or instruction of the access request is not stored in the L2 cache (L2 cache), the L2 cache can retrieve the target data or instruction from the memory located at the level below the L2 cache (L2 cache). For example, the memory located at the level below the L2 cache (L2 cache) can be the L3 cache (L3 cache), the L4 cache (L4 cache), or the main memory.

[0053] Step S06: The miss address cache memory writes the target data or instruction of the access request into the first level cache (L1 Cache).

[0054] Step S07: The first-level cache (L1 Cache) provides the target data or instruction of the access request to the processor core for further processing by the processor core.

[0055] FIG1B is a schematic diagram of a pipeline of a processor core, where the dashed line with an arrow represents a redirected instruction flow.

[0056] As shown in Figure 1B, the processor core (e.g., CPU core) of a single-core or multi-core processor uses pipeline technology to improve instruction-level parallelism. The processor core internally includes multiple pipeline stages. For example, after the pipeline inputs program counters from various sources and the next program counter (PC) is selected by a multiplexer (Mux), the instruction corresponding to the program counter undergoes branch prediction, instruction fetch, instruction decoding, instruction dispatch and rename, instruction execution, and instruction retirement. Waiting queues are set between each pipeline stage as needed. These queues are typically first-in, first-out (FIFO) queues. For example, after the branch prediction unit, there's a branch prediction (BP) FIFO queue to store branch prediction results; after the instruction fetch unit, there's an instruction cache (IC) FIFO to cache fetched instructions; after the instruction decode unit, there's a decode (DE) FIFO to cache decoded instructions; and after the instruction dispatch and rename unit, there's a finish (RT) FIFO to cache instructions waiting for completion confirmation. The processor core's pipeline also includes an instruction queue to cache instructions waiting for execution by the instruction execution unit after dispatch and renaming.

[0057] To support high operating frequencies, each pipeline stage may contain multiple pipeline stages (operation cycles), with each pipeline stage performing limited operations to further improve the performance of the processor core. Although each pipeline stage performs limited operations, this allows each clock to be kept to a minimum, thereby improving the performance of the CPU core by increasing the CPU's operating frequency. Each pipeline stage can also further improve the performance of the processor core by accommodating more instructions (i.e., superscalar technology). Superscalar refers to a method of executing multiple instructions in parallel within a cycle. A processor with increased instruction-level parallelism that can process multiple instructions within a cycle is called a superscalar processor. Superscalar processors add additional resources to conventional scalar processors, creating multiple pipelines, each of which executes its own assigned instructions to achieve parallelism.

[0058] Similarly, to improve the parallelism of instruction execution in the processor, the processor can also adopt simultaneous multi-threading (SMT) technology. The processor's pipeline structure for instruction execution (also referred to as "pipeline") can support the simultaneous execution of two or more (hardware) threads, such as SMT2 (supporting up to two concurrent threads), SMT4 (supporting up to four concurrent threads), or SMT8 (supporting up to eight concurrent threads). In the pipeline of a processor that supports simultaneous multi-threading technology, the processor's computing resources are shared by multiple threads. For example, each thread can have its own independent logical register; in the queues of various control functions of the pipeline, they can be shared by multiple threads, such as the instruction scheduling queue, or they can be statically divided among multiple threads, such as the instruction reordering queue. Simultaneous multi-threading technology can use the parallelism between threads to improve pipeline resource utilization.

[0059] Processors that use Simultaneous Multithreading (SMT) can operate in either SMT or single-threaded mode. For example, in SMT mode, when one thread encounters a wait state, other threads can continue executing. This effectively improves hardware resource utilization, thereby enhancing the CPU core's multithreading capabilities, overall performance, and performance-per-watt ratio.

[0060] In SMT mode, different hardware threads share the cache, but their memory access behavior is uneven. Simultaneously running threads may have different cache access preferences: some threads frequently access certain data blocks, while others may only traverse the data once or twice, never accessing them again for an extended period. The data blocks used by the former need to remain in the cache, while those used by the latter do not. Some SMT solutions fail to distinguish between these two situations, leading to intense competition for cache space and "cache conflicts," where threads with poor cache access preferences frequently flush the data blocks of threads with good cache access preferences, resulting in low cache utilization.

[0061] To improve cache space utilization, modern processors generally adopt a cache bypass strategy. For example, some data blocks with poor spatial locality are prevented from entering the cache to reserve more space for other data blocks with better spatial locality.

[0062] Some cache bypass technologies (such as dead block prediction and reuse distance prediction) use a certain hardware structure to record the reuse history of data blocks, and predict whether the data block can be bypassed based on the reuse history. For dead blocks (that is, data blocks that will not be accessed again) or data blocks with a large reuse distance (that is, the time interval between two accesses is very long), cache bypass is performed.

[0063] The inventors of the present disclosure have noted that the above-mentioned prediction-based cache bypass method requires a relatively complex hardware structure to record the history of data blocks and make predictions, and usually predictions need to be made for each data block. In SMT mode, running multiple threads at the same time means that it is necessary to monitor the more complex data blocks and program memory access behaviors caused by these threads at the same time. However, in some solutions, SMT does not distinguish between multiple threads and cannot solve the cache conflict problem between threads in the case of concurrent multi-threading (especially SMT4). Therefore, there are problems of high hardware overhead, high power consumption, and low accuracy in SMT mode.

[0064] At least one embodiment of the present disclosure provides a cache management method, which includes: obtaining a cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor; in response to the cache access preference of the first thread for the first cache satisfying a cache bypass condition, preventing a data block accessed by the first thread from being loaded into the first cache.

[0065] The cache management method provided by at least one embodiment of the present disclosure can obtain the cache access preferences of different threads for at least one cache, and bypass the threads with low cache access preferences to access the cache, thereby resolving contention and conflicts between threads for the cache, improving the utilization efficiency of the cache, and achieving low overhead and low power consumption of hardware storage.

[0066] The cache management method provided in accordance with the present disclosure is described below in a non-restrictive manner through multiple embodiments and examples. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present disclosure.

[0067] FIG2 is a flow chart of a cache management method provided by at least one embodiment of the present disclosure. As shown in FIG2 , the cache management method provided by the embodiment of the present disclosure includes the following steps S101 to S102.

[0068] Step S101: obtaining a cache access preference of a first thread among a plurality of threads running in a multi-threaded processor to a first cache of the multi-threaded processor.

[0069] For example, the cache management method shown in FIG2 is applied to a processor that supports simultaneous multi-threading technology, which can be a single-core or multi-core central processing unit (CPU), a graphics processing unit (GPU), an AI accelerator, etc., and the present disclosure does not limit this. For example, the cache management method shown in FIG2 can be applied to various multi-threaded processors, such as a multi-threaded processor including 2 threads, a multi-threaded processor including 4, 6, or 8 threads, etc., and the embodiments of the present disclosure do not limit this and can be set according to actual needs.

[0070] It should be noted that the term "first thread" refers to the thread currently being described, and does not necessarily refer to the first thread among multiple threads running in a multi-threaded processor. It can be any of the multiple threads mentioned above. In addition, the term "thread" in this disclosure refers to a hardware thread in a multi-threaded processor, which can be based on a software thread in application software, for example, but is not limited to multiple different software threads. A hardware thread is the actual computing resource provided by the processor core. Each hardware thread has its own register set and execution unit, and can independently execute instruction streams.

[0071] For example, the cache hierarchy of a multi-threaded processor can be configured according to different architectural requirements and performance goals. It can include only a first-level cache (L1 Cache) or further include a second-level cache (L2 Cache). In high-performance processors, it can also further include a third-level cache (L3 Cache) or even a fourth-level cache (L4 Cache) and other more complex multi-level cache structures. The embodiments of the present disclosure do not limit this.

[0072] It should be noted that "first cache" is used to refer to the cache that is currently being described, and it does not necessarily have to be the first-level cache (L1 Cache) in a multi-threaded processor. It can be the first-level cache (L1 Cache) or any level of cache in a multi-level cache structure, such as the second-level cache (L2 Cache), the third-level cache (L3 Cache), etc.

[0073] For example, if there is only one level of cache in a multi-threaded processor, the cache access preference corresponding to each of the multiple threads running in the multi-threaded processor can be obtained. For example, if there are multiple levels of cache in a multi-threaded processor, the cache access preference of each of the multiple threads for each level of cache can be obtained.

[0074] It should be noted that in the case of multiple cache levels in a multi-threaded processor, it is not necessary to obtain cache access preferences for each cache level, and can be set according to actual needs. For example, for a cache with a large capacity, cache bypass may not be necessary, which will not be discussed in detail later.

[0075] Step S102: In response to the cache access preference of the first thread to the first cache satisfying a cache bypass condition, preventing the data block accessed by the first thread from being loaded into the first cache.

[0076] For example, when a thread needs to access a data block in memory, it is not directly obtained from the memory, but is first searched from the cache. If the required data exists in the cache, it is read directly; if not, it needs to be read from the memory and written to the appropriate cache level. For example, the first thread reads a data block from the memory. If the cache access preference of the first thread for the first cache meets the cache bypass condition, cache bypass is performed to avoid writing the data block read from the memory to the first cache, thereby ensuring efficient use of the cache space and avoiding unnecessary replacement pressure on other data blocks with good spatial locality and higher cache value.

[0077] For example, the cache bypass condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold. The preset threshold can be flexibly set according to actual application requirements, and the present disclosure does not impose any restrictions on this.

[0078] For example, cache access preference can be reflected based on indicators such as the thread's data block reuse rate, the thread's cache hit rate, and the probability that the thread's cache access instruction causes pipeline stalls.

[0079] FIG3 is a flowchart of a cache management method provided by at least one embodiment of the present disclosure.

[0080] For example, as shown in FIG3 , a multi-threaded processor has a three-level cache structure (L1 cache, L2 cache, and last-level cache (LLC)). To simplify the description, the three-level cache structure here adopts an inclusive design, that is, the lower-level cache includes the higher-level cache. For example, a data block located in the L1 cache must also be located in the L2 cache (L2 cache) and the last-level cache (LLC). For each of the multiple threads, the cache management method provided by the embodiment of the present disclosure includes the following steps S1 to S9:

[0081] Step S1: Determine whether the cache access preference of the thread to the L1 cache is lower than a preset threshold. If yes, proceed to step S2; if no, proceed to step S3.

[0082] Step S2: Bypassing the L1 Cache.

[0083] Step S3: Do not bypass the L1 cache, and execute step S6.

[0084] Step S4: Determine whether the cache access preference of the thread to the L2 cache is lower than a preset threshold. If yes, proceed to step S5; if no, proceed to step S6.

[0085] Step S5: Bypassing the L2 Cache.

[0086] Step S6: Do not bypass the L2 cache and execute step S9.

[0087] Step S7: Determine whether the cache access preference of the thread to the last level cache (LLC) is lower than a preset threshold. If yes, execute step S8; if not, execute step S9.

[0088] Step S8: Bypass the last level cache (LLC).

[0089] Step S9: Do not bypass the last level cache (LLC).

[0090] For example, if a thread's cache access preference for a data block at a certain level of cache is lower than that of other threads and is below a preset threshold, it is determined that the thread is likely executing a "cache-flushing" code snippet, such as traversing a large array or performing random memory accesses. The data blocks accessed by this thread are stored in the cache and will not be accessed again, but will occupy cache storage space. Therefore, cache bypass is performed on the data blocks accessed by this thread, actively avoiding loading them into the cache to save cache space for other threads with higher cache access preferences.

[0091] For example, a thread's data block reuse rate can be used to reflect its cache access preference. For a thread with a low data block reuse rate, this means that the data blocks read from memory and loaded into the cache by the thread are rarely reused (by the thread or other threads) after entering the cache. Therefore, cache bypass can be implemented for the data blocks accessed by the thread. This allows the data blocks accessed by the thread to not be loaded into the cache if certain conditions are met, thus avoiding cache resource usage and improving cache utilization.

[0092] For example, in step S101 , the data block reuse rate is calculated based on the reuse of the data block loaded into the first cache by the first thread from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

[0093] For example, step S101 includes steps S201 to S203.

[0094] Step S201: for a first time period, recording a first value of the number of times a data block loaded by a first thread into a first cache is re-accessed from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

[0095] Step S202: for the first time period, record a second value of the number of times a data block loaded by the first thread into the first cache is not accessed again from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

[0096] Step S203: Calculate the reuse rate of the data block loaded into the cache by the first thread based on the first value and the second value. For example, data block reuse rate = first value / (first value + second value).

[0097] For example, the first time period can be flexibly set based on actual application requirements, and this disclosure does not impose any restrictions on this. For example, if two counters are used to record the first and second values, respectively, the setting of the first time period can depend on the maximum count value of the counters included in the counter group. For example, when the maximum count value of the counter is set to 1024, the first time period can be set to the time range from the last time the counter was cleared to the time when 1024 cache replacements occurred. That is, during the first time period, the two counters respectively record the cache hits and misses during the 1024 cache replacements.

[0098] For example, for the first cache, a first counter and a second counter are provided for the first thread, the first counter is configured to record the first value, and the second counter is configured to record the second value.

[0099] For example, the first counter and the second counter may be saturation counters. For example, when the count value of one of the first counter and the second counter reaches the maximum, in order to ensure the accuracy of the counter, the following processing methods may be used:

[0100] 1. The count values ​​of the first counter and the second counter no longer increase and remain unchanged at the current value;

[0101] 2. The count values ​​of the first counter and the second counter are reset to zero;

[0102] 3. The count values ​​of the first counter and the second counter are halved at the same time (for example, reduced by the same value or reduced by the same proportion) to ensure normal counting.

[0103] It should be noted that the embodiment of the present disclosure does not limit the maximum count value of the counter, which can be set according to actual needs.

[0104] For example, in a multi-threaded processor, a counter group is provided for each thread to record the data block reuse of each level of cache by the corresponding thread, thereby calculating the corresponding data block reuse rate.

[0105] FIG4 is a schematic diagram of a counter group provided by at least one embodiment of the present disclosure.

[0106] For example, the number of counters set in the above manner depends on the maximum number of threads and cache levels supported by the processor. For example, as shown in Figure 4, a processor core can support a maximum of K threads (threads 0 to K-1) and is equipped with a three-level cache structure (L1 cache, L2 cache, and last-level cache (LLC)). Therefore, 6K saturation counters are required. For example, taking thread 0 as an example, an L1 reuse counter and an L1 non-reuse counter are set to record thread 0's reuse of data blocks in the L1 cache (L1 cache), an L2 reuse counter and an L2 non-reuse counter are set to record thread 0's reuse of data blocks in the L2 cache (L2 cache), and an LLC reuse counter and an LLC non-reuse counter are set to record thread 0's reuse of data blocks in the last-level cache (LLC). The situation for other threads is similar to that for thread 0 and will not be further described here. For example, when a new thread is started, its corresponding set of counters needs to be cleared.

[0107] For example, the first cache includes a plurality of cache lines (or cache blocks), and each of the plurality of cache lines includes a data block, a first field, and a second field. Here, the data block corresponds to the cache line.

[0108] FIG5 is a schematic diagram of a cache line provided by at least one embodiment of the present disclosure.

[0109] For example, the first field is configured to indicate the thread that loads the data block into the first cache. For example, as shown in Figure 5, the first field is the thread ID of the thread that loads the data block into the first cache. For example, the size of the first field is 2 bits. If thread 1 loads the data block into the first cache, the value of the first field is 01. It should be noted that the size of the first field can be set according to actual needs and is not limited in this embodiment of the present disclosure.

[0110] For example, the second field is configured to indicate whether the data block has been accessed again from the time it enters the first cache to the time it is replaced out of the first cache. For example, as shown in FIG5 , the second field is a reuse information bit (Reuse Bit), and the size of the second field is 1 bit. For example, when the reuse information bit is a first value (e.g., 0), it indicates that the data block has not been accessed again from the time it enters the first cache to the time it is replaced out of the first cache; when the reuse information bit is a second value (e.g., 1), it indicates that the data block has been accessed again from the time it enters the first cache to the time it is replaced out of the first cache.

[0111] It should be noted that the positions of the first domain and the second domain in the cache line can be set according to actual conditions, and the embodiments of the present disclosure do not limit this.

[0112] For example, as shown in Figure 5, each cache line also includes a tag item (Tag) and other identification bits. For example, other identification bits may include a replacement identification bit (Replacement Bit), a dirty data identification bit (Dirty Bit), a valid identification bit (Valid Bit), etc., and may also include more identification bits, which are not limited in the embodiments of the present disclosure. The replacement identification bit (Replacement Bit) is used to indicate whether the corresponding data block is replaced out of the cache during cache replacement, the dirty data identification bit (Dirty Bit) is used to indicate whether the data in the corresponding data block has been modified, and the valid identification bit (Valid Bit) is used to indicate whether the data in the corresponding data block is valid. For example, a cache line may also include other information, which is not limited in the embodiments of the present disclosure.

[0113] It should be noted that in the embodiment of the present disclosure, in the case of multi-level cache, for the cache lines of each level of cache, reference can be made to the relevant description of the cache lines of the first cache in the above embodiment and the relevant content of the cache lines shown in Figure 5, which will not be repeated here.

[0114] For example, when cache replacement is required, the cache replacement logic circuit reads the identification bits (including the reuse information bit (Reuse Bit)) of several alternative data blocks, checks whether the data blocks have been accessed again, and then obtains the reuse status of the data blocks taken into the cache of this layer by each thread.

[0115] For example, when a data block among multiple data blocks is replaced out of the first cache, in response to the second field corresponding to the data block indicating that the data block was accessed again from the time it entered the first cache to the time it was replaced out of the first cache, the first value corresponding to the thread indicated by the first field corresponding to the data block is incremented by 1; in response to the second field corresponding to the data block indicating that the data block was not accessed again from the time it entered the first cache to the time it was replaced out of the first cache, the second value corresponding to the thread indicated by the first field corresponding to the data block is incremented by 1.

[0116] For example, assuming that a reuse bit (Reuse Bit) of 0 indicates that a data block was not accessed again from the time it entered the cache until it was replaced out of the cache, and a reuse bit (Reuse Bit) of 1 indicates that the data block was accessed again from the time it entered the cache until it was replaced out of the cache, for a data block read from memory by thread 0 and loaded into the L1 cache, when the data block is replaced out of the L1 cache, if the reuse bit (Reuse Bit) of the data block is 1, the value of the L1 reuse counter corresponding to thread 0 is incremented by 1; if the reuse bit (Reuse Bit) of the data block is 0, the value of the L1 non-reuse counter corresponding to thread 0 is incremented by 1. At this time, for thread 0 and the L1 cache, the data block reuse rate = the value of the L1 reuse counter / (the value of the L1 reuse counter + the value of the L1 non-reuse counter).

[0117] For example, the cache hit rate of a thread can be used to reflect its cache access preference. A lower hit rate indicates a lower cache access preference for that thread. Bypassing threads with low cache access preference from the cache can improve cache utilization. This method also offers lower implementation overhead than calculating data block reuse rates.

[0118] For example, in step S101 , for the first cache, a hit rate of the first thread accessing the cache is calculated.

[0119] For example, step S101 includes steps S301 to S303.

[0120] Step S301: for a first time period, record a third value of the number of cache hits when a first thread accesses a first cache.

[0121] Step S302: for the first time period, record a fourth value of the number of cache misses when the first thread accesses the first cache.

[0122] Step S303: Calculate the hit rate of the first thread accessing the first cache based on the third value and the fourth value, for example, hit rate = third value / (third value + fourth value).

[0123] For example, for the relevant content of the first time period, reference may be made to the relevant description of the first time period in the above embodiment, which will not be repeated here.

[0124] For example, for the first cache, a third counter and a fourth counter are provided for the first thread. The third counter is configured to record the third value, and the fourth counter is configured to record the fourth value. For details about the third and fourth counters, refer to the description of the first and second counters in the above embodiment, and will not be repeated here.

[0125] For example, in a multi-threaded processor, each thread provides a counter group for recording the hit status of the corresponding thread on each level of cache, thereby calculating the corresponding hit rate.

[0126] FIG6 is a schematic diagram of a counter group provided by at least one embodiment of the present disclosure.

[0127] For example, the number of counters set in the above manner depends on the maximum number of threads and cache levels supported by the processor. For example, as shown in Figure 6, a processor core can support a maximum of K threads (threads 0 to K-1) and is equipped with a three-level cache structure (L1 cache, L2 cache, and last-level cache (LLC)). Therefore, 6K saturation counters are required. For example, taking thread 0 as an example, an L1 hit counter and an L1 miss counter are set to record thread 0's hits to the L1 cache (L1 cache), an L2 hit counter and an L2 miss counter are set to record thread 0's hits to the L2 cache (L2 cache), and an LLC hit counter and an LLC miss counter are set to record thread 0's hits to the last-level cache (LLC). The situation for other threads is similar to that for thread 0 and will not be further described here. For example, when a new thread is started, its corresponding set of counters needs to be cleared.

[0128] For example, the cache access preference of a thread can be reflected based on the criticality of the instructions accessing the cache by the thread. For example, the cache access preference of a thread can be reflected based on the probability that the instructions accessing the cache by the thread cause pipeline stalls (e.g., Reorder Buffer (ROB) Stall). If the proportion of ROB Stall is small, it means that the criticality of the instructions accessing the cache by the thread is low. The lower the criticality, the lower the cache access preference of the thread. Bypassing threads with low cache access preference from the cache can also achieve the purpose of improving cache utilization.

[0129] For example, in step S101 , for the first cache, the probability that an instruction accessing the cache by the first thread causes pipeline stall is calculated.

[0130] For example, step S101 includes steps S401 to S403.

[0131] Step S401: for a first time period, record a fifth value of the pipeline stall caused by the instruction accessed by the first thread to the cache.

[0132] Step S402: for the first time period, record a sixth value of the pipeline stall caused by the instruction accessed by the first thread to the cache.

[0133] Step S403: Calculate the probability that the instruction accessed by the first thread causes pipeline stall based on the fifth value and the sixth value. For example, the probability that the instruction accessed by the first thread causes pipeline stall = fifth value / (fifth value + sixth value).

[0134] For example, for the relevant content of the first time period, reference may be made to the relevant description of the first time period in the above embodiment, which will not be repeated here.

[0135] For example, for the first cache, a fifth counter and a sixth counter are provided for the first thread. The fifth counter is configured to record the fifth value, and the sixth counter is configured to record the sixth value. For details about the fifth and sixth counters, refer to the description of the first and second counters in the above embodiments, and will not be repeated here.

[0136] For example, in a multi-threaded processor, each thread provides a counter group to record situations where instructions accessed by the corresponding thread cause pipeline stalls, thereby calculating the corresponding probability. For example, the relevant content of the counter group can refer to the relevant description of the counter group in the above embodiment, and will not be repeated here.

[0137] FIG7 is a schematic block diagram of a cache management device according to at least one embodiment of the present disclosure. As shown in FIG7 , the cache management device 10 according to the embodiment of the present disclosure includes an acquisition module 11 and a processing module 12 .

[0138] For example, the acquisition module 11 is configured to acquire a cache access preference of a first thread among multiple threads running in the multi-threaded processor to a first cache of the multi-threaded processor.

[0139] For example, the processing module 12 is configured to prevent the data block accessed by the first thread from being loaded into the first cache in response to the cache access preference of the first thread for the first cache satisfying the cache bypass condition.

[0140] For details about the acquisition module 11 and the processing module 12, please refer to the description of steps S101 and S102 shown in FIG2 of the embodiment of the cache management method, and will not be repeated here. The various modules shown in FIG7 can be implemented by software, hardware, firmware, or any combination thereof.

[0141] For example, in at least one embodiment of the present disclosure, the cache bypass condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

[0142] For example, in at least one embodiment of the present disclosure, the acquisition module 11 includes a first calculation module, which is configured to calculate a data block reuse rate based on the reuse of a data block loaded into the first cache by the first thread from the time it enters the first cache to the time it is replaced out of the first cache.

[0143] For example, in at least one embodiment of the present disclosure, the first computing module includes a first recording unit, a second recording unit, and a first computing unit. The first recording unit is configured to record, for a first time period, a first numerical value of the number of times a data block loaded by a first thread into a first cache is accessed again from the time it enters the first cache to the time it is replaced out of the first cache; the second recording unit is configured to record, for the first time period, a second numerical value of the number of times a data block loaded by a first thread into a first cache is not accessed again from the time it enters the first cache to the time it is replaced out of the first cache; and the first computing unit is configured to calculate a reuse rate of the data block loaded by the first thread into the cache based on the first numerical value and the second numerical value, wherein the data block reuse rate = first numerical value / (first numerical value + second numerical value).

[0144] For example, in at least one embodiment of the present disclosure, the first computing module also includes a first providing unit, which is configured to provide a first counter and a second counter for the first thread with respect to the first cache; the first counter is configured to record a first value; and the second counter is configured to record a second value.

[0145] For example, in at least one embodiment of the present disclosure, the first cache includes multiple cache lines, each of the multiple cache lines includes a data block, a first domain, and a second domain; the first domain is configured to indicate a thread that loads the data block into the first cache; and the second domain is configured to indicate whether the data block has been accessed again from the time it enters the first cache to the time it is replaced out of the first cache.

[0146] For example, in at least one embodiment of the present disclosure, when a data block among multiple data blocks is replaced out of the first cache, the first providing unit is further configured to, in response to the second domain corresponding to the data block indicating that the data block was accessed again from the time it entered the first cache to the time it was replaced out of the first cache, increment the first value corresponding to the thread indicated by the first domain corresponding to the data block by 1; and in response to the second domain corresponding to the data block indicating that the data block was not accessed again from the time it entered the first cache to the time it was replaced out of the first cache, increment the second value corresponding to the thread indicated by the first domain corresponding to the data block by 1.

[0147] For example, in at least one embodiment of the present disclosure, the acquisition module 11 includes a second calculation module, and the second calculation module is configured to calculate, for the first cache, a hit rate of the first thread accessing the cache.

[0148] For example, in at least one embodiment of the present disclosure, the second calculation module includes a third recording unit, a fourth recording unit, and a second calculation unit. The third recording unit is configured to record, for a first time period, a third value of the number of cache hits of the first thread accessing the first cache; the fourth recording unit is configured to record, for the first time period, a fourth value of the number of cache misses of the first thread accessing the first cache; and the second calculation unit is configured to calculate a hit rate of the first thread accessing the first cache based on the third value and the fourth value, where hit rate = third value / (third value + fourth value).

[0149] For example, in at least one embodiment of the present disclosure, the second computing module further includes a second providing unit, which provides a third counter and a fourth counter for the first thread with respect to the first cache; the third counter is configured to record a third value; and the fourth counter is configured to record a fourth value.

[0150] For example, in at least one embodiment of the present disclosure, the acquisition module 11 includes a third calculation module, which is configured to calculate, for the first cache, a probability that an instruction of the first thread accessing the cache will cause a pipeline stall.

[0151] FIG8 is a schematic block diagram of a processor provided by at least one embodiment of the present disclosure.

[0152] As shown in Figure 8, the processor 20 provided in the embodiment of the present disclosure includes a pipeline 21, a first cache 22, and a cache controller 23. For example, the processor 20 can be a single-core or multi-core central processing unit (CPU), a graphics processing unit (GPU), an AI accelerator, etc., and the present disclosure does not limit this.

[0153] In at least one embodiment of the present disclosure, the pipeline 21 is configured to run multiple threads; the present application has no limitation on the pipeline of the processor, for example, see the exemplary structure and working method shown in FIG. 1B .

[0154] In at least one embodiment of the present disclosure, the cache controller 23 is configured to obtain a cache access preference of a first thread among multiple threads running in a multi-threaded processor with respect to a first cache 22 of the multi-threaded processor, and, in response to the cache access preference of the first thread with respect to the first cache 22 satisfying a cache bypass condition, prevent a data block accessed by the first thread from being loaded into the first cache 22. For example, the cache controller 23 may be integrally formed with the first cache 22. For example, when a processor includes multiple levels of cache, each level of cache may have a corresponding cache manager.

[0155] For example, the first cache 22 includes multiple cache lines, each of the multiple cache lines includes a data block, a first domain and a second domain; the first domain is configured to indicate the thread that loads the data block into the first cache 22; the second domain is configured to indicate whether the data block has been accessed again from the time it enters the first cache 22 to the time it is replaced out of the first cache 22.

[0156] For the relevant contents of the first domain and the second domain, please refer to the relevant description of FIG5 in the embodiment of the cache management method, which will not be repeated here.

[0157] For example, the processor 20 provided in at least one embodiment of the present disclosure further includes a counter group. For example, the counter group includes a first counter and a second counter for the first cache 22. The first counter is configured to record a first value, and the second counter is configured to record a second value. The first value represents the number of times a data block loaded by the first thread into the first cache 22 is re-accessed from the time it enters the first cache 22 to the time it is replaced from the first cache 22; the second value represents the number of times a data block loaded by the first thread into the first cache 22 is not re-accessed from the time it enters the first cache 22 to the time it is replaced from the first cache 22. The counter group can be implemented, for example, using registers or static random access memory (SRAM).

[0158] For relevant contents about the counter group, please refer to the relevant descriptions of FIG. 4 and FIG. 6 in the embodiment of the cache management method, which will not be repeated here.

[0159] For example, the processor 20 provided in at least one embodiment of the present disclosure further includes a multi-level cache. For example, the multi-level cache includes a first cache 22. For example, the multi-level cache may include only a first-level cache (L1 Cache), or may further include a second-level cache (L2 Cache). In high-performance processors, a third-level cache (L3 Cache) or even a fourth-level cache (L4 Cache) may further include a more complex multi-level cache structure. The embodiments of the present disclosure do not impose any restrictions on this, and the multi-level cache may be configured according to actual needs.

[0160] At least one embodiment of the present disclosure further provides an electronic device, which includes the processor of any of the above embodiments. Figure 9 is a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure.

[0161] The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device 900 shown in FIG9 is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0162] For example, as shown in Figure 9, in some examples, the electronic device 900 includes a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which may include a processor of any of the above-mentioned embodiments, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the computer system are also stored in the RAM 903. The processor 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0163] For example, the following components may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 908 including, for example, a magnetic tape, hard disk, etc.; and a communication device 909, which may also include, for example, a network interface card such as a LAN card or modem. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or wired to exchange data, performing communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. Removable media 911, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in the drive 910 as needed, so that computer programs read therefrom can be installed into the storage device 908 as needed. Although FIG. 9 shows the electronic device 900 including various devices, it should be understood that implementation or inclusion of all of the devices shown is not required. More or fewer devices may alternatively be implemented or included.

[0164] For example, the electronic device 900 may further include a peripheral interface (not shown in the figure), etc. The peripheral interface may be various types of interfaces, such as a USB interface, a lightning interface, etc. The communication device 909 may communicate with a network and other devices through wireless communication, such as the Internet, an intranet and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communications may use any of a variety of communication standards, protocols, and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0165] For example, the electronic device 900 can be any device such as a mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, navigator, server, etc., or it can be any combination of data processing devices and hardware, and the embodiments of the present disclosure are not limited to this.

[0166] Although the present disclosure has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the embodiments of the present disclosure. Therefore, such modifications or improvements, as long as they do not depart from the spirit of the present disclosure, are within the scope of protection claimed by the present disclosure.

[0167] Regarding this disclosure, the following points need to be explained:

[0168] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0169] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present disclosure, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale.

[0170] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0171] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A cache management method, comprising: Obtaining a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor; as well as In response to the cache access preference of the first thread to the first cache meeting a cache bypass condition, the data block accessed by the first thread is prevented from being loaded into the first cache.

2. The cache management method according to claim 1, wherein: The cache bypass condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

3. The cache management method according to claim 1 or 2, wherein: The obtaining of a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor includes: The data block reuse rate is calculated according to a reuse condition of the data block loaded by the first thread into the first cache from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

4. The cache management method according to claim 3, wherein: Calculating the data block reuse rate according to a reuse situation of the data block loaded by the first thread into the first cache from the time the data block enters the first cache to the time the data block is replaced out of the first cache includes: For the first time period, Recording a first value of the number of times a data block loaded by the first thread into the first cache is re-accessed from the time the data block enters the first cache to the time the data block is replaced out of the first cache; a second value recording the number of times a data block loaded by the first thread into the first cache is not accessed again from the time the data block enters the first cache to the time the data block is replaced out of the first cache; and The data block reuse rate loaded into the cache by the first thread is calculated based on the first value and the second value, wherein the data block reuse rate=first value / (first value+second value).

5. The cache management method according to claim 4, wherein: The calculating of the data block reuse rate according to the reuse of the data block loaded by the first thread into the first cache from the time the data block enters the first cache to the time the data block is replaced out of the first cache further includes: For the first cache, a first counter and a second counter are provided for the first thread, wherein the first counter is configured to record the first value, and the second counter is configured to record the second value.

6. The cache management method according to claim 5, wherein: The first cache includes a plurality of cache lines, each cache line of the plurality of cache lines includes a data block, a first domain, and a second domain; The first domain is configured to instruct a thread to load the data block into the first cache; The second field is configured to indicate whether the data block is accessed again from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

7. The cache management method according to claim 6, wherein: When one of the plurality of data blocks is replaced from the first cache, In response to the second field corresponding to the data block indicating that the data block has been accessed again from the time the data block enters the first cache to the time the data block is replaced out of the first cache, incrementing the first value corresponding to the thread indicated by the first field corresponding to the data block by 1; as well as In response to the second field corresponding to the data block indicating that the data block has not been accessed again from the time it enters the first cache to the time it is replaced out of the first cache, the second value corresponding to the thread indicated by the first field corresponding to the data block is incremented by 1.

8. The cache management method according to claim 1 or 2, wherein: The obtaining of a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor includes: For the first cache, a hit rate of the first thread accessing the cache is calculated.

9. The cache management method according to claim 8, wherein: Calculating, for the first cache, a hit rate of the first thread accessing the cache includes: For the first time period, Recording a third value of the number of cache hits when the first thread accesses the first cache; Recording a fourth value of a number of cache misses when the first thread accesses the first cache; A hit rate of the first thread accessing the first cache is calculated based on the third value and the fourth value, wherein the hit rate=the third value / (the third value+the fourth value).

10. The cache management method according to claim 9, wherein: Calculating a hit rate of the first thread accessing the cache for the first cache further includes: For the first cache, providing a third counter and a fourth counter for the first thread; The third counter is configured to record the third value; The fourth counter is configured to record the fourth value.

11. The cache management method according to claim 1 or 2, wherein: The obtaining of a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor includes: For the first cache, a probability that an instruction accessing the cache by the first thread causes a pipeline stall is calculated.

12. A cache management device, comprising: An acquisition module configured to acquire a cache access preference of a first thread among a plurality of threads running in the multi-threaded processor to a first cache of the multi-threaded processor; as well as The processing module is configured to prevent a data block accessed by the first thread from being loaded into the first cache in response to the cache access preference of the first thread to the first cache satisfying a cache bypass condition.

13. The cache management device according to claim 12, wherein: The cache bypass condition includes that the cache access preference is lower than that of other threads and lower than a preset threshold.

14. The cache management device according to claim 12 or 13, wherein: The acquisition module includes: The first calculation module is configured to calculate a data block reuse rate according to a reuse condition of a data block loaded by the first thread into the first cache from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

15. The cache management device according to any one of claims 12 to 14, wherein: The acquisition module includes: The second calculation module is configured to calculate, for the first cache, a hit rate of the first thread accessing the cache.

16. A processor comprising: Pipelines, are configured to run multiple threads; First cache; as well as A cache controller is configured to obtain a cache access preference of a first thread among multiple threads running in a multi-threaded processor for a first cache of the multi-threaded processor, and in response to the cache access preference of the first thread for the first cache satisfying a cache bypass condition, prevent a data block accessed by the first thread from being loaded into the first cache.

17. The processor of claim 16, wherein: The first cache includes a plurality of cache lines, each cache line of the plurality of cache lines includes a data block, a first domain, and a second domain; The first domain is configured to instruct a thread to load the data block into the first cache; The second field is configured to indicate whether the data block is accessed again from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

18. The processor according to claim 16 or 17, further comprising a counter group, wherein The counter group includes a first counter and a second counter for the first cache, The first counter is configured to record a first value, The second counter is configured to record a second value, The first value represents the number of times a data block loaded by the first thread into the first cache is accessed again from the time the data block enters the first cache to the time the data block is replaced out of the first cache; The second value represents the number of times that the data block loaded by the first thread into the first cache is not accessed again from the time the data block enters the first cache to the time the data block is replaced out of the first cache.

19. The processor according to any one of claims 16 to 18, further comprising a multi-level cache, wherein: The multi-level cache includes the first cache.

20. An electronic device comprising the processor according to any one of claims 16 to 19.

Images