Cache memory system employing a multi-level hierarchical cache coherency architecture

Abstract

Cache memory systems employing a multi-level hierarchy cache coherency architecture, and related methods and computer readable media. A processor-based system includes multiple independent dies, each die having a processor and a local cache memory that logically forms part of a global cache memory in a system address space. To provide single point cache coherency in the global cache memory, the processor-based system includes a proxy cache controller circuit in each die, and a global cache controller circuit. The global cache controller circuit can communicate with the proxy cache controller circuits to maintain single point cache coherency in the global cache memory. Thus, a single point cache coherency protocol can be implemented. However, the proxy cache controller circuits can also be able to locally service memory requests within their own die only, to provide lower latency memory transactions, while still being able to maintain cache coherency.

Description

[0001] Priority Statement

[0002] This application claims priority to U.S. Patent Application Serial No. 18 / 422,951, filed January 25, 2024, entitled “CACHE MEMORY SYSTEM EMPLOYINGA MULTIPLE-LEVEL HIERARCHY CACHE COHERENCY ARCHITECTURE”, the contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to a cache memory system disposed in a processor-based system, the cache memory system including multiple caches organized in a cache hierarchy to support access to cached data, and more specifically to maintaining cache coherency in the cache memory system. This cache memory system is particularly suitable for multi-die processor-based systems. Background Technology

[0004] To provide low-latency retrieval of instructions and / or data (e.g., compared to the latency of transactions to the main memory system), microprocessors typically include cache memory resources. This cache memory system includes one or more cache memories (also referred to as "caches") that can be arranged in a hierarchical manner. For example, this cache memory system may include a Level 0 (L0) cache, a Level 1 (L1) cache, a Level 2 (L2) cache, and a Level 3 (L3) cache located within the central processing unit (CPU) or each CPU core. In multi-core or multi-processor systems, each CPU core or processor typically has L0 and / or L1 caches to store frequently accessed data to improve performance. The L0 cache can be the relatively smallest and lowest-latency cache, and these caches increase in capacity and latency until the L3 cache, which may have the largest capacity but also the longest latency compared to the other caches. In some respects, one or more levels of a cache hierarchy may have separate instruction and data caches (i.e., the L0 cache level may include separate L0 instruction and L0 data caches), while other levels of the cache hierarchy may contain both instructions and data. Some cache levels in a cache memory system may be "private" to the microprocessor, or, in the case of a multi-core microprocessor, private to one or more individual cores (meaning such private caches are only visible and accessible to the associated microprocessor or individual core). Other cache levels in a cache memory system, although physically located in the same location as a particular microprocessor, may be shared and used by one or more other microprocessors in the system.

[0005] Maintaining cache coherence is essential in cache memory systems. Cache coherence refers to the consistency of data stored in different caches within a cache memory system. The goal of cache coherence is to ensure that all processors in the system see a consistent view of memory, despite the existence of multiple caches that can store copies of data. When multiple cores or processors access and modify the same memory location, the data in their respective caches may become inconsistent. Cache coherence mechanisms are introduced to prevent or resolve these inconsistencies. For example, the Advanced Microprocessor Bus Architecture (AMBA) Coherent Hub Interface (CHI) specification from Arm Holdings, plc defines a hardware-based protocol for maintaining consistency of the system memory address space among multiple cache agents (e.g., CPU cores) in a processor-based system. The AMBA CHI specification addresses component systems in a processor-based system that have specific roles and responsibilities for maintaining consistency of the cache memory system within that processor-based system. The AMBA CHI specification requires that a hardware coherence manager (also referred to as a "home node" or "home agent") be defined as a point of coherence (PoC) in the cache memory system. The hardware consistency manager is responsible for ensuring the consistency of a set of system addresses in the processor-based system. According to the architecture in the AMBA CHI specification, if a requesting node (e.g., a CPU core) cannot service a memory request by accessing its local private cache, the requesting node sends the cache request to the hardware consistency manager, which acts as the sole consistency manager. The hardware consistency manager then initiates snoop requests to other remote nodes as instructed by the AMBA CHI protocol, propagating memory read and write requests to the appropriate memory controllers.

[0006] In a single-socket processor-based system where the CPU core is provided in a single semiconductor die (“die”), the hardware coherence manager can communicate snooping requests to the CPU core within that single die. However, in multi-die and / or multi-socket processor-based systems comprising two (2) or more processors, these processors share a single system memory address space contained in separate dies, thus requiring a die-to-die (D2D) communication interface to allow inter-die and inter-socket communication. To maintain cache coherence, the hardware coherence manager must communicate with multiple processors in separate dies via the D2D communication interface. This can cause significant performance issues for memory access due to the additional latency and bandwidth limitations of inter-die and inter-socket communication. Summary of the Invention

[0007] The aspects disclosed herein include cache memory systems employing a multi-level cache coherency architecture, which are particularly suitable for, but not limited to, multi-die processor-based systems. Related methods and computer-readable media are also disclosed herein. In an exemplary aspect, the processor-based system includes multiple independent dies (e.g., compute dies on multiple sockets), each die having a processor (e.g., having multiple central processing unit (CPU) cores) configured to execute instructions to perform tasks. Each die includes a local cache memory, as part of the cache memory system, for storing cached data to serve memory access requests from the processor (e.g., its CPU cores) on its die, where possible. In this way, lower-latency memory transactions can be supported within each die if memory access requests can be served solely by managing memory transactions in the local cache memory on its die. However, it is desirable that these local cache memories distributed across the multiple dies logically form a global cache memory and that cache coherency protocols can be implemented to maintain cache coherency. These local cache memories distributed across multiple bare dies logically form a global cache memory, which can also support a larger capacity global cache memory for the processor-based system.

[0008] In this regard, to provide single-point cache coherence in the processor-based system while providing a global cache memory logically comprising local cache memories distributed across the multiple dies, the processor-based system includes multiple proxy cache controller circuits located on each die and a global cache controller circuit. Each of these proxy cache controller circuits is configured to serve memory access requests from the processor on its die, spanning the entire system address space of the processor-based system. The global cache controller circuit is configured to communicate with each of these proxy cache controller circuits to maintain cache coherence on the global cache memory logically formed by these distributed local cache memories. The global cache controller circuit may be contained on a separate die different from these dies, or contained within one of the multiple dies. Therefore, if a memory access request can be adequately serviced within a given bare die while maintaining cache coherence of the global cache memory (e.g., a read request that successfully hits a cache line in the local cache memory, or access after acquiring exclusive ownership of the cache line), then, while maintaining cache coherence, the proxy cache controller circuit can autonomously service memory access requests from the local cache memory contained within its bare die without communicating with the global cache controller circuit, thereby supporting lower-latency memory transactions. However, if a memory access request cannot be serviced within a given bare die (e.g., a write request for a cache line not exclusively owned by a local cache memory), then the proxy cache controller circuit is configured to communicate with the global cache controller circuit. The global cache controller circuit is configured to communicate hierarchically with other proxy cache controller circuits that manage access to local cache memories, each of which is part of the global cache memory, as needed to maintain cache coherence of the global cache memory.

[0009] In this way, as an example, a cache coherence protocol based on single-point cache coherence can be implemented in a multi-die processor-based system through the global cache controller circuitry. However, where feasible, these proxy cache controller circuits in each die can still serve memory access requests contained within their respective dies to achieve lower-latency memory transactions. In other words, since these proxy cache controller circuits can only serve memory access requests that can be performed within their respective dies and still maintain the cache coherence of the global cache memory, there is no need to configure these proxy cache controller circuits to always communicate memory access requests through the global cache controller circuitry to maintain the cache coherence of the global cache memory. Communicating with the global cache controller circuitry for memory transactions incurs the cost of additional latency for those memory transactions.

[0010] In other exemplary aspects, the cache memory system is configured to perform snooping in a hierarchical manner. In one example, each proxy cache controller circuit maintains a local cache line directory that tracks the cache locations of the cache lines between CPU cores on the same die. The global cache controller circuit also maintains a global cache line directory that tracks the cache locations of the cache lines between these local cache memories that logically form the global cache memory in these dies. If other cache devices are present in the processor-based system, the local cache memories associated with these other cache devices are also tracked by the global cache controller circuit using the global cache line directory. Therefore, when processing a coherent memory access request, each proxy cache controller circuit is configured to access its local cache line directory to determine whether the memory access request is for a cache line contained within its local cache memory and whether the proxy cache controller circuit needs to initiate a snooping request through the global cache controller circuit. The global cache controller circuitry is configured to send snooping requests to these proxy cache controller circuitry (or other coupled cache devices) on these bare dies, which in turn can be communicated to a processor on the bare die to update the local cache memory on that bare die.

[0011] In other exemplary aspects, the processor-based system includes a global cache controller circuit for each system address space addressable by a processor. Therefore, if all processors on these dies are configured to address the entire system address space of the processor-based system, only one global cache controller circuit is required. However, if any processor on these dies is configured to address different system address spaces of the processor-based system, multiple global cache controller circuits can be provided and allocated to each specific system address space, and the proxy cache controller circuit allocated to a given processor is configured to communicate with the global cache controller circuit associated with the system address space of that processor.

[0012] In this regard, in one exemplary aspect, a semiconductor die (die) is provided. The die includes a processor comprising a plurality of CPU cores and a local cache memory comprising a plurality of cache memories, each cache memory being allocated to one of the plurality of CPU cores. The die also includes a proxy cache controller circuitry communicatively coupled to the plurality of CPU cores. The proxy cache controller circuitry is configured to: receive the memory access request from the first CPU core in response to a first CPU core being unable to serve a memory access request containing a memory address in a first cache memory allocated to the first CPU core. The proxy cache controller circuitry is further configured to: determine whether the memory access request can be served in any other cache memory among the plurality of cache memories allocated to other CPU cores of the plurality of CPU cores. In response to determining that the memory access request cannot be served in any of the plurality of other cache memories, the proxy cache controller circuitry is further configured to: communicate the memory access request to a global cache controller circuitry located on a second bare die, and update the cache state of the memory address associated with the memory access request.

[0013] In another exemplary aspect, a method for maintaining cache coherence in a multi-level cache memory system is provided. The method includes: in a proxy cache controller circuit located on a first die and communicatively coupled to a plurality of CPU cores, receiving a memory access request containing a memory address from a first CPU core in response to a first CPU core being unable to serve a memory access request in a first cache memory among a plurality of cache memories allocated to the first CPU core. The method further includes: determining whether the memory access request can be served in any other cache memory among the plurality of cache memories respectively allocated to other CPU cores of the plurality of CPU cores. In response to determining that the memory access request cannot be served in any of the plurality of other cache memories, the method further includes: communicating the memory access request to a global cache controller circuit located on a second die and updating the cache state of the memory address associated with the memory access request.

[0014] In another exemplary aspect, a non-transient computer-readable medium is provided. The non-transient computer-readable medium stores computer-executable instructions that, when executed by a proxy cache controller circuit located on a first die and communicatively coupled to a plurality of CPU cores, cause the proxy cache controller circuit to: receive the memory access request containing a memory address from the first CPU core in response to a first CPU core being unable to serve a memory access request in a first cache memory among a plurality of cache memories allocated to the first CPU core; determine whether the memory access request can be served in any other cache memory among the plurality of cache memories respectively allocated to the plurality of other CPU cores; and in response to determining that the memory access request cannot be served in any of the plurality of other cache memories: communicate the memory access request to a global cache controller circuit located on a second die; and update the cache state of the memory address associated with the memory access request.

[0015] In another exemplary aspect, a processor-based system is provided. The processor-based system includes a first semiconductor die (die) comprising: a first processor including one or more first CPU cores; a first local cache memory including one or more first cache memories, each of the one or more first cache memories being allocated to a first CPU core among the one or more first CPU cores; and a first proxy cache controller circuitry communicatively coupled to the one or more first CPU cores. The processor-based system further includes a second die comprising: a second processor including one or more second CPU cores; a second local cache memory including one or more second cache memories, each of the one or more second cache memories being allocated to a second CPU core among the one or more second CPU cores; and a second proxy cache controller circuitry communicatively coupled to the one or more second CPU cores. The processor-based system further includes a cache memory system comprising: a global cache controller circuitry; the first proxy cache controller circuitry; and the second proxy cache controller circuitry. The first CPU core in the one or more first CPU cores is configured to: initiate a memory access request to a memory address in a first cache memory of the one or more first cache memories allocated to the first CPU core; and, in response to the first CPU core being unable to serve the memory access request in the first cache memory, communicate the memory access request to the first proxy cache controller circuit. The first proxy cache controller circuit in the first CPU core is configured to: determine whether the memory access request can be served in any of the one or more first cache memories of the other first CPU cores allocated to the one or more first CPU cores; and, in response to determining that the memory access request cannot be served in any of the plurality of other first cache memories: communicate the memory access request to the global cache controller circuit. The global cache controller circuit is configured to determine whether the memory access request can be served in any of the one or more second cache memories. In response to determining that the memory access request can be served in one of the one or more second cache memories, the global cache controller circuitry is configured to communicate the memory access request to the second proxy cache controller circuitry so that it can be served in the second cache memory.The global cache controller circuit is configured to update the cache state of the memory address associated with the memory access request based on the memory access request being served in the second local cache memory.

[0016] Those skilled in the art will understand the scope of this disclosure and recognize its other aspects after reading the following detailed description of preferred embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings form part of this specification, illustrate several aspects of this disclosure, and together with the specification serve to explain the principles of this disclosure.

[0018] Figure 1 is a block diagram of an exemplary processor-based system comprising a multi-level caching system for multiple bare dies (e.g., compute dies), each bare die having a processor, a local cache memory as part of a global cache memory, and a proxy cache controller circuit, wherein each proxy cache controller circuit is capable of serving memory requests locally only within its bare die, where cache coherency is possible, to provide lower latency memory transactions, and wherein the processor-based system further includes a global cache controller circuit configured to communicate with these proxy cache controller circuits to maintain single-point cache coherency in the global cache memory;

[0019] Figure 2A is an exemplary flowchart illustrating how the proxy cache controller circuit in the multi-level cache system of Figure 1 executes a memory read request while maintaining cache coherency in its bare die, and is configured to communicate the memory access request to the global cache controller circuit to maintain cache coherency in the global cache memory when the request cannot be served in its bare die.

[0020] Figure 2B is an exemplary flowchart illustrating how the proxy cache controller circuit in the multi-level cache system of Figure 1 executes a memory write request to obtain exclusive ownership of the cache state of the data to be written while maintaining cache consistency in its bare die, and is configured to communicate the memory write request to the global cache controller circuit to maintain cache consistency in the global cache memory.

[0021] Figure 2C is an exemplary flowchart illustrating how the proxy cache controller circuit in the multi-level cache system of Figure 1 executes a memory atomic request while maintaining cache consistency in its bare die, and is configured to communicate the memory atomic request to the global cache controller circuit to maintain cache consistency in the global cache memory when the request cannot be served in its bare die.

[0022] Figure 3A illustrates an exemplary process of executing a memory read request in the multi-level cache system of Figure 1, which results in a cache miss to the local cache memory in the requester's die and involves communication between the proxy cache controller circuitry in the die and the global cache controller circuitry to retrieve the requested valid cache line from the system memory.

[0023] Figure 3B is an exemplary process of executing a memory read request in the multi-level cache system of Figure 1, which results in a cache hit to the local cache memory located in the same die as the requested central processing unit (CPU) core.

[0024] Figure 3C illustrates an exemplary process of executing a memory read request in the multi-level cache system of Figure 1, which results in a cache miss in the local cache memory in the die where the requested CPU core resides, and involves communication between the proxy cache controller circuitry in that die and the global cache controller circuitry to retrieve the requested valid cache line from the local cache memory in another die.

[0025] Figure 4A illustrates an exemplary process of executing a memory write request in the multi-level cache system of Figure 1, which results in a cache hit owned by an agent in the local cache memory located in the same bare die as the requesting CPU core.

[0026] Figure 4B illustrates an exemplary process of executing a memory write request in the multi-level cache system of Figure 1, which results in a cache miss to the local cache memory in the bare die where the requesting CPU core resides.

[0027] Figure 4C is an exemplary process in the multi-level cache system of Figure 1, in which a memory write-back (i.e., storage) request is executed due to the eviction of a local cache memory, and the process involves communication between the proxy cache controller circuitry in the bare die and the global cache controller circuitry.

[0028] Figure 5A is an exemplary process for performing an atomic memory request, which involves accessing local cache memory in the same bare die through a proxy cache controller circuit.

[0029] Figure 5B is an exemplary process for performing an atomic memory request, which involves accessing a local cache memory in another bare die via a global cache controller circuit.

[0030] Figure 6 is a block diagram of another exemplary computer system including a processor-based system. The processor-based system includes multiple bare dies, each bare die having a processor, a local cache memory as part of a global cache memory of a multi-level cache coherence system, and a proxy cache controller circuit. Each proxy cache controller circuit is capable of serving memory requests locally only within its bare die, where cache coherence can be maintained, to provide lower latency memory transactions. The processor-based system also includes a global cache controller circuit configured to communicate with these proxy cache controller circuits to maintain single-point cache coherence in the global cache memory. Detailed Implementation

[0031] The various aspects disclosed herein include cache memory systems employing a multi-level cache coherence architecture, particularly suitable for (but not limited to) multi-die processor-based systems. Related methods and computer-readable media are also disclosed. In an exemplary aspect, the processor-based system includes multiple separate dies (e.g., compute dies on multiple sockets), each die having a processor (e.g., having multiple central processing unit (CPU) cores) configured to execute instructions to perform tasks. Each die includes a local cache memory as part of the cache memory system for storing cached data to service memory access requests issued by the processor on that die (e.g., its CPU cores), where possible. In this way, lower-latency memory transactions can be supported within each die if memory access requests can complete memory transactions solely through the local cache memory on their die. However, it is desirable that these local cache memories distributed across multiple dies logically form a global cache memory, and that a cache coherence protocol can be implemented to maintain cache coherence. By logically forming a global cache memory from local cache memories distributed across multiple bare dies, it is also possible to support a larger capacity global cache memory for this processor-based system.

[0032] In this regard, to provide single-point cache coherency for the processor-based system while simultaneously providing a global cache memory logically comprising local cache memories distributed across multiple dies, the processor-based system includes multiple proxy cache controller circuits and a global cache controller circuit on each die. These proxy cache controller circuits are configured to service memory access requests from the processor on their respective die, spanning the entire system address space of the processor-based system. The global cache controller circuit is configured to communicate with each proxy cache controller circuit to maintain cache coherency of the global cache memory logically formed by these distributed local cache memories. The global cache controller circuit may be contained on a separate die separate from these dies, or within one of the multiple dies. Therefore, if a memory access request can be fully serviced within a given bare die while maintaining cache coherence of the global cache memory (e.g., a read request that successfully hits a cache line in the local cache memory, or an access made after acquiring exclusive ownership of that cache line), the proxy cache controller circuitry can autonomously service the memory access request from the local cache memory included in its own bare die, without communicating with the global cache controller circuitry, thus supporting lower-latency memory transactions, provided cache coherence is maintained. However, if a memory access request cannot be serviced within a given bare die (e.g., a write request to a cache line not exclusively owned by the local cache memory), the proxy cache controller circuitry is configured to communicate with the global cache controller circuitry. The global cache controller circuitry is configured to communicate hierarchically with other proxy cache controller circuitry as needed, which manage access to their respective local cache memories, which are part of the global cache memory, to maintain cache coherence of that global cache memory.

[0033] In this way, as an example, a cache coherence protocol based on single-point cache coherence can be implemented in a multi-die processor-based system using a global cache controller circuit. However, the proxy cache controller circuit in each die can still service memory access requests contained within its die, where possible, to achieve lower-latency memory transactions. In other words, because the proxy cache controller circuit can service memory access requests only within its die while still maintaining global cache coherence, it is not necessary to configure the proxy cache controller circuit to always communicate memory access requests through the global cache controller circuit to maintain global cache coherence. Communicating with the global cache controller circuit for memory transactions incurs the additional latency cost of those memory transactions.

[0034] in this regard, Figure 1 This is a block diagram of an exemplary multi-die processor-based system 100 (“processor-based system 100”), which includes a multi-level cascade cache system 102. As discussed in more detail below, the multi-level cascade cache system 102 is configured to maintain cache coherence among cache memories distributed across a plurality of dies 104(0)-104(D), each of which has a respective first processor 106(0)-106(D). In this example, dies 104(0)-104(D) are also referred to as “computing” dies 104(0)-104(D). A “computing die” refers to any die containing a processing circuitry system, such as a processor and / or CPU core for executing instructions to perform computer-related tasks, such as CPU tasks, graphics processing unit (GPU) tasks, etc. In this example, the processor-based system 100 is a multi-socket processor-based system comprising multiple processors 106(0)-106(D) distributed across independent computing dies 104(0)-104(D). Figure 1In the diagram, die 104(0) is shown as “Computation Die A”, and die 104(D) is shown as “Computation Die B”. Multi-socket processor-based systems offer several advantages, including higher processing power, scalability through additional slots, and the ability to support larger memory capacities within a single address space—where memory circuitry can be shared in a shared address space but distributed across multiple computation dies 104(0)–104(D) and / or other dies. Each processor 106(0)–106(D) includes its own multiple central processing unit (CPU) cores 108(0)(0)–108(0)(C) to 108(D)(0)–108(D)(C), all configured to execute instructions to perform computation-related tasks within their respective computation dies 104(0)–104(D). Please note that although two (2) compute dies 104(0) and 104(D) are shown in the figure, each with its own processor 106(0) and 106(D), the processor-based system 100 is shown as having a total of “D+1” compute dies 104(0)-104(D) and their respective processors 106(0)-106(D), where “D” can be any positive integer. Furthermore, although... Figure 1 The diagram shows that each of the respective processors 106(0)-106(D) contains four (4) CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C), but note that each processor 106(0)-106(D) is shown as having a total of “C+1” CPU cores, where “C” can be any positive integer.

[0035] like Figure 1As further shown, in the processor-based system 100, a multi-level cache system 102 forms a global cache memory 110, which includes respective local cache memories 112(0)-112(D) disposed on each compute die 104(0)-104(D). In this example, each of the local cache memories 112(0)-112(D) includes a plurality of cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C), which are associated with or allocated to (e.g., contained in) one or more CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) in the respective processors 106(0)-106(D). In this way, each CPU core 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) can locally access cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) to serve memory access requests. However, if a memory access request for a given CPU core 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) to its respective cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) results in a cache miss, then another cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) in its respective CPU core 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) can subsequently be accessed as the next level cache memory. If a cache miss also occurs in the cache memory 114(0)(0)-114(0)(C)-114(D)(0)-108(D)(C) associated with different CPU cores 108(0)(0)-108(0)(C)-108(D)(0)-108(D)(C) in its processor 106(0)-106(D), the memory access request may be communicated off-die to an input / output (I / O) die 116 that may contain the requested cached data, and / or to the local cache memory 112(0)-112(D) of another processor 106(0)-106(D) located on its respective compute die 104(0)-104(D).Please note that bare cores 104(0)-104(D) may also contain other cache memories that are at a different, higher cache level relative to the local cache memories 112(0)-112(D) and can be accessed by their respective proxy cache controller circuits 120(0)-120(D) to serve cache requests. The disclosure herein is by no means limited to including local cache memories in the respective CPU cores 108(0)(0) through 108(D)(C).

[0036] In this way, if the local cache memories 112(0)-112(D) can service the memory access request, the operation can be performed without sending such requests to external compute dies, thereby reducing memory access latency. However, if the local cache memories 112(0)-112(D) cannot service the memory access request, a request can still be sent to external compute dies to attempt to service the memory access request from another local cache memory 112(0)-112(D) and / or from system memory 118. Therefore, the global cache memory 110, which includes distributed local cache memories 112(0)-112(D), provides a single addressable memory system to reduce memory addressing complexity in the memory system of the processor-based system 100, while the local cache memories 112(0)-112(D) are distributed across multiple compute dies 104(0)-104(D) to improve processing power and scalability. By logically forming a global cache memory 110 from the local cache memories 112(0)-112(D) distributed across multiple compute bare dies 104(0)-104(D), a larger capacity global cache memory 110 can be supported for the processor-based system 100. However, since the logical global cache memory 110 composed of the local cache memories 112(0)-112(D) can be addressed across the entire system memory address space of the processor-based system 100, cache coherence of the global cache memory 110 and its local cache memories 112(0)-112(D) must also be maintained to ensure the integrity of the cache system.

[0037] In this regard, in order to provide a single point of cache coherency for the multi-level cache system 102, while providing a global cache memory 110 that logically includes local cache memories 112(0)-112(D) distributed across the various compute dies 104(0)-104(D), the processor-based system 100 in this example includes a proxy cache controller circuit 120(0)-120(D) on each of the respective compute dies 104(0)-104(D), and a global cache controller circuit 122 coupled to the respective CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) on the respective compute dies 104(0)-104(D). In this example, the global cache controller circuit 122 resides on a separate I / O die 116, but in another example it could reside on any of the compute dies 104(0)-104(D). As discussed in more detail below, the proxy cache controller circuits 120(0)-120(D) are all configured to serve memory access requests across the entire system address space of the processor-based system 100, targeting their respective local cache memories 112(0)-112(D) on their compute dies 104(0)-104(D). In this way, if a cache miss occurs for a given memory access request of CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) to their respective cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C), the memory access request can be communicated to the proxy cache controller circuit 120(0)-120(D) on their respective compute die 104(0)-104(D), which in turn can determine whether the memory access request can be serviced by another cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) in their respective local cache memories 112(0)-112(D) on their respective compute dies 104(0)-104(D).Each compute die 104(0)-104(D) includes its own on-die network circuitry 124(0)-124(D) for communication requests between the CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) on its compute die 104(0)-104(D) and their respective proxy cache controller circuitry 120(0)-120(D) and other CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) on its compute die 104(0)-104(D). In this way, each proxy cache controller circuit 120(0)-120(D) on its respective compute bare die 104(0)-104(D) is able to service memory access requests between its respective local cache memories 112(0)-112(D) and also maintain cache coherence of its respective local cache memories 112(0)-112(D).

[0038] Continue to refer to Figure 1As described above, if a cache miss occurs because the proxy cache controller circuits 120(0)-120(D) are unable to service a memory access request for the local cache memory 112(0)-112(D) on their respective compute dies 104(0)-104(D), the proxy cache controller circuits 120(0)-120(D) forward the request to the global cache controller circuit 122 outside the die. In this respect, each compute die 104(0)-104(D) includes a respective die-to-die (D2D) interface circuit 126(0)-126(D) coupled to the die-in-die network circuit 124(0)-124(D) of its respective compute die 104(0)-104(D). The proxy cache controller circuits 120(0)-120(D) are configured to communicate memory access requests to the global cache controller circuit 122 via the following communication methods: communicating via their in-die network circuits 124(0)-124(D) to the D2D interface circuits 126(0)-126(D) on their respective compute dies 104(0)-104(D), and then communicating to the global cache controller circuit 122 via their respective D2D interface circuits 128(0)-128(D) on the I / O die 116 containing the global cache controller circuit 122. The I / O die 116 includes in-die network circuitry 130 to provide an interface between the D2D interface circuits 128(0)-128(D) and the global cache controller circuit 122. Global cache controller circuit 122 is configured to communicate memory access requests to system memory 118 via memory controller 129 coupled to system memory 118 and respective memory interface circuits 131, in order to service memory access requests issued to system memory 118, for example, in the event of a complete cache miss and / or memory write-back to all local cache memories 112(0)-112(D) in global cache memory 110. Global cache controller circuit 122 is also configured to communicate with each agent cache controller circuit 120(0)-120(D) to maintain cache coherence of the global cache memory 110 logically formed by these distributed local cache memories 112(0)-112(D).

[0039] Therefore, if a memory access request can be fully serviced within a given compute bare die 104(0)-104(D), and its respective proxy cache controller circuits 120(0)-120(D) can still maintain cache coherence of the global cache memory 110, then the proxy cache controller circuits 120(0)-120(D) can autonomously service the memory access request from their respective local cache memories 112(0)-112(D), and (in this example) there is no need to communicate with the global cache controller circuit 122 outside the bare die to support lower latency memory transactions. An example of this service is a read request that successfully hits a cache line in a given local cache memory 112(0)-112(D), or an access made after obtaining exclusive ownership of that cache line. However, if a memory access request cannot be served within a given compute bare die 104(0)-104(D), and its respective proxy cache controller circuits 120(0)-120(D) cannot otherwise maintain cache coherence of the global cache memory 110, then the proxy cache controller circuits 120(0)-120(D) are configured to communicate with the global cache controller circuit 122. An example of this service is a write request to a cache line exclusively owned by an unaccessed local cache memory 112(0)-112(D). The global cache controller circuit 122 is configured to communicate hierarchically with other proxy cache controller circuits 120(0)-120(D) on other compute bare dies 104(0)-104(D) as needed. These proxy cache controller circuits manage access to their respective local cache memories 112(0)-112(D) which are part of the global cache memory 110 in order to maintain cache coherence of the global cache memory 110.

[0040] In this way, as an example, a cache coherence protocol based on single-point cache coherence can be implemented in a multi-die processor-based system 100 through a global cache controller circuit 122, while the proxy cache controller circuits 120(0)-120(D) in each compute die 104(0)-104(D) can still serve memory access requests contained in the local cache memory 112(0)-112(D) of their compute dies 104(0)-104(D) to achieve lower latency memory transactions. In other words, because the proxy cache controller circuits 120(0)-120(D) can service memory access requests only within their compute die 104(0)-104(D) while still maintaining cache coherence of the global cache memory 110, there is no need to configure the proxy cache controller circuits 120(0)-120(D) to always communicate memory access requests to the external global cache controller circuit 122 to maintain cache coherence of the global cache memory 110. Communicating with the global cache controller circuit 122 for memory transactions incurs the cost of additional latency for that memory transaction.

[0041] Figure 2A It is explained in Figure 1 A flowchart of an exemplary process 200 in which proxy cache controller circuits 120(0)-120(D) execute memory access request 132 in a multi-level cache system 102 of a processor-based system 100. Figure 2AIn the exemplary process 200, the process 200 may be a memory read request for executing a memory read request process 200R. As described above, in response to a cache miss for a memory access from CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) to their allocated cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C), the proxy cache controller circuits 120(0)-120(D) are able to execute a memory access request in order to maintain cache coherence in their respective local cache memories 112(0)-112(D) in their compute die 104(0)-104(D). If the proxy cache controller circuits 120(0)-120(D) are unable to execute the memory access request 132 in response to a cache miss in their respective compute dies 104(0)-104(D), then the proxy cache controller circuits 120(0)-120(D) are configured to communicate the memory access request 132 to the global cache controller circuit 122, so that it can be serviced by another proxy cache controller circuit 120(0)-120(D) in another compute dies 104(0)-104(D) to maintain cache coherence in the global cache memory 110. If the memory access request 132 cannot be serviced by another proxy cache controller circuit 120(0)-120(D) in another compute dies 104(0)-104(D), then the memory access request 132 can be serviced by the system memory 118.

[0042] In this regard, such as Figure 2A As shown, taking the proxy cache controller circuit 120(0) in the compute bare die 104(0) as an example (but applicable to any proxy cache controller circuit 120(0)-120(D)), the proxy cache controller circuit 120(0) receives a memory access request 132 (e.g., memory read request 132R) containing a memory address 134 (e.g., memory read address 134R) from the first CPU core 108 among the plurality of CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C). Figure 2AStep 202). In response to the first CPU core 108 being unable to service the memory access request 132 within the first cache memory 114 of the plurality of cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) allocated to the first CPU core 108, the proxy cache controller circuit 120(0) receives the memory access request 132 (e.g., memory read request 132R). Figure 2A Step 202 in the process. The process 200, 200R also includes the agent cache controller circuit 120(0) determining whether the memory access request 132 (e.g., memory read request 132R) can be served (e.g., cache hit) in any of the plurality of cache memories 114 (0)(0)(0)(0)(0)(0)(C) to 114(D)(0)(0)(108(D)(C)) allocated to other CPU cores 108 (0)(0)(0)(0)(0)(C) to 108(D)(0)(108(D)(C)). Figure 2A Step 204). In response to determining that the memory access request 132 (e.g., memory read request 132R) cannot be serviced in any other cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) (e.g., a cache miss) ( Figure 2A In step 206), the proxy cache controller circuit 120(0) can directly communicate the memory access request 132 (e.g., read access request 137R), or cause the memory access request 132 (e.g., read access request 137R) to be communicated to the global cache controller circuit 122 located on the second (I / O) bare die 116. Figure 2A Step 208 in the middle.

[0043] The memory access snooping request 136 communicating from the proxy cache controller circuit 120(0) to the global cache controller circuit 122 may also communicate or contain other information, such as information to identify the proxy cache controller circuit 120(0). In this way, the global cache controller circuit 122 can know from which proxy cache controller circuits 120(0)-120(D) received the request, and conversely, which proxy cache controller circuits 120(0)-120(D) should respond to in response to the received request. For example, in the case of memory read request 132R, if the proxy cache controller circuits 120(0)-120(D) issue a memory access snoop request 136 to the global cache controller circuit 122 in order to read data 136R, the global cache controller circuit 122 will use the identification of the proxy cache controller circuits 120(0)-120(D) to know which proxy cache controller circuit 120(0)-120(D) should return the read data 136R to.

[0044] As discussed in more detail below, if service cannot be provided in other cache memories 114(0)(1)-114(D)(C), the global cache controller circuit 122 can service the memory access request 132 (e.g., memory read request 132R) from another local cache memory 112(1)-112(D) or system memory 118 in another compute die 104(0)-104(D). The proxy cache controller circuit 120(0) is configured to update the cache state of the memory address associated with the memory access request 132 (e.g., memory read request 132R) to indicate that the cache memory 114(0) on the compute die 104(0) where the proxy cache controller circuit 120(0) resides has a copy of the data 138 (e.g., read data 138R) of the memory access request 132 (e.g., memory read request 132R). Figure 2A Step 210 in the middle.

[0045] Figure 2B It is explained in Figure 1In the multi-level cache system 102 of the processor-based system 100, a flowchart illustrates an exemplary memory write request process 200W in which proxy cache controller circuits 120(0)-120(D) execute a memory write request 132W to acquire exclusive ownership of the memory write address 134W to be written. This memory write request process 200W is capable of executing the memory write request process 200W when each of the proxy cache controller circuits 120(0)-120(D) does not have exclusive ownership of the memory write address 134W in its respective local cache memory 112(0)-112(D). As described above, in response to a memory write access from CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) not having exclusive ownership of the memory write address 134W in the cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) allocated to them, the proxy cache controller circuits 120(0)-120(D) are able to execute the memory write request in order to maintain cache coherence in their respective local cache memory 112(0)-112(D) in their compute die 104(0)-104(D). If the proxy cache controller circuits 120(0)-120(D) are unable to execute the memory write request 132W with the exclusive ownership cache state of the memory write address 134W in their local cache memory 112(0)-112(D) on their compute bare die 104(0)-104(D), then the proxy cache controller circuits 120(0)-120(D) are configured to communicate an exclusive snoop request for the memory write request 132W to the global cache controller circuit 122 to request the exclusive cache state of the memory write address 134W to be written, thereby obtaining the exclusive ownership cache state of the memory write address 134W from another proxy cache controller circuit 120(0)-120(D) or system memory 118 in another compute bare die 104(0)-104(D) in order to maintain cache coherence in the global cache memory 110.

[0046] In this regard, such as Figure 2BAs shown, taking the proxy cache controller circuit 120(0) in the compute die 104(0) as an example (but applicable to any proxy cache controller circuit 120(0)-120(D)), the proxy cache controller circuit 120(0) receives a memory write request 132W from the first CPU core 108 among the plurality of CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C), the memory write request 132W containing an exclusive ownership request 140 for the memory write request 132W containing the memory write address 134W. Figure 2B Step 220). In response to whether the cache line for the memory write address 134W in the cache memory 114(0)(0)-114(0)(C) corresponding to the memory write request 132W is in a shared state or does not exist in the first cache memory 114, the proxy cache controller circuit 120(0) is able to receive the memory write request 132W. The proxy cache controller circuit 120(0) determines whether the write data 138W for the memory write request 132W is in a shared state in the other cache memories 114 of the plurality of CPU cores 108 allocated to the other CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C). Figure 2B Step 222 in the middle.

[0047] In response to determining that the write data 138W for the memory write request 132W is in a shared state within one or more other cache memories 114 of the plurality of cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C), Figure 2BIn step 224), the proxy cache controller circuit 120(0) initiates snoop invalidate 142 to the one or more other cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) to invalidate the write data 138W in the one or more other cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) corresponding to the memory write address 134W of the memory write request 132W. Figure 2B Step 226 in the process. Next, the proxy cache controller circuit 120(0) communicates to the first CPU core 108 the exclusive ownership status of the memory write address 134W for the memory write request 132W. Figure 2B Step 228). In response to receiving an exclusive ownership state for memory write address 134W for the memory write request 132W, the first CPU core 108 then writes the write data 138W for the memory write request 132W into the first cache memory 114. Figure 2B Step 230 in the middle.

[0048] As discussed in more detail below, if in Figure 2B In step 226, it is determined that the write data 138W for the memory write request 132W is not in a shared state within one or more other cache memories 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C). Figure 2B In step 224), the proxy cache controller circuit 120(0) communicates the memory write request 132W to the global cache controller circuit 122 so that it can obtain the exclusive ownership cache state of the memory write address 134W from other local cache memories 112(1)-112(D) in other compute bare dies 104(1)-104(D) to maintain cache consistency.

[0049] Figure 2C It is explained in Figure 1In the multi-level cache system 102, the proxy cache controller circuits 120(0)-120(D) execute an exemplary process 200A of memory atomic request 132A while maintaining cache coherence in their compute bare dies 104(0)-104(D). The proxy cache controller circuits 120(0)-120(D) are configured to communicate memory atomic requests 132A that cannot be serviced within their compute bare dies 104(0)-104(D) to the global cache controller circuit 122 to maintain cache coherence in the global cache memory 110.

[0050] In this regard, such as Figure 2C As shown, taking the proxy cache controller circuit 120(0) in compute die 104(0) as an example (but applicable to any proxy cache controller circuit 120(0)-120(D)), the process 200A includes the proxy cache controller circuit 120(0) receiving a memory atomic request 132A. The memory atomic request 132A contains a memory write address 134A from the CPU core 108 on its compute die 104(0) and an atomic operation 144. In response to the cache line in cache memory 114 allocated to the CPU core 108 corresponding to the memory atomic request 132A being in an invalid state I, the proxy cache controller circuit 120(0) receives the memory atomic request 132A. Figure 2C Step 240). The proxy cache controller circuit 120(0) determines whether the memory atomic request 132A is a cache hit in another cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) in the computed bare die 104(0)-104(D). Figure 2C (Step 242 in the original text). For example, the proxy cache controller circuit 120(0) can first determine whether the memory atomic request 132A is a cache hit in another cache memory 114(0)(0)-114(0)(C) in the same compute die 104(0). If not, the proxy cache controller circuit 120(0) can communicate the memory atomic request 132A to the global cache controller 122 to determine whether the memory atomic request 132A is a cache hit in another cache memory 114(1)(0)-114(D)(C) in another separate compute die 104(1)-104(D).

[0051] Continue to refer to Figure 2C For example, in response to determining that the memory atomic request 132A is a cache hit in another cache memory 114(0)(0)-114(0)(C) within the same compute die 104(0), Figure 2C In step 244), the proxy cache controller circuit 120(0) communicates a clean unique snoop request 140 for the memory write address 134A for the memory atomic request 132A to the CPU core 108 assigned to the other cache memory 114(0)(0)-114(0)(C) to 114(D)(0)-114(D)(C) where the cache hit occurred. Figure 2C Step 246). In response to determining that the memory atomic request 132A is a cache hit in another cache memory 114(0)(0)-114(0)(C) in another compute bare die 104(1)-104(D) ( Figure 2C In step 244), the proxy cache controller circuit 120(0) communicates the memory atomic request 132A to the global cache controller circuit 122 so that it can be serviced. Figure 2C Step 246). Next, the proxy cache controller circuit 120(0) receives read data 138R from another cache memory 114(0)(0)-114(0)(C) on the same compute die 104(0), for write address 134A of that memory, or receives read data 138R from the global cache controller 122 originating from another cache memory 114(1)(0)-114(D)(C) stored in another compute die 104(1)-104(D). Figure 2C Step 248). The proxy cache controller circuit 120(0) receives atomic operation 144 from CPU core 108. Figure 2C Step 250 in the process. Next, the proxy cache controller circuit 120(0) performs the atomic operation 144 on the read data 138R to generate the write data 138W. Figure 2C Step 252), and store the written data 138W ( Figure 2C (Step 254 in the text). The written data 138W can be written back to system memory 118.

[0052] Now we will combine Figures 3A to 5B , for Figure 1A more specific example of a memory transaction performed in a multi-level cache system 102 is described. Figures 3A to 3C Described in Figure 1 An exemplary process for executing a memory read request in a multi-level caching system 102. Figures 4A to 4C Described in Figure 1 An exemplary process for executing a memory write request in a multi-level caching system 102. Figure 5A and Figure 5B Described in Figure 1 An exemplary process for performing atomic memory requests in a multi-level caching system 102. Figures 3A to 5B The exemplary process described herein is illustrated by an example involving the following components: computing two (2) CPU cores 108(0)(0) and 108(0)(C) and proxy cache controller circuit 120(0) in bare die 104(0); global cache controller circuit 122; and computing two (2) CPU cores 108(D)(0) and 108(D)(C) and proxy cache controller circuit 120(D) in bare die 104(D). However, please note that Figures 3A to 5B The process may involve computing any CPU core 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) on the bare die 104(0)-104(D) and its respective proxy cache controller circuit 120(0)-120(D).

[0053] Figure 3A It is explained in Figure 1A flowchart of an exemplary memory read request process 300A executing a memory read request in a multi-level cache system 102, the memory read request causing a cache miss in the local cache memory 112(0)-112(D) of the respective compute dies 104(0)-104(D) of the respective CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) that issued the request. In this example, CPU core (0)(0) is labeled as Core-A0, representing core 0 of compute die A 104(0). CPU core (0)(C) is labeled as "Core-A3", representing core C in compute die A 104(0), where "C" can be "3" or represent any number of C+1 CPU cores 108 in compute die A 104(0). In this example as well, CPU core (D) (0) is labeled "Core-B0", representing core 0 of compute die B 104 (D). CPU core (D) (C) is labeled "Core-B3", representing core C in compute die B 104 (D), where "D" can represent any number of compute dies D+1, and "C" can be "3" or represent any number of CPU cores 108 in compute die B 104 (D). As discussed in more detail below, such a memory read request involves communication between the proxy cache controller circuits 120(0)-120(D) in the respective compute bare cores 104(0)-104(D) of the CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) that issued the request and the global cache controller circuit 122, in order to retrieve the requested valid cache line from the system memory 118.

[0054] In this regard, refer to Figure 3A The memory read request process 300A involves: the requesting CPU core 108(0)(0) communicating a shared memory read request 132R to the proxy cache controller circuit 120(0) on its compute die 104(0), because the requesting CPU core 108(0)(0) has determined, based on the tag L2 associated with the memory read address 134R in its allocated cache memory 114(0)(0), that the memory read address 134R is invalid in the cache memory 114(0)(0). Figure 3A (302 in the example). For example, this is in Figure 3AThe symbol “L2[X]=I” is used to represent this, where “L2” is the cache memory 114(0)(0), “X” is the index to enter the cache memory 114(0)(0) based on the memory read address 134R, and “I” indicates an invalid state. In other words, the memory read request 132R of the requesting CPU core 108(0)(0) causes a cache miss in its allocated cache memory 114(0)(0). Therefore, the requesting CPU core 108(0)(0) issues the memory read request 132R to the proxy cache controller circuit 120(0) on its local computing die 104(0), which also causes a cache miss.

[0055] Next, in this example, and similarly... Figure 3A As shown, in response to the cache miss, the proxy cache controller circuit 120(0) initiates the memory read request 132R to the global cache controller circuit 122 via the D2D interface circuit 126(0). In this example, the global cache controller circuit 122 is located on the I / O bare die 116. Figure 3A (304 in the example). This is because, in this example, the proxy cache controller circuit 120(0) has determined that none of the other cache memories 114(0)(1)-114(0)(C) on the compute die 104(0) contain a valid and shared copy of the read data 138R associated with the memory read address 134R of the memory read request 132R. This is indicated by an invalid cache state (CS(0)=I) for the memory read address 134R. For example, this is in Figure 3AThe cache state is represented by the symbol "ST[X]=I", where "ST" is the cache state, "X" is the index of the cache memory 114(0)(0) based on the memory read address 134R, and "I" indicates an invalid state. In this example, the local cache line directory ST(0) (which is a memory circuit) is associated with and accessible by the proxy cache controller circuit 120(0). The local cache line directory ST(0) contains a cache status indicator CS(0), which is updated along with the cache status of the cache line associated with the memory read address contained in its local cache memory 112(0), which is composed of cache memories 114(0)(0)-114(0)(C) on the computing die 104(0) allocated to each CPU core 108(0)(0)-108(0)(C).

[0056] In this example, a memory read request 132R communicated by the proxy cache controller circuit 120(0) to the global cache controller circuit 122 includes an instruction to the requesting CPU core 108(0)(0) (shown as Core-A0) to communicate the read data 138R associated with the memory read request 132R back to the requesting CPU core 108(0)(0) upon receipt of the read data 138R associated with the memory read request 132R. Figure 3A (304 in the example). This design is implemented in this example because otherwise these CPU cores 108(0)(0)-108(0)(C) would be transparent to the global cache controller circuit 122, as the global cache controller circuit 122 can execute standard cache coherence protocols. Otherwise, the global cache controller circuit 122 would only be aware of the proxy cache controller circuits 120(0)-120(D). This feature allows the global cache controller circuit 122 to provide a mechanism that enables the finally acquired read data 138R to be communicated directly back to the requesting CPU core 108(0)(0), thereby reducing memory access latency.

[0057] In this example, the global cache controller circuit 122 then initiates a read access request 137R, which includes identification of the requester CPU core 108(0)(0) (Core-A0), resulting in access to the read data 138R from system memory 118. Figure 3A(Step 306 in the example). In this example, the global cache controller circuit 122 initiates the read access request 137R to the memory controller 129, which is able to obtain the requested read data 138R for the memory read address 134R from the system memory 118. This is because the global cache controller circuit 122 determines by accessing its global cache line directory ST(G) (which is the memory circuit): none of the other cache memories 114(1)(0)-114(D)(C) on the other compute bare dies 104(1)-104(D) contain a valid and shared copy of the read data 138R associated with the memory read address 134R of the memory read request 132R. This is indicated by an invalid global cache state (CS(G)=I) for the memory read address 134R. The global cache line directory ST(G) contains a global cache status indicator CS(G), which is updated along with the cache status of the cache line associated with the memory read address contained therein, the memory read address being contained in its global cache memory 110, which is composed of cache memories 114(0)(0)-114(D)(C) on compute bare dies 104(0)-104(D).

[0058] Next, as Figure 3A As shown, the global cache controller circuit 122 is configured to send a response data message (RepSepData) to the proxy cache controller circuit 120(0) to acknowledge receipt of the memory read request 132R. Figure 3A (308 in the middle), then, the proxy cache controller circuit 120(0) sends an acknowledgment (CompAck) back to the global cache controller circuit 122 ( ). Figure 3A(310 in the example). The global cache controller circuit 122 updates the cache status indicator CS(G) in its global cache line directory ST(G)[X] for the memory read address 134R to “0001”, indicating that the proxy cache controller circuit 120(0) will have a copy of the read data 138R for the memory read address 134R, and its cache status indicator CS(G) indicates a uniqueclean (UC) cache state. In this example, the cache status indicator bits use the least significant bit to represent the cache status indicator for CPU core 0 and the most significant bit to represent the cache status indicator for CPU core C in their respective bare dies 104(0)-104(D). This data is used for further processing by the global cache controller circuit 122 to determine which proxy cache controller circuits 120(0)-120(D) control access to memory addresses addressable in the processor system 100. The proxy cache controller circuit 120(0) is configured to send a response data message (RepSepData) to the requesting CPU core 108(0)(0) to acknowledge receipt of the memory read request 132R. Figure 3A (312 in the middle), then the requester CPU core 108(0)(0) sends an acknowledgment (CompAck) back to the proxy cache controller circuit 120(0). Figure 3A(314 in the example). These messages can be part of standard cache coherence protocols, such as the Coherent Hub Interface (CHI) cache coherence protocol. As illustrated in the examples in this paper, this concept also applies to cache coherence protocols other than CHI. The proxy cache controller circuit 120(0) and the requester CPU core 108(0)(0) are configured to update their respective tag L2 and local cache line directory ST(0) to indicate that the local cache memory 112(0) associated with the proxy cache controller circuit 120(0) and the cache memory 114(0)(0) associated with the requester CPU core 108(0)(0) will have a shared copy of the read data 138R for the memory read address 134R. This is indicated by updating the cache status indicator CS(0) for the local cache line directory ST(0)[X] to “0001”, where the least significant bit indicates that the cache memory 114(0)(0) associated with the requesting CPU core 108(0)(0) will have a valid copy of the read data 138R, and proxy-owned (PO) = “1” (true), meaning that the memory read address 134R is owned by the proxy cache controller circuit 120(0) and the cache status (S) = “UC” (exclusive clean). This is also indicated by the flag L2[X] for the CPU core 108(0)(0) having a “UC” cache status for the memory read address 134R, meaning that the cache memory 114(0)(0) has an exclusive clean copy of the read data 138R. These flags and indicators are used in other processes to indicate the cache status and condition of the operation.

[0059] Next, as Figure 3A As shown, the memory controller 129, using the identification of the requester CPU core 108(0)(0) (Core-A0) received in the previous self-read access request 137R, directly communicates the retrieved read data 138R for the memory read request 132R to the requester CPU core 108(0)(0). Figure 3A (318 in the middle).

[0060] Figure 3B It is explained in Figure 1A flowchart of an exemplary memory read request process 300B executing a memory read request in a multi-level cache system 102, the memory read request causing a cache hit in the local cache memory 114 in the same compute die 104 as the CPU core 108 that issued the request. In this example, Figure 3B The memory read request process 300B involves the requester CPU core 108(0)(0) communicating a shared memory read request 132R to the proxy cache controller circuit 120(0) on its compute die 104(0), because the requester CPU core 108(0)(0) has determined, based on the tag L2[X] associated with the memory read address 134R in its allocated cache memory 114(0)(0), that the memory read address 134R is invalid in the cache memory 114(0)(0) (I)(). Figure 3B (320 in the original text). In other words, the memory read request 132R of the requesting CPU core 108(0)(0) causes a cache miss in its allocated cache memory 114(0)(0). Therefore, the requesting CPU core 108(0)(0) issues the memory read request 132R to the proxy cache controller circuit 120(0) on its local compute die 104(0).

[0061] Next, as Figure 3B As shown, in this example, the proxy cache controller circuit 120(0) determines that the cache memory 114(0)(C) allocated to the CPU core 108(0)(C) has a valid copy of the read data 138R for the memory read request 132R. This is based on the fact that the proxy cache controller circuit 120(0) has a cache status indicator CS(0) indicating "1000" and having an SC cache status in its local cache line directory ST(0)[X] for the memory read address 134R, thereby determining from the local cache line directory ST(0)[X] that the CPU core 108(0)(C) controls a shared copy of the read data 138R for the memory read address 134R that is in a shared clean (SC) cache state. In this response, the proxy cache controller circuit 120(0) initiates a read access snooping request 136R to the CPU core 108(0)(C). Figure 3B322 in the middle). This causes the proxy cache controller circuit 120(0) to update its local cache line directory ST(0)[X] for the memory read address 134R to indicate that both CPU cores 108(0)(0) and 108(0)(C) can now access a valid copy of the read data 138R for the memory read address 134R, which is in a shared clean (SC) cache state. Figure 3B (324 in the middle).

[0062] In this example, the global cache controller circuit 122 does not necessarily need to consider the fact that cache lines previously in a clean unique state in CPU core 108(0)(C) on die 104(0) are now in a shared clean cache state in two (2) CPU cores on die 104(0). From the perspective of the global cache controller circuit 122, the memory read address 132R is still exclusively owned by a specific proxy cache controller circuit 120(0). The memory read address 132R can be shared by multiple CPU cores 108(0)(0) to 108(0)(C) on die 104(0), and consistency is managed by the proxy cache controller circuit 120(0), thus making the memory read address 132R effectively shared on die 104(0). The proxy cache controller circuit 120(0) abstracts all CPU cores 108(1)(0) to 108(D)(C) from the global cache controller circuit 122. To the global cache controller circuit 122, the proxy cache controller circuit 120(0) appears to be a single CPU core. Similarly, when the proxy cache controller 120(0) grants exclusive ownership of a certain address to CPU cores 108(0)(0) to 108(0)(C), those CPU cores 108(0)(0) to 108(0)(C) are allowed to internally demote their cache state to shared (however, if they want to modify that address, they must subsequently initiate another ownership claim). The same applies to the other proxy cache controller circuits 120(1)-120(D).

[0063] Furthermore, in response to the read access snooping request 136R issued by the proxy cache controller circuit 120(0) to the CPU core 108(0)(C), the CPU core 108(0)(C) accesses its cache memory 114(0)(C) for the memory read address 134R and provides the associated read data 138R to the requesting CPU core 108(0)(0). Figure 3B (326 in the example). In this example, the CPU core 108(0)(C) is configured to forward the read data 138R directly to the requester CPU core 108(0)(0). Figure 3B Alternatively, the read data 138R can be communicated back to the proxy cache controller circuit 120(0) to be provided to the requester CPU core 108(0)(0). The requester CPU core 108(0)(0) acknowledges receipt of the read data 138R (326). Figure 3B (328 in the middle).

[0064] Figure 3C It is explained in Figure 1 A flowchart of an exemplary memory read request process 300C executing a memory read request in a multi-level cache system 102, the memory read request causing a cache miss in a local cache memory 114 in a compute die 104 where the requesting CPU core 108 resides. The memory read request process 300C involves communication between a proxy cache controller circuit 120(0) and a global cache controller circuit 122 to retrieve the requested valid cache line from another local cache memory 114 in another compute die 104.

[0065] In this regard, refer to Figure 3C The memory read request process 300C involves: the requesting CPU core 108(0)(0) communicating a shared memory read request 132R to the proxy cache controller circuit 120(0) on its compute die 104(0), because the requesting CPU core 108(0)(0) has determined, based on the tag L2 associated with the memory read address 134R in its allocated cache memory 114(0)(0), that the memory read address 134R is invalid in the cache memory 114(0)(0). Figure 3C (330 in the text). For example, this is in Figure 3CThe symbol “L2[X]=I” is used to represent this, where “L2” is the cache memory 114(0)(0), “X” is the index to enter the cache memory 114(0)(0) based on the memory read address 134R, and “I” indicates an invalid state. In other words, the memory read request 132R of the requesting CPU core 108(0)(0) causes a cache miss in its allocated cache memory 114(0)(0). Therefore, the requesting CPU core 108(0)(0) issues the memory read request 132R to the proxy cache controller circuit 120(0) on its local computing die 104(0).

[0066] Next, in this example, and similarly... Figure 3C As shown, the proxy cache controller circuit 120(0) initiates the memory read request 132R to the global cache controller circuit 122 via the D2D interface circuit 126(0). In this example, the global cache controller circuit 122 is located on the I / O bare die 116. Figure 3C (332 in the example). This is because, in this example, the proxy cache controller circuit 120(0) has determined that none of the other cache memories 114(0)(1)-114(0)(C) on the compute die 104(0) contain a valid and shared copy of the read data 138R associated with the memory read address 134R of the memory read request 132R. This is indicated by an invalid cache state in the cache status indicator CS(0)=I for the memory read address 134R. For example, this is in Figure 3CThe symbol “ST(0)[X]=I” is used to represent this, where “ST” is the local cache line directory, which has a cache status indicator CS(0) that is indexed by “X” into the cache memory 114(0)(0) based on the memory read address 134R and is invalid (I). In this example, the local cache line directory ST(0) is associated with and accessible by the proxy cache controller circuit 120(0). The local cache line directory ST(0) contains a cache status indicator CS(0), which is updated along with the cache status of the cache line associated with the memory read address contained in its local cache memory 112(0), which is composed of cache memories 114(0)(0)-114(0)(C) on the computing die 104(0) allocated to each CPU core 108(0)(0)-108(0)(C).

[0067] In this example, a memory read request 132R communicated by the proxy cache controller circuit 120(0) to the global cache controller circuit 122 includes an instruction to the requesting CPU core 108(0)(0) (shown as Core-A0) to communicate the read data 138R associated with the memory read request 132R back to the requesting CPU core 108(0)(0) upon receipt of the read data 138R associated with the memory read request 132R. Figure 3C (332 in the example). This design is implemented in this example because otherwise these CPU cores 108(0)(0)-108(0)(C) would be transparent to the global cache controller circuit 122, as the global cache controller circuit 122 can execute standard cache coherence protocols. Otherwise, the global cache controller circuit 122 would only be aware of the proxy cache controller circuits 120(0)-120(D). This feature allows the global cache controller circuit 122 to provide a mechanism to communicate the finally acquired read data 138R directly back to the requesting CPU core 108(0)(0) to reduce memory access latency.

[0068] In this example, the global cache controller circuit 122 then initiates a read access snooping request 136R, which includes identification of the proxy cache controller circuit 120(0) that issued the request. This will result in the self-computed bare die 104(D) accessing the read data 138R via communication with the proxy cache controller circuit 120(D). Figure 3C (334 in the example). In this example, the global cache controller circuit 122 initiates the read access snooping request 136R to the proxy cache controller circuit 120(D), which is able to obtain the requested read data 138R from its local cache memory 112(D). This is because the global cache controller circuit 122 determines by accessing its global cache line directory ST(G) that the local cache memory 112(D) on the compute die 104(D) contains a valid copy of the read data 138R associated with the memory read address 134R of the memory read request 132R. This is indicated by the global cache line directory ST(G) containing a cache status indicator CS(G) indicating “1000” and having a unique clean (UC) cache status, which indicates that the local cache memory 112(D) on the compute die 104(D) contains a valid and unique clean copy of the read data 138R.

[0069] Next, as Figure 3C As shown, the global cache controller circuit 122 is configured to initiate the read access snooping request 136R to the proxy cache controller circuit 120(D). Figure 3C (334 in the original text). The read access snooping request 136R contains an instruction to the proxy cache controller circuit 120(0), which acts as the requester proxy cache controller circuit, so that the proxy cache controller circuit 120(D) is aware of the requester proxy cache controller circuit and can communicate the requested read data 138R directly back to the requester proxy cache controller circuit 120(0). Then, the proxy cache controller circuit 120(D) initiates the read access snooping request 136R to the CPU core 108(D)(0). Figure 3C(336 in the text). The proxy cache controller circuit 120(D) determines, based on the local cache line directory ST[D][X] having a cache status indicator CS(D) indicating "0001" and having an exclusive clean (UC) cache state, that the cache memory 114(D)(0) associated with the CPU core 108(D)(0) contains a valid copy of the read data 138R for the memory read address 134R. This indicates that the cache memory 114(D)(0) associated with the CPU core 108(D)(0) contains a valid copy of the read data 138R. The CPU core 108(D)(0) initiates a snoop response acknowledgment (SnpRespData) from its cache memory 114(D)(0) to provide the read data 138R for the memory read address 134R. Figure 3C (338 in the middle). Then, the proxy cache controller circuit 120(D) communicates the read data 138R back to the requesting proxy cache controller circuit 120(0) ( Figure 3C (340 in the original text). Since the compute bare die 104(0) now has a copy of the read data 138R for that memory read address 134R, the proxy cache controller circuit 120(D) updates the cache state of that memory read address 134R to a shared clean (SC) cache state. The proxy cache controller circuit 120(D) also communicates a shared cache state snoop response (SnpRespSC) to the global cache controller circuit 122. Figure 3C (342 in the middle).

[0070] In response to the proxy cache controller circuit 120(0) receiving a copy of the read data 138R for the memory read address 134R from the proxy cache controller circuit 120(D), Figure 3C (340) The proxy cache controller circuit 120(0) initiates an acknowledgment (CompAck) to the global cache controller circuit 122. Figure 3C(344 in the text). The global cache controller circuit 122 updates its cache status indicator CS(G) to ST(G)[X] = “1001” to indicate that both proxy cache controller circuits 120(0) and 120(D) control a valid copy of the read data 138R for the memory read address 134R, and also updates its cache status to a shared dirty (SD) state. This data is used for other processing by the global cache controller circuit 122 to know which proxy cache controller circuits 120(0)-120(D) control access to the addressable memory address in the processor-based system 100. The proxy cache controller circuit 120(0) is configured to communicate the read data 138R for the memory read address 134R to the requester CPU core 108(0)(0) ( Figure 3C (346 in the middle), then the requester CPU core 108(0)(0) sends an acknowledgment (CompAck) back to the proxy cache controller circuit 120(0). Figure 3C (348 in the original text). The proxy cache controller circuit 120(0) and the requester CPU core 108(0)(0) are configured to update their respective tag L2 and local cache line directory ST(0) to indicate that the local cache memory 112(0) associated with the proxy cache controller circuit 120(0) and the cache memory 114(0)(0) associated with the requester CPU core 108(0)(0) will have a shared copy of the read data 138R for the memory read address 134R. This is indicated by updating the cache status indicator CS(0) of the local cache line directory ST(0)[X] to “0001”, where the least significant bit indicates that the cache memory 114(0)(0) associated with the requesting CPU core 108(0)(0) will have a valid copy of the read data 138R, and proxy owned (PO) = “0” (false), meaning that the memory read address 134R is shared by the proxy cache controller circuit 120(0) and the cache status (S) = “SD” (shared dirty). This is also indicated by the flag L2[X] for the CPU core 108(0)(0) having an “SD” cache status for the memory read address 134R, meaning that the cache memory 114(0)(0) has a shared dirty copy of the read data 138R. These flags and indicators are used in other processes to indicate the cache status and condition of operations.

[0071] In this example, CPU core 108(D)(0) is initially in a UD (unique dirty) state. The proxy cache controller 120(D) and global cache controller 122 are unaware, and do not need to be aware, that CPU core 108(D)(0) has modified a memory location—it is sufficient that the memory read address 134R was previously given a unique state for the consistency protocol. Then, when CPU core 108(D)(0) receives a snoop request, it retains a clean shared copy of the memory location and passes a dirty shared copy—in CHI, PD = pass dirty. This allows the snoopee to pass the dirty state to either its proxy cache controller circuit 120(D) or the requesting CPU core 108(D)(0). Therefore, in this example, the dirty state is ultimately passed to the requesting CPU core 108(D)(0).

[0072] Figure 4A It is explained in Figure 1 A flowchart of an exemplary memory write request process 400A executing a memory write request in a multi-level caching system 102, the memory write request causing a cache hit owned by the proxy to occur in cache memory 114 in the same compute die 104 as the CPU core 108 that issued the request. In this respect, as Figure 4A As shown, the requester CPU core 108(0)(0) initiates a clean exclusive request 136W to the proxy cache controller circuit 120(0) on its same compute die 104(0) in response to a memory write request 132W. Figure 4A (402 in the original text). This is to request a clean exclusive cache state for memory write address 134W associated with memory write request 132W. This is because the requesting CPU core 108(0)(0) expects to store write data 138W to memory write address 134W, but as indicated by the marker L2[X] (which indicates that the “X” for memory write address 134W in cache memory 114(0)(0) is in SC cache state), the cache state for memory write address 134W is in a shared clean (SC) state. In order for the write data 138W to be stored in cache memory 114(0)(0), cache memory 114(0)(0) needs to have an exclusive cache state for memory write address 134W.

[0073] Subsequently, as shown in Figure 4A, in response to the clean exclusive request 136W, the proxy cache controller circuit 120(0) determines that the cache memory 114(0)(C) allocated to the CPU core 108(0)(C) has a shared cache state for the memory write address 134W. This is indicated by the cache state indicator CS(0) in its local cache line directory ST(0)[X] for the memory write address 134W, which has "1001" and a cache state of shared clean (SC), indicating that both cache memories 114(0)(0) and 114(0)(C) have a shared cache state for the memory write address 134W. The cache status indicator CS(0) also indicates that the memory write address 134W is owned by the proxy by setting PO to "1" or true, which means that there is no other copy of the write data 138W from the global cache controller circuit 122 to be invalidated in another computing chip 104(1)-104(D). In response, the proxy cache controller circuit 120(0) initiates SnpCleanInvalid 142 (404 in FIG4A) to the CPU core 108(0)(C) so that the CPU core 108(0)(C) invalidates the memory write address 134W in its allocated cache memory 114(0)(C) as indicated by the updated invalid (I) cache status. The CPU core 108(0)(C) then sends a snoop invalid response (SnpResp_I) (406 in Figure 4A) to the proxy cache controller circuit 120(0), which in turn causes the proxy cache controller circuit 120(0) to send a complete exclusive clean state (Comp_UC) response to the CPU core 108(0)(0) to indicate that the CPU core 108(0)(0) now has a clean exclusive cache state for the memory write address 134W (408 in Figure 4A).

[0074] In this manner, the CPU core 108(0)(0) can now update the cache status in the cache status indicator CS(0) for the memory write address 134W in the local cache line directory ST(0)[X] to an exclusive clean (UC) state (i.e., S=UC). The proxy cache controller circuit 120(0) updates its cache status indicator CS(0) for the memory write address 134W in the local cache line directory ST(0)[X] to indicate that only the CPU core 108(0)(0) has a copy of the write data 138W for the memory write address 134W on the compute die 104(0), as indicated by "0001", and the proxy own (PO) state is "1" (true) and the cache status is exclusive clean (UC). The CPU core 108(0)(0) is now permitted to submit a memory write request 132W for memory write address 134W, which will subsequently change the cache state for memory write address 134W in the local cache line directory ST(0)[X] to dirty until the write data 138W for memory write address 134W is written back to system memory 118. When the write operation is complete, the CPU core 108(0)(0) sends a completion acknowledgment (CompAck) back to the proxy cache controller circuit 120(0). Figure 4A (410 in the example). In this example, because the memory write request 132W results in a cache hit in the cache memory 114 of the same compute die 104 as the requesting CPU core 108(0)(0), the CPU core 108(0)(0) can acquire exclusive ownership of the memory write address 134W without the involvement of the global cache controller circuit 122.

[0075] Figure 4B illustrates an exemplary memory write request process 400B in the multi-level cache system 102 of Figure 1, which executes a memory write request, resulting in a cache miss in the local cache of the requesting CPU core 108's compute die 104. As described below, because the memory write request is sent to a proxy cache controller circuit that does not have the memory write address of the memory write request in a proxy-owned state, the memory write request process 400B involves communication between the proxy cache controller circuit 120 and the global cache controller circuit 122 in the compute die 104 to invalidate the memory write address in the cache memory 114 of another compute die 104, thereby gaining exclusive ownership of the requested cache line.

[0076] In this regard, referring to Figure 4B, the memory write request process 400B involves the requesting CPU core 108(0)(0) sending a clean exclusive request 136W (420 in Figure 4B) for memory write request 132W to the proxy cache controller circuit 120(0) on its same compute die 104(0). This is to request a clean exclusive cache state for memory write address 134W associated with memory write request 132W. This is because the requesting CPU core 108(0)(0) wants to store write data 138W to memory write address 134W, but the cache state for memory write address 134W is a shared clean (SC) state, as indicated by the SC cache state of “X” for memory write address 134W in cache memory 114(0)(0) indicated by marker L2[X]. In order to store the written data 138W in the cache memory 114(0)(0), the cache memory 114(0)(0) needs to have an exclusive cache state for the write address 134W of the memory.

[0077] Subsequently, as shown in Figure 4B, in response to the clean exclusive request 136W, the proxy cache controller circuit 120(0) determines that the cache memory 114(0)(C) allocated to the CPU core 108(0)(C) has a shared cache state for the memory write address 134W. This is indicated by a cache state indicator CS(0) with “1001” in its local cache line directory ST(0)[X]='1001' for the memory write address 134W and a cache state of shared clean (SC), which indicates that both cache memories 114(0)(0) and 114(0)(C) have a shared cache state for the memory write address 134W. In response, the proxy cache controller circuit 120(0) initiates SnpCleanInvalid 142 (422 in Figure 4B) to the CPU core 108(0)(C), causing the CPU core 108(0)(C) to invalidate the memory write address 134W in its allocated cache memory 114(0)(C), as indicated by the updated flag L2[X] for the memory write address 134W to an invalid (I) cache state. The CPU core 108(0)(C) then sends a SnpResp_I response (424 in Figure 4B) back to the proxy cache controller circuit 120(0).

[0078] Subsequently, as shown in Figure 4B, the proxy cache controller circuit 120(0) initiates the clean exclusive request 136W (426 in Figure 4B) to the global cache controller circuit 122. This is because, based on the cache status indicator CS(0) in the local cache line directory ST(0)[X] indicating proxy ownership (PO) = "0" or false, the proxy cache controller circuit 120(0) determines that the memory write address 134W is not owned by the proxy. Therefore, this means that the write data 138W for the memory write address 134W may exist in at least one other compute die 104(1)-104(D), which must also be invalidated through a snooping clean invalidation process. In this regard, as shown in Figure 4B, based on the determination result in the cache status indicator CS(G) for the memory write address 134W in the global cache line directory ST(G)[X], that is, based on the cache status indicator value "1001", the computed bare die 104(D) has a valid copy of the write data 138W for the memory write address 134W for the memory write request 132W, thereby determining the proxy cache controller circuit 120(D) (428 in Figure 4B).

[0079] In response to the proxy cache controller circuit 120(D) receiving the clean exclusive request 136W from the global cache controller circuit 122, the proxy cache controller circuit 120(D) initiates a snpCleanInvalid 142 (430, 432 in Figure 4B) to the CPU cores 108(D)(0)-108(D)(C). This causes the CPU cores 108(D)(0)-108(D)(C) to invalidate the memory write address 134W in their allocated cache memory 114(D)(0)-114(D)(C), as indicated by updating the flag L2[X] for the memory write address 134W to the invalid (I) cache state. The CPU cores 108(D)(0)-108(D)(C) then send a snoop invalid response (SnpResp_I) (434, 436 in Figure 4B) back to the proxy cache controller circuit 120(D), which in turn causes the proxy cache controller circuit 120(D) to update the cache status indicator CS(D) for the memory write address 134W in the local cache line directory ST(D)[X] to an invalid cache status (I). The proxy cache controller circuit 120(D) then sends a snoop invalid response (SnpResp_I) (438 in Figure 4B) back to the global cache controller circuit 122 to indicate that the memory write address 134W has been successfully invalidated in the compute die 104(D). The global cache controller circuit 122 then updates the global cache status indicator CS(G) in the global cache line directory ST(G)[X]='0001' for the memory write address 134W to indicate that only the compute die 104(0) will have a valid clean exclusive state for the memory write address 134W.

[0080] Subsequently, as shown in Figure 4B, the global cache controller circuit 122 initiates a Complete Exclusive Clean Status (Comp_UC) response to the proxy cache controller circuit 120(0) to indicate that the memory write address 134W is now owned by the proxy cache controller circuit 120(0) (440 in Figure 4B). The proxy cache controller circuit 120(0) updates the cache status indicator CS(0) for the memory write address in its local cache line directory ST(0)[X] to “0001” to indicate that the compute die 104(0) has an exclusive copy of the write data 138W for the memory write address 134W. The proxy cache controller circuit 120(0) also updates the cache status indicator CS(0) in its local cache line directory ST(0)[X] to indicate that the memory write address 134W is owned by the proxy (PO="1") and is in a clean exclusive (UC) state in the compute die 104(0). The proxy cache controller circuit 120(0) then initiates a Complete Clean Exclusive (Comp_UC) response to the requesting CPU core 108(0)(0) to indicate that the memory write address 134W is now in a clean exclusive state, so that the requesting CPU core 108(0)(0) can now store the write data 138W for the memory write address 134W in its cache memory 114(0)(0) and update its flag L2[X] for the memory write address 134W to the clean exclusive (UC) state (442 in Figure 4B). When the write operation is complete, the CPU core 108(0)(0) sends a completion acknowledgment (CompAck) back to the proxy cache controller circuit 120(0) (444 in Figure 4B).

[0081] Figure 4C illustrates an exemplary memory write-back request process 400C in the multi-level cache system 102 of Figure 1, where a memory write-back request (e.g., dirty cache eviction) is executed due to the eviction of local cache memory 112, which involves communication between the proxy cache controller circuitry 120 and the global cache controller circuitry 122 in the compute die 104. The memory write-back request process 400C is executed if eviction by CPU core 108 is required due to the storage of write data 138W in its allocated cache memory 114 (e.g., according to memory write processes 400A, 400B in Figures 4A and 4B).

[0082] In this regard, as shown in Figure 4C, in response to evicting cached data from its allocated cache memory 114(0)(0), the requesting CPU core 108(0)(0) initiates a write-back request 132WB to the proxy cache controller circuitry 120(0) on its compute die 104(0), the write-back request including the memory write address 134W(0) to be evicted. Figure 4C (450 in Figure 4C). The proxy cache controller circuit 120(0) sends a write-back response (CmopDBIDResp) to the requesting CPU core 108(0)(0) to acknowledge receipt of the write-back request 132WB (452 ​​in Figure 4C). This causes the requesting CPU core 108(0)(0) to update its tag L2[X] for the memory write address 134W to be evicted to an invalid state (I). The requesting CPU core 108(0)(0) then sends a copy of the write data 138W to be evicted to the proxy cache controller circuit 120(0) in a copy-back response (CopyBackWrData) (454 in Figure 4C). The proxy cache controller circuit 120(0) then initiates a write clean request (in this example, WriteCleanFull-X) for the write data 138W at memory write address 134W to the global cache controller circuit 122 (456 in Figure 4C). The global cache controller circuit then sends a writeback response (CmopDBIDResp) back to the proxy cache controller circuit 120(0) to acknowledge receipt of the write clean request (WriteCleanFull-X) (458 in Figure 4C). A write clean full request (WriteCleanFull) is used in this case instead of a writeback full request (WriteBackFull) because there are other CPU cores 108(0)(0)-108(0)(C) on the compute die 104(0) that have copies of the memory write address 132W. Therefore, from the perspective of the global cache controller circuit 222, the proxy cache controller 120(0), although performing a copy back operation, still retains a copy of the cache line at memory write address 132. The proxy cache controller circuit 120(0) then sends a copy of the write data 138W to be written back to the global cache controller circuit 122 in a copy back response (CopyBackWrData) (460 in FIG. 4C).

[0083] Subsequently, as shown in Figure 4C, the global cache controller circuit 122 updates the cache status indicator CS(G) in the global cache line directory ST(G)[X] for the memory write address 134W to be in a shared clean (SC) cache state. The global cache controller circuit 122 then initiates a write-no-sneak request (WriteNoSnp-X) to the memory controller 129 for the memory write address 134W to be written back, in order to establish a write-back of the written data 138W evicted to the system memory 118 (462 in Figure 4C). The memory controller 129 then sends a write-back response (CmopDBIDResp) back to the global cache controller circuit 122 to acknowledge receipt of the write-no-sneak request (WriteNoSnp-X) (464 in Figure 4C). The global cache controller circuit 122 then sends a copy of the write data 138W to be written back to the memory controller 129 in a copy back response (NonCopyBackWrData) so that it can be written to the system memory 118 (step 466 in FIG4C).

[0084] The multi-level cache system 102 in the processor-based system 100 in Figure 1 can also be configured to perform atomic memory operations and maintain cache coherence. Atomic memory operations involve read-modify-write operations that are performed in a single step relative to other threads, such that each of these operations is executed before any other thread can access the memory address of the atomic operation.

[0085] In this regard, Figure 5A illustrates an exemplary memory atomic request process 500A for memory atomic request 132A, which involves accessing the local cache memory 112 in the same compute die 104 via proxy cache controller circuitry 120, without accessing the global cache controller circuitry 122. This is because the read data requested for this memory atomic request 132A is valid and contained in the same compute die 104 as the proxy cache controller circuitry 120.

[0086] Referring to Figure 5A, in this example, the requesting CPU core 108(0)(0) initiates a memory atomic request 132A (step 502 in Figure 5A) to the proxy cache controller circuit 120(0). This is because, in this example, the requesting CPU core 108(0)(0) has determined that the memory write address 134W in the cache memory 114(0)(0) it allocated is invalid, based on the L2 tag associated with the memory read address 134R in the cache memory 114(0)(0). For example, this is indicated in Figure 5A by the symbol “L2[X]=I”, where “L2” is the cache memory 114(0)(0), “X” is the index into the cache memory 114(0)(0) based on the memory write address 134W, and “I” indicates an invalid state. In other words, the memory atomic request 132A of the requesting CPU core 108(0)(0) has resulted in a cache miss in its allocated cache memory 114(0)(0). Therefore, the requesting CPU core 108(0)(0) initiates the memory atomic request 132A to the proxy cache controller circuit 120(0) on its local compute die 104(0). The proxy cache controller circuit 120(0) initiates a memory atomic response (DBIDResp) (504 in Figure 5A).

[0087] Subsequently, as shown in Figure 5A, the proxy cache controller circuit 120(0) determines, based on the cache status indicator CS(0) for the memory write address 134W in its local cache line directory ST(0)[X], that the cache memory 114(0)(C) allocated to the CPU core 108(0)(C) contains a proxy-owned copy of the memory write address 134W in an exclusive clean (UC) cache state. Therefore, the atomic memory operation can be performed on the compute die 104(0) without involving the global cache controller circuit 122. In this regard, the proxy cache controller circuit 120(0) initiates a snoop-unique invalidation request (SnpUnique) (506 in FIG. 5A) to the CPU core 108(0)(C) to subsequently receive the read data 138R stored at memory write address 134W from the cache memory 114(0)(C) allocated to the CPU core 108(0)(C) in a copy return response (SnpRespData_I) (508 in FIG. 5A). The CPU core 108(0)(C) then invalidates its exclusive copy of the read data 138R at memory write address 134W in the cache memory 114(0)(C) (508 in FIG. 5A). The CPU core 108(0)(C) sets its flag L2[X] for memory write address 134W to an invalid cache state (I).

[0088] As shown in Figure 5A, in response to the requester CPU core 108(0)(0) receiving the memory atomic response (DBIDResp) (504 in Figure 5A), the requester CPU core 108(0)(0) sends transaction data (NonCopyBackWrData) to the proxy cache controller circuit 120(0). This transaction data includes read data 138R (510 in Figure 5A) for the atomic operation (e.g., increment operation, compare-and-swap operation) for the memory atomic request 132A. At this time, the proxy cache controller circuit 120(0) can perform the memory atomic operation by using the received read data 138R (NonCopyBackWrData) and subsequently write the result of the atomic operation for the memory atomic request 132A as write data 138W to the memory write address 134A (512 in Figure 5A). For example, the proxy cache controller circuit 120(0) can write the write data 138W back to system memory 118 or, for example, back to shared cache memory (e.g., L3 cache memory). After the write operation, the proxy cache controller circuit 120(0) sends an atomic completion response (CompData_I) and a copy of the original data at memory write address 134A 114(0)(0) to the requesting CPU core 108(0)(0) to indicate that the atomic memory operation has been completed (514 in FIG5A). The proxy cache controller circuit 120(0) also updates the cache status indicator CS(0) in the local cache line directory ST(0)[X] for the memory write address 134A to “0001” to indicate that the cache memory 114(0)(0) has the original data at the memory write address 134A of the memory atomic request 132A, but the write data 138W is in an exclusive dirty (UD) cache state.

[0089] Figure 5B illustrates an exemplary memory atomic request process 500B for memory atomic request 132A, which involves access via proxy cache controller circuitry 120 and global cache controller circuitry 122, because the data requested for memory atomic request 132A is not stored in the compute bare die 104 with proxy cache controller circuitry 120. Referring to Figure 5B, in this example, the requesting CPU core 108(0)(0) initiates memory atomic request 132A (520 in Figure 5B) to proxy cache controller circuitry 120(0). This is because, in this example, the requesting CPU core 108(0)(0) has determined, based on the L2 tag associated with memory read address 134R in cache memory 114(0)(0), that the memory write address 134W in its allocated cache memory 114(0)(0) is invalid. For example, this is indicated in Figure 5B by the symbol “L2[X]=I”, where “L2” is the cache memory 114(0)(0), “X” is the index into the cache memory 114(0)(0) based on the memory write address 134W, and “I” indicates an invalid state. In other words, the memory atomic request 132A of the requesting CPU core 108(0)(0) has resulted in a cache miss in its allocated cache memory 114(0)(0). Therefore, the requesting CPU core 108(0)(0) initiates the memory atomic request 132A to the proxy cache controller circuit 120(0) on its local compute die 104(0). The proxy cache controller circuit 120(0) sends a memory atomic response (DBIDResp) back to the requesting CPU core 108(0)(0) (522 in Figure 5B). In response that the requesting CPU core 108(0)(0) has received the memory atomic response (DBIDResp) (522 in FIG5B), the requesting CPU core 108(0)(0) sends transaction data to the proxy cache controller circuit 120(0) as the read data 138R (NonCopyBackWrData) for the atomic operation (e.g., increment operation, compare and swap operation) of the memory atomic request 132A to be performed on the read data 138R at the memory write address 134A, for later use in performing the atomic memory operation (524 in FIG5B).

[0090] Subsequently, as shown in Figure 5B, the proxy cache controller circuit 120(0) determines, based on the cache status indicator CS(0) in its local cache line directory ST(0)[X], that the memory write address 134W is invalid, meaning a cache miss. Therefore, the atomic memory operation cannot be performed on the compute die 104(0) without involving the global cache controller circuit 122. In this regard, the proxy cache controller circuit 120(0) initiates the memory atomic request 132A (526 in Figure 5A) to the global cache controller circuit 122. The global cache controller circuit 122 sends back a memory atomic response (DBIDResp) (528 in Figure 5B) to the proxy cache controller circuit 120(0). In response that the proxy cache controller circuit 120(0) has received the memory atomic response (DBIDResp) (528 in FIG5B), the proxy cache controller circuit 120(0) forwards the memory atomic operation (NonCopyBackWrData) (e.g., increment operation, compare and swap operation) to the global cache controller circuit 122 for later use in performing the atomic memory operation (530 in FIG5B).

[0091] Subsequently, as shown in Figure 5B, the global cache controller circuit 122 initiates a ReadNoSnp-X request to the memory controller 129 to obtain the read data 138R at the memory write address 134A of the memory atomic request 132A, because the global cache controller circuit 122 has determined that no other compute bare die 104(1)-104(D) has a valid copy of the read data 138R at the memory write address 134A (532 in Figure 5B). This is because the cache status indicator CS(G) for the memory write address 134A in the global cache line directory ST(G)[X] is invalid. The global cache controller circuit 122 receives the read data 138R at the memory write address 134A from the memory controller 129 (534 in Figure 5B). At this time, the global cache controller circuit 122 can perform the atomic operation (536 in FIG. 5B) by performing the memory atomic operation (NonCopyBackWrData) on the received read data 138R, and then initiate a write no-snipping request (WriteNoSnp-X) to the system memory 118, writing the result as write data 138W to the memory write address 134A of the memory atomic request 132A (538 in FIG. 5B). The global cache controller circuit 122 then receives the write request completion (CompDBIDResp) returned from the memory controller 129 (540 in FIG. 5B).

[0092] After performing the write operation in response to the memory atomic request 132A, the global cache controller circuit 122 sends an atomic completion response (CompData_I) and a copy of the original read data 138R to the proxy cache controller circuit 120(0) to indicate that the memory atomic operation has been completed (542 in FIG. 5B). The proxy cache controller circuit 120(0) sends an atomic completion response (CompData_I) and a copy of the written data 138W to be written to the cache memory 114(0)(0) to indicate that the memory atomic operation has been completed (544 in FIG. 5B). The proxy cache controller circuit 120(0) also updates the cache status indicator CS(0) in the local cache line directory ST(0)[X] for the memory write address 134A to invalid (I) to indicate that the write data 138W was not cached because it was written back to system memory 118 in step 538 of FIG5B.

[0093] Figure 6 illustrates an example of another multi-die processor-based system 600 (“processor-based system 600”), which may be the same as or similar to the multi-die processor-based system 100 in Figure 1. The processor-based system 600 includes a multi-level cascade cache system 602 configured to maintain cache coherency across a plurality of dies 604(0)-604(D) (e.g., similar to compute dies 104(0)-104(D) in Figure 1), where each die has its own first processor 606(1)-606(D) (e.g., similar to processors 106(0)-106(D) in Figure 1). In this example, the processor-based system 600 is a multi-socket processor-based system comprising a plurality of processors 606(0)-606(D) distributed among their respective independent dies 604(1)-604(D). Each processor 606(0)-606(D) includes its own multiple CPU cores 608(0)(0)-608(0)(C) to 608(D)(0)-608(D)(C) (e.g., similar to CPU cores 108(0)(0)-108(0)(C) to 108(D)(0)-108(D)(C) in Figure 1), which are configured to execute instructions to perform computation-related tasks in their respective die 604(0)-604(D), where “D”+1 is the number of dies 604(0)-604(D) and their respective processors 606(0)-606(D), and “D” can be any positive integer.

[0094] As further shown in Figure 6, the multi-level cache system 602 in the processor-based system 600 forms a global cache memory, which includes a respective local cache memory 612(0)-612(D) provided on each die 604(0)-604(D) (e.g., similar to the local cache memory 112(0)-112(D) in Figure 1). If a cache miss occurs for a memory access request initiated by a given CPU core 608(0)(0)-608(0)(C) to 608(D)(0)-608(D)(C) to its respective die 604(0)-604(D), the memory access request may be communicated off-die to I / O die 616 (e.g., similar to I / O die 116 in Figure 1), which may contain the requested cache data and / or provide access to another local cache memory 612(0)-612(D) of another respective processor 606(0)-606(D) on its respective die 604(0)-604(D). To provide single-point cache coherency for the multi-level cache system 602, in this example, the processor-based system 600 includes proxy cache controller circuitry 620(0)-620(D) (e.g., similar to proxy cache controller circuitry 120(0)-120(D) in FIG. 1) on each respective die 604(0)-604(D) and a global cache controller circuitry 622 (e.g., similar to global cache controller circuitry 122 in FIG. 1), coupled to respective CPU cores 608(0)(0)-608(0)(C) to 608(D)(0)-608(D)(C) on their respective dies 604(0)-604(D). In this example, the global cache controller circuitry 622 is located on a separate I / O die 616, but in another example it could be located on any of these dies 604(0)-604(D).

[0095] Each of the proxy cache controller circuits 620(0)-620(D) is configured to serve memory access requests for the local cache memory 612(0)-612(D) on its respective bare die 604(0)-604(D) through the entire system address space of the processor-based system 600. In this manner, if a cache miss occurs for a given memory access request initiated by CPU cores 608(0)(0)-608(0)(C) to 608(D)(0)-608(D)(C), the memory access request can be communicated to the proxy cache controller circuit 620(0)-620(D) on their respective die 604(0)-604(D), which can then determine whether the memory access request can be serviced by another local cache memory 612(0)-612(D) on its own respective compute die 104(0)-104(D). In this way, the proxy cache controller circuits 620(0)-620(D) on each of their respective bare dies 604(0)-604(D) can serve memory access requests between their respective local cache memories 612(0)-612(D) and also maintain cache coherence for their respective local cache memories 612(0)-612(D).

[0096] Referring again to Figure 6, and as described above, if a cache miss occurs because the proxy cache controller circuits 620(0)-620(D) are unable to service the memory access request via the local cache memory 612(0)-612(D) on their respective bare dies 604(0)-604(D), the proxy cache controller circuits 620(0)-620(D) can forward the request to the global cache controller circuit 622 outside the bare die. In this respect, each bare die 604(0)-604(D) includes a respective D2D interface circuit 626(0)-626(D) coupled to the system bus 630. The proxy cache controller circuits 620(0)-620(D) are configured to communicate memory access requests to the global cache controller circuit 622 via the D2D interface circuits 626(0)-626(D) on their respective bare dies 604(0)-604(D), and to the global cache controller circuit 622 via the respective D2D interface circuits 628 on the I / O bare die 616 that includes the global cache controller circuit 622. The global cache controller circuit 622 is configured to communicate memory access requests to the system memory 618 via a memory controller 629 coupled to the system memory 618 so that, in the event of a complete cache miss and / or for memory write-back of the local cache memory 612(0)-612(D) (as an example), the system memory 618 will service the memory access requests. The global cache controller circuit 622 is also configured to communicate with each of the agent cache controller circuits 620(0)-620(D) to maintain cache consistency of the multi-level cache system 602.

[0097] In this way, as an example, a cache coherence protocol based on single-point cache coherence can be implemented in the multi-die processor-based system 600 through the global cache controller circuit 622. However, the proxy cache controller circuits 620(0)-620(D) in each die 604(0)-604(D) can still serve memory access requests contained in the local cache memory 612(0)-612(D) of their respective dies 604(0)-604(D) to achieve lower latency memory transactions. In other words, because the proxy cache controller circuits 620(0)-620(D) can only serve memory access requests that can be processed within their own die 604(0)-604(D) while still maintaining cache coherence, there is no need to configure the proxy cache controller circuits 620(0)-620(D) to always communicate memory access requests to the external global cache controller circuit 622 to maintain cache coherence. Communicating with the global cache controller circuit 622 for memory transactions would incur the cost of additional latency for those memory transactions.

[0098] Any process (including but not limited to processes 200, 200W, and 200A in Figures 2A-2C, processes 300A-300C in Figures 3A-3C, processes 400A-400C in Figures 4A-4C, and processes 500A-500B in Figures 5A-5B) can be executed in the multi-level cache system 602, wherein the proxy cache controller circuits 120(0)-120(D) mentioned therein are the proxy cache controller circuits 620(0)-620(D) in Figure 6, and the global cache controller circuit 122 mentioned therein is the global cache controller circuit 622 in Figure 6.

[0099] Referring again, each of the processors 606(0)-606(D) is coupled to the system bus 630 and can be coupled to the manager / subordinate or initiator / target devices included in the processor-based system 600. Although not shown in Figure 6, multiple system buses 630 may be provided, each forming a different fabric. As is well known, the processors 606(0)-606(D) can communicate with these other devices by exchanging address, control, and data information on the system bus 630. For example, the processors 606(0)-606(D) can communicate memory write and read requests to be serviced by the local cache memories 612(0)-612(D) and / or the system memory 618, which serve as slave devices, by interacting with their respective proxy cache controller circuits 620(0)-620(D) and the global cache controller circuit 622. The memory controller 629 interfaces between the global cache controller circuit 622 and the system memory 618 and can provide storage in one or more memory chips, such as dynamic random access memory (DRAM) chips for storing primary and / or redundant write data.

[0100] Other master devices and slave devices can be connected to the system bus 630 of the processor-based system 600. As shown in Figure 6, these devices may include, as an example, one or more input devices 632, one or more output devices 634, one or more network interface devices 636, and one or more display controllers 638. Input devices 632 may include any type of input device, including but not limited to input buttons, switches, voice processors, etc. Output devices 634 may include any type of output device, including but not limited to audio, video, other visual indicators, etc. Network interface devices 636 may be any device configured to allow the exchange of data with network 640. Network 640 may be any type of network, including but not limited to wired or wireless networks, private or public networks, local area networks (LANs), wireless local area networks (WLANs), wide area networks (WANs), Bluetooth™ networks, and the Internet. Network interface devices 636 may be configured to support any type of communication protocol required.

[0101] The processors 606(0)-606(D) can also be configured to access the display controller 638 via the system bus 630 to control information sent to one or more displays 642. The display controller 638 sends information to be displayed to the display 642 via one or more video processors 644, which process the information to be displayed into a format suitable for the display 642. The display 642 can include any type of display, including but not limited to cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light-emitting diode (LED) display, etc. The processors 606(0)-606(D) and their allocated local cache memories 612(0)-612(D), the proxy cache controller circuits 620(0)-620(D), the global cache controller circuit 622, the system memory 618, the network 640, the input device 632, and / or the display controller 638 can include computer instructions 646 as a non-transient computer-readable medium 648 to control their respective functions.

[0102] Those skilled in the art will further understand that the various illustrative logic blocks, modules, circuits, and algorithms described in conjunction with the various aspects of this disclosure can be implemented as electronic hardware, instructions stored in memory or other computer-readable media and executed by a processor or other processing device, or a combination of both. As an example, the master and slave devices described herein can be applied to any circuit, hardware component, integrated circuit (IC), or IC chip. The memory disclosed herein can be of any type and capacity and can be configured to store any type of information as required. To clearly illustrate this interchangeability, various illustrative components, blocks, modules, circuits, and steps have been described above in general form according to their functionality. How these functions are implemented depends on the specific application, design choices, and / or design constraints imposed on the system as a whole. Those skilled in the art can implement the described functions in alternative ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this disclosure.

[0103] The various illustrative logic blocks, modules, and circuits described in conjunction with the various aspects disclosed herein can be implemented or performed using a processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The processor may be a microprocessor, but alternatively, it may also be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration).

[0104] The various aspects disclosed herein can be embodied in hardware and in instructions stored in hardware, and can reside, for example, in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CD-ROMs, or any other form of non-transitory computer-readable medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a remote station. Alternatively, the processor and the storage medium can also reside as discrete components in a remote station, base station, or server.

[0105] It should also be noted that the operational steps described in any exemplary aspect herein are described for the purpose of providing examples and discussion. The described operations can be performed in many different orders other than the order illustrated. Furthermore, the operations described in a single operational step can actually be performed in many different steps. Additionally, one or more operational steps discussed in the exemplary aspects can be combined. It should be understood that the operational steps illustrated in the flowcharts can be modified in many different ways, which will be apparent to those skilled in the art. Those skilled in the art will also understand that information and signals can be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or light particles, or any combination thereof.

[0106] The foregoing description provided in this disclosure is intended to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A semiconductor bare die (bare die), comprising: A processor, comprising: Multiple central processing unit (CPU) cores; and Local cache memory, comprising multiple cache memories, each allocated to one of the multiple CPU cores; and The proxy cache controller circuitry is communicatively coupled to the multiple CPU cores; and The proxy cache controller circuit is configured as follows: In response to the first CPU core being unable to service the memory access request in the first cache memory allocated to the first CPU core in the plurality of cache memories, a memory access request including a memory address is received from the first CPU core; Determine whether the memory access request can be serviced in any other cache memory among the plurality of cache memories allocated to other CPU cores in the plurality of CPU cores; and In response to determining that the memory access request cannot be serviced in any of the other cache memories: The memory access request is communicated to the global cache controller circuitry on the second bare die; and Update the cache state of the memory address associated with the memory access request.

2. The bare core according to claim 1, wherein, The proxy cache controller circuit is configured as follows: In response to the cache line in the first cache memory corresponding to the memory read request being invalid, the memory access request including the memory read address and containing the memory read request is received from the first CPU core among the plurality of CPU cores. Determine whether the memory read request is a cache miss in each of the other cache memories allocated by the other CPU cores in the plurality of CPU cores; as well as In response to determining that the memory read request is a cache miss in each of the other cache memories: The memory read request is communicated to the global cache controller circuitry on the second bare die.

3. The bare core according to claim 2, wherein, In response to determining that the memory read request is a cache miss in each of the other cache memories, the proxy cache controller circuitry is further configured to: The global cache controller circuit receives read data for the memory read request.

4. The bare core according to claim 2, wherein, In response to determining that the memory read request is a cache hit in a second cache memory among the plurality of cache memories, the proxy cache controller circuitry is further configured to: The read snooping shared request is communicated to the second CPU core among the plurality of CPU cores allocated to the second cache memory; and This causes the read data used for the memory read request to be communicated to the first CPU core; The first CPU core is configured to write the read data used for the memory read request into the first cache memory.

5. The bare core according to claim 4, wherein, The proxy cache controller circuit is configured to cause the read data for the memory read request to be communicated to the first CPU core by causing the second CPU core to forward the read data for the memory read request to the first CPU core.

6. The bare core according to claim 4, wherein, The proxy cache controller circuitry is configured to cause the read data for the memory read request to be communicated to the first CPU core by performing the following operations: Receive the read data for the memory read request from the second CPU core; as well as The read data used for the memory read request is forwarded to the first CPU core.

7. The bare die according to claim 2 further includes a local cache line directory, the local cache line directory including a plurality of local cache line entries, each local cache line entry being indexable by a memory address, and each local cache line entry including a cache status indicator indicating whether the cache memory in the plurality of cache memories contains a valid cache line associated with its associated memory address. in, The proxy cache controller circuitry is configured to determine whether a memory read request is a cache miss by performing the following actions: Determine whether the cache status indicator in the local cache line entry indexed in the local cache line directory for the memory read address requested by the memory read indicates that read data exists in each of the other cache memories allocated by the other CPU cores in the plurality of CPU cores.

8. The bare die according to claim 4 further includes a local cache line directory, the local cache line directory including a plurality of local cache line entries, each local cache line entry being indexable by a memory address, and each local cache line entry including a cache status indicator indicating whether the cache memory in the plurality of cache memories contains a valid cache line associated with its associated memory address. in: The proxy cache controller circuitry is configured to determine whether a memory read request is a cache hit by performing the following actions: Determine whether the cache status indicator in the local cache line entry indexed by the memory read address of the memory read request in the local cache line directory indicates that the read data exists in the second cache memory among the plurality of cache memories; and The proxy cache controller circuitry is also configured to update the cache status indicator in the local cache line entry corresponding to the memory read address of the memory read request in the local cache line directory, to indicate that the read data for the memory read address exists in the first cache memory allocated to the first CPU core.

9. The bare core according to claim 1, wherein, The proxy cache controller circuit is configured as follows: The memory access request, which includes an exclusive ownership request for a memory write request and contains a memory write address, is received from the first CPU core among the plurality of CPU cores. Determine whether the write data for the memory write request is in a shared state in the other cache memory allocated by the other CPU cores in the plurality of CPU cores; as well as In response to determining that the write data for the memory write request is in a shared state in one or more other cache memories among the plurality of cache memories: Initiate a snoop clean invalidation to one or more other cache memories among the plurality of cache memories, so as to invalidate the write data in one or more other cache memories corresponding to the memory write address of the memory write request; as well as The exclusive ownership status of the memory write address used for the memory write request is communicated to the first CPU core; The first CPU core is configured to, in response to receiving the exclusive ownership state of the memory write address for the memory write request, store the write data for the memory write request in the first cache memory.

10. The bare core according to claim 9, wherein, The proxy cache controller circuit is configured to receive the memory access request in response to the cache line in the first cache memory corresponding to the memory write request being in a shared state.

11. The bare core according to claim 9, wherein, The proxy cache controller circuit is configured to receive the memory access request in response to the absence of a cache line in the first cache memory corresponding to the memory write request.

12. The bare core according to claim 9, wherein: The proxy cache controller circuit is also configured to, in response to determining that the data used for the memory write request is not owned by the proxy cache controller circuit: The exclusive ownership request for the memory write address used for the memory write request is communicated to the global cache controller circuitry; as well as The exclusive ownership status of the memory write address used for the memory write request; as well as The first CPU core is configured to, in response to receiving the exclusive ownership state of the memory write address for the memory write request, store the write data for the memory write request in the first cache memory.

13. The bare die according to claim 9, further comprising: The local cache line directory includes: Multiple local cache line entries, each indexable by a memory address, and each local cache line entry including a cache status indicator indicating whether the cache memory in the plurality of cache memories contains a valid cache line associated with its associated memory address; and Specifically, the proxy cache controller circuit is configured to determine whether the write data for the memory write request is in a shared state in the other cache memory among the plurality of cache memories by performing the following operations: Determine whether the cache status indicator in the local cache line entry indexed by the memory write address of the memory write request in the local cache line directory indicates that the write data exists in each of the other cache memories allocated by the other CPU cores in the plurality of CPU cores; and Determine whether the write data existing in one of the multiple cache memories is in a shared state.

14. The bare core according to claim 1, wherein, The proxy cache controller circuit is also configured as follows: In response to the evicting of a cache line corresponding to a memory write address from the first cache memory, the memory access request containing the memory write address and the write data, including a write-back request, is received from the first CPU core among the plurality of CPU cores. The write-back request is communicated to the global cache controller circuit. as well as In response to the global cache controller circuit writing the write data of the write-back request back to the system memory, a write-back response is received from the global cache controller circuit.

15. The bare core according to claim 13, wherein, In response to receiving the write-back response, the proxy cache controller circuitry is configured to update the cache state for the memory write address to a shared state.

16. The bare core according to claim 13, wherein, The first CPU core is configured to invalidate the written data in the first cache memory.

17. The bare core according to claim 1, wherein, The proxy cache controller circuit is configured as follows: In response to a cache line in the first cache memory allocated to the first CPU core being invalid, a memory atomic request containing a memory write address is received from the first CPU core among the plurality of CPU cores; Determine whether the atomic memory request is a cache hit in the second cache memory of the plurality of cache memories allocated to another CPU core; and In response to determining that the memory atomic request is a cache hit in the second cache memory, the proxy cache controller circuitry is further configured to: The exclusive ownership request for the memory write address used for the memory atomic request is communicated to the second CPU core; Receive read data from the second cache memory for the address to be written to the memory; Receive atomic operations for the memory atomic request from the first CPU core; Perform this atomic operation on the read data to generate the write data; as well as Store the written data.

18. The bare core according to claim 1, wherein, The proxy cache controller circuit is also configured as follows: In response to a cache line in the first cache memory allocated to the first CPU core being invalid, a memory atomic request containing a memory write address is received from the first CPU core among the plurality of CPU cores; Determine whether the memory atomic request is a cache miss in each of the other cache memories allocated by the other CPU cores in the plurality of CPU cores; In response to determining that the memory atomic request is a cache miss in each of the other cache memories, the proxy cache controller circuitry is further configured to: The memory atomic request is communicated to the global cache controller circuit; and Receive valid read data for the memory write address from the global cache controller circuit; Receive atomic operations for the memory atomic request from the first CPU core; Perform the atomic operation on the read data to generate the write data for the memory atomic operation; as well as The written data is stored for atomic operations on the memory.

19. A method for maintaining cache coherency in a multi-level cache memory system, comprising: In response to the first CPU core being unable to serve a memory access request in a first cache memory among the multiple cache memories allocated to the first CPU core, the proxy cache controller circuitry on the first die and communicatively coupled to the multiple central processing unit (CPU) cores receives the memory access request containing a memory address from the first CPU core among the multiple CPU cores. Determine whether the memory access request can be served in any other cache memory among the plurality of cache memories allocated to each of the other CPU cores in the plurality of CPU cores; as well as In response to determining that the memory access request cannot be served in any of the other cache memories: The memory access request is communicated to the global cache controller circuitry on the second bare die; and Update the cache state of the memory address associated with the memory access request.

20. A non-transient computer-readable medium storing computer-executable instructions, which, when executed by a proxy cache controller circuit located on a first die and communicatively coupled to a plurality of central processing unit (CPU) cores, cause the proxy cache controller circuit to: In response to the first CPU core being unable to serve a memory access request in the first cache memory among the plurality of cache memories allocated to the first CPU core, the memory access request containing the memory address is received from the first CPU core among the plurality of CPU cores. Determine whether the memory access request can be served in any other cache memory among the plurality of cache memories allocated to each of the other CPU cores in the plurality of CPU cores; and In response to determining that the memory access request cannot be served in any of the other cache memories: The memory access request is communicated to the global cache controller circuitry on the second bare die; and Update the cache state of the memory address associated with the memory access request.

21. A processor-based system, comprising: First Semiconductor bare die (bare die) includes: The first processor includes one or more first central processing unit (CPU) cores; A first local cache memory, comprising one or more first cache memories, each first cache memory allocated to a first CPU core in the one or more first CPU cores; and The first proxy cache controller circuit is communicatively coupled to the one or more first CPU cores; and The second bare core includes: The second processor includes one or more second CPU cores; The second local cache memory includes one or more second cache memories, each of which is allocated to a second CPU core in the one or more second CPU cores; and The second proxy cache controller circuit is communicatively coupled to the one or more second CPU cores; and Cache memory systems include: Global cache controller circuit; The first proxy cache controller circuit; and The second proxy cache controller circuit; The first CPU core in one or more first CPU cores is configured as follows: Initiate a memory access request to a memory address in the first cache memory of the one or more first cache memories allocated to the first CPU core; In response to the first CPU core being unable to serve the memory access request in the first cache memory, the memory access request is communicated to the first proxy cache controller circuit. The first proxy cache controller circuit in the first CPU core is configured as follows: Determine whether the memory access request can be served in any of the one or more first cache memories allocated by the other first CPU core in the one or more first CPU cores; and In response to determining that the memory access request cannot be served in any of the other first cache memories: The memory access request is communicated to the global cache controller circuit. The global cache controller circuit is configured as follows: Determine whether the memory access request can be served in any of the one or more second cache memories; In response to determining that the memory access request can be served in the second cache memory of the one or more second cache memories: The memory access request is communicated to the second proxy cache controller circuitry for service in the second cache memory; and Based on the memory access request served in the second local cache memory, update the cache state of the memory address associated with the memory access request.

22. The processor-based system according to claim 21, wherein: The second proxy cache controller circuit is configured as follows: Receive memory access prying requests from the global cache controller circuitry; and Forward the memory access snooping request to the second CPU core in the one or more second CPU cores allocated to the second cache memory in the one or more second cache memories; and The second CPU core is configured as follows: The memory access request is served from the second cache memory in one or more second cache memories.

23. The processor-based system according to claim 21, wherein: The first proxy cache controller circuit in the first CPU core is configured as follows: Determine whether the memory access request, which includes a memory read request, is a cache miss in each of the one or more first cache memories; as well as In response to determining that the memory read request is a cache miss in each of the one or more first cache memories: The memory read request is communicated to the global cache controller circuit. as well as The global cache controller circuit is configured as follows: Determine whether the memory read request is a cache miss in the second local cache memory; as well as In response to determining that the memory read request is a cache miss in the second local cache memory: A read snooping share request for the memory read request is initiated to the second proxy cache controller circuitry to be served in the second local cache memory; as well as Based on the memory read request served in the second local cache memory, update the cache state of the memory address associated with the memory read request.

24. The processor-based system according to claim 23, wherein: The second proxy cache controller circuit is configured as follows: Receive the read snooping share request from the global cache controller circuitry; and The read snooping share request is forwarded to the second CPU core in the one or more second CPU cores allocated to the second cache memory in the one or more second cache memories; The second CPU core is configured as follows: Access the second cache memory in the one or more second cache memories at the memory address of the memory read request; as well as The read data at the memory address of the memory read request in the second cache memory is communicated to the second proxy cache controller circuit; as well as The second proxy cache controller circuit is configured to communicate the read data for the memory read request to at least one of the global cache controller circuit and the first proxy cache controller circuit.

25. The processor-based system according to claim 23, wherein: The first bare die also includes a first local cache line directory, which includes multiple first local cache line entries. Each first local cache line entry is indexable by a first memory address, and each first local cache line entry includes a first cache status indicator for indicating whether the first cache memory in the one or more first cache memories contains a valid cache line associated with its first memory address; and It also includes a global local cache line directory, which includes multiple global cache line entries, each of which can be indexed by a memory address and includes a global cache status indicator for indicating the local cache memory containing a valid cache line associated with the second memory address. The first proxy cache controller circuit is configured to determine whether a memory read request is a cache miss by performing the following operations: Determine whether the first cache status indicator in the first local cache line entry of the memory address index requested by the memory read in the first local cache line directory indicates that read data exists in any of the one or more first cache memories; and The global cache controller circuit is configured as follows: Whether a memory read request is a cache miss in the second local cache memory is determined by being configured to perform the following operations: Determine whether the global cache status indicator in the global cache line entry of the memory address index requested by the memory read in the global local cache line directory indicates that the read data exists in the second local cache memory.

26. The processor-based system according to claim 25, wherein: The first proxy cache controller circuit is further configured to update the first cache status indicator in the first local cache line entry corresponding to the memory address of the memory read request in the first local cache line directory, to indicate that the read data for the memory address exists in the first cache memory allocated to the first CPU core; as well as The global cache controller circuit is also configured to update the global cache status indicator in the global cache line entry corresponding to the memory address of the memory read request in the global local cache line directory, to indicate that the read data for the memory read request exists in the first local cache memory and the second local cache memory.

27. The processor-based system according to claim 21, wherein: The first proxy cache controller circuit is configured as follows: The memory access request is received from the first CPU core among the plurality of CPU cores, which includes an exclusive ownership request for a memory write request containing a memory write address. Determine whether the write data used for the memory write request is not owned by the first proxy cache controller circuit; as well as In response to determining that the write data for the memory write request is not owned by the first proxy cache controller circuitry: The exclusive ownership request for the memory write address used for the memory write request is communicated to the global cache controller circuitry; and The global cache controller circuit is also configured as follows: Receive the exclusive ownership request for the memory write address initiated by the first proxy cache controller circuit for the memory write request; Initiate a snoop clean invalidation to one or more other cache memories among the plurality of cache memories, so as to invalidate the write data in one or more other cache memories corresponding to the memory write address of the memory write request; as well as The exclusive ownership status of the memory write address used for the memory write request is communicated to the first proxy cache controller circuitry. The first proxy cache controller circuit is also configured as follows: In response to receiving the exclusive ownership state of the memory write address for the memory write request, the exclusive ownership state of the memory write address for the memory write request is communicated to the first CPU core; as well as The first CPU core is configured to, in response to receiving the exclusive ownership state of the memory write address for the memory write request, store the write data for the memory write request in the first cache memory.

28. The processor-based system according to claim 27, further comprising: A global local cache line directory includes multiple global cache line entries, each of which can be indexed by a memory address and includes a global cache status indicator for indicating the local cache memory containing a valid cache line associated with a second memory address. The global proxy cache controller circuitry is configured to determine whether the write data for the memory write request is shared in other cache memories among the plurality of cache memories by performing the following operations: Determine whether the global cache status indicator in the first global cache line entry of the memory write address index of the memory write request in the global cache line directory indicates that the write data exists in each of the other cache memories allocated by the other CPU cores in the plurality of CPU cores; as well as In response to determining that the write data exists in at least one of the plurality of cache memories allocated by the other CPU cores in the plurality of CPU cores, the snoop clean invalidation is initiated to the one or more other cache memories in the plurality of cache memories to invalidate the write data in the one or more other cache memories corresponding to the memory write address of the memory write request.

29. The processor-based system according to claim 21, wherein, The global cache controller circuit is also configured as follows: The write data used for the memory write request is stored in system memory.

30. The processor-based system according to claim 21, wherein: The first proxy cache controller circuit is configured as follows: In response to a cache line in the first cache memory allocated to the first CPU core being invalid, a memory atomic request containing a memory write address is received from the first CPU core among the plurality of CPU cores; Determine whether the memory atomic request is a cache miss in each of the other cache memories allocated by the other CPU cores in the plurality of CPU cores; In response to determining that the memory atomic request is a cache miss in each of the other cache memories, the proxy cache controller circuitry is further configured to: The memory atomic request is communicated to the global cache controller circuit; and The global cache controller circuit is configured as follows: Receive atomic operations for the memory atomic request from the first proxy cache controller circuit; Initiate a memory read prying request to the memory controller coupled to the system memory for the atomic request of the memory at the memory write address; as well as Receive valid read data stored in the system memory at the write address of that memory; Perform the atomic operation on the read data to generate the write data for the memory atomic operation; as well as The write data used for the atomic operation of this memory is stored in the system memory.

31. The processor-based system according to claim 30, wherein, The proxy cache controller circuit is also configured to communicate the write data to the first proxy cache controller circuit.

Images