Inter-core cache stashing and target discovery

Abstract

A method and apparatus are disclosed for transferring data from a first processor core to a second processor core. The first processor core executes a stash instruction having a first operand associated with a data address of the data. The second processor core is determined to be a stash target of a stash message based on the data address or the second operand. The stash message is sent to the second processor core to notify the second processor core of the written data. In response to receiving the stash message, the second processor core can select to store the data in its cache. The data can be included in the stash message or can be retrieved in response to a read request by the second processing core. The second processor core can be determined by prediction based at least in part on monitored data transactions.

Description

[Technical field]

[0001] In a multi-core data processing system, data produced by a processing thread executing on a first processor core may be consumed by a processing thread executing on a second processor core. During operation, a producer core writes the data to a local cache or shared memory. A subsequent read operation by the second processor core on the newly written data results in a cache miss because the data has not been loaded into the second processor's local cache. The movement of the data to the consuming core's local cache causes a processing delay or latency. [Brief description of the drawings]

[0002] The accompanying drawings provide visual representations used to more fully explain various exemplary embodiments and may be used to enable those skilled in the art to better understand the disclosed exemplary embodiments and their advantages, in which like reference numerals identify corresponding or similar elements.

[0003] [Figure 1] FIG. 1 is a simplified block diagram of a multi-core data processing system, in accordance with various representative embodiments. [Diagram 2] 1 is a flowchart of a method for data transfer in a multi-core data processing system, according to various representative embodiments. [Diagram 3] 1 is an interaction chart of an exemplary inter-core stash operation, according to various representative embodiments. [Figure 4] 11 is an interaction chart for a further example of an inter-core stash operation, according to various representative embodiments. [Diagram 5] 1 illustrates an example of a Stash Target Table (STT), according to various representative embodiments. [Figure 6] 1 illustrates a further example of a Stash Target Table (STT), according to various representative embodiments. [Figure 7] 1 is an interaction chart illustrating a method for updating a stash target table, according to various representative embodiments. [Figure 8] 1 is an interaction chart illustrating a method for updating a stash target table, according to various representative embodiments. [Figure 9] 1 is an interaction chart illustrating a method for updating a stash target table, according to various representative embodiments. [Figure 10] 1 is an interaction chart illustrating a method for updating a stash target table, according to various representative embodiments. [Figure 11] 1 is an interaction chart illustrating a method for updating a stash target table, according to various representative embodiments. [Figure 12] 1 is a flowchart of a method for creating and updating a stash target table by a home agent, according to various representative embodiments. [Figure 13] 1 is an interaction chart of an exemplary inter-core stash operation, according to various representative embodiments. [Figure 14] 1 is an interaction chart of an exemplary inter-core stash operation, according to various representative embodiments. [Figure 15] 1 is an interaction chart of an exemplary stash target registration operation according to various representative embodiments. [Figure 16] 1 is an interaction chart of an exemplary inter-core stash operation, according to various representative embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] The various apparatus, devices, and methods described herein provide mechanisms for inter-core cache stashing in a multi-core data processing system.

[0005] While the present disclosure is susceptible of embodiments in many different forms, specific embodiments are shown in the drawings and described in detail herein, with the understanding that the embodiments shown and described herein should be considered as providing examples of the principles of the disclosure and are not intended to limit the disclosure to the specific embodiments shown and described. In the following description, like reference numerals are used to describe the same, similar or corresponding parts in the several views of the drawings. For simplicity and clarity of illustration, reference numerals may be repeated among the drawings to indicate corresponding or similar elements.

[0006] In this document, terms indicating relationships, such as first and second or top and bottom, may be used only to distinguish one entity or operation from another entity or action, without necessarily requiring or implying an actual such relationship or order between such entities or actions. The terms "comprise," "comprising," "include," "including," "has," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes at least those elements, and may also include other elements that are not expressly listed or that are inherent to such process, method, article, or apparatus. An element preceded by "comprise...a" does not, without further constraints, exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the element.

[0007] References throughout this specification to "one embodiment," "particular embodiment," "embodiment," "implementation(s)," "aspect(s)," or similar terms mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of such phrases in various places throughout this specification do not necessarily refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments, without limitation.

[0008] As used herein, the term "or" should be interpreted as inclusive or to mean any one or any combination. Thus, "A, B, or C" means any of the following: A; B; C; A and B; A and C; B and C; A, B, and C. Exceptions to this definition occur only when combinations of elements, features, steps, or acts are in some respect mutually exclusive by nature.

[0009] As used herein, the term "configured to," when applied to an element, indicates that the element can be designed or constructed to perform a specified function or has the required structure to enable the element to be reconfigured or adapted to perform that function.

[0010] 1 is a simplified block diagram of a multi-core data processing system 100, according to various representative embodiments. A first processor core 102, also referred to as a sender core or producer core, includes at least one local cache 104 (which may be private to that core or cluster of cores, for example), and a second processor core 106 includes at least one local cache 108. The second processor core 106 is also referred to as a receiver core or consumer core. In general, the data processing system 100 may include any number of processor cores, some of which may be configured in clusters and share lower level caches. The processor cores are coupled by an interconnect fabric 110, which is configured to pass messages between the processor cores and other devices coupled to the interconnect fabric. Also coupled to the interconnect fabric 110 is a home agent 112. Optionally, the home agent 112 may include a system level cache 114. Shared memory 116 is coupled to interconnect fabric 110 via memory controller 118. Processor cores access shared memory 116 by sending request messages to home agent 112. The home agent manages a set or range of memory addresses and acts as a point of data coherency and serialization, ensuring that copies of data stored in caches are consistent and up-to-date. It will be apparent to one skilled in the art that any number of other components may be included in the data processing system, such as I / O devices, network interfaces, additional memory, processing accelerators, etc. The system may be integrated into a single integrated module, a multi-chip module, or may be comprised of distributed components.

[0011] An embodiment of the present disclosure relates to transferring data written or produced by a sending processor core 102 to a cache 108 of a second processor core 106 before being read by the second processor core 106. When the second processor core executes an instruction to read data, if the data is already stored in the cache 108, the access time is significantly reduced. Thus, the disclosed mechanism provides an improvement to the operation of a data processing system. The data may include information in one cache line (e.g., 64 bytes) and may be identified by the upper bits of a data address or some other identifier. As used herein, the term "data address" refers to the address of a data element (e.g., a word or a byte), which is an identifier (e.g., the upper bits of an element address) of a data block that contains the address. If the data block is larger than the width of a data channel in the interconnect fabric, the data may be transferred in two or more flits (flow control digits) through the data channel.

[0012] Data may be associated with a data address. The data address may be, for example, a physical address in a backing memory or store, or an address of an input / output device. A processor core may operate in a virtual address space that is mapped to a physical address space. A virtual address may be mapped to a physical address through one or more intermediate addresses. To speed up processing, copies of the data may be stored in one or more caches within a data processing system. Such data is said to be "associated" with a data address. However, the data may be modified or written by the processor core and may not match the data stored at the associated address at a given time.

[0013] Data produced by a processing thread executing on a first processor core 102 can be consumed by a processing thread executing on a second processor core 106. In previous multi-core data processing systems, a producer core writes data to a local cache 104, a home agent 114, or a shared memory 116. A read operation by a consuming core 106 on the newly written data results in a cache miss. The data must then be moved to the consuming core's local cache, causing a processing delay or latency.

[0014] Inter-core stashing, as disclosed herein, provides a mechanism for transferring data from a first processor core 102 to the local cache 108 of a second processor core 106 before the data is read by the second processor core, thereby increasing the probability of a cache hit when reading the data.

[0015] Inter-core stashing mitigates performance degradation caused by cache misses caused by consumer processes, for example, in a producer-consumer communication pattern. In one embodiment, when a producer core 102 completes writing to a cache line, the producer core 102 executes a stash instruction that initiates a transfer of the cache line to the consumer core's private cache 108. If the transfer completes before the consumer core issues a request for the cache line, the consumer will get a hit in its private cache, significantly reducing access latency for the cache line.

[0016] In some applications, the producer thread knows which core is executing the consumer thread and can identify that core as the target of the stash operation. In one embodiment, the target core of the stash operation is specified by the producer processor core.

[0017] In other applications, it may be difficult or expensive for a producer thread to determine the target core or to propagate that information through multiple layers of software. For example, it may be difficult to propagate information from a Message Passing Interface (MPI) library to a low-level communication library such as a Unified Communication X (UCX) library.

[0018] In a further embodiment, the target core is identified automatically based on previous data usage by the processor core. In this embodiment, transactions from the processor core that allocate and deallocate data to the cache are monitored by the stash target table (STT) control circuitry 120 and can be used to update information in the stash target table 122. The stash target table 122 can be a memory such as a set associative cache. The cache can be indexed by the physical address or intermediate address of the cache line and can be fully or partially tagged. The stash target table stores a number of entries. Each entry stores an identifier of the stash target(s) for that address. The stash target table control circuitry 120 and the stash target table 122 can be implemented in the home agent or one or more processor cores and are used to identify one or more processor cores as stash target cores for data associated with a specified data address.

[0019] The home agent 112 may also include a snoop filter 124 that also monitors transactions from the processor cores that allocate and deallocate data to the cache. The STT 122 may be implemented by adding an additional field to entries in the snoop filter's cache that identifies the predicted stash target.

[0020] Thus, according to an embodiment of the present disclosure, data processing system 100 includes interconnect fabric 110, with multiple processor cores (102, 106) interconnected by the interconnect fabric. At least one processor core is configured to write data associated with a first data address to a cache and execute a first stash instruction for the first data address. Stash target table control circuitry 120 is configured to monitor transactions from the processor cores that allocate and deallocate data to the processor core's cache, and to create and update entries in stash target table 122. Stash target table control circuitry 120 is also configured to, in response to the first stash instruction, predict that one or more processor cores are stash target cores for the data associated with the first data address, and to send a stash message to the stash target cores informing the stash target cores of the data associated with the first data address and produced by the first processor core. A second processor core of the one or more stash target cores is configured to store data associated with the first data address in a cache of the second processor core in response to receiving the stash message.

[0021] 1, an embodiment of the data processing system may also include a home agent 112 that is coupled to the processor cores by interconnect fabric 110 and provides a point of coherence and serialization for the data processing system. In this embodiment, the home agent includes stash target table control circuitry 120. In alternative embodiments, stash target table control circuitry 120 may be implemented in one or more of the processor cores.

[0022] The stash instruction executed by the producer thread specifies a data address. The instruction may or may not also specify an identifier of the intended stash target core. In the absence of an explicit target in the instruction, the stash target may be determined in various ways. For example, the receiving core may register itself with the home agent as a stash target, or the home agent may predict the stash target. In some embodiments, the prediction is based on the home agent monitoring shared-to-unique, unique-to-shared transitions of data addresses, and read operations on previously stashed data addresses.

[0023] A method of data transfer is disclosed in which a first processor core (sending core) of a data processing system writes data associated with a first data address to a first cache. The write may include, for example, generating data and associating it with the first data address in the cache, or modifying data in the cache associated with the first address. In either case, the first processor core may claim ownership of the data associated with the first address and cause copies of the data in other caches to be invalidated. The first data address may be, for example, a virtual address or a physical address.

[0024] FIG. 2 is a flowchart 200 of a method for data transfer in a multi-core data processing system, according to various representative embodiments. Following start block 202 of FIG. 2, a stashing operation begins with execution of a stash instruction by a producer processor core in block 204. The stash instruction has one input register or operand that specifies the data address to be stashed, and optionally another input register or operand that identifies the stash target(s). The operation is performed on a cache line in the cache of the first processor core that holds the data associated with the address. The stash instruction may be included in the Instruction Set Architecture (ISA) of the data processing system. Referring again to FIG. 2, if the stash instruction includes an identifier for the stash target, as indicated by the yes branch from decision block 206, then a stash message is sent over the interconnect fabric to the identified stash target (recipient or consumer processor core) in 208. The stash message identifies the data address and provides a notification or hint to the stash target that the data associated with the address is available. The stash message may be sent to the stash target directly or via a home agent. If sent via a home agent, the home agent generates the stash message and determines the target from the message from the target identifier. If the stash target chooses to accept the data, as indicated by the YES branch from decision block 210, then in block 212 the stash target prefetches the data associated with the data address. Execution of the stash instruction is then completed, as indicated by block 214. Execution is also completed if the stash target determines not to accept the data, as indicated by the NO branch from decision block 210.

[0025] If the stash instruction does not include an identifier for the stash target, as indicated by the NO branch from decision block 206, the stash target is predicted from previous data transactions in the data processing system. In block 216, a stash target table (STT) is accessed based on the data address to identify one or more stash targets. In one embodiment, the STT is accessed by a producer processor core. In a further embodiment, the producing processor core sends a stash request over the interconnect fabric to a home agent, which accesses the STT. The stash request includes the first data address. As described above, the home agent serves as a point of coherency and serialization for memory shared between processor cores or other addressable elements of the data processing system. The home agent may include a snoop filter that records the presence and coherency state of data in various caches of the data processing system. Entries in the snoop filter's caches are updated by monitoring transactions from processor cores that allocate and deallocate data to those caches. The home agent may also include a system level cache or a last level cache. The home agent records which processing cores share copies of the cache line (which may be recorded, for example, as a presence vector) and, if applicable, which of the sharers is designated as having ownership of the cache line.

[0026] If an entry for the data address is found in the stash target table and the entry identifies one or more second processor cores as targets, as indicated by the YES branch from decision block 218, flow continues to block 208, where a stash message is sent to each of the target processor cores. The stash message notifies the target processor cores that newly written data associated with the first data address is available. The target processor cores can either further ignore the stash message or prefetch the data associated with the first data address into a cache of the second processor core, which may be, for example, a local or private cache of the second processor core, or a lower level cache of a cluster of cores. If the second processor core executes an instruction to read the data before the data is evicted from the cache, a cache hit occurs and the time to read the data is reduced. The cache hit improves the performance of the data processing system by reducing the time to perform a read operation on the data. If an entry for the data address is not found in the stash target table, as indicated by the NO branch from decision block 218, or if the found entry does not specify a target, execution of the stash instruction is completed, as indicated by block 214. The stash target table may, however, be updated as described in more detail below.

[0027] 3 is an exemplary inter-core stash operation interaction chart 300, according to various representative embodiments. The chart shows a sender or producer core S timeline 302, a home agent timeline 304, and a receiver or consumer core R timeline 306, with time flowing downwards. At 308, the sender core writes data associated with data address A to the sender core's cache line. At 310, the sender core executes a stash instruction for data address A, indicating core R as the stash target. The stash instruction may take the following form: STASH data_address, target_id where the operand data_address may be a virtual address, an indicator of a range of virtual addresses, or some other indicator of one or more data addresses. The operand target_id identifies one or more processor cores as cache targets. It may be, for example, a numeric identifier or a bit vector, in which the bit positions correspond to the stash targets.

[0028] A message 312 is sent to the home agent. In response, the home agent sends a stash message 316 to the target core R. The stash message 316 indicates that the data associated with data address A has been modified. Core R can choose to ignore the message or to prefetch the data associated with data address A. If core R chooses to prefetch the data associated with data address A, core R sends a read request 318 for the data to the home agent. In response, the home agent sends a message 320 to the sending core requesting that the data associated with data address A be sent to core R. Messages from the home agent to the processor cores are sometimes referred to as "snoop" messages. The sending core completes the stash operation by sending the data associated with data address A to the receiving core, core R, in message 322. The sent data is stored in the local cache of core R. When the receiving core R performs the read operation at 324, the data is already in the local cache, so the read results in a cache hit and the data can be loaded with minimal latency. In this manner, the performance of the data processing system is improved.

[0029] In a further embodiment, when a stash command specifies a stash target, the stash message may be sent directly from the sending core to the receiving core. In this case, after sending the data to core R, an acknowledgment may need to be sent from the sending core to the home agent to enable the home agent to monitor data coherency.

[0030] 4 is an interaction chart 400 for a further example of an inter-core stash operation, according to various representative embodiments. The chart shows a sender or producer core S timeline 402, a home agent timeline 404, and a receiver or consumer core R timeline 406, with time flowing downwards. At 408, the sender core writes data associated with data address A to the sender core's cache line. At 410, the sender core executes a stash instruction for data address A without the direction of the stash target. The stash instruction may take the following form: STASH data_address In the formula, the operand data_address may be, for example, a virtual address.

[0031] In FIG. 3, the home agent determined the stash target directly from message 312. However, in FIG. 4, message 412 includes data address A but does not include a target identifier. In this embodiment, the home agent determines the stash target(s) based on the data address A indicated in the stash message 412 by accessing a stash target table at 414. If the table includes an entry for data address A, and that entry identifies one or more stash targets, a stash message 416 is sent to the receiving core identified for data address A. The method continues as described with reference to FIG. 3, with interactions 418-424 corresponding to elements 318-324. If core R selects to prefetch the data associated with data address A, core R sends a read request 418 for the data to the home agent. The home agent in response sends a message 420 to the sending core requesting that the data associated with data address A be sent to core R. The sending core completes the stash operation by sending the data associated with data address A to the receiving core, core R, in message 422. The sent data is stored in the local cache of core R. When the receiving core, core R, performs the read operation at 424, the data is already in the local cache, so the read results in a cache hit and the data can be loaded with minimal latency.

[0032] In the approach described with reference to Figure 3, an identifier in a stash message is used to determine the stash target, while in the approach described with reference to Figure 4, a stash target table is used to determine the stash target. Some example mechanisms for creating and maintaining the stash target table are described below with reference to Figures 7-11. In general, a home agent creates and maintains stash targets by monitoring data coherency operations on data transfers and data addresses.

[0033] 3 and 4 thus illustrate an example method of data transfer in a data processing system. According to the method, a first processor core writes data associated with a first data address to a first cache and sends a first stash request including the first data address to a home agent of the data processing system. The home agent or producer core determines a second processor core to receive the stash message based at least in part on the first stash request and sends the stash message to the second processor core. Upon receiving the stash message, the second processor core may select to prefetch the data associated with the first data address into a cache of the second processor core. In one embodiment, the first stash request includes an identifier of the second processor core. In a further embodiment, the stash target is determined by accessing a first entry in a stash target table that is associated with the first data address.

[0034] FIG. 5 illustrates an example of a stash target table (STT) 500, according to various representative embodiments. Each entry in the table 500 includes a data address 502 and an identifier 504 of one or more stash targets. Initially, an entry may have a data address but no stash target. The data address 502 may be a physical address or an intermediate address such as an apparent physical address or a ghost physical address. The data address may be, for example, an identifier of a cache line or a cache page. More generally, memory may be logically divided in regions with a specified granularity (smaller than, larger than, or equal to a cache line), in which case the data address may be an identifier of the region. A data processing system may include multiple SSTs using different granularities.

[0035] FIG. 6 illustrates a further example of a stash target table (STT) 600, according to various representative embodiments. Each entry in the table includes a base data address 602, an identifier 604 of one or more stash targets, and a size 606. The base address and size together indicate a range of size addresses (e.g., size cache lines, size words, or size bytes) from base_address to base_addess+size-1. Thus, the first entry in STT 600 indicates stash targets for addresses A, A+1, A+2, and A+3, equivalent to the first four entries in table 500 of FIG. 5. In this example, the stash targets are specified as bit vectors. For example, the identifier for the second entry in table 600 is the bit vector 00001010, with the 1 position indicating stash targets 2 and 4 for data address B.

[0036] The range of entries specified by base data address 602 and size 606 may be constrained so as not to cross specified memory boundaries, allowing implementation as a set-associative table, requiring searching only one set for each table lookup operation.

[0037] The STT may be located in the home agent or in each of several processor cores. Alternatively, the stash target information may be stored in a snoop filter cache of the home agent. For each cache line, the snoop filter cache may include an entry with data fields that record the sharers, owners, and coherency state for several cache lines. Additional fields may be provided to record the stash target for the cache line. This is efficient for applications where most of the cache lines have a cache target, but is less efficient for applications where only a small percentage of the cache lines have a stash target.

[0038] In the remainder of this document, various exemplary methods for creating and updating an STT are disclosed. These examples are described with reference to a home agent. However, equivalent methods may be used to maintain an STT in a processor core.

[0039] FIG. 7 is an interaction chart 700 of a method for updating a stash target table, according to various representative embodiments. According to this method, a receiving processor core sends a target registration message for a data address to a home agent. In response, the home agent creates an entry in the stash target table, with a first entry identifying the receiving processor core as the target of the data associated with the data address. Referring to FIG. 7, the chart shows a timeline 702 of a sender or producer core S, a timeline 704 of a home agent, and a timeline 706 of a receiver or consumer core R, with time flowing downwards. At 708, the receiving core R executes a register instruction that indicates to the hardware that software running on the receiving core (future target) expects to read data that will be stashed by another core. The data may be specified by a virtual address. For example, the register instruction may take the form: STASH_TARGET virtual_address The core executing this instruction is identified as the target of a future stash for the cache line containing the physical address corresponding to this virtual address.

[0040] Alternatively, the STASH_TARGET instruction may specify a virtual address range, for example, of the following form: STASH_TARGET base_virtual_address,size This instruction identifies all cache lines associated with the corresponding physical address as potential targets for a stashing operation targeted at the core executing the STASH_TARGET instruction.

[0041] Execution of the REGISTER instruction results in the transmission of a message 710 from the receiving core R to the home agent. Note that the virtual address may be translated into an intermediate address or a physical address for inclusion in the message 710.

[0042] In some applications, threads are pinned to a core. In other applications, the STASH_TARGET instruction may need to be executed or re-executed after the consumer thread is moved. This is to ensure that the reported target core is correct when the actual stash operation is performed. There are various opportunities to execute a STASH_TARGET instruction when initializing, reading, or recycling data buffers. Therefore, it may not be necessary to proactively execute a STASH_TARGET instruction immediately after each thread move. It is common for threads to be executed by a core for longer than the program iterations that follow stashing. As a result, only a small percentage of stashing opportunities may be lost by not registering proactively.

[0043] At 712, the home agent creates or updates an entry in the stash target table for the data address A that indicates the receiving core R as the stash target.

[0044] Again, the snoop filter in the home agent records metadata associated with each cache line (including the coherency state of the cache line in various caches, which cores have copies of the cache line, and which core owns the cache line, if any), so the metadata can be extended to include a stash target field (e.g., a bit vector) that stores the stash target for the cache line.

[0045] In an alternative embodiment, the stash target table described above may be used to record the reported target core information. This stash target table need only record the data for which the cache line is to be stashed, which may be a small fraction of the total number of cache lines stored in the snoop filter cache. The stash target table may be a set associative cache indexed by the cache line physical address, fully or partially tagged, and storing one or more stash targets for each entry. A recency-based cache replacement policy may be used, such that the least recently used entries in the table are eventually replaced, including entries that have become stale due to thread movement, etc. The only potential impact of an inaccurate stash target due to a stale entry is the occurrence of the same cache miss that would occur without the stash. The target will be corrected when the consumer thread re-executes a new STASH_TARGET instruction for the line.

[0046] As described above, the stash target table can specify address ranges. This reduces the number of entries required, for example, when stashing large messages or when the communication library operates with large buffers. When address ranges are specified, address range overlaps between entries can be detected and old entries can be updated and / or invalidated to prevent overlaps.

[0047] 8 is a further exemplary update interaction chart 800 of a stash target table, according to various representative embodiments. The chart shows, with time flowing downwards, a sender or producer core S timeline 802, a home agent timeline 804, and a receiver or consumer core R timeline 806. In this embodiment, the home agent updates the stash target table when a change in the data coherency state occurs.

[0048] A core (sender) wishing to write to a cache line must request the line in its native state and invalidate all copies of the line in the private caches of other cores. This establishes the core as having ownership of the cache line. Cores that get the invalidation message (sharers of the line) are previous readers or writers of the line, and are therefore potential stashing targets for the newly created cache line when the sender performs a stash operation on the line. Typically the time between writing a buffer cache line and stashing it is short (often less than a few thousand cycles), which creates an opportunity to identify cores whose copies of the cache line have recently been invalidated as targets for subsequent stashing operations.

[0049] 8, at 808, sending core S initiates a write operation for the data associated with data address A. In this example, since the cache line is shared with another core, it sends message 810 to the home agent to request ownership of the appropriate cache line for core S. In response, the home agent accesses its snoop filter and identifies receiving core R as a sharer of the cache line. The home agent then sends invalidation message 812 to receiving core R to invalidate the cache line in the receiving core's cache.

[0050] From this transaction, it is clear that the sending core is a producer of the data associated with data address A, while the receiving core was, at least formerly, a consumer (or producer) of the data. Based on this knowledge, the home agent creates or updates an entry in the stash target table, at 814, indicating that receiving core R is a potential stash target of the data associated with data address A. The operation is completed by receiving core R acknowledging an invalidation message to the home agent in message 816, which in turn sends an acknowledgment to sending core S in message 818. The data in the local cache of sending core S is now in an inherent coherency state, and sending core S has ownership of the cache line.

[0051] This approach allows the stash target table to be updated by the home agent without the core needing to know the identity of the stash target or the stash source, which is useful when it is difficult for software to access the stash target ID, for example in a virtualized environment, and when there are multiple layers of libraries, such as a networking stack.

[0052] This mechanism is useful, for example, for use with a communication library that reuses a buffer for communication between the same two cores as before, in which case the snoop filter will repeatedly see the same shared-to-unique transition for a given address (cache line).

[0053] If there is more than one sharer when the uniquing request is received at the home agent, the home agent may insert multiple targets into the stash target table.

[0054] As shown in Figure 9, another opportunity to predict a stash target exists when a dirty cache line (i.e., a cache line containing data that has been written but not yet written to memory) is requested and sent by another core. The requesting core is a potential stash target of the stashing operation. The home agent can insert the stash target into the stash target table once it receives an acknowledgment for the send from the sender according to the cache coherence protocol.

[0055] 9 is an interaction chart 900 of a further method for updating a stash target table, according to various representative embodiments. The chart shows a sender or producer core S timeline 902, a home agent timeline 904, and a receiver or consumer core R timeline 906, with time flowing downwards. In this embodiment, the home agent again updates the stash target table when a data coherency state change occurs. However, in this case, the table is updated when a cache transitions from a unique dirty state to a shared dirty state.

[0056] 9, receiving core R requests data associated with data address A by sending a request 910 to the home agent at 908. The home agent accesses its snoop filter, identifies sending core S as the owner of the cacheline, and sends a snoop request 912 to sending core S. In response, sending core S sends the cacheline to receiving core R in message 914 and acknowledges the transaction to the home agent in message 916. From this transaction, and the knowledge that the data is "dirty" (i.e., has been modified), it is clear that sending core S is the producer of the data associated with data address A, and receiving core R is the consumer of the data. Thus, at 918, the home agent creates an entry in the stash target table for data address A that identifies receiving core R as the stash target.

[0057] The approach of Figure 9 works if the updated stash target table entry has not been evicted from the stash target table by the time the sender writes to the cache line and executes the stash instruction. However, it is recognized that not all sent cache lines will be actual targets of future stashing operations.

[0058] 10 is an interaction chart 1000 of further exemplary updates of a stash target table, according to various representative embodiments. The chart shows a timeline 1002 of a sender or producer core S, a timeline 1004 of a home agent, and a timeline 1006 of a receiver or consumer core R, with time flowing downwards. When a stash command without a stash target is executed at 1008, a corresponding message 1010 is sent to the home agent. The stash target table is accessed at 1012 to determine that the receiver core R is the stash target. A stash message 1014 is sent to the stash target. In this example, the receiver core R chooses to reject the stash message and sends a message 1016 acknowledging receipt of the stash message and rejecting the data associated with data address A. The home agent updates the stash target table at 1018 to remove receiver core R as a stash target for data address A.

[0059] FIG. 11 is an interaction chart 1100 of a further method for updating a stash target table, according to various representative embodiments. In the illustrated approach, an entry is inserted into the stash target table only for the cache line specified in the stash instruction. The chart shows a timeline 1102 of a sender or producer core S, a timeline 1104 of a home agent, and a timeline 1106 of a receiver or consumer core R, with time flowing downwards. When a stash instruction without a stash target is executed at 1108, a corresponding message 1110 is sent to the home agent. In the example shown in FIG. 11, the cache line is in an inherently dirty state; that is, only the sender core has a valid copy of the data. The data is dirty because it has not yet been written back to memory. In response to the stash message, the home agent creates an entry in the stash target table at 1112 for the data address A (assuming that such an entry does not already exist). However, the new entry created at 1112 does not specify any stash target.

[0060] Later, at 1114, receiving core R initiates a read operation. Since the data is not in core R's local cache, a request 1116 for the data is sent to the home agent. The home agent sends a snoop message 1118 to sending core S, which delivers the requested data to receiving core R in message 1120. Sending core S then acknowledges the transaction to the home agent in message 1122. At 1124, the home agent updates the entry for data address A to indicate receiving core R as a stash target for data address A.

[0061] It will be apparent to one skilled in the art that other events may be used to identify targets in the stash target table, including, for example, eviction of data from the recipient's private cache, transitions from shared to private, and sending of dirty cache lines, such that the targets become available for future stash instructions.

[0062] The stash targets in the stash target table may be reset upon execution of a specified number of stash instructions, allowing the system to re-learn the stash targets for the next iteration. Alternatively, this may be done using feedback from the recipient as to whether the stash was useful.

[0063] The stash target table's capacity is bounded so that it can follow a recency-based replacement policy for its entries (such as Least Recently Used). Software can take the stash target table capacity into account when determining how many lines it can reasonably stash without specifying a stash target.

[0064] 12 is a flowchart 1200 of a method for creating and updating a stash target table by a home agent, according to various representative embodiments. In block 1202, the home agent receives a message from a processor core specifying a data address. If the home agent determines that there is currently no entry associated with the data address in the stash target table, as indicated by the NO branch from decision block 1204, flow proceeds to decision block 1206. If the message received from the processor core is not a stash message or a register target message, as indicated by the NO branch from decision block 1206, no updates are made to the stash target table, as indicated by block 1208. If the message from the processor core is a stash message or a register target message, a new entry for the data address is created in the stash target table in block 1210, as indicated by the YES branch from decision block 1206, and flow proceeds to decision block 1212. If an entry for the data address already exists in the table, as indicated by the YES branch from decision block 1204, flow also proceeds to decision block 1212.

[0065] If the received message is a request to read modified or dirty data from a producer core or an explicit request to register the core as a stash target, as indicated by the yes branch from decision block 1212, then in block 1214, an identifier of the core that sent the message to the home agent is inserted into the entry for the specified data address in the stash target table.

[0066] If the received message is a request from the producer core for ownership of the cache line (i.e., a request to make the cache line unique) or a stash request from the producer core, as indicated by the YES branch from decision block 1216, the home agent determines from the snoop filter which other cores, if any, are sharing the cache line and inserts identifiers of those cores into the stash target table entry for the data address, in block 1218. If the message is some other message, as indicated by the NO branch from decision block 1216, the update of the stash target table is completed for the received message, as indicated by block 1220.

[0067] In this manner, the stash target table may be created and updated by the home agent in response to messages received from the processor cores. If the sending core knows the target of the stash operation, an identifier of the target may be included in the stash message. If the recipient of the stash operation knows the source of the updated data, the recipient may register with the home agent. In other cases, the home agent determines the stash target based on interactions between the processor cores.

[0068] In one embodiment, the home agent generates an entry in the stash target table for the data address in response to receiving a stash request for the data address from the first processing core. When data associated with the data address is stored in a cache of one or more second processing cores, the data in the cache is invalidated in response to a request by the first processing core to make the data in the cache of the first processing core unique. The home agent updates the entry to identify the one or more second processing cores as targets for future stash requests for the data address.

[0069] An entry may also be created or updated when a read request for data associated with the data address is routed by the home agent from the second processing core to the first processing core, with the updated entry identifying the second processing core as the target of a stash request for the data address.

[0070] More generally, in response to a coherency state of data associated with a first data address changing from unique to shared or from shared to unique in a cache of a first processor core, an entry in the stash target table for the first data address is updated by the home agent.

[0071] For example, the home agent may generate an entry in the stash target table for a data address in response to a stash message from a first processing core, and later, in response to sending data associated with the data address to a second processing core, update the entry to identify the second processing core as a stash target for the data address.

[0072] FIG. 13 is an interaction chart 1300 of an inter-core stash operation, according to various representative embodiments. The chart shows a timeline 1302 of a sending or producer core S, a timeline 1304 of a first receiving core R1, and a timeline 1306 of a second receiving core R2, with time flowing downwards. At 1308, the sending core writes data associated with data address A to a cache line of the sending core. At 1310, the sending core executes a stash instruction for data address A, indicating cores R1 and R2 as stash targets. The sending core S sends a stash message 1312 to the receiving core R1 and a stash message 1314 to the receiving core R2. Alternatively, the stash message may be broadcast to all other receiving cores. The stash message provides a notification or hint to the receiving core that data associated with address A is available at the sending core S. In the example shown, receiving core R1 chooses to reject the data at 1316, while receiving core R2 chooses to accept the data at 1318. Receiving core R2 requests the data in message 1320, and the data is returned from sending core S in send message 1322. Receiving core R2 then executes a read instruction for the data associated with address A at 1324. The instruction results in a cache hit, since the data has already been stashed in receiving core R2's local cache. In this embodiment, a home agent is not required. Each core can monitor data transactions to track data coherency. Additionally, each core can maintain a stash target table.

[0073] FIG. 14 is a further inter-core stash operation interaction chart 1400 according to various representative embodiments. The chart shows a timeline 1402 of a sending or producer core S, a timeline 1404 of a first receiving core R1, and a timeline 1406 of a second receiving core R2, with time flowing downwards. At 1408, the sending core writes data associated with data address A to a cache line of the sending core. At 1410, the sending core executes a stash instruction for data address A without specifying a stash target. At 1412, the sending core accesses the stash target table. If an entry exists in the stash target table for data address A, a stash message is sent to each of the stash targets specified in the entry. In the illustrated example, the entry specifies receiving core R1 as the only target, and a stash message 1414 is sent to receiving core R1. In this embodiment, the stash message includes the data written at 1408. At 1416, the receiving core R1 can choose to accept the data and store it in its local cache, or reject the data. Alternatively, the receiving core R1 can choose to reject the data, in which case a response such as an acknowledgment may be specified by the protocol.

[0074] It will be apparent to one skilled in the art that the stash message of FIG. 13 can alternatively include data rather than notifications, and the stash message of FIG. 14 can alternatively include notifications rather than data.

[0075] FIG. 15 is an interaction chart 1500 of an exemplary stash target registration operation, according to various representative embodiments. The chart shows a timeline 1502 of a first processor core, a timeline 1504 of a second processor core, and a timeline 1506 of a third processor core, with time flowing downwards. At 1508, core 3 executes a registration instruction to register with other cores as a stash target for data address A. A registration message is then broadcast to other cores in the processing system. In this example, a registration message 1510 is sent to core 2, and a registration message 1512 is sent to core 1. Upon receiving the messages, the cores record the data address in their respective stash target tables (core 2 at 1514 and core 1 at 1516). The table entry for address A then identifies core 3 as the stash target. In this manner, a stash table can be created at each core. Similarly, data transactions between cores can be monitored and used to predict which cores will benefit from receiving data or stash notifications.

[0076] As mentioned above, a stash message can contain data or a stash notification. An example is shown in FIG.

[0077] 16 is an interaction chart 1600 of a further example of an inter-core stash operation, according to various representative embodiments. The chart shows a sender or producer core S timeline 1602, a home agent timeline 1604, and a receiver or consumer core R timeline 1606, with time flowing downwards. At 1608, the sender core writes data associated with data address A to the sender core's cache line. At 1610, the sender core executes a stash instruction for data address A without the direction of the stash target. The stash instruction may take the following form: STASH data_address where the operand data_address may be a virtual address or some other indicator of a cache line (such as an address tag), or the stash instruction may specify one or more stash targets.

[0078] In the example shown in FIG. 16, message 1612 includes data address A and data associated with data address A, but does not include a target identifier. Thus, at 1614, the data is stored in the home agent's system cache. Also at 1614, the home agent determines the stash target(s) by accessing a stash target table based on data address A specified in stash message 1612. If the table includes an entry for data address A, and the entry specifies one or more stash targets, then stash message 1616 is sent to the receiving core specified for data address A. When core R selects to prefetch the data associated with data address A, core R sends a read request 1618 for the data to the home agent. In response, the home agent sends the requested data to receiving core R in message 1620. The data is stored in core R's local cache. When the receiving core, core R, performs a read operation at 1622, the data is already in its local cache, so the read results in a cache hit and the data can be loaded with minimal latency.

[0079] Numerous details have been described to provide an understanding of the embodiments described herein. The embodiments may be practiced without these details. In other instances, well-known methods, procedures, and components have not been described in detail to avoid obscuring the described embodiments. The present disclosure should not be considered as limited in scope to the embodiments described herein.

[0080] Those skilled in the art will recognize that the present disclosure has been described by way of example. The present disclosure may be implemented using equivalents of hardware components, such as dedicated hardware and / or dedicated processors, that are equivalent to the present disclosure as described and claimed. Similarly, alternative equivalent embodiments of the present disclosure may be constructed using dedicated processors and / or dedicated hardwired logic.

[0081] Dedicated or reconfigurable hardware components used to implement the disclosed mechanisms may be described by instructions in a hardware description language (HDL), such as, for example, VHDL, Verilog, or RTL (Register Transfer Language), or by a netlist of components and connections. The instructions may be at the functional level, the logical level, or a combination thereof. The instructions or netlist may be input to an automated design or manufacturing process (sometimes called high-level synthesis) that interprets the instructions and creates digital hardware that implements the described functions or logic.

[0082] The HDL instructions or netlist may be stored in a non-transitory computer readable medium such as Electrically Erasable Programmable Read Only Memory (EEPROM); non-volatile memory (NVM); mass storage devices such as hard disk drives, floppy disk drives, optical disk drives, etc.; optical storage devices, magnetic storage devices, magneto-optical storage devices, flash memory, core memory, and / or other equivalent storage technologies without departing from this disclosure. Such alternative storage devices should be considered equivalents.

[0083] Various embodiments described herein are implemented using dedicated hardware, configurable hardware, or programmed processors that execute programming instructions, broadly described in flow chart format, that may be stored on any suitable electronic storage medium or transmitted over any suitable electronic communication medium. Combinations of these elements may also be used. Those skilled in the art will appreciate that the above-described processes and mechanisms may be implemented in any number of variations without departing from the present disclosure. For example, the order of certain operations performed can often be changed, additional operations can be added, or operations can be removed, without departing from the present disclosure. Such variations are contemplated and considered equivalent.

[0084] The various representative embodiments described in detail herein are presented by way of example and not limitation, and those skilled in the art will understand that various changes in form and detail of the described embodiments may be made resulting in equivalent embodiments that remain within the scope of the appended claims.

Claims

1. writing, by a first processor core of the data processing system, data associated with the first data address to a first cache; executing, by the first processor core, a stash instruction having as a first operand a virtual address corresponding to the first data address; determining, based at least in part on the first data address or a second operand of the stash instruction, that a second processor core of the data processing system is a stash target of a stash message; sending the stash message including the first data address to the second processor core; in response to receiving the stash message, the second processor core stores the data associated with the first data address in a cache within the second processor core; (b) the method of data transfer, including

2. 2. The method of claim 1, further comprising: the first processor core determining that the second processor core is the stash target; and sending the stash message to the second processor core.

3. The method of claim 1 , wherein the second operand of the stash instruction comprises an identifier for the second processor core.

4. The method of claim 1 , wherein the stash message includes the data associated with the first data address.

5. the first cache is a cache of the first processor core; The second processor core storing the data associated with the first data address in the cache of the second processor core, the second processor core requesting the data associated with the first data address; the first processor core sending the data associated with the first data address to the second processor core; The method of claim 1 , comprising:

6. 2. The method of claim 1, wherein the determining comprises accessing a first entry in a stash target table that is associated with the first data address.

7. Prior to the determination, sending, by the second processor core, a target registration message for the first data address; creating the first entry in the stash target table, the first entry identifying the second processing core as a target for data associated with the first data address; The method of claim 6 further comprising:

8. Prior to the determination, 7. The method of claim 6, further comprising: in response to receiving a prior stash request for the first data address from the first processing core, generating the first entry in the stash target table for the first data address.

9. updating the first entry in the stash target table in response to a coherency state of the data associated with the first data address in a cache of the first processor core changing from unique to shared or from shared to unique; The method of claim 6 further comprising:

10. Prior to the writing, In response to a request by the first processing core for ownership of the data associated with the first data address, invalidating data associated with the first data address stored in caches of the one or more third processing cores; updating the first entry in the stash target table to identify the one or more third processing cores as a stash target for the first data address; The method of claim 6 further comprising:

11. and transmitting, by the first processor core, a first stash request to a home agent of the data processing system, the first stash request including the first data address, the home agent being configured to: determine that the second processor core of the data processing system is the stash target of the stash message; and transmit the stash message to the second processor core. The method of claim 1.

12. 12. The method of claim 11, wherein the determining comprises accessing a first entry in a stash target table that is associated with the first data address.

13. Prior to the determination, generating, by the home agent, the first entry in the stash target table for the first data address in response to receiving a prior stash request for the first data address from the first processing core; Further comprising: The method of claim 12.

14. Prior to the determination, sending, by the home agent, a read request for the data associated with the first data address from the second processor core to the first processor core; updating or creating, by the home agent, the first entry in the stash target table to identify the second processing core as a stash target for the first data address; sending, by the first processor core, the data associated with the first data address to the second processor core for storage in the cache of the second processor core; The method of claim 13 further comprising:

15. updating the first entry in the stash target table in response to a coherency state of the data associated with the first data address in a cache of the first processor core changing from unique to shared or from shared to unique; The method of claim 13 further comprising:

16. The second processor core storing the data associated with the first data address in the cache of the second processor core, sending, by the second processor core, a request to the home agent; the home agent sending a message to the first processor core; the first processor core sending the data associated with the first data address to the second processor core; The method of claim 11 , comprising:

17. monitoring transactions from a plurality of processor cores of a data processing system that allocate and deallocate data to caches of the plurality of processor cores; predicting, based at least in part on the monitored transactions, that one or more processor cores of the plurality of processor cores are stash target cores for data associated with a first data address; executing, by a first processor core of the plurality of processor cores, a first stash instruction for the first data address; sending a stash message to the stash target core notifying the stash target core of the data associated with the first data address and produced by the first processor core; in response to receiving the stash message, a second one of the one or more stash target cores stores the data associated with the first data address in a cache of the second processor core; (b) the method of data transfer, including

18. prior to executing the first stash instruction; executing, by the first processor core, a second stash instruction for the first data address; storing an entry in a stash target table for the first data address in response to execution of the second stash instruction; updating the entry in the stash target table to identify the second processing core as a stash target core in response to the second processing core becoming or ceasing to be a sharer of the data associated with the first data address; The method further comprising: predicting that one or more of the plurality of processor cores are stash target cores for data associated with the first data address includes accessing the entry in the stash target table for the first data address.

20. The method of claim 17.

19. prior to executing the first stash instruction; executing, by the second processor core, a register instruction to the first data address; creating or updating, in response to the register instruction, an entry in a stash target table identifying the second processor core as a stash target core for the first data address; The method further comprising: predicting that one or more of the plurality of processor cores are stash target cores for data associated with the first data address includes accessing the entry in the stash target table for the first data address.

20. The method of claim 17.

20. 1. A data processing system comprising: an interconnect fabric; a plurality of processor cores interconnected by the interconnect fabric, a first processor core of the plurality of processor cores configured to write data associated with a first data address to a cache and execute a first stash instruction for the first data address; 1. A stash target table control circuit, comprising: monitoring transactions from a plurality of processor cores of a data processing system that allocate and deallocate data to caches of the plurality of processor cores; Create and update entries in the stash target table, predicting, in response to the first stash instruction, one or more processor cores of the plurality of processor cores to be a stash target core for data associated with a first data address; sending a stash message to the stash target core, the stash message being associated with the first data address and configured to notify the stash target core of the data produced by the first processor core; a stash target table control circuit configured to Equipped with a second processor core of the one or more stash target cores is configured to, in response to receiving the stash message, store the data associated with the first data address in a cache of the second processor core; Data processing system.

21. a home agent coupled to the plurality of processor cores by the interconnect fabric and configured to provide a point of coherence and serialization for the data processing system; the home agent includes the stash target table control circuit; 21. The data processing system of claim 20.

22. 21. The data processing system of claim 20, wherein the first processor core includes the stash target table control circuitry.

23. The stash target table control circuitry includes: creating an entry in the stash target table in response to a stash instruction from the first processor core or a register request from the second processor core; inserting the second processor core into the entry as a stash target in response to a read request or a stash register request sent from the second processor core; inserting sharers of the data other than the first processor core as stash targets into the entry in response to a stash instruction from the first processor core or a change in a coherency state of the data associated with a first data address from shared to owned by the first processor core.

21. The data processing system of claim 20, configured to:

Images