Centralized delivery of multicast requests in a data processing system

Abstract

Centralized delivery of multicast requests in a data processing system The data processing system includes a master, a central request agent, and multiple snoopers communicatively coupled to a system fabric for communicating requests subject to retry. The master issues multicast requests intended for the multiple snoopers on the system fabric. The central request agent receives the multicast requests on the system fabric, assigns the multicast requests to specific state machines among multiple state machines in the central request agent, and provides a coherence response to the master indicating successful completion of the multicast requests. The central request agent repeatedly issues the multicast requests on the system fabric in association with a machine identifier that identifies the specific state machine until the coherence response indicates that the multicast request was successfully received by all of the multiple snoopers.

Description

[Background technology]

[0001] The present invention relates generally to data processing, and more particularly to the delivery of multicast requests, such as translation entry invalidation requests, in a multi-threaded data processing system.

[0002] A conventional multiprocessor (MP) computer system includes multiple processing units (which may each include one or more processor cores and their various cache memories), input / output (I / O) devices, and data storage, which may include both system memory (which may be volatile or nonvolatile) and nonvolatile mass storage. To provide sufficient addresses for memory-mapped I / O operations and for data and instructions utilized by operating systems and application software, MP computer systems typically refer to an effective address space that includes a number of effective addresses that is significantly greater than the number of physical storage locations in memory-mapped I / O devices and system memory. Thus, to perform memory-mapped I / O or access system memory, processor cores in a computer system that utilizes effective addressing are required to translate the effective addresses into real addresses that are assigned to particular I / O devices or physical storage locations within system memory.

[0003] In the POWER™ RISC architecture, the effective address space is partitioned into multiple uniformly sized memory pages, where each page has its own associated address descriptor called a page table entry (PTE). The PTE corresponding to a particular memory page stores the base effective address of the memory page and the associated base real address of a page frame, thereby allowing the processor core to translate any effective address within the memory page into a real address in system memory. PTEs created in system memory by the operating system and / or hypervisor software are collected in a page frame table.

[0004] To expedite the translation of effective addresses to real addresses during the processing of memory-mapped I / O and memory access instructions (hereinafter both simply referred to as "memory referent instructions"), conventional processor cores often employ, among other translation structures, a cache called a translation lookaside buffer (TLB) to buffer recently accessed PTEs within the processor core. Naturally, as data is moved in or out of physical storage locations in system memory (e.g., in response to the invocation of a new process or context switch), entries in the TLB must be updated to reflect the presence of the new data, and TLB entries associated with data that has been removed from system memory (e.g., paged out to non-volatile mass storage) must be invalidated. In many conventional processors, such as the POWER™ line of processors available from IBM Corporation, invalidation of TLB entries is a software responsibility and is accomplished through the execution of an explicit TLB invalidate entry instruction (e.g., TLBIE in the POWER™ instruction set architecture (ISA)).

[0005] In an MP computer system, invalidating a PTE cached in one processor core's TLB is complicated by the fact that each other processor core has its own respective TLB, which may also cache a copy of the target PTE. To maintain a consistent view of system memory across all processor cores, invalidating a PTE in one processor core requires invalidating the same PTE, if present, in the TLBs of all other processor cores. In many conventional MP computer systems, invalidating PTEs in all processor cores in the system is accomplished by executing a TLB invalidate entry instruction in the initiating processor core and broadcasting a TLB invalidate entry request from the initiating processor core to each other processor core in the system. The TLB invalidate entry instruction (or multiple instructions, if multiple PTEs are to be invalidated) may be followed by one or more synchronization instructions in the initiating processor core's instruction sequence that ensure the TLB entry invalidation has been executed by all processor cores.

[0006] The present disclosure recognizes that, if not handled properly, the broadcast of a TLB invalidation entry request from an initiating processor core to each other processor core in a system can cause livelock in a system that allows interconnect operations to be retried by the recipients. Accordingly, it would be useful and desirable to provide improved techniques for managing the distribution of multicast requests, such as translation entry invalidation requests, in a data processing system that allows interconnect operations to be retried. Summary of the Invention

[0007] As briefly described above, data processing systems that allow multicast requests on a system interconnect to be retried by snoopers suffer from so-called “ping-pong” livelock. Ping-pong livelock occurs when a first subset of snoopers accepts a multicast request and potentially begins processing it, and a second subset of snoopers issues a retry response to the multicast request indicating that they are currently unable to process the request. In response to the retry response, the master that initiated the multicast request will reissue the multicast request on the system interconnect, potentially resulting in one or more snoopers in the second subset accepting the request and one or more snoopers in the first subset issuing a retry response. Furthermore, one or more snoopers in the first subset may additionally accept the request again and resume processing the request upon reissue by the master.

[0008] This scenario illustrates two potential problems with multicast requests in data processing systems where interconnect requests may be retried: First, a multicast request is not guaranteed to be accepted and processed by all snoopers necessary to complete the multicast request, and second, a multicast request may be repeatedly processed by one or more snoopers if it is retried and reissued.

[0009] To avoid ping-pong livelock for multicast requests, the present disclosure employs a central request agent and associated communication protocol that ensures that a given multicast request is eventually accepted by each associated snooper, that an associated snooper will process a multicast request only once even if it is retried, and that issuance of other multicast requests of the same type will not prevent a given multicast request from being accepted by all associated snoopers. The central request agent has a given number of state machines for tracking multicast requests, and the snoopers have a corresponding number of snoop machines.

[0010] Multicast requests issued on the system interconnect by various masters distributed within the data processing system compete for acceptance by the state machine of the central request agent. Once a multicast request is accepted by a given state machine in the central request agent, the central request agent forwards the multicast request to corresponding snoop machines in different snoopers distributed within the data processing system. Once a request is forwarded by the centralized request agent, one or more snoopers may provide a retry coherence response to the multicast request, while one or more other snoopers accept and process the multicast request. A snooper will provide a retry response if the snoop machine in the snooper is still busy working on a previous multicast request assigned to the same snoop machine. Eventually, the multicast request forwarded by the central request agent can be snooped and processed in all involved snoopers, as indicated to the central request agent by a successful combined response. Note that the snooper will not issue a retry coherence response to a snooped multicast request if the multicast request is the same multicast request that the snooper is currently processing or has just completed processing.

[0011] According to one embodiment, a data processing system includes a master, a central request agent, and multiple snoopers communicatively coupled to a system fabric for communicating requests subject to retry. The master issues a multicast request over the system fabric intended for the multiple snoopers. The central request agent receives the multicast request over the system fabric, assigns the multicast request to a specific state machine among multiple state machines in the central request agent, and provides a coherence response to the master indicating successful completion of the multicast request. The central request agent repeatedly issues the multicast request over the system fabric in association with a machine identifier that identifies the specific state machine until a coherence response indicates that the multicast request was successfully received by all of the multiple snoopers.

[0012] In at least one embodiment, the multicast request comprises a translation entry invalidation request.

[0013] In at least one embodiment, each of the plurality of snoopers has a plurality of snoop machines corresponding in number to the plurality of state machines in the central request agent, and each of the plurality of snoopers assigns the multicast request received from the central request unit to a particular snoop machine among the plurality of snoop machines that corresponds to the particular state machine.

[0014] In at least one embodiment, prior to said repeatedly issuing said multicast request, said central request agent modifies said multicast request to indicate that said multicast request is to be forwarded from said central request agent.

[0015] In at least one embodiment, in response to the central request agent repeatedly issuing the multicast request, one or more of the plurality of snoopers provides a null coherence response indicating successful receipt of a prior issuance of the multicast request by the central request agent.

[0016] In at least one embodiment, the multicast request includes multiple periods of operation on the system fabric. In such an embodiment, the central request agent receives at least a first period of operation and a corresponding second period of operation and assigns the first and second periods of operation to the particular state machine. The central request agent marks the multicast request as available for distribution to the multiple snoopers based on the assignment of both the first and second periods of operation to the particular state machine.

[0017] In at least one embodiment, the central request agent issues an epoch indication of the first operating period of the multicast request on the system fabric, and one of the plurality of snoopers discards a previously snooped multicast request based on a mismatched epoch indication.

[0018] The disclosed embodiments may be implemented as a method, an integrated circuit, a data processing system, and / or a design structure. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a schematic block diagram of an exemplary data processing system according to one embodiment.

[0020] [Figure 2] FIG. 2 is a more detailed block diagram of an exemplary processing unit according to one embodiment.

[0021] [Figure 3] FIG. 2 is a detailed block diagram of a processor core and lower-level cache memory according to one embodiment.

[0022] [Figure 4] FIG. 4A is a first exemplary translation entry invalidation instruction sequence according to one embodiment.

[0023] FIG. 4B is a second exemplary translation entry invalidation instruction sequence according to one embodiment.

[0024] [Figure 5] 1 is a schematic logic flowchart of an exemplary method for processing a translation entry invalidate instruction by a processor core of a multiprocessor data processing system according to one embodiment.

[0025] [Figure 6] 1 is a schematic logic flowchart of an exemplary method by which sidecar logic of a processing unit processes a translation entry invalidation request according to one embodiment.

[0026] [Figure 7] 1 is a schematic logic flowchart of an exemplary method for a snooper of a processing unit to handle translation entry invalidation requests and translation synchronization requests, according to one embodiment;

[0027] [Figure 8] 1 is a schematic logic flow diagram of an exemplary method by which an arbiter of a processing unit processes a translation entry invalidation request, according to one embodiment.

[0028] [Figure 9] 1 is a schematic logic flowchart of an exemplary method by which a translation sequencer of a processor core processes a translation entry invalidation request according to one embodiment.

[0029] [Figure 10]1 is a schematic logic flowchart of an exemplary method for a store queue of a processing unit to process a translation invalidation completion request according to one embodiment.

[0030] [Figure 11] 1 is a schematic logic flow diagram of an exemplary method for a processor core to process a transform synchronization instruction according to one embodiment.

[0031] [Figure 12] 1 is a schematic logic flowchart of an exemplary method by which sidecar logic of a processing unit processes a transformation synchronization request, according to one embodiment.

[0032] [Figure 13] 1 is a schematic logic flowchart of an exemplary method for a processing core to process a page table synchronization instruction, according to one embodiment.

[0033] [Figure 14] 1 is a schematic logic flowchart of an exemplary method for a processing unit to process a page table synchronization request according to one embodiment.

[0034] [Figure 15] 1 is a schematic logic flowchart of an exemplary method by which snooper logic of a processing unit processes translation entry invalidation requests, translation invalidation completion requests, and page table synchronization requests, according to one embodiment.

[0035] [Figure 16] 2 is a space-time diagram of interconnect activity on the system fabric of the data processing system of FIG. 1 according to one embodiment.

[0036] [Figure 17] FIG. 10 illustrates an exemplary format of a tag portion of an interconnect operation according to one embodiment.

[0037] [Figure 18] 2 is a schematic block diagram of an exemplary embodiment of a central request agent, according to one embodiment.

[0038] [Figure 19] 4 is a schematic logic flow diagram of an exemplary method for an initiating processing unit or central request agent to issue a translation entry invalidation request according to a first embodiment.

[0039] [Figure 20] 4 is a schematic logic flow diagram of an exemplary method for a central request agent to receive a translation entry invalidation request from an initiating processing unit according to a first embodiment.

[0040] [Figure 21] 4 is a schematic logic flow diagram of an exemplary method for a central request agent to select a translation entry invalidation request for issuance according to a first embodiment.

[0041] [Figure 22] 4 is a schematic logic flow diagram of an exemplary method for a central request agent to issue a translation entry invalidation request according to a first embodiment;

[0042] [Figure 23] 4 is a schematic logic flow diagram of an exemplary method in which a processing unit receives a translation entry invalidation request from a central request agent according to a first embodiment.

[0043] [Figure 24] 10 is a schematic logic flow diagram of an exemplary method for an initiating processing unit or central request agent to issue a translation entry invalidation request according to a second embodiment;

[0044] [Figure 25]10 is a schematic logic flow diagram of an exemplary method for a central request agent to receive a translation entry invalidation request from an initiating processing unit according to a second embodiment;

[0045] [Figure 26] 10 is a schematic logic flow diagram of an exemplary method for a central request agent to select a translation entry invalidation request for issuance according to a second embodiment.

[0046] [Figure 27] 10 is a schematic logic flow diagram of an exemplary method for a central request agent to issue a translation entry invalidation request according to a second embodiment;

[0047] [Figure 28] 29 together with FIG. 30 form a schematic logic flow diagram of an exemplary method by which a processing unit receives a translation entry invalidation request for processing, according to a second embodiment. [Figure 29] 29 together with FIG. 28 form a schematic logic flow chart of an exemplary method by which a processing unit receives a translation entry invalidation request for processing, according to a second embodiment.

[0048] [Figure 30] FIG. 1 is a data flow diagram illustrating a design process. DETAILED DESCRIPTION OF THE INVENTION

[0049] Referring now to the figures, wherein like reference numerals refer to like and corresponding parts throughout, and with particular reference to FIG. 1 , there is shown a schematic block diagram illustrating an exemplary data processing system 100 according to one embodiment. In the illustrated embodiment, data processing system 100 is a cache-coherent symmetric multiprocessor (SMP) data processing system that includes multiple processing nodes 102 for processing data and instructions. Processing nodes 102 are coupled to a system interconnect 110 for communicating address, data, and control information. System interconnect 110 may be implemented, for example, as a bus-based interconnect, a switched interconnect, or a hybrid interconnect.

[0050] In the illustrated embodiment, each processing node 102 is implemented as a multi-chip module (MCM) that houses multiple (e.g., four) processing units 104a-104d, each preferably implemented as a respective integrated circuit. The processing units 104 within each processing node 102 are communicatively coupled to each other and to system interconnect 110 by local interconnect 114, which, like system interconnect 110, may be implemented with, for example, one or more buses and / or switches. System interconnect 110 and local interconnect 114 together form the system fabric.

[0051] As described in more detail below with reference to FIG. 2 , processing units 104 each include a memory controller 106 coupled to local interconnect 114 to provide an interface to a respective system memory 108. Data and instructions residing in system memory 108 may generally be accessed, cached, and modified by processor cores in any processing unit 104 of any processing node 102 in data processing system 100. System memory 108 therefore forms the lowest level of memory storage in the distributed shared memory system of data processing system 100. In alternative embodiments, one or more memory controllers 106 (and system memory 108) may be coupled to system interconnect 110 rather than local interconnect 114.

[0052] 1 further illustrates that data processing system 100 includes at least one central request agent 120 coupled to the system fabric. As discussed further below with reference to FIG. 18 et seq., central request agent 120 can be utilized to deliver certain types of multicast requests over the system fabric while avoiding the development of livelock. In some examples, data processing system 100 may include a single central request agent 120, which may be conveniently coupled to system interconnect 110.

[0053] Those skilled in the art will appreciate that the SMP data processing system 100 of Figure 1 may include many additional, not-shown, components, such as interconnect bridges, non-volatile storage, ports for connection to networks or attached devices, etc. Because such additional components are not necessary for an understanding of the described embodiments, they are not shown in Figure 1 or further discussed herein. However, it should also be understood that the improvements described herein are applicable to data processing systems of a variety of architectures and are not limited in any way to the generalized data processing system architecture shown in Figure 1.

[0054] 2, a more detailed block diagram of an exemplary processing unit 104 according to one embodiment is shown. In the embodiment shown, each processing unit 104 is an integrated circuit that includes one or more processor cores 200 for processing instructions and data. In a preferred embodiment, each processor core 200 supports simultaneous multithreading (SMT) and is therefore capable of independently executing multiple hardware threads of execution at the same time.

[0055] The operation of each processor core 200 is supported, at its lowest level, by a multi-level memory hierarchy having a shared system memory 108 accessed via an integrated memory controller 106. As shown, the shared system memory 108 stores a page frame table 220 that stores a plurality of page table entries (PTEs) 222 for performing effective-to-real address translations to enable access to storage locations in the system memory 108. At its upper level, the multi-level memory hierarchy includes one or more levels of cache memory, which in the exemplary embodiment include a store-through level 1 (L1) cache 302 (see FIG. 3 ) within and dedicated to each processor core 200, and a respective store-in level 2 (L2) cache 230 for each processor core 200. While the cache hierarchy shown includes only two levels of cache, those skilled in the art will understand that alternative embodiments may include additional levels (L3, L4, etc.) of on-chip or off-chip, private or shared, in-line or look-aside cache that may be fully inclusive, partially inclusive, or non-inclusive to the contents of the higher levels of cache.

[0056] Each processing unit 104 further includes an integrated and distributed fabric controller 216 that is responsible for controlling the flow of operations over the system fabric, including local interconnect 114 and system interconnect 110, and for implementing the coherency communications required to implement a selected cache coherency protocol. Processing unit 104 also includes an integrated I / O (input / output) controller 214 that supports the attachment of one or more I / O devices (not shown).

[0057] Referring now to FIG. 3, a more detailed block diagram of an exemplary embodiment of a processor core 200 and its associated L2 cache 230 is shown, according to one embodiment.

[0058] In the illustrated embodiment, processor core 200 includes one or more execution units 300 that execute instructions from multiple simultaneous hardware threads of execution. The instructions may include, for example, arithmetic, logical, and memory reference instructions, as well as translation entry invalidate instructions (hereinafter referred to by the POWER™ ISA mnemonic TLBIE (Translation Lookaside Buffer Invalidate Entry)) and associated synchronization instructions. Execution units 300 can generally execute instructions of hardware threads in any order, as long as the data dependencies and explicit ordering mandated by synchronization instructions are observed.

[0059] Processor core 200 additionally includes a memory management unit (MMU) 308 responsible for translating target effective addresses determined by execution of memory-referencing instructions in execution unit 300 into real addresses. MMU 308 performs effective-to-real address translations by referencing one or more translation structures 310, such as a translation lookaside buffer (TLB), a block address table (BAT), a segment lookaside buffer (SLB), etc. The number and type of these translation structures vary between implementations and architectures. When present, a TLB reduces the latency associated with effective-to-real address translations by caching PTEs 222 retrieved from page frame table 220. A translation sequencer 312 associated with translation structure 310 handles invalidation of effective-to-real translation entries maintained within translation structure 310 and manages such invalidation for ongoing memory-referencing instructions in processor core 200.

[0060] Processor core 200 additionally includes various storage facilities shared by the multiple hardware threads supported by processor core 200. The storage facilities shared by the multiple hardware threads include an L1 store queue (L1 STQ) 304 that temporarily buffers store and synchronization requests generated by execution of corresponding store and synchronization instructions by execution unit 300. L1 cache 302 is a store-through cache, meaning that coherence is entirely determined at a lower level in the cache hierarchy (e.g., in L2 cache 230), so requests flow through L1 STQ 304 and then proceed via bus 318 to L2 cache 230 for processing. The storage facilities of processor core 200 shared by the multiple hardware threads additionally include a load miss queue (LMQ) 306 that temporarily buffers load requests that miss in L1 cache 302. Because such load requests have not yet been satisfied, they may hit the wrong memory page if the address translation entry used to obtain the target real address of the load request is invalidated before the load request is satisfied. Thus, if a PTE or other translation entry is to be invalidated, any load requests in LMQ 306 that depend on that translation entry must be drained from LMQ 306 and satisfied before the effective address translated by the associated translation entry can be reassigned.

[0061] 3, L2 cache 230 includes a cache array 332 and an L2 directory 334 of the contents of cache array 332. Assuming that cache array 332 and L2 directory 334 are conventionally set associative, storage locations in system memory 108 are mapped to particular congruence classes in cache array 332 using predetermined index bits in the system memory (real) address. The particular memory blocks stored within cache lines of cache array 332 are recorded in L2 directory 334, which stores one directory entry per cache line. Although not explicitly shown in FIG. 3, those skilled in the art will understand that each directory entry in cache directory 334 includes various fields, such as a tag field that identifies the real address of the memory block held in the corresponding cache line of cache array 332, a status field that indicates the coherency state of the cache line, a least recently used (LRU) field that indicates the replacement order for the cache line relative to other cache lines in the same congruence class, and an inclusiveness bit that indicates whether the memory block is held in the associated L1 cache 302.

[0062] The L2 cache 230 additionally includes an L2 STQ 320 that receives store modification requests and synchronization requests from the L1 STQ 304 via interface 321 and buffers such requests. Note that the L2 STQ 320 is a unified store queue that buffers requests for all hardware threads of the associated processor core 200. Thus, all of a thread's store requests, TLBIE requests, and associated synchronization requests flow through the L2 STQ 320. While the L2 STQ 320 contains multiple entries in most embodiments, the L2 STQ 320 is required to function in a deadlock-free manner regardless of its depth (i.e., even when implemented as a single-entry queue). To this end, the L2 STQ 320 is coupled by interface 321 to associated sidecar logic 322, which contains one request buffer entry (referred to herein as a "sidecar") 324 for each hardware thread supported by the associated processor core 200. Thus, the number of sidecars 324 is unrelated to the number of entries in the L2 STQ 320. As further described herein, the use of sidecar 324 allows potentially deadlock-causing requests to be removed from L2 STQ 320 so that deadlocks do not occur during translation entry invalidation.

[0063] L2 cache 230 further includes dispatch / response logic 336 that receives local load and store requests initiated by the associated processor core 200 via buses 327 and 328, respectively, and remote requests snooped on local interconnect 114 via bus 329. Such requests, including local and remote load requests, store requests, TLBIE requests, and associated synchronization requests, are processed by dispatch / response logic 336 and then dispatched to the appropriate state machine for processing.

[0064] In the illustrated embodiment, the state machine implemented within the L2 cache 230 for processing requests includes multiple read-claim (RC) machines 342, which independently and simultaneously process load (LD) and store (ST) requests received from an associated processor core 200. To process remote memory access requests originating from processor cores 200 other than the associated processor core 200, the L2 cache 230 also includes multiple snoop (SN) machines 344. Each snoop machine 344 is capable of independently and simultaneously handling remote memory access requests snooped from the local interconnect 114. As will be appreciated, processing of memory access requests by the RC machine 342 may require replacement or invalidation of memory blocks within the cache array 332 (and the L1 cache 302). Accordingly, the L2 cache 230 also includes a castout (CO) machine 340, which manages the eviction and write-back of memory blocks from the cache array 332.

[0065] In the illustrated embodiment, L2 cache 230 additionally includes multiple translation snoop (TSN) machines 346, which are utilized to process TLBIE requests and associated synchronization requests. It should be understood that in some embodiments, TSN machines 346 may be implemented in another subunit of processing unit 104, such as a non-cacheable unit (NCU) (not shown) that handles non-cacheable memory access operations. In at least one embodiment, to simplify implementation of a consensus protocol (as discussed further herein) that coordinates the processing of multiple concurrent TLBIE requests within data processing system 100, a similar number of TSN machines 346 are implemented in each L2 cache 230.

[0066] All TSN machines 346 are coupled to an arbiter 348 that selects requests to be handled by the TSN machines 346 for transmission to a translation sequencer 312 in the processor core 200 via a bus 350. In at least some embodiments, the bus 350 is implemented as a unified bus that not only transmits requests for the TSN machines 346, but also returns data from the L2 cache 230 to the processor core 200, and performs other operations. Note that the translation sequencer 312 must accept requests from the arbiter 348 in a non-blocking manner to avoid deadlock.

[0067] 16, a time-space diagram of an exemplary interconnect operation on the interconnect fabric of data processing system 100 of FIG. 1 is shown. An interconnect operation begins when a master 1600 (e.g., read-claim (RC) machine 342 of L2 cache 230, a master in I / O controller 214, central request agent 120, a master in a non-cacheable unit (NCU) not shown, etc.) issues a request 1602 on the interconnect fabric. In the example given in FIG. 17, request 1602 includes an address field 1702 for specifying a target real address to be accessed, a transaction type (ttype) field 1710 for specifying the type of access to be made, a tag field (tag) 1720 for identifying master 1600 initiating request 1602, and an other field 1740 for specifying additional request information. In the illustrated example, tag field 1720 includes a topographical identifier (ID) 1722 that uniquely identifies the master's physical location within data processing system 100, a machine ID 1724 that identifies which state machine (e.g., CO 340, RC 342, sidecar 324, etc.) is the master, and a forwarding (F) field 1726 that is set to indicate that the request is forwarded from central request agent 120 and reset otherwise. In some embodiments, tag field 1720 may further include an epoch (E) field 1728 and an operation type (O) field 1730, as discussed further below.

[0068] Common types of requests 1602 include those described below in Table I. Table I [Table 1]

[0069] 16, a request 1602 is received by snoopers 1604a-1604n distributed throughout data processing system 100 (e.g., snoop machine 344 of L2 cache 230, central request agent 120, and snoopers in memory controller 106). Each snooper 1604 that receives and processes request 1602 may provide a respective partial response (Presp) 1606 representing at least that snooper's 1604 response to request 1602. The snooper in memory controller 106 determines which partial response 1606 to provide based, for example, on whether the snooper is responsible for the request address and whether it has available resources to process the request. The L2 cache 230 may determine its partial response 1606 based on, for example, the availability of a snoop machine 344 to handle the request, the availability of its L2 directory 334, and the coherency state associated with the target real address in the L2 directory 334.

[0070] The partial responses 1606 of snoopers 1604a-1604n are logically combined, either incrementally or all at once, by one or more instances of response logic 1622 to determine a system-wide combined response (Cresp) 1610 to request 1602. While not explicitly shown, Cresp 1610 preferably includes a tag field 1720 that corresponds to that of the original request 1602 to allow master 1600 to match its request 1602 and the combined response 1610. In one embodiment assumed below, the instance of response logic 1622 responsible for generating Cresp 1610 is located in the processing unit 104 that houses the master 1600 that issued the request 1602. Response logic 1622 provides Cresp 1610 to master 1600 and snooper 1604 via the interconnect fabric to indicate a system-wide coherence response (e.g., success, failure, retry, etc.) to request 1602. If Cresp 1610 indicates success of request 1602, Cresp 1610 may indicate, for example, the data source of the target memory block of request 1602, the coherence state in which the requested memory block will be cached by master 1600 (or other caches), and whether a "cleanup" operation is required to invalidate the requested memory block in one or more caches.

[0071] In response to receiving Cresp 1610, one or more of master 1600 and snooper 1604 may perform one or more additional actions to process request 1602. These additional actions may include providing data to master 1600, invalidating or otherwise updating the coherence state of cached data in one or more L2 caches 230, performing a castout operation, writing data back to system memory 108, etc. If required by request 1602, the requested or target memory block may be transmitted to or from master 1600 before or after generation of Cresp 1610 by response logic 1622.

[0072] 4A , a first exemplary translation entry invalidation instruction sequence 400 that may be executed by processor core 200 of data processing system 100, according to one embodiment, is shown. The purpose of instruction sequence 400 is to: (a) disable a translation entry (e.g., PTE 222) in page frame table 220 so that the translation entry is not subject to reloading by any MMU 308 of data processing system 100; (b) invalidate any copies of the translation entry (or other translation entries that translate the same effective address as the translation entry) cached by any processor core 200 in data processing system 100; and (c) flush all outstanding memory access requests that depend on the old translation entry before the effective address is reallocated. If the translation is updated before a store request that depends on the old translation entry is flushed, the store request may corrupt the memory page identified by the old translation entry. Similarly, if a load request that relies on an old translation entry and misses the L1 cache 302 is not satisfied before the translation is reallocated, the load request will read data from a different memory page than intended and therefore observe data that was not intended to be visible to the load request.

[0073] An instruction sequence 400, which may be preceded and followed by any arbitrary number of instructions, begins with one or more store (ST) instructions 402. Each store instruction 402, when executed, generates a store request that, when propagated to the associated system memory 108, marks the target PTE 222 as invalid in page frame table 220. Once the store request marks the PTE 222 as invalid in page frame table 220, MMU 308 will no longer load the invalidated translation from page frame table 220.

[0074] One or more store instructions 402 in instruction sequence 400 are followed by a heavy weight synchronization (i.e., HWSYNC) instruction 404, which is a barrier that ensures that a subsequent TLBIE instruction 406 is not reordered by processor core 200 to execute before any of the store instructions 402. HWSYNC instruction 404 therefore ensures that when processor core 200 reloads PTE 222 from page frame table 220 after TLBIE instruction 406 invalidates a cached copy of PTE 222, processor core 200 is guaranteed to have observed the invalidation due to store instruction 402 and therefore will not use or reload target PTE 222 into translation structure 310 until the effective address translated by target PTE 222 has been reallocated and set valid.

[0075] The HWSYNC instruction 404 in the instruction sequence 400 is followed by at least one TLBIE instruction 406 which, when executed, generates a corresponding TLBIE request that invalidates any translation entries that translate the target effective address of the TLBIE request in all translation structures 310 in the data processing system 100. One or more TLBIE instructions 406 are followed in the instruction sequence 400 by a translation synchronization (i.e., TSYNC) instruction 408 which, prior to execution of the thread proceeding to the subsequent instruction, ensures that the TLBIE request generated by execution of the TLBIE instruction 406 has finished invalidating all translations of the target effective address in all translation structures 310 in the data processing system 100 and that all previous memory access requests that depend on the now-invalidated translations have been drained.

[0076] The instruction sequence 400 ends with a second HWSYNC instruction 410, which implements a barrier that prevents any memory-referencing instructions that follow the HWSYNC instruction 410 in program order from executing until the TSYNC instruction 406 completes its processing. In this way, any more recent memory-referencing instructions that request a translation of the target effective address of the TLBIE request will receive the new translation rather than the old translation that was invalidated by the TLBIE request. Note that the HWSYNC instruction 410 does not have any functionality directly related to invalidating the target PTE 222 in the page frame table, invalidating the translation entry in the translation structure 310, or flushing memory-referencing instructions that depend on the old translation.

[0077] To facilitate understanding of the invention disclosed herein, the progression of the TLBIE instruction 406 and the TLBIE requests generated therefrom will be described from inception to completion with reference to Figures 5-10. Figures 11 and 12 additionally show the progression of the TSYNC instruction 408 and its corresponding TSYNC requests, which ensure that the invalidation requested by the TLBIE request is completed on all snooping processor cores 200.

[0078] 5, a schematic logic flow diagram of an exemplary method by which an initiating processor core 200 of multiprocessor data processing system 100 processes a translation entry invalidation (e.g., TLBIE) instruction is shown, according to one embodiment. The illustrated process represents processing performed in a single hardware thread, meaning that multiple of these processes may be executing simultaneously (i.e., in parallel) on a single processor core 200, and further, multiple of these processes may be executing simultaneously on various different processing cores 200 throughout data processing system 100. As a result, multiple different address translation entries buffered in various processor cores 200 of data processing system 100 may be invalidated by different initiating hardware threads at the same time.

[0079] The illustrated process begins at block 500 and thereafter proceeds to block 501, which depicts execution of a TLBIE instruction 406 in instruction sequence 400 by execution unit 300 of processor core 200. Execution of TLBIE instruction 406 determines a target effective address at which all translation entries buffered in translation structure 310 throughout data processing system 100 will be invalidated. In response to execution of TLBIE instruction 406, processor core 200 stops dispatching any additional instructions in the initiating hardware thread because, in the exemplary embodiment of FIG. 3 , sidecar logic 322 includes only a single sidecar 324 per thread, meaning that at most one TLBIE request per thread can be active at a time. In other embodiments having multiple sidecars 324 per thread, multiple simultaneously active TLBIE requests per thread can be supported.

[0080] At block 504, a TLBIE request corresponding to the TLBIE instruction 406 is generated and issued to the L1 STQ 304. The TLBIE request may include, for example, a transaction type indicating the type of request (i.e., TLBIE), the effective address for which cached translations are to be invalidated, and an indication of the initiating processor core 200 and hardware thread that issued the TLBIE request. Processing of the request in the L1 STQ 304 progresses, and the TLBIE request eventually moves from the L1 STQ 304 to the L2 STQ 320 via bus 318, as shown at block 506. The process then proceeds to block 508, which illustrates the initiating processor core 200 continuing to refrain from dispatching instructions in the initiating hardware thread until it receives a TLBCMPLT_ACK signal from the storage subsystem via bus 325, indicating that processing of the TLBIE request by the initiating processor core 200 has completed. (The generation of the TLBCMPLT_ACK signal is described below with reference to block 1010 of FIG. 10.) Note also that because dispatching of instructions in the initiating thread is halted, there can be no contention for the initiating thread's sidecar 324 due to a TSYNC request corresponding to TSYNC instruction 408, since for any given thread, only one of the two types of requests can be present in the L2 STQ 320 and sidecar logic 322 at a time.

[0081] In response to determining at block 508 that a TLBCMPLT_ACK signal has been received, the process proceeds from block 508 to block 510, which indicates that processor core 200 resumes dispatching instructions in the initiating thread; therefore, releasing the thread at block 510 allows processing of TSYNC instruction 408 (which is the next instruction in instruction sequence 400) to begin as described below with reference to Figure 11. The process of Figure 5 then ends at block 512.

[0082] 6, there is shown a schematic logic flow diagram of an exemplary method by which sidecar logic 322 of L2 cache 230 processes translation entry invalidation (e.g., TLBIE) requests for an associated hardware thread of processor core 200, according to one embodiment. The process of FIG. 6 is performed on a per-thread basis.

[0083] 6 begins at block 600 and then proceeds to block 602, which illustrates the sidecar logic 322 determining whether the TLBIE request for the hardware thread of the associated processor core 200 is loaded in the L2 STQ 320. If not, the process repeats at block 602. However, in response to determining that the TLBIE for the hardware thread of the associated processor core 200 is loaded in the L2 STQ 320, the sidecar logic 322 removes the TLBIE request from the L2 STQ 320 and moves the TLBIE request via the interface 321 to the sidecar 324 corresponding to the initiating thread (block 604). Removing the TLBIE request from the L2 STQ 320 ensures that a deadlock does not occur due to the L2 STQ 320 being unable to receive an incoming request from the associated processor core 200, allowing such a request to flow through the L2 STQ 320.

[0084] At block 606, the sidecar 324 participates in a consensus protocol via the interface 326 and the local interconnect 114 to ensure that one (and only one) TSN machine 346 in every L2 cache 230 receives the TLBIE request. In some implementations, the consensus protocol may be conventional. In other embodiments described below with reference to FIGS. 18-29, the consensus protocol employed by the sidecar 324 to deliver the TLBIE request utilizes a central request agent 120 to manage the delivery of the TLBIE request to the L2 cache 230. Following block 606, the process returns to block 602 as described.

[0085] 7, there is shown a schematic logic flow diagram of an exemplary method for processing TLBIE and TSYNC requests by TSN machines 346, according to one embodiment. The illustrated process is performed independently and concurrently for each TSN machine 346.

[0086] The process begins at block 700 and then proceeds to blocks 702 and 720. Block 702 and subsequent block 704 depict the TSN machine 346 buffering the TLBIE request and assuming the TLBIE_active state in response to notification of receipt of the TLBIE request via the consensus protocol. The TLBIE request, broadcast via system fabric 110, 114 in block 606 of FIG. 6 to the L2 cache 230 of the initiating processor core 200 and the L2 caches of all other processor cores 200 in the data processing system 100, is received by the L2 cache 230 via bus 329, processed by dispatch / response logic 336, and then assigned to the TSN machine 346. The TSN machine 346, assuming the TLBIE_active state, notifies the associated arbiter 348 that the TLBIE request is ready to be processed, as described further below with reference to block 802 of FIG. 8.

[0087] Block 706 indicates that the TSN machine 346 remains in the TLBIE_active state until the associated processor core 200 has completed processing of the TLBIE request (i.e., invalidating the associated translation entry in translation structure 310 and flushing the associated memory reference request from processor core 200), as indicated by receipt of a TLBCMPLT_ACK signal via bus 330. In response to receiving the TLBCMPLT_ACK signal, the TLBIE_active state is reset and the TSN machine 346 is released for redistribution (block 708). The process of FIG. 7 then returns from block 708 to block 702 as described above.

[0088] Referring now to blocks 720-724, the TSN machine 346 determines in block 720 whether it is in the TLBIE_active state established in block 704. If not, the process repeats at block 720. On the other hand, if the TSN machine 346 is in the TLBIE_active state established in block 704, the TSN machine 346 monitors to determine whether a TSYNC request for the initiating hardware thread of that TLBIE request has been detected (block 722). If a TSYNC request is not detected, the process continues to repeat at blocks 720-722. On the other hand, in response to detecting a TSYNC request for the initiating hardware thread of that TLBIE request while the TSN machine 346 is in the TLBIE_active state, the TSN machine 346 provides a retry coherence response via the system fabric 110, 114, as shown in block 724. As discussed below with reference to block 1208 of FIG. 12 , a retry coherence response by any TSN machine 346 handling the TLBIE request for the initiating hardware thread forces the TSYNC request to be reissued by the source L2 cache 230 and prevents the initiating hardware thread from proceeding to the HWSYNC instruction 410 until the TSYNC request completes without a retry coherence response. A TSYNC request completes without a retry coherence response when all processor cores 200 other than the initiating processor core 200 have completed their processing of the TLBIE request. (A TSYNC request is not issued by the initiating processor core 200 until the initiating processor core 200 has completed processing the TLBIE request due to instruction dispatch being halted for processing the TLBIE request, as discussed above with reference to block 508 of FIG. 5 .)

[0089] 8, there is shown a schematic logic flow diagram of an exemplary method by which the arbiter 348 of the L2 cache 230 processes a TLBIE request, according to one embodiment. The process begins at block 800 and then proceeds to block 802, which depicts the arbiter 348 determining whether any of its TSN machines 346 are in the TLBIE_active state. If not, the process of FIG. 8 repeats at block 802. However, in response to determining that one or more of its TSN machines 346 are in the TLBIE_active state, the arbiter 348 selects one of the TSN machines 346 in the TLBIE_active state that has not previously forwarded the TLBIE request and transmits the TLBIE request via the bus 350 to the translation sequencer 312 of the associated processor core 200 (block 804). To avoid deadlock, the translation sequencer 312 is configured to accept the TLBIE request within a fixed time and not to arbitrarily delay accepting the TLBIE request.

[0090] The process proceeds from block 804 to block 806, which depicts the arbiter 348 waiting to receive a TLBCMPLT_ACK message indicating that the associated processor core 200 has invalidated one or more associated translation entries in translation structure 310 in response to the TLBIE request and has drained any associated memory reference requests that may have had their target addresses translated by the invalidated translation entries. Thus, in block 806, the arbiter 348 is waiting for a TLBCMPLT_ACK message, as is both the initiating thread (block 508) and the TSN machines 346 (block 706) in each of the L2 caches 230. In response to receiving the TLBCMPLT_ACK message in block 806, the process returns to block 802 as described above. Note that by the time the process returns to block 802, the previously selected TSN machine 346 will not still be in the TLBIE_active state due to a TLBIE request that has already been processed, because the TLBIE_active state will have been reset as shown in blocks 706-708 before the process returns to block 802.

[0091] 8 (and particularly blocks 802 and 806) ensures that only one TLBIE request is processed by processor core 200 at a time. The serial processing of TLBIE requests by processor core 200 eliminates the need to tag TLBCMPLT_ACK messages to associate them with TLBIE requests, simplifying the instruction marking mechanism, as discussed below with reference to FIG. 9. However, those skilled in the art will recognize that in other embodiments, processor core 200 may be configured to process multiple TLBIE requests simultaneously, with some additional complexity.

[0092] 9, a high-level logic flow diagram of an exemplary method by which translation sequencer 312 of initiating or snooping processor core 200 processes a TLBIE request, according to one embodiment, is shown. The process shown in FIG. 9 begins at block 900 and thereafter proceeds to block 902, which depicts translation sequencer 312 awaiting receipt of a TLBIE request forwarded by arbiter 348, as described above with reference to block 804 of FIG. 8. In response to receiving the TLBIE request, translation sequencer 312 invalidates one or more translation entries (e.g., PTEs or other translation entries) in translation structure 310 that translate the target effective address of the TLBIE request (block 904). Additionally, at block 906, translation sequencer 312 marks all memory reference requests to be flushed from processor core 200.

[0093] In a less sophisticated embodiment, in block 906, the translation sequencer 312 marks all memory reference requests of all hardware threads in the processor core 200 that had their target addresses translated, under the assumption that any of such memory reference requests could have had their target addresses translated by one or more translation entries invalidated by the TLBIE request received in block 902. Thus, in this embodiment, the marked memory reference requests would include all store requests in the L1 STQ 304 and all load requests in the LMQ 306. This embodiment advantageously eliminates the need to implement comparators for every entry in the L1 STQ 304 and LMQ 306, but may lead to higher latency due to long drain times.

[0094] A more sophisticated embodiment implements a comparator for every entry in L1 STQ 304 and LMQ 306. In this embodiment, each comparator compares a subset of the effective address bits specified by the TLBIE request (and not translated by MMU 308) with the corresponding real address bits of the target real address specified in the associated entry in L1 STQ 304 or LMQ 306. Only memory reference requests for which the comparator finds a match are marked by translation sequencer 312. This more sophisticated embodiment therefore reduces the number of memory access requests that are marked, at the expense of additional comparators.

[0095] In some implementations of the less precise and more precise marking embodiments, the markings applied by translation sequencer 312 apply only to requests within processor core 200 and persist only until the marked requests are ejected from processor core 200. In such implementations, L2 cache 230 may revert to pessimistically assuming that all in-progress store requests in L2 cache 230 could have had their addresses translated by translation entries invalidated by the TLBIE request, and may force all such store requests to be ejected prior to processing the store requests using a new translation of the target effective address of the TLBIE request. In other implementations, the more precise markings applied by translation sequencer 312 may also extend to in-progress store requests in L2 cache 230.

[0096] 9 proceeds from block 906 to block 908, which depicts translation sequencer 312 waiting for the requests marked in block 906 to be drained from processor core 200. In particular, translation sequencer 312 waits until all load requests marked in block 906 have returned their requested data to processor core 200 and all store requests marked in block 906 have been issued to L2 STQ 320. In response to all marked requests being drained from processor core 200, translation sequencer 312 inserts a TLBCMPLT request into L2 STQ 320 to indicate that translation sequencer 312 has completed processing of the TLBIE request (block 910). The process of FIG. 9 then ends at block 912.

[0097] 10, a high-level logic flow diagram of an exemplary method by which L2 STQ 320 processes a TLBCMPLT request is shown, according to one embodiment. The process of FIG. 10 begins at block 1000 and thereafter proceeds to block 1002, which illustrates L2 STQ 320 receiving and enqueuing a TLBCMPLT request issued by its associated processor core 200, as described above with reference to block 910 of FIG. 9, into one of its entries. As shown in block 1004, following receipt of the TLBCMPLT request, L2 STQ 320 waits until all older store requests for all hardware threads have been drained from L2 STQ 320. Once all of the older store requests have been drained from the L2 STQ 320, the process proceeds from block 1004 to block 1006, which shows the L2 STQ 320 sending a TLBCMPLT_ACK signal via bus 330 to the TSN machine 346 that issued the TLBIE request and to the arbiter 348, which are waiting for confirmation of completion of processing of the TLBIE request, as described above with reference to blocks 706 and 806.

[0098] In block 1008, the L2 STQ 320 determines whether the associated processor core 200 is the initiating processor core of the TLBIE request whose completion is signaled by the TLBCMPLT request, e.g., by examining thread identification information in the TLBCMPLT request. If it is not the initiating processor core (meaning the process is executing in the L2 cache 230 associated with the snooping processing core 200), processing of the TLBIE request is complete and the L2 STQ 320 removes the TLBCMPLT request from the L2 STQ 320 (block 1014). The process then ends in block 1016.

[0099] On the other hand, if the L2 cache 320 determines in block 1008 that the associated processor core 200 is the initiator processor core 200 of the TLBIE request buffered in the sidecar logic 322, the process proceeds from block 1008 to block 1009, which depicts the L2 STQ 320 issuing a TLBCMPLT_ACK signal to the sidecar logic 322 via bus 330. In response to receiving the TLBCMPLT_ACK signal, the sidecar logic 322 issues a TLBCMPLT_ACK signal to the associated processor core 200 via bus 325. As described above with reference to block 508 of FIG. 5, receipt of the TLBCMPLT_ACK signal releases the initiator thread of the processor core 200 to resume dispatching new instructions (i.e., the TSYNC instruction 408, whose behavior is described with reference to FIG. 11 ). The associated sidecar 324 then deletes the completed TLBIE request (block 1012), and the process proceeds to blocks 1014 and 1016 as described.

[0100] Referring now to FIG. 11, there is shown a simplified logic flow diagram of an exemplary manner in which processor core 200 processes a transform synchronization (eg, TSYNC) instruction, according to one embodiment.

[0101] The illustrated process begins at block 1100 and thereafter proceeds to block 1101, which depicts execution of a TSYNC instruction 408 in instruction sequence 400 by execution unit 300 of processor core 200. In response to the execution of TSYNC instruction 408, processor core 200 halts dispatching of any subsequent instructions in the hardware thread (block 1102). As described above, in the exemplary embodiment of FIG. 3, dispatching is halted because sidecar logic 322 includes only a single sidecar 324 per hardware thread of processor core 200, meaning that at most one TLBIE or TSYNC request per thread can be active at a time.

[0102] At block 1104, a TSYNC request corresponding to the TSYNC instruction 408 is generated and issued to the L1 STQ 304. The TSYNC request may include, for example, a transaction type indicating the type of request (i.e., TSYNC) and an indication of the initiating processor core 200 and hardware thread that issued the TSYNC request. Processing of the request in the L1 STQ 304 progresses, and the TSYNC request eventually moves from the L1 STQ 304 to the L2 STQ 320 via bus 318, as shown in block 1106. The process then proceeds to block 1108, which illustrates the initiating processor core 200 continuing to refrain from dispatching instructions in the initiating hardware thread until it receives a TSYNC_ACK signal from the storage subsystem via bus 325, indicating that processing of the TSYNC request by the initiating processor core 200 has completed. (Generation of the TSYNC_ACK signal is described below with reference to block 1210 of FIG. 12.) Note again that because dispatching of instructions in the initiating thread is halted, there can be no contention for the initiating hardware thread's sidecar 324 by another TLBIE request, since for any given thread, only one of the two types of requests can be in the L2 STQ 320 and sidecar logic 322 at a time.

[0103] In response to determining at block 1108 that a TSYNC_ACK signal has been received, the process proceeds to block 1110, which indicates that processor core 200 resumes dispatching instructions in the initiating thread; therefore, releasing the thread at block 1110 allows processing of HWSYNC instruction 410 (the next instruction in instruction sequence 400) to begin. The process of FIG. 11 then ends at block 1112.

[0104] 12, a high-level logic flow diagram of an exemplary method by which sidecar logic 324 processes a TSYNC request is shown, according to one embodiment. The process begins at block 1200 and thereafter proceeds to block 1202, which illustrates sidecar logic 324 monitoring via interface 321 for notification that a TSYNC request has been enqueued in L2 STQ 320. In response to receiving notification via interface 321 that a TSYNC request has been enqueued in L2 STQ 320, sidecar logic 322 moves the TSYNC request via interface 321 to the initiating hardware thread's sidecar 324 (block 1204). In response to receiving the TSYNC request, the sidecar 324 issues the TSYNC request on the system fabric 110, 114 via the interface 326 (block 1206) and then monitors the coherence response to the TSYNC request to determine whether any TSN machines 346 have provided a retry coherence response (block 1208) as previously described with respect to block 724 of Figure 7. As described above, the TSN machines 346 provide a retry coherence response if the TSN machines are still in the TLBIE_active state and are waiting for their snooping processor core 200 to complete processing of a previous TLBIE request for the same initiating processor core 200 and hardware thread. It may be noted that by the time the TSYNC request is issued, the issuing processing unit's TSN machine 346 is no longer in the TLBIE_active state and will not issue a retry coherence response because the TLBCMPLT_ACK signal resets the issuing processor core's TSN machine 346 to the inactive state in block 1006 before the TLBCMPLT_ACK is issued to the initiating processor core 200 in block 1010.Receipt of the TLBCMPLT_ACK signal by processor core 200 causes initiating processor core 200 to resume dispatch instructions after TLBIE instruction 406, thus causing it to execute TSYNC instruction 408, which generates a TSYNC request. However, initiating processor core 200 may complete processing of the TLBIE request well before snooping processing cores 200 have completed invalidating their translation entries and flushing memory-referencing instructions that depend on, or are possibly marked as dependent on, the invalidated translation entries. Thus, the TSYNC request ensures that invalidation of translation entries in snooping processing cores 200 and flushing of memory-referencing instructions that depend on the invalidated translation entries are completed before initiating processor core 200 executes HWSYNC instruction 410.

[0105] When all snooping processor cores 200 have completed their processing of the TLBIE request, the TSYNC request will eventually complete without a retry coherence response. In response to the TSYNC request completing without a retry coherence response in block 1208, the sidecar 324 issues a TSYNC_ACK signal to the initiating processor core 200 via the bus 325 (block 1210). As described above with reference to block 1108, in response to receiving the TSYNC_ACK signal, the initiating processor core 200 executes the HWSYNC instruction 410, which completes the ordering requirements of the initiating thread with respect to more recent memory reference instructions. Following block 1210, the sidecar 324 deletes the TSYNC request (block 1212), and the process returns to block 1202 as described.

[0106] Here, the instruction sequence 400 of FIG. 4A and the associated processing with reference to FIGS. 5 to 12 have been described in detail. Here, reference is made to FIG. 4B, which shows an alternative code sequence 420 that reduces the number of instructions, particularly synchronization instructions, in the conversion invalidation sequence. As shown, the instruction sequence 420 includes one or more store instructions 422 that invalidate the PTE 222 in the page frame table 220, a HWSYNC instruction 424, and one or more TLBIE instructions 426 that invalidate the cached translation entries for the specified effective addresses in all processor cores 200. Therefore, instructions 422 to 426 correspond to instructions 402 to 406 of the instruction sequence 400 in FIG. 4A. The instruction sequence 420 further includes a PTESYNC instruction 430 that follows immediately after the TLBIE instruction 426. The PTESYNC instruction 430 combines the operations performed by the TSYNC instruction 408 and the HWSYNC instruction 410 in the instruction sequence 400 of FIG. 4A into a single instruction. That is, the execution of the PTESYNC instruction 430 ensures the system-wide completion of the TLBIE request generated by the TLBIE instruction 426 (similar to the TSYNC request generated by the execution of the TSYNC instruction 408), and performs instruction ordering with respect to the most recent memory reference instruction (similar to the HWSYNC request generated by the execution of the HWSYNC instruction 410), by generating a PTESYNC request that is broadcast to all processing units 104 of the data processing system 100.

[0107] Given the similarity between the instruction sequences 420 and 400, the processing of the instruction sequence 420 is the same as that of the instruction sequence 400 given in FIGS. 5 to 12, except for the processing related to the PTESYNC request generated by the execution of the PTESYNC instruction 430, which is described below with reference to FIGS. 13 to 15.

[0108] 13, there is shown a schematic logic flow diagram of an exemplary method by which processing core 200 processes page table synchronization (e.g., PTESYNC) instruction 430, according to one embodiment. As described above, PTESYNC instruction 430 and the PTESYNC requests generated by its execution have two functions: ensuring system-wide completion of TLBIE requests generated by TLBIE instruction 426, and enforcing instruction ordering with respect to more recent memory referencing instructions.

[0109] The illustrated process begins at block 1300 and thereafter proceeds to block 1301, which illustrates processor core 200 generating a PTESYNC request by executing a PTESYNC instruction 430 in instruction sequence 420 in execution unit 300. The PTESYNC request may include, for example, a transaction type indicating the type of request (i.e., PTESYNC) and an indication of the initiating processor core 200 and hardware thread that issued the PTESYNC request. In response to executing PTESYNC instruction 430, processor core 200 halts dispatching of any more recent instructions in the initiating hardware thread (block 1302). As described above, in the exemplary embodiment of FIG. 3 , dispatching is halted because sidecar logic 322 includes only a single sidecar 324 per hardware thread of processor core 200, which means that in this embodiment, at most one TLBIE or PTESYNC request per thread can be active at a time.

[0110] 13 proceeds in parallel with block 1303 and blocks 1304-1312. Block 1303 represents the initiating processor core 200 performing a load ordering function for the PTESYNC request by waiting for all appropriate older load requests of all hardware threads (i.e., those that would be architecturally required by HWSYNC to receive their requested data prior to completing processing of the HWSYNC request) to be drained from the LMQ 306. Waiting for these load requests to be satisfied in block 1303 ensures that the set of load requests identified in block 906 receives data from the correct memory page rather than a reallocated memory page (even if the target address was on the memory page being reallocated).

[0111] In parallel with block 1303, processor core 200 also issues a PTESYNC request corresponding to PTESYNC instruction 430 to L1 STQ 304 (block 1304). The process proceeds from block 1304 to block 1308, which illustrates processor core 200 performing the store ordering function of the PTESYNC request by waiting until all appropriate older store requests of all hardware threads (i.e., those architecturally required by HWSYNC to be drained from L1 STQ 304) have been drained from L1 STQ 304. Once the store ordering performed in block 1308 is complete, the PTESYNC request is issued from L1 STQ 304 to L2 STQ 320 via bus 318, as shown in block 1310.

[0112] The process then proceeds from block 1310 to block 1312, which depicts the initiating processor core 200 monitoring to detect receipt of a PTESYNC_ACK signal from the storage subsystem over the bus 325, indicating that processing of the PTESYNC request by the initiating processor core 200 has completed. (Generation of the PTESYNC_ACK signal is described below with reference to block 1410 of FIG. 14.) Note again that because instruction dispatching within the initiating hardware thread remains halted, only one TLBIE request or PTESYNC request can be present in the L2 STQ 320 and sidecar logic 322 at a time for any given thread, so there can be no contention for the initiating hardware thread's sidecar 324 with another TLBIE request.

[0113] Only in response to affirmative determinations at both blocks 1303 and 1312 does the process of Figure 13 proceed to block 1314, which indicates that processor core 200 resumes dispatching instructions in the initiating thread; thus, releasing the thread at block 1314 allows processing of instructions following PTESYNC instruction 430 to begin. The process of Figure 13 then ends at block 1316.

[0114] 14, a high-level logic flow diagram of an exemplary method by which the L2 STQ 320 and sidecar logic 322 of a processing unit 104 process a PTESYNC request is shown, according to one embodiment. The process of FIG. 14 begins at block 1400 and thereafter proceeds to block 1402, which illustrates the L2 STQ 320 monitoring for receipt of a PTESYNC request from the L1 STQ 304, as described above with reference to block 1310 of FIG. 13. In the second embodiment of FIG. 4B, in response to receiving a PTESYNC request, the L2 STQ 320 and sidecar logic 324 cooperate to perform two functions: (1) store ordering for store requests within the L2 STQ 320, and (2) ensuring completion of TLBIE requests in all of the other processing cores 200. 14 embodiment, these two functions are executed in parallel along two paths shown in blocks 1403, 1405 and blocks 1404, 1406, and 1408, respectively. In an alternative embodiment, these functions may instead be serialized by first performing the ordering functions shown in blocks 1403 and 1405, and then ensuring completion of TLBIE requests in blocks 1404, 1406, and 1408. (Note that attempting to serialize the ordering of these functions by ensuring completion of TLBIE requests prior to performing store ordering may create deadlock.)

[0115] Referring now to blocks 1403-1405, the L2 STQ 320 performs store ordering on the PTESYNC request to ensure that all appropriate older store requests in the L2 STQ 320 have been drained from the L2 STQ 320. The set of store requests being ordered in block 1403 includes a first subset that may have had their target addresses translated by translation entries invalidated by earlier TLBIE requests. This first subset corresponds to those marked in block 906. Additionally, the set of store requests being ordered in block 1403 includes a second subset that includes architecturally defined store requests that will be ordered by HWSYNC. Once all such store requests have been drained from the L2 STQ 320, the L2 STQ 320 removes the PTESYNC request from the L2 STQ 320 (block 1405). Removal of the PTESYNC request allows store requests more recent than the PTESYNC request to flow through the L2 STQ 320.

[0116] Referring now to block 1404, the sidecar logic 322 detects the presence of a PTESYNC request in the L2 STQ 320 and copies the PTESYNC request to the appropriate sidecar 324 via interface 321, prior to removing the PTESYNC request from the L2 STQ 320 in block 1405. The process then proceeds in a loop shown in blocks 1406 and 1408, where the sidecar logic 322 continues to issue PTESYNC requests on the system fabric 110, 114 until no processor cores 200 respond with a retry coherence response (i.e., until all previous TLBIE requests for the same processor core and hardware thread have been completed by all snooping processor cores 200).

[0117] Only in response to completion of both the functions depicted in blocks 1403, 1405 and blocks 1404, 1406, and 1408 does the process proceed to block 1410, which illustrates sidecar logic 322 issuing a PTESYNC_ACK signal to the associated processor core via bus 325. Sidecar logic 322 then removes the PTESYNC request from sidecar 324 (block 1412), and the process returns to block 1402 as described.

[0118] Referring now to FIG. 15, there is shown a schematic logic flow diagram of an exemplary method by which the TSN machine 346 processes TLBIE requests, TLBCMPT_ACK signals, and PTESYNC requests, according to one embodiment. As indicated by like reference numerals, FIG. 15 is the same as previously described FIG. 7, except for block 1522. Block 1522 indicates that while in the TLBIE_active state established in block 704, the TSN machine 346 monitors to determine whether a PTESYNC request specifying an initiating processor core and hardware thread matching the TLBIE request has been detected. If not, the process continues to iterate in the loop including blocks 720 and 1522. Meanwhile, in response to the TSN machine 346 detecting a PTESYNC request specifying a processor core and initiating hardware thread matching the TLBIE request while in the TLBIE_active state, the TSN machine 346 provides a retry coherence response, as shown in block 724. As discussed above, a retry coherence response by any TSN machine 346 handling the TLBIE request for the initiating hardware thread forces the PTESYNC request to be retried, preventing the initiating hardware thread from executing any memory referencing instructions more recent than the PTESYNC instruction 430 until the PTESYNC request completes without a retry coherence response.

[0119] As briefly discussed above, avoiding ping-pong livelock is a design concern for data processing system interconnect architectures that allow retries of multicast requests. According to disclosed embodiments, ping-pong livelock can be avoided through the implementation of a central request agent 120 and associated communication protocols to manage the delivery of specific multicast requests that are subject to retry by a snooper. In the following discussion, an example of a suitable design for the central request agent 120 is described with reference to FIG. 18. The flow of an exemplary multicast request (e.g., a TLBIE request) delivered to a snooper via the central request agent 120 according to a first embodiment is then described with reference to FIGS. 19-23. In this first embodiment, it is assumed that the broadcast of a multicast request by the initiating master 1600 requires only a single period of operation on the system fabric. FIGS. 24-29 illustrate the flow of a multicast request in a second embodiment in which the broadcast of a multicast request by the master 1600 can require multiple periods of operation on the system fabric.

[0120] 18, a schematic block diagram of an exemplary embodiment of central request agent 120 is shown, according to one embodiment. In this example, central request agent 120 includes dispatch logic 1802, which is coupled to receive multicast requests of predetermined transaction types (ttypes) over system fabric 1800 (including system interconnect 110 and local interconnect 114). In some embodiments, these multicast requests include TLBIE requests, as previously described; in other embodiments, the multicast requests may include multicast requests of alternative or additional transaction types. Non-multicast requests and multicast requests other than the predetermined ttypes are ignored by dispatch logic 1802 of central request agent 120.

[0121] Central request agent 120 additionally includes a plurality of request forwarding machines (RFMs) 1804, which manage the forwarding of multicast requests accepted by dispatch logic 1802 to snoopers distributed within data processing system 100. In a preferred embodiment, each snooper associated with a given ttype of multicast request delivered by central request agent 120 has a corresponding plurality of snoop machines in number and identifiers for those implemented in central request agent 120. Thus, for example, central request agent 120 may implement eight RFMs 1804 for handling delivery of TLBIE requests, which means that each L2 cache 230 implements eight TSNs 346, each uniquely and respectively corresponding to a respective one of the RFMs 1804 and identifiable by a common machine identifier (e.g., which may be specified in RFM ID field 1704 of the forwarded multicast request). Each RFM 1804 has an associated request buffer 1806 for buffering at least the address field 1702, ttype field 1710, and tag field 1720 of the multicast request currently being distributed for handling. In some embodiments, each RFM 1804 also has an optional associated epoch indication 1808 (e.g., a single bit) that can be utilized to track chronological epochs in the operation of data processing system 100.

[0122] 19, there is shown a schematic logic flow diagram of an exemplary method for an initiating processing unit 104 or central request agent 120 to issue a translation entry invalidation request (e.g., a TLBIE request) on the system fabric according to a first embodiment. For the initiating processing unit 104, the process is performed by the sidecar logic 322 at block 606 of FIG. 6.

[0123] The process of FIG. 19 begins at block 1900 and thereafter proceeds to block 1902, which illustrates a master issuing a TLBIE request having the format given in FIG. 17 to system fabric 1800. When the TLBIE request is issued by sidecar logic 322 of processing unit 104, forward (F) field 1726 is reset (e.g., to b'0'); when it is reissued by central request agent 120 on behalf of the original initiating master, forward field 1726 is set (e.g., to b'1'). As shown in block 1904, the master monitors receipt of a Cresp for the TLBIE request issued in block 1902. In response to receiving the Cresp, the master determines in block 1906 whether the Cresp indicates a retry. If a retry is indicated, the process returns to block 1902 and subsequent blocks, which illustrate the master reissueing the TLBIE request on system fabric 1800. On the other hand, if the master determines in block 1906 that Cresp does not indicate a retry (ie, Cresp indicates a success), the process of FIG.

[0124] As will become apparent from the following discussion, a TLBIE request as originally issued on the system fabric by sidecar logic 322 will receive a cresp indicating retry if central request agent 120 does not have an RFM 1804 available to handle forwarding of the TLBIE request to the final snooper (e.g., L2 cache 230). As long as an RFM 1804 is available within central request agent 120 to handle forwarding of the TLB request, no cresp indicating retry is provided. On the other hand, a TLBIE request as forwarded on the system fabric by RFM 1804 of central request agent 120 will receive a cresp indicating retry until all relevant snoopers (e.g., L2 cache 230) are able to successfully allocate the TSN 346 specified by the TLBIE request to process the TLBIE request.

[0125] 20, there is shown a schematic logic flow diagram of an exemplary method by which central request agent 120 receives a translation entry invalidation request (e.g., a TLBIE request) from initiating processing unit 104, according to a first embodiment. The illustrated process is performed for each TLBIE request received by central request agent 120 via the system fabric that causes forwarding (F) field 1726 to be reset.

[0126] The process of Figure 20 begins at block 2000 in response to receiving a TLBIE request on system fabric 1800 and then proceeds to block 2004, which depicts dispatch logic 1802 determining whether RFM 1804 is available for distribution to handle the TLBIE request. If not, dispatch logic 1802 provides a partial response indicating a retry on system fabric 1800 (block 2010). The retry partial response will cause response logic 1622 to generate a Cresp indicating a retry, which will cause the TLBIE request to be reissued on system fabric 1800, as discussed above with reference to block 1906 of Figure 19. On the other hand, if the dispatch logic 1802 determines in block 2004 that an RFM 1804 of the central request agent 120 is available for distribution to handle the TLBIE request, the dispatch logic 1802 assigns the TLBIE request to the available RFM 1804, logs the TLBIE request in the associated request buffer 1806, and marks the TLBIE request as completed (i.e., ready for forwarding) (block 2006). Additionally, in block 2008, the dispatch logic 1802 provides a null partial response on the system fabric 1800, thereby allowing the TLBIE request to complete successfully without retries, as discussed above with reference to block 1906 of FIG. 19. Following block 2008 or block 2010, the process of FIG. 20 ends in block 2012.

[0127] 21, a schematic logic flow diagram of an exemplary method by which the central request agent 120 selects a translation invalidation entry request (e.g., a TLBIE request) for issuance according to a first embodiment is shown. The process of FIG. 21 begins at block 2100 and then proceeds to block 2102, which illustrates the central request agent 120 determining whether any RFMs 1804 have been allocated to handle TLBIE requests that have completed but not yet been forwarded to the final snooper. If not, the process simply repeats at block 2102. On the other hand, if the central request agent 120 makes a positive determination at block 2102, the central request agent 120 selects one of the RFMs 1804 identified at block 2102 and signals the selected RFM 1804 to forward the TLBIE request over the system fabric 1800 using the process of FIG. 22 (block 2104). Following block 2104, the process of FIG. 21 returns to block 2102 as described above.

[0128] 22, there is shown a schematic logic flow diagram of an exemplary method for a selected RFM 1804 of a central request agent 120 to issue a translation entry invalidation request (e.g., a TLBIE request) according to a first embodiment. The illustrated process is invoked, for example, at block 2104 of FIG. 21.

[0129] The process of FIG. 22 begins at block 2200 and thereafter proceeds to block 2202, which shows the selected RFM 1804 modifying the tag 1720 of its buffered TLBIE request by setting the forwarding field 1726 to indicate that the request is being forwarded from the central request agent 120. In addition, the RFM 1804 preferably updates the TLBIE request to include its unique RFM identifier (ID). For example, in the embodiment of FIG. 17, the RFM ID may conveniently be specified in the lower-order bits of the address field 1702, these lower-order bits being unused for the TLBIE request because the TLBIE request operates on logical memory pages having a minimum page size of, for example, 4 kB. The selected RFM 1804 then issues (and reissues, if necessary) the modified TLBIE request on the system fabric using the previously described process of FIG. 19 until the TLBIE request is accepted by the TSN 346 in each processing unit 104 corresponding to the specified RFM ID (block 2204). The selected RFM 1804 is then released for redistribution (block 2206), and the process of FIG.

[0130] 23, there is shown a schematic logic flow diagram of an exemplary method by which processing unit 104 receives a translation entry invalidation request (e.g., a TLBIE request) from central request agent 120, according to a first embodiment. The illustrated process is performed by processing unit 104 for each snooped TLBIE request that has the forwarding field 1726 set to indicate central request agent 120 as the initiating master. Other requests are simply ignored by the illustrated process.

[0131] 23 begins at block 2300 in response to processing unit 104 snooping a TLBIE request issued by central request agent 120, and then proceeds to block 2302, which depicts processing unit 104 determining whether the TLBIE request is the same request currently being processed or most recently processed by TSN 346 corresponding to RFM ID 1704 specified in the TLBIE request. In response to a positive determination at block 2302, the process proceeds to block 2308, described below. On the other hand, if processing unit 104 makes a negative determination at block 2302, processing unit 104 additionally determines at block 2304 whether the specified TSN 346 (i.e., the TSN 346 corresponding to RFM ID 1704) is currently active, meaning that the specified TSN 346 is still busy working on a previously received TLBIE request. In response to a positive determination in block 2304, the processing unit 104 issues a retry partial response on the system fabric 1800, which will cause the central request agent 120 to generate a retry Cresp, which will cause the central request agent 120 to reissue the TLBIE request (block 2310).

[0132] Returning to block 2304, in response to determining that the specified TSN 346 is not currently active, the processing unit 104 assigns the TLBIE request to the specified TSN 346 for handling and marks the TLBIE request as completed (i.e., ready for processing by the TSN 346) (block 2306). The steps illustrated in block 2306 cause the TSN 346 to set TLBIE_active as discussed above in blocks 702-704 of FIG. 7 and to forward the TLBIE request to the associated processor core 200 as discussed above in blocks 802-804 of FIG. 8. The process of FIG. 23 then proceeds to block 2308, which illustrates the processing unit 104 issuing a null partial response on the system fabric 1800. Following block 2308 or 2310, the process of FIG. 23 ends at block 2312.

[0133] In the process of Figure 23 (and similar Figure 28 below), processing units 104 can safely provide null partial responses to previously snooped TLBIE requests that are reissued on system fabric 1800 because central request agent 120 is configured to not send a new TLBIE request from a given RFM 1804 until it receives a successful (i.e., non-retry) Cresp for the preceding TLBIE request, where a successful Cresp indicates that the preceding TLBIE request has been accepted by all associated processing units 104. Any TSN 346 that has completed processing for a given TLBIE request earlier than corresponding TSNs 346 in other processing units 104 simply maintains an idle state waiting for the other TSNs to complete their processing of TLBIE requests and allowing a new TLBIE request to be issued by the corresponding RFM 1804.

[0134] In the description of the first embodiment with reference to Figures 19-23, it is implicitly assumed that a multicast request (e.g., a TLBIE request) can be transmitted over the system fabric 1800 in a single period of operation. In some implementations, this may not be true for all multicast requests given the relative size of the multicast request to the width of the system fabric 1800. However, the disclosed techniques utilizing a central request agent to distribute multicast requests can be extended for use in such environments, as described with reference to the second embodiment presented herein and subsequent Figures. In the second embodiment, exemplary multicast requests include those requesting only a single period of operation (referred to as TLBIE_OP1_Only) and those requesting two periods of operation (referred to as TLBIE_OP1 and TLBIE_OP2).

[0135] 24, there is shown a schematic logic flow diagram of an exemplary method for an initiation processing unit 104 or central request agent 120 to issue a translation entry invalidation request (e.g., a TLBIE request) according to a second embodiment. For the initiation processing unit 104, the process of FIG. 24 is performed by the sidecar logic 322 at block 606 of FIG. 6.

[0136] The process of FIG. 24 begins at block 2400 and then proceeds in parallel to blocks 2402 and 2410. Block 2402 depicts a master issuing a TLBIE_OP1_Only or TLBIE_OP1 request on the system fabric, having the format given in FIG. 17. When issued by the sidecar logic 322 of the processing unit 104, the forward (F) field 1726 is reset (e.g., to b'0'); when issued again by the central request agent 120 on behalf of the original initiating master, the forward field 1726 is set (e.g., to b'1'). As shown in block 2404, the master monitors receipt of a Cresp for the multicast request issued in block 2402. In response to receiving the Cresp, the master determines in block 2406 whether the Cresp indicates a retry. If so, the process returns to block 2402 and subsequent blocks, which indicate the master reissues the TLBIE_OP1_Only and TLBIE_OP1 requests. On the other hand, if the master determines in block 2406 that Cresp does not indicate a retry but instead indicates success, the process of FIG. 24 proceeds to junction 2422.

[0137] Referring now to block 2410, the master additionally determines whether the transaction type of the multicast request indicates that only a single operation period is required. In various implementations, this information may be conveyed, for example, in the ttype field 1710 or in the operation (O) field 1730. In response to a positive determination at block 2410, the process proceeds to join point 2422. On the other hand, if the master makes a negative determination at block 2410, meaning that the TLBIE request includes both a TLBIE_OP1 request and a TLBIE_OP2 request, the master handles the multiple operation periods of the multicast request according to implementation-dependent ordering requirements, if any. That is, as indicated by the positive determination at block 2412, if a given implementation requires that a TLBIE_OP1 request be issued before the corresponding TLBIE_OP2 request, the master postpones issuing the TLBIE_OP2 request until handling of the TLBIE_OP1 request reaches join point 2422 (block 2414). On the other hand, if a given implementation does not require a TLBIE_OP1 request to be issued prior to the corresponding TLBIE_OP2 request, as indicated by a negative result in block 2412, the master issues the TLBIE_OP2 request on the system fabric independent of the timing of the issuance of the corresponding TLBIE_OP1 request (block 2416). As described above, when issued by the sidecar logic 322, the forward (F) field 1726 is reset (e.g., to b'0'), and when issued again by the central request agent 120 on behalf of the original initiating master, the forward field 1726 is set (e.g., to b'1'). As indicated in block 2418, the master monitors receipt of a Cresp for the TLBIE_OP2 request issued in block 2416. In response to receiving the Cresp, the master determines in block 2420 whether the Cresp indicates a retry.If so, the process returns to block 2416 and subsequent blocks, which indicate the master reissues the TLBIE_OP2 request. On the other hand, if the master determines in block 2420 that Cresp does not indicate a retry, the process of Figure 24 proceeds to join point 2422. Once both legs of the process reach join point 2422, the process of Figure 24 ends at block 2424.

[0138] As discussed above with reference to FIG. 19 , a TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request as originally issued on the system fabric by sidecar logic 322 receives a cresp indicating retry only if central request agent 120 does not have an RFM 1804 available to handle forwarding of the multicast request to the final snooper (e.g., L2 cache 230). A successful cresp is provided as long as an RFM 1804 is available within central request agent 120 to handle forwarding of the TLBIE request. On the other hand, a multicast request as forwarded on the system fabric by RFM 1804 of central request agent 120 will receive a cresp indicating retry until all associated snoopers (e.g., L2 cache 230) are able to receive the multicast request and successfully distribute the specified TSN 346 to handle the multicast request.

[0139] 25, there is shown a schematic logic flow diagram of an exemplary method by which central request agent 120 receives a translation entry invalidation request (e.g., a TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request) from initiating processing unit 104, according to a second embodiment. The illustrated process is performed for each TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request received by central request agent 120 via system fabric 1800 that has caused its forwarding (F) field 1726 to be reset. Other types of requests are simply ignored by central request agent 120.

[0140] 25 begins at block 2500 in response to central request agent 120 receiving a TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request on system fabric 1800. The process then proceeds from block 2500 to block 2502, which illustrates dispatch logic 1802 determining whether the received request is a TLBIE_OP1_Only request. In response to a positive determination at block 2502, the process proceeds to block 2504 and subsequent blocks. In response to a negative determination at block 2502, meaning the received request is a TLBIE_OP1 or TLBIE_OP2 request, the process proceeds to block 2520 and subsequent blocks.

[0141] Referring to block 2504, the dispatch logic 1802 determines whether an RFM 1804 is currently available for distribution to handle the TLBIE_OP1_Only request. If not, the dispatch logic 1802 provides a partial response indicating a retry on the system fabric 1800 (block 2510). A retry partial response will cause the response logic 1622 to generate a Cresp indicating a retry, which will cause the TLBIE_OP1_Only request to be reissued on the system fabric 1800, as shown in block 2406 of FIG. 24. On the other hand, if the dispatch logic 1802 determines in block 2504 that at least one RFM 1804 of the central request agent 120 is available for distribution to handle the TLBIE_OP1_Only request, the dispatch logic 1802 assigns the TLBIE_OP1_Only request to an available RFM 1804, logs the TLBIE_OP1_Only request in the associated request buffer 1806, and marks the TLBIE_OP1_Only request as completed (i.e., ready for forwarding) (block 2506). Additionally, in block 2508, the dispatch logic 1802 provides a null partial response over the system fabric 1800, thereby allowing the TLBIE_OP1_Only request to complete without retries, as discussed above with reference to block 2406 of FIG. 24. Following block 2508 or block 2510, the process of FIG. 25 ends in block 2530.

[0142] Referring now to block 2520, the dispatch logic 1802 determines whether any RFM 1804 holds an uncompleted TLBIE_OP2 or TLBIE_OP1 in its request buffer 1806 corresponding to a TLBIE_OP1 or TLBIE_OP2 request, respectively, received via the system fabric 1800. The corresponding request will have a matching tag 1720 except that the operation type field 1730 is set to indicate another type of request (e.g., TLBIE_OP1 or TLBIE_OP2). In response to a positive determination at block 2520, the dispatch logic 1802 assigns the received TLBIE_OP1 or TLBIE_OP2 request received at block 2502 to the RFM 1804 that buffers the corresponding TLBIE_OP2 or TLBIE_OP1 request and marks both the TLBIE_OP1 and TLBIE_OP2 requests as completed (block 2522). The process then proceeds to block 2529, which is described below.

[0143] Returning to block 2520, in response to determining that no RFM 1804 currently buffers an outstanding TLBIE_OP2 or TLBIE_OP1 corresponding to the TLBIE_OP1 or TLBIE_OP2 request received via system fabric 1800, dispatch logic 1802 determines, in block 2524, whether an RFM 1804 is available for distribution to handle the TLBIE_OP1 or TLBIE_OP2 request. If not, dispatch logic 1802 provides a partial response indicating a retry on system fabric 1800 (block 2526). A retry partial response will cause response logic 1622 to generate a Cresp signal indicating a retry, which will cause the TLBIE_OP1 or TLBIE_OP2 request to be reissued on system fabric 1800, as shown in block 2406 or block 2420 of FIG. 24 . On the other hand, if the dispatch logic 1802 determines in block 2524 that an RFM 1804 of the central request agent 120 is available for distribution to handle the received TLBIE_OP1 or TLBIE_OP2 request, the dispatch logic 1802 assigns the TLBIE_OP1 or TLBIE_OP2 request to the available RFM 1804 and logs the request in the associated request buffer 1806 (block 2528). Additionally, in block 2529, the dispatch logic 1802 provides a null partial response over the system fabric 1800, thereby allowing the TLBIE_OP1 or TLBIE_OP2 request to complete without retries, as discussed above with reference to block 2406 or block 2420 of FIG. 24. Following block 2526 or block 2529, the process of FIG. 25 ends in block 2530.

[0144] 26, there is shown a schematic logic flow diagram of an exemplary method by which the central request agent 120 selects a translation entry invalidation request (e.g., a TLBIE_OP1_Only or TLBIE_OP1 / TLBIE_OP2 request) for issuance according to a second embodiment. The process of FIG. 26 begins at block 2600 and thereafter proceeds to block 2602, which illustrates the central request agent 120 determining whether any RFMs 1804 have been distributed to handle multicast requests (e.g., TLBIE_OP1_Only or TLBIE_OP1 / TLBIE_OP2 requests) that have completed but not yet been forwarded to the final snooper. If not, the process simply repeats at block 2602. On the other hand, if the central request agent 120 makes an affirmative determination at block 2602, the central request agent 120 selects one of the RFMs 1804 identified at block 2602 and signals the selected RFM 1804 to forward the multicast request over the system fabric according to the process of Figure 27 (block 2604). Following block 2604, the process of Figure 26 returns to block 2602 as described.

[0145] 27, there is shown a schematic logic flow diagram of an exemplary method for a central request agent to issue a translation entry invalidation request on system fabric 1800 according to a second embodiment. The illustrated process is invoked, for example, at block 2604 of FIG. 26.

[0146] The process of Figure 27 begins at block 2700 and thereafter proceeds to block 2702, which shows the selected RFM 1804 modifying the tag 1720 of the buffered TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request to be forwarded by setting the forwarding field 1726 to indicate the central request agent 120 as the source. In addition, the RFM 1804 preferably updates the TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request to include its unique RFM ID 1704, for example, in the low-order bits of the address field 1702. If an epoch indication 1808 is implemented in the central request agent 120, the selected RFM 1804 also configures the epoch field 1728 of the tag 1720 with the value of its associated epoch indication 1808. Following the tag modification indicated in block 2702, the selected RFM 1804 then issues (and reissues, if necessary) the modified multicast request on the system fabric 1800 using the previously described process of FIG. 24 until the multicast request is accepted by the TSN 346 in each processing unit 104 corresponding to the specified RFM ID 1704 (block 2704). In optional block 2706, the selected RFM 1804 toggles the value of its associated epoch indication 1808. Updating the epoch indication 1808 between issuing each pair of at least TLBIE_OP1 and TLBIE_OP2 requests allows processing units 104, which may be powered off and on asynchronously to the issuance of multicast requests, to detect whether snooped TLBIE_OP1 and TLBIE_OP2 form a corresponding pair. Following block 2704, or optional block 2706, if present, the selected RFM 1804 is released for redistribution (block 2708), and the process of FIG.

[0147] 28-29, there is shown a schematic logic flow diagram of an exemplary method by which a processing unit 104 receives a translation entry invalidation request (e.g., TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2) for processing, according to a second embodiment. The illustrated process is performed by each processing unit 104 for each snooped TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request that has the forwarding field 1726 set in its tag 1720 to indicate central request agent 120 as the initiating master. Other requests are simply ignored by the process of FIGS. 28-29.

[0148] The process of Figure 28 begins at block 2800 in response to processing unit 104 snooping a TLBIE_OP1_Only, TLBIE_OP1, or TLBIE_OP2 request issued by central request agent 120. The process then proceeds to block 2801, which depicts processing unit 104 determining whether the snooped request is a TLBIE_OP1_Only request. If so, the process proceeds to block 2802 and subsequent blocks of Figure 28; if not, which means the snooped request is a TLBIE_OP1 or TLBIE_OP2 request, the process passes through page connector A to block 2902 and subsequent blocks of Figure 29.

[0149] Referring now to block 2802, the processing unit 104 determines whether the snooped TLBIE_OP1_Only request is the same request currently being processed or most recently processed by the TSN 346 corresponding to the RFM ID 1704 specified in the snooped request. In response to a positive determination at block 2802, the process proceeds to block 2808, described below. On the other hand, if the processing unit 104 makes a negative determination at block 2802, the processing unit 104 additionally determines at block 2804 whether the TSN 346 corresponding to the specified RFM ID 1704 is currently active, meaning that the specified TSN 346 is still busy working on a previously snooped multicast request. In response to a positive determination at block 2804, the processing unit 104 issues a retry partial response on the system fabric 1800, which causes the central request agent 120 to generate a retry Cresp that causes the TLBIE_OP1_Only request to be reissued (block 2810).

[0150] Returning to block 2804, in response to determining that the TSN 346 corresponding to the RFM ID 1704 specified in the snooped TLBIE_OP1_Only request is not currently active, the processing unit 104 assigns the snooped TLBIE_OP1_Only request to the specified TSN 346 for handling and marks the TLBIE_OP1_Only request as completed (i.e., ready for processing by the TSN 346) (block 2806). The steps illustrated in block 2806 cause the TSN 346 to set TLBIE_active as discussed above in blocks 702-704 of FIG. 7 and forward the TLBIE_OP1_Only request to the associated processor core 200 as discussed above in blocks 802-804 of FIG. The process of Figure 28 then proceeds to block 2808, which depicts processing unit 104 issuing a null partial response on system fabric 1800. Following block 2808 or 2810, the process of Figure 28 ends at block 2812.

[0151] 29 block 2902, the processing unit 104 determines whether the snooped TLBIE_OP1 or TLBIE_OP2 request is the same request currently being processed or most recently processed by the TSN 346 corresponding to the RFM ID 1704 specified in the snooped request. If the use of epochs is implemented, the determination made in block 2902 includes determining whether the epoch field 1728 of the snooped request matches the epoch field 1728 of the currently being processed or most recently processed request. In response to a positive determination at block 2902, the process proceeds to block 2920, described below. On the other hand, if the processing unit 104 makes a negative determination at block 2902, the snooping processing unit 104 optionally additionally determines at block 2904 whether the specified TSN 346 is currently being served with an outstanding request that specifies a different epoch than the one specified in the epoch field 1728 of the snooped request. In response to a positive determination at block 2904, the processing unit 104 cancels the outstanding prior multicast request and discards it from the specified TSN 346 (block 2906). Following block 2906 or a negative determination at block 2904, the process proceeds to block 2910.

[0152] Block 2910 depicts processing unit 104 determining whether TSN 346 corresponding to the specified RFM ID 1704 is currently assigned an uncompleted TLBIE_OP2 or TLBIE_OP1, respectively, corresponding to a TLBIE_OP1 or TLBIE_OP2 request snooped via system fabric 1800. The corresponding request will have a matching tag 1720 except that operation type field 1730 is set to indicate another type of request (e.g., TLBIE_OP1 or TLBIE_OP2). In response to a positive determination at block 2910, processing unit 104 assigns the snooped TLBIE_OP2 or TLBIE_OP1 to the specified TSN 346 and marks the TLBIE_OP1 and TLBIE_OP2 requests as completed and therefore ready to be processed by TSN 346 (block 2912). Marking the TLBIE_OP1 and TLBIE_OP2 requests as shown in block 2912 causes TSN 346 to set TLBIE_active as discussed above in blocks 702-704 of Figure 7 and to forward the TLBIE_OP1 and TLBIE_OP2 requests to the associated processor core 200 as discussed above in blocks 802-804 of Figure 8. The process then proceeds to block 2920, described below.

[0153] Returning to block 2910, in response to a negative determination, the process proceeds to block 2914, which depicts the processing unit 104 determining whether the specified TSN 346 is active, meaning that the specified TSN 346 is still busy working on a previously snooped request. In response to a positive determination at block 2914, the processing unit 104 issues a retry partial response on the system fabric 1800, which will cause the generation of a retry Cresp (block 2918) that will cause the central request agent 120 to reissue the TLBIE_OP1 or TLBIE_OP2 request, as described above with reference to blocks 2406 and 2420 of FIG. 24 .

[0154] Returning to block 2914, in response to determining that the specified TSN 346 is not currently active, the processing unit 104 assigns the snooped TLBIE_OP1 or TLBIE_OP2 request to the specified TSN 346 for handling and marks the request as incomplete (block 2916). The process of Figure 29 then proceeds to block 2920, which depicts the processing unit 104 issuing a null partial response on the system fabric 1800. Following block 2918 or 2820, the process of Figure 29 ends at block 2922.

[0155] Referring now to FIG. 30 , a block diagram of an exemplary design flow 3000 used, for example, in semiconductor IC logic design, simulation, testing, layout, and manufacturing is shown. The design flow 3000 includes processes, machines, and / or mechanisms for processing design structures or devices to generate logically or otherwise functionally equivalent representations of the design structures and / or devices described above. The design structures processed and / or generated by the design flow 3000 may be encoded on a machine-readable transmission or storage medium to include data and / or instructions that, when executed or otherwise processed on a data processing system, generate logically, structurally, mechanically, or otherwise functionally equivalent representations of hardware components, circuits, devices, or systems. The machine includes, but is not limited to, any machine used in an IC design process, such as designing, manufacturing, or simulating a circuit, component, device, or system. For example, the machine may include a lithography machine, a machine and / or equipment for generating a mask (e.g., an electron beam writer), a computer or equipment for simulating a design structure, any apparatus used in a manufacturing or testing process, or any machine for programming a functionally equivalent representation of a design structure into any medium (e.g., a machine for programming a programmable gate array).

[0156] The design flow 3000 may vary depending on the type of representation being designed. For example, a design flow 3000 for building an application specific integrated circuit (ASIC) may differ from a design flow 3000 for designing a standard component, or from a design flow 3000 for instantiating a design into a programmable array, such as a programmable gate array (PGA) or field programmable gate array (FPGA) offered by Altera® Inc. or Xilinx® Inc.

[0157] FIG. 30 illustrates multiple such design structures, including an input design structure 3020, that are preferably processed by the design process 3010. The design structure 3020 may be a logic simulation design structure generated and processed by the design process 3010 to generate a logically equivalent functional representation of a hardware device. The design structure 3020 may also or alternatively include data and / or program instructions that, when processed by the design process 3010, generate a functional representation of the physical structure of the hardware device. Whether representing functional and / or structural design features, the design structure 3020 may be generated using electronic computer-aided design (ECAD) as implemented by a core developer / designer. Once encoded on a machine-readable data transmission, gate array, or storage medium, the design structure 3020 may be accessed and processed by one or more hardware and / or software modules in the design process 3010 to simulate or otherwise functionally represent an electronic component, circuit, electronic or logic module, apparatus, device, or system, such as those illustrated herein. Thus, design structure 3020 may include files or other data structures containing human- and / or machine-readable source code, compiled structures, and computer-executable code structures that, when processed by a design or simulation data processing system, functionally simulate or otherwise represent circuit or other levels of hardware logic design. Such data structures may include hardware-description language (HDL) design entities or other data structures that conform to and / or are compatible with lower-level HDL design languages ​​such as Verilog and VHDL, and / or higher-level design languages ​​such as C or C++.

[0158] Design process 3010 preferably employs and incorporates hardware and / or software modules for synthesizing, transforming, or otherwise processing functional equivalents of the design / simulation of components, circuits, devices, or logic structures shown herein to generate netlist 3080, which may include design structures such as design structure 3020. Netlist 3080 may include, for example, a compiled or otherwise processed data structure representing a list of wires, discrete components, logic gates, control circuits, I / O devices, models, etc., describing connections to other elements and circuits in an integrated circuit design. Netlist 3080 may be synthesized using an iterative process in which netlist 3080 is resynthesized one or more times depending on the design specifications and parameters for the device. As with the other design structure types described herein, netlist 3080 may be recorded on a machine-readable storage medium or programmed into a programmable gate array. The medium may be a non-volatile storage medium such as a magnetic or optical disk drive, a programmable gate array, compact flash, or other flash memory. Additionally or alternatively, the medium may be system or cache memory, or buffer space.

[0159] Design process 3010 may include hardware and software modules for processing a variety of input data structure types, including netlist 3080. Such data structure types may, for example, exist in library elements 3030 and include sets of commonly used elements, circuits, and devices, including models, layouts, and symbolic representations, for a given manufacturing technology (e.g., different technology nodes, 30 nm, 45 nm, 90 nm, etc.). Data structure types may further include design specifications 3040, characterization data 3050, verification data 3060, design rules 3070, and test data files 3085, which may include input test patterns, output test results, and other test information. Design process 3010 may also include standard mechanical design processes, such as stress analysis, thermal analysis, machine event simulation, process simulation for operations such as casting, molding, and die press forming. Those skilled in the art of mechanical design will appreciate the range of possible mechanical design tools and applications that can be used in design process 3010 without departing from the scope and spirit of the present invention. Design process 3010 may include modules for performing standard circuit design processes such as timing analysis, verification, design rule checking, place and route operations, and the like.

[0160] Design process 3010 employs and incorporates logic and physical design tools, such as HDL compilers and simulation model building tools, to process design structure 3020 with some or all of the illustrated supporting data structures, along with any additional mechanical design or data (if applicable), to generate second design structure 3090. Design structure 3090 resides on a storage medium or programmable gate array in a data format used for the exchange of mechanical device and structure data (e.g., information stored in IGES, DXF, Parasolid XT, JT, DRG, or any other suitable format for storing or rendering such mechanical design structures). Like design structure 3020, design structure 3090 preferably resides on a transmission or data storage medium and includes one or more files, data structures, or other computer-encoded data or instructions that, when processed by an ECAD system, generate a logically or otherwise functionally equivalent form of one or more of the invention embodiments illustrated herein. In one embodiment, the design structure 3090 may include a compiled, executable HDL simulation model that functionally simulates one or more of the devices illustrated herein.

[0161] Design structure 3090 may employ a data format and / or symbolic data format used for the exchange of integrated circuit layout data (e.g., information stored in GDSII (GDS2), GL1, OASIS, map files, or any other suitable format for storing such design data structures). Design structure 3090 may include information such as, for example, symbolic data, map files, test data files, design content files, fabrication data, layout parameters, wires, metal levels, vias, shapes, data for routing fabrication lines, and any other data required by a manufacturer or other designer / developer to produce the devices or structures described above and shown herein. Design structure 3090 may then proceed to stage 3095, where, for example, design structure 3090 proceeds to tapeout, is released to fabrication, is released to a mask house, is sent to another design house, is returned to a customer, etc.

[0162] As described, in at least one embodiment, a data processing system includes a master, a central request agent, and multiple snoopers communicatively coupled to a system fabric for communicating requests subject to retry. The master issues multicast requests intended for the multiple snoopers over the system fabric. The central request agent receives the multicast requests over the system fabric, assigns the multicast requests to specific state machines among multiple state machines in the central request agent, and provides a coherence response to the master indicating successful completion of the multicast request. The central request agent repeatedly issues the multicast requests over the system fabric in association with a machine identifier that identifies the specific state machine until the coherence response indicates that the multicast request was successfully received by all of the multiple snoopers.

[0163] Although embodiments have been described in which the various tag fields (e.g., forward, epoch, and action fields) are explicitly shown as separate tag fields for ease of explanation, in other embodiments the same information may be conveyed using other means. For example, in some embodiments, the number of code points in the encoded machine ID may be expanded for state machines issuing multicast requests that are distributed via a central request agent.

[0164] Although aspects of the claimed invention have been described with reference to embodiments in which a TLBIE request is employed as an example of a multicast request that may be distributed by a central request agent, those skilled in the art will appreciate that the disclosed techniques may also be applied to other types of multicast requests. For example, the disclosed techniques may be applied to ICBI (Instruction Cache Block Invalidate) requests, which request invalidation of specified instructions in a non-coherent instruction cache.

[0165] It should also be understood that not all multicast requests need to be delivered via a central request agent. For example, in the described example, PTESYNC and TSYNC commands are not delivered to the snooper via a central request agent because the snooper (e.g., an L2 cache) does not accept and process TSYNC or PTESYNC requests, and therefore these commands are not subject to ping-pong livelock.

[0166] While various embodiments have been specifically shown and described, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the appended claims, and that all of these alternative implementations fall within the scope of the appended claims. For example, while aspects have been described in terms of a computer system executing program code directing the functions of the present invention, it should be understood that the present invention may alternatively be implemented as a program product including a computer-readable storage device storing program code that can be processed by a processor of the data processing system to cause the data processing system to perform the described functions. A computer-readable storage device may include volatile or non-volatile memory, optical or magnetic disks, or the like, but excludes non-statutory subject matter such as a propagated signal per se, a transmission medium per se, and forms of energy per se.

[0167] As an example, a program product may include data and / or instructions that, when executed or otherwise processed on a data processing system, generate a logically, structurally, or otherwise functionally equivalent representation (including a simulation model) of a hardware component, circuit, device, or system disclosed herein. Such data and / or instructions may include Hardware Description Language (HDL) design entities or other data structures that conform to and / or are compatible with lower-level HDL design languages ​​such as Verilog and VHDL, and / or higher-level design languages ​​such as C or C++. Additionally, the data and / or instructions may employ data formats and / or symbolic data formats used for the exchange of integrated circuit layout data (e.g., information stored in GDSII (GDS2), GL1, OASIS, map files, or any other suitable format for storing such design data structures).

Claims

1. 1. A method for distributing multicast requests in a data processing system comprising a master, a central request agent, and a plurality of snoopers communicatively coupled to a system fabric for communicating requests subject to retry, the method comprising: issuing a multicast request on the system fabric by the master, the multicast request being intended for the plurality of snoopers; receiving, by the central request agent over the system fabric, the multicast request, assigning the multicast request to a particular state machine among a plurality of state machines in the central request agent, and providing a coherence response to the master indicating successful completion of the multicast request; and repeatedly issuing the multicast request over the system fabric by the central request agent in association with a machine identifier that identifies the particular state machine until a coherence response indicates that the multicast request was successfully received by all of the plurality of snoopers. A method comprising:

2. The method of claim 1 , wherein the multicast request comprises a translation entry invalidation request.

3. each of the plurality of snoopers having a plurality of snoop machines corresponding in number to the plurality of state machines in the central request agent; The method comprises: and assigning, based on the machine identifier, each of the plurality of snoopers to a particular snoop machine among the plurality of snoop machines, the particular snoop machine corresponding to the particular state machine, the multicast request received from a central request agent. The method of claim 1 further comprising:

4. and prior to said step of repeatedly issuing, said central request agent modifying said multicast request to indicate that said multicast request is to be forwarded from said central request agent. The method of claim 1 further comprising:

5. in response to the central request agent repeatedly issuing the multicast request, one or more of the plurality of snoopers providing a null coherence response indicating successful receipt of a prior issuance of the multicast request by the central request agent. The method of claim 1 further comprising:

6. The method of claim 1 , wherein the multicast request comprises multiple periods of activity on the system fabric.

7. the receiving step includes the central request agent receiving at least a first operating period and a corresponding second operating period; the assigning step includes the central request agent assigning the first and second operating periods to the particular state machine; 7. The method of claim 6, further comprising the step of the central request agent marking the multicast request as available for delivery to the plurality of snoopers based on the assignment of both the first and second operating periods to the particular state machine.

8. the repeatedly issuing step includes the central request agent issuing the first period of operation of the multicast request on the system fabric along with an epoch indication; 7. The method of claim 6, further comprising one of the plurality of snoopers discarding a previously snooped multicast request based on a non-matching epoch indication.

9. 1. A data processing system comprising a central request agent communicatively coupled to a system fabric to which a plurality of snoopers and masters are communicatively coupled, The central request agent has a plurality of state machines, the central request agent comprising: receiving a multicast request intended for the plurality of snoopers from the master on the system fabric; in response to receiving the multicast request, assigning the multicast request to a particular state machine among the plurality of state machines and providing a coherence response indicating successful completion of the multicast request; and repeatedly issuing the multicast request over the system fabric in association with a machine identifier that identifies the particular state machine until a coherence response indicates that the multicast request was successfully received by all of the plurality of snoopers.

1. A data processing system configured to run

10. 10. The data processing system of claim 9, wherein the multicast request comprises a translation entry invalidation request.

11. further comprising the plurality of snoopers; each of the plurality of snoopers having a plurality of snoop machines corresponding in number to the plurality of state machines in the central request agent; 10. The data processing system of claim 9, wherein each of the plurality of snoopers is configured to assign the multicast request received from a central request agent to a particular snoop machine among the plurality of snoop machines that corresponds to the particular state machine based on the machine identifier.

12. The central request agent: modifying the multicast request to indicate that the multicast request is to be forwarded from the central request agent prior to said repeatedly issuing the multicast request.

10. The data processing system of claim 9 configured to execute:

13. The plurality of snoopers further comprises: providing a null coherence response indicating successful receipt of a prior issuance of said multicast request by said central request agent; 10. The data processing system of claim 9 configured to execute:

14. 10. The data processing system of claim 9, wherein the multicast request comprises multiple periods of activity on the system fabric.

15. The multicast request includes at least a first operating period and a corresponding second operating period; the assigning includes the central request agent assigning the first and second operating periods to the particular state machine; The central request agent: marking the multicast request as available for delivery to the plurality of snoopers based on the assignment of both the first and second operating periods to the particular state machine.

15. The data processing system of claim 14 configured to execute:

16. 15. The data processing system of claim 14, wherein said repeatedly issuing comprises said central request agent issuing said first operating period of said multicast request on said system fabric with an epoch indication.

17. the system fabric; the master; and the plurality of snoopers 10. The data processing system of claim 9, further comprising:

18. A design structure tangibly embodied in a machine-readable storage device for designing, manufacturing, or testing an integrated circuit, comprising: a central request agent communicatively coupled to a system fabric to which a plurality of snoopers and masters are communicatively coupled; The central request agent has a plurality of state machines, the central request agent comprising: receiving a multicast request intended for the plurality of snoopers from the master on the system fabric; in response to receiving the multicast request, assigning the multicast request to a particular state machine among the plurality of state machines and providing a coherence response indicating successful completion of the multicast request; and repeatedly issuing the multicast request over the system fabric in association with a machine identifier that identifies the particular state machine until a coherence response indicates that the multicast request was successfully received by all of the plurality of snoopers. A design structure that is configured to perform

19. 20. The design structure of claim 18, wherein the multicast request comprises a translation entry invalidation request.

20. The central request agent: modifying the multicast request to indicate that the multicast request is to be forwarded from the central request agent prior to said repeatedly issuing the multicast request.

20. The design structure of claim 18 configured to perform:

21. 20. The design structure of claim 18, wherein the multicast request comprises multiple periods of activity on the system fabric.

22. The multicast request includes at least a first operating period and a corresponding second operating period; the assigning includes the central request agent assigning the first and second operating periods to the particular state machine; The central request agent: marking the multicast request as available for delivery to the plurality of snoopers based on the assignment of both the first and second operating periods to the particular state machine.

22. The design structure of claim 21 configured to perform:

23. 22. The design structure of claim 21, wherein the repeatedly issuing comprises the central request agent issuing the first operating period of the multicast request on the system fabric with an epoch indication.

Images