CACHE STORAGE ACCESS

Abstract

A method of data processing in a multiprocessor data processing system (200) comprising several vertical cache memory hierarchies supporting a plurality of processor cores (102a, 102b), a system memory (132), and a system connection connected to the system memory (132) and the several vertical cache memory hierarchies, wherein the method comprises: in response to receiving a request, loading and reserving from a first processor core (102a, 102b), outputting through a first cache memory in a first vertical cache memory hierarchy supporting the first processor core (102a, 102b), on a system connection, a memory access request for a target cache memory row of the request to load and reserve;In response to the memory access request and prior to receiving a system-wide coherence response for the memory access request, the first cache memory receives the target cache memory row from a second cache memory in a second vertical cache memory hierarchy through cache-to-cache intervention and an early specification of the system-wide coherence response for the memory access request; and in response to the early specification of the system-wide coherence response and prior to receiving the system-wide coherence response, the first cache memory initiates processing to update the target cache memory row in the first cache memory.

Description

TECHNICAL AREA

[0001] The present invention relates to data processing and in particular to improving the performance of the data processing system by reducing the data transfer interval in a multi-processor data processing system based on an early specification of a system-wide coherence response. BACKGROUND

[0002] A conventional symmetric multiprocessor (SMP) computer system, such as a server computer system, contains multiple processing units, all connected by a system link that typically includes one or more address, data, and control buses. Connected to the system link is a system memory, which represents the lowest level of shared memory in the multiprocessor computer system and is generally accessible for read and write operations by all processing units. To reduce access latency to instructions and data located in the system memory, each processing unit is typically further supported by a corresponding multi-level vertical cache hierarchy, the lower level(s) of which can be shared by one or more processor cores.

[0003] Because multiple processor cores can request write access to the same memory block (e.g., cache memory row or sector), and because cached memory blocks that are modified are not immediately synchronized with system memory, the cache memory hierarchies of multiprocessor computer systems typically implement a cache memory coherence protocol to ensure at least a minimum required level of coherence between the different processor cores' "views" of the system memory contents. The minimum required coherence level is determined by the selected memory consistency model, which defines rules for the apparent order and visibility of updates to the distributed shared memory.For all memory consistency models on the continuum between models with weak consistency and models with strong consistency, cache memory coherence requires at least that the unit can no longer access the old ("outdated") copy of the memory block after a processing unit accesses a copy of a memory block and subsequently accesses an updated copy of the memory block.

[0004] A cache memory coherence protocol typically defines a set of coherence states stored in conjunction with cached copies of memory blocks, as well as the events that trigger transitions between coherence states and the coherence states to which transitions are made. Coherence protocols can generally be classified as directory-based or watchdog-based protocols. In directory-based coherence protocols, a common central directory maintains coherence by controlling access to memory blocks through the cache memories and updating or invalidating copies of the memory blocks held in the various cache memories.In contrast, monitoring-based coherence protocols implement a distributed design paradigm in which each cache maintains a private directory of its contents, monitors the system connection for memory access requests targeting memory blocks held in the cache ("snoops"), and responds to the memory access requests by updating its private directory and, if necessary, by transmitting coherence message(s) and / or its copy of the memory block.

[0005] The cache memory states of the coherence protocol can, for example, include those of the well-known MESI protocol (Modified, Exclusive, Shared, Invalid) or a variant thereof. The MESI protocol allows a cache memory data row to be assigned to one of four states: "M" (modified), "E" (exclusive), "S" (shared), or "I" (invalid). The Modified state indicates that a memory block is valid only in the cache memory that contains the Modified memory block, and that the memory block is not consistent with system memory. The Exclusive state indicates that the assigned memory block is consistent with system memory, and that the assigned cache memory is the only cache memory in the data processing system that contains the assigned memory block.The Shared state indicates that the associated memory block is resident in its associated cache and possibly in one or more other caches, and that all copies of the memory block are consistent with system memory. Finally, the Invalid state indicates that both the data and the address appendage associated with a coherence element are invalid.

[0006] In surveillance-based coherence protocols, it is common for cache stores to respond to a request that is snooped on the connection by providing an individual coherence response. These individual coherence responses are then combined or otherwise processed to determine a final system-wide coherence response for the request. This response might indicate, for example, whether the request will be successful or needs to be retried, whether a data source is responsible for feeding a target cache data row to the requesting cache identified in the request, the coherence state of the target cache row across one or more subsequent caches, and so on.In a conventional data processing system using a monitoring-based coherence protocol, the minimum transfer interval in which a cache memory data row is obtained (intervened in) from one cache memory in a vertical cache memory hierarchy supporting one processor core to another cache memory in a different vertical cache memory hierarchy supporting a different processor core via the system link is the time between issuing a request from one cache memory and receiving the system-wide coherence response by that cache memory. SUMMARY

[0007] According to one embodiment, the minimum transfer interval is reduced in which a cache memory data row can be obtained from one cache memory in a vertical cache memory hierarchy to a cache memory in another vertical cache memory hierarchy via the system connection.

[0008] In at least one embodiment, a multiprocessor data processing system includes multiple vertical cache memory hierarchies supporting multiple processor cores, system memory, and a system connection. In response to a load and reserve request from a first processor core, a first cache memory supporting the first processor core issues a memory access request on the system connection for a target cache memory row of the load and reserve request. In response to the memory access request, and prior to receiving a system-wide coherence response for the memory access request, the first cache memory receives the target cache memory row and an early indication of the system-wide coherence response for the memory access request from a second cache memory in a second vertical cache memory hierarchy via cache-to-cache intervention.In response to the early specification and before receiving the system-wide coherence response, the first cache initiates processing to update the target cache line in the first cache. List of characters

[0009] Embodiments of the invention are now described only by way of example with reference to the accompanying drawings, in which: Fig. 1 is a representation of a relevant section of a processing unit according to an embodiment; Fig. 2 a representation of a relevant section of an exemplary data processing system according to an embodiment; Fig. 3 is a time-space representation of an exemplary operation that includes a request phase, a partial response phase (Presp), and a linked response phase (Cresp), which is implemented in the data processing system of Fig. 2 are implemented; Fig.4. A time-space representation of an exemplary operation in the data processing system of Fig. 2 is. Fig. 5. A more detailed block representation of an L2 cache of Fig. 1 according to one embodiment; Fig. 6 is a flowchart of a conventional process by which a read claim (RC) control routine of a lower-level cache memory serves a memory access request of a connected processor core via a link operation; Fig. 7 is a time-lapse representation of a conventional link operation in which, prior to receiving the associated response for the link operation request, the lower-level cache memory receives a target cache memory row specified by the request through a cache-to-cache intervention; Fig.8. An exemplary data acquisition on the connection structure of the data processing system of Fig. 2 illustrates which, according to one embodiment, includes an early indication of the linked response of a linking operation; Fig. 9 is an overview flowchart of an exemplary process by which a read request (RC) control routine of a lower-level cache memory serves a memory access request of a connected processor core via a link operation; Fig. 10. A timeline of an exemplary link operation is where, prior to receiving the associated response for the link operation request, the lower-level cache memory receives a target cache memory row specified by the request through a cache-to-cache intervention; Fig.11 illustrates an example of the use of Load and Reserve instructions and Conditional Save instructions to synchronize updates to a cache memory row in a multithreaded data processing system; the Fig. 12 to Fig. 13 jointly form an overview flowchart of an exemplary process by which a read request (RC) control routine of a lower-level cache memory serves requests to load and reserve and requests to conditionally store a connected processor core according to an embodiment; Fig. 14 a timeline of an operational scenario under the process of Fig. 12 to Fig. 13 is where the target cache memory line of a request is received as a load and reserve before the associated linked response, which in turn receives conditional memory before the processing of the request is completed; Fig.15 a timeline of an operational scenario under the process of Fig. 12 to Fig. 13 is where the target cache memory line of a request is received under Load and Reserve before the associated linked response, which in turn is received under Conditional Store after the request has finished processing; Fig. 16 an alternative embodiment of the process of Fig. Figure 13 illustrates how a lower-level cache assigns a different RC control routine to serve a conditional save request on a connected processor core, rather than using the same RC control routine that handled the previous load and reserve request; Fig.17 a logical overview flowchart of a process by which an RC control routine of a lower-level cache memory performs a request-conditional storage of a connected processor core in the embodiment of Fig. 16 served; Fig. 18 a time-lapse representation of an operational scenario under the process of Fig. 12 and Fig. 16 is where the target cache memory line of a request is received as a load and reserve before the associated linked response; and Fig. Figure 19 illustrates an exemplary design process according to one embodiment. DETAILED DESCRIPTION

[0010] With reference to the characters and in particular with reference to Fig. Figure 1 is a block diagram illustrating an exemplary embodiment of a processing unit. 100An embodiment of a multi-processor data processing system is illustrated. In the illustrated embodiment, the processing unit is... 100 to a single integrated circuit that has two processor cores 102a , 102b for the independent processing of instructions and data. (Of course, in other embodiments the number of processor cores may vary.) 102 (vary.) Each processor core 102 contains an instruction sequencing unit (ISU) 104 for retrieving and organizing instructions for execution and one or more execution units 106 to execute instructions. For example, the execution units can 106 They contain one or more floating-point units (FPUs), one or more load storage units (LSUs), and one or more integer units (IUs). The execution units 106The executed instructions may include, for example, fixed-point and floating-point arithmetic instructions, logical instructions, and instructions that require read and / or write access to data in a memory block.

[0011] The operation of each processor core 102a , 102b is supported by a multi-level memory hierarchy, which at its lowest level has one or more shared system memories. 132 (of those in Fig. (only one is shown) and has a vertical cache hierarchy with one or more cache levels at its upper levels. As shown, the processing unit contains 100 an integrated memory controller (IMC) 124 , which allows read and write access to a system memory 132 in response to operations that occur on a connection structure (described below) of monitoring devices (snoopers) 126to be tracked down (snooped).

[0012] In the illustrative embodiment, the vertical cache memory hierarchy of the processing unit contains 100 a cache memory 108 the Level 1 (LI) memory in each processor core 102a , 102b and a cache memory 110 the level- 2 (L2), which is accessed by all processor cores 102a , 102b the processing unit 100 shared. (In other designs, each processor core can 102(have its own private L2 cache memory 110.) Although the illustrated cache memory hierarchy contains only two levels of cache memory, the person skilled in the art will recognize that alternative embodiments may contain additional levels (e.g. level three (L3), level four (L4), etc.) of an in-chip or in-chip external in-line or look-aside cache memory that contains the contents of the upper levels of cache memory completely, partially, or not at all.

[0013] What's next in Fig. Figure 1 shows the processing unit. 100 an integrated connection logic 120 , through which the processing unit 100 It can be connected to the connection structure of a larger multi-processor data processing system. In the illustrated embodiment, the connection logic supports 120 any number t1of first-tier connection lines, which in this case contain incoming and outgoing lines 'X', 'Y', and 'Z'. The connection logic 120 furthermore supports an arbitrary number t2 of second layer lines, which are in Fig. 4 are designated as incoming and outgoing lines 'A' and 'B'. Each processing unit can be connected to these first and second layer lines. 100 for bidirectional data exchange with up to t1 / 2 + 12 / 2 (in this case five) other processing units 100 To be connected. The logic of connection. 120 contains a requirements logic (denoted by 'R') 121a , a logic partial answer (denoted by 'P') 121b , a logically linked answer (labeled 'C') 121c and a data logic (labeled 'D') 121dfor processing and forwarding information during various phases of operations on the connection. The connection logic also includes... 120 a configuration register (labeled 'CR') 123, which contains a plurality of mode bits used to configure the processing unit 100 These mode bits preferably contain: ( 1 ) a first set of one or more mode bits that select a desired line information mapping for the lines of the first and second layers; ( 2 ) a second set of mode bits that specify which of the lines of the first and second layers of the processing unit 100 with other processing units 100 are connected; and ( 3 ) a third set of mode bits that define a programmable duration of a protection window extension.

[0014] Each processing unit 100It also contains an instance of the response logic. 122 , which implements part of a distributed monitoring-based coherence signaling mechanism that maintains cache coherence between the cache hierarchy of the processing unit 100 and those of other processing units 100 maintains. Ultimately, each processing unit contains 100 an integrated I / O (input / output) controller 128 , which supports the connection of one or more I / O units, such as the I / O unit 130. The I / O controller 128 In response to requests, the I / O unit can output 130 operations and receive data on links 'X', 'Y', 'Z', A', and 'B'.

[0015] In Fig. Figure 2 is a block diagram of an exemplary embodiment of a data processing system. 200 with multiple processing units 100as shown in the present invention. As shown, the data processing system includes 200 eight processing nodes 202a0 until 202d0 and 202a1 until 202d1 , each of which can be implemented as a multi-chip module (MCM) comprising a package that includes four processing units 100 It contains the processing units. 100 in each processing node 202 are connected for point-to-point data exchange via the 'X', 'Y', and 'Z' lines of the processing units as shown. Each processing unit 100 can also be used with processing units 100 in two different processing nodes 202 for point-to-point data exchange via links 'A' and 'B' of the processing units. Although they are in Fig.2 is represented by a double arrow, however it should be clear that each pair of links 'X', 'Y', 'Z', 'A' and 'B' is preferably (but not necessarily) implemented as two unidirectional lines instead of one bidirectional line.

[0016] General terms for forming the in Fig. The topologies shown in the two examples can be specified as follows: Node[I][K].Chip[J].Line[K] is connected to Node[J][K].Chip[I].Line[K] for all I ≠ J; and Node[I][K].Chip[I]Line[K] is connected to Node[I][not K].Chip[I].Line[not K ]; and Node[I][K].Chip[I].Line[not K ] is connected to either: (1) no connection (is reserved for future expansions); or (2) Node[ extra ][ not K ].Chip[ I ].Line[ K ], in the case that all lines are fully utilized (i.e. nine 8-way nodes form a 72-way system); and where I and J belong to the set {a, b, c, d} and K belongs to the set {0, 1}.

[0017] Of course, alternative expressions can be defined to form other functionally equivalent topologies. Furthermore, it should be clear that the topology presented is representative, but not exhaustive, for the topologies of data processing systems in which the present invention is implemented, and that other topologies are possible. In such alternative topologies, for example, the number of lines in the first and second layers, which are connected to each processing unit, may differ. 100 are connected to handle any number, and the number of processing nodes 202 The number of processing units in each layer (i.e., I) does not have to be equal to the number of processing units. 100 per processing node 100 (i.e., J) be.

[0018] The expert will recognize that the SMP data processing system100 The illustration may contain many additional components not shown, e.g., connecting bridges, non-volatile storage devices, connectors for connecting to networks or associated units, etc. Since such additional components are not necessary for understanding the present invention, they are omitted in the illustration. Fig. 2 are not shown or are not explained further here.

[0019] In Fig. Figure 3 is a time-space representation of an exemplary connection operation on the connection structure of the data processing system. 200 from Fig. 2 shown. The connection operation begins when a master 300 (e.g. a read claim (RC) control routine) 512 one L2 -Cache storage 110 (see e.g. Fig. 5) or a master in an I / O controller 128) a request 302 outputs on the connection structure. The requirement 302It preferably contains at least one transaction type that specifies the type of desired access and a resource identifier (e.g., a real-world target address) that specifies a resource to be accessed by the request. Common types of requests include those listed below in Table I. TABLE I Requirement Description READ Requests a copy of the image of a memory block for query purposes. RWITM (Read with the intention to change) Requests a unique copy of the image of a memory block with the intention of updating (modifying) it, and may require the destruction of other copies. DCLAIM (Data Claim) Requests permission to replace an existing query-only copy of the memory block with a unique copy with the intent to update (modify) it, and may require the destruction of other copies. DCBZ (Data Cache Block Zero) Requests permission to create a new, unique copy of a memory block without regard to its current state and subsequently modify its contents; may require the destruction of other copies. CASTOUT Copies the image of a memory block from a higher memory level to a lower memory level in preparation for destroying the copy at the higher level. WRITE Requests permission to create a new unique copy of a memory block without regard to its current state and to immediately copy the image of the memory block from a higher-level memory to a lower-level memory in preparation for destroying the higher-level copy. PARTIAL WRITE Requests the permission to create a new unique copy of a sub-memory block without regard to its current state and to immediately copy the image of the sub-memory block from a higher-level memory to a lower-level memory, in order to destroy the higher-level copy. to prepare

[0020] Further details regarding these operations and an exemplary protocol of cache memory coherence, which facilitates efficient handling of these operations, may be found in U.S. Patent No. 7,774,555, which is incorporated herein by reference in its entirety for all purposes.

[0021] The requirement 302 is monitored by the monitoring facilities 304 (e.g. monitoring control routines) 511 the L2 -Cache memory 110 (see e.g. Fig.5) and the monitoring units 126 the EVICs 124 ) received, which are distributed across the entire data processing system 200 are distributed. In general, with some exceptions, surveillance units detect 304 in the same L2 -Cache memory 110 like the master 300 the requirement 302 the requirement 302 not (i.e., there is generally no self-monitoring), since a requirement 302 The data is only transmitted on the connection structure if the request 302 not internally by a processing unit 100 can be operated. Monitoring units 304 , which meet the requirements 302 each receiving and processing a corresponding partial response (Presp) 306 ready to provide the response of at least this monitoring unit 304 upon request 302 represents a monitoring unit. 126 in an EVIC 124determines the partial answer 306 , which is provided, for example, based on whether the monitoring unit 126 who is responsible for the request address and whether they have the resources to handle the request. L2 -Cache memory 110 can his partial answer 306 for example, based on the availability of a monitoring and control routine 511 to address the requirement, the availability of its L2 -Cache directory 508 (see e.g. Fig. 5) and the associated coherence state, where the actual target address is found in the directory 508 the L2 -Cache storage.

[0022] The partial answers 306 the monitoring units 304 are processed either stepwise or all at once by one or more instances of the response logic 122logically linked to produce a system-wide linked response (Cresp) 310 upon request 302 to determine. In one embodiment, it is assumed below that the instance of the response logic is 122 , which are responsible for creating the cresp 310 is responsible in the processing unit 100 is located, which is the Master 300 contains the requirement 302 has issued. The response logic 122 Cresp presents 310 the Master 300 and monitoring units 304 via the connection structure to provide the system-wide coherence response (e.g., success, failure, retry, etc.) to the request 302 to specify. If Cresp 310 Success of the requirement 302 Cresp indicates 310 for example, a data source for a target storage block of the request 302 , a coherence state in which the requested memory block is provided by the master300 (or other cache stores) to be cached, and whether "cleanup" operations are required that would invalidate the requested memory block in one or more cache stores.

[0023] In response to receiving Cresp 310 lead one or more of Master 300 and monitoring units 304 typically one or more additional actions are required to fulfill the request. 302 to operate. These additional actions can deliver data to the master. 300 , invalidating or otherwise updating the coherence state of data contained in one or more L2 -Cache storage 110, executing castout operations, writing data back to system memory 132 etc. If the requirement 302 If required, a requested or target memory block can be created before or after the creation of a cresp. 310through the answer logic 122 to or from the master 300 be transferred.

[0024] The following description contains the partial answer 306 a monitoring unit 304 upon a request 302 and the actions taken by the monitoring unit 304 in response to the request 302 and / or their linked answer 310The operation is described with reference to whether this monitoring unit is a highest point of coherence (HPC), a lowest point of coherence (LPC), or neither with respect to the request (destination) address specified by the request. An LPC is defined here as a storage unit or an I / O unit that serves as a repository for a memory block. In the absence of an HPC for the memory block, the LPC contains the true image of the memory block and is authorized to grant or deny requests to create an additional cached copy of the memory block. For a typical request in the embodiment of the data processing system of Fig. 1 and Fig. 2. The LPC is the memory controller. 124 for system memory 132, which contains the referenced memory block. An HPC is defined here as a uniquely identified entity that caches a true image of the memory block (which may be consistent with the corresponding memory block at the LPC) and is authorized to grant or deny a request to modify the memory block. Descriptively, the HPC can also provide a copy of the memory block to a requester in response to an operation that does not modify the memory block. Therefore, for a typical request in the implementation of the data processing system, it may be Fig. 1 and Fig. 2. At the HPC, possibly by one L2-Cache memory 110. While other indicators can be used to designate an HPC for a memory block, in a preferred embodiment of the present invention, the HPC optionally designates a memory block in which one or more selected cache memory coherence states are found in the cache memory directory of a L2 -Cache memory can be used.

[0025] Still referring to Fig. 3 is the HPC, if applicable, for a memory block that is referenced in a request. 302 is referred to, or in the absence of an HPC, the LPC of the memory block is preferably responsible for transferring the coherence ownership of a memory block in response to a request. 302 to protect if necessary. In the exemplary scenario described in Fig. As shown in section 3, a monitoring unit protects 304nin the case of HPC (or, in the absence of HPC, LPC) for the memory block that is defined by the request address of the request 302 specified is the transfer of coherence ownership of the requested (target) memory block to the master. 300 during a protective window 312a , which extends from the time when the monitoring device 304n their partial answer 306 determined until the monitoring device 304n the cresp 310 receives and during a subsequent window expansion 312b , which occur over a programmable period of time after receiving the Cresp 310 through the monitoring device 304n extends. During the protective window 312a and the window extension 312b protects the monitoring device 304n the transfer of coherence ownership of the target memory block from the monitoring device 304n to Master's 300by providing partial answers 306 (e.g., repeated partial responses) to other requests that specify the same request address. Such partial responses 306 Prevent other masters from acquiring coherence ownership of the target memory block until coherence ownership is successfully obtained from the monitoring device. 304n to the Master 300 was transferred. If necessary, the master can 300 after receiving the linked response 310 also a protective window 313 to initiate the process of capturing coherence ownership of the target memory block. The protection window 313 ensures that each subsequent master requesting the target memory block receives a new value for the target memory block provided by the master 300 is generated instead of receiving an outdated value.

[0026] Since all monitoring devices 304Given limited resources to handle the CPU and I / O demands described above, several different levels of partial responses and corresponding creps are possible. For example, if a monitoring device 126 in a memory controller 124 , which is responsible for a requested memory block, has a queue available to process a request, the monitoring device 126 respond with a partial answer indicating that it is capable of serving as an LPC for the request. If, on the other hand, the monitoring device 126 If there is no queue available to handle the request, the monitoring facility may 126 respond with a partial answer indicating that it is the LPC for the memory block, but is currently unable to serve the request. Similarly, a L2-Cache memory 110, an available monitoring control routine 511 and access to the directory 508 the L2 -Cache memory is requested to handle a request. Failure to access one (or both) of these resources results in a partial response (and the corresponding cresp), signaling an inability to serve the request due to the absence of a required resource.

[0027] What's next in Fig. The monitoring device can be shown in section 3. 304n Data (for example, for a READ or RWITM request) to the master 300 (e.g. a L2 -Return cache memory (110) before or after the master 300 the cresp (for the READ or RWITM request) from the response logic 122 receives.

[0028] In Fig. Figure 4 is a time-space representation of an exemplary operational sequence in the data processing system. 200 from Fig. Figure 2 shows the different processing units. 100 in the data processing system 200 equipped with two location identifiers - a first one that identifies the processing node 202 identifies to which the processing unit 100 belongs, and a second one, which is the specific processing unit 100 in the processing node 202 identifies. Thus, for example, the processing unit 100a0c refers to the processing unit 100c of the processing node 202a0 Additionally, each processing unit 100 provided with a functional identifier that defines its function relative to the other processing units 100 indicates those involved in the operation. These functional identifiers include: ( 1 ) a local master (LM) that controls the processing unit 100 designated which triggers the operation, ( 2) a local hub (LH) which is a processing unit 100 designated as those located in the same processing node 202 how the local master is located and which is responsible for transferring the operation to another processing node 202 responsible (a local master can also be a local hub), ( 3 ) a distant hub (RH) that includes a processing unit 100 designated as those located in a different processing node 202 as the local master and the one responsible for forwarding the operation to other processing units 100 in its processing node 202 to distribute, and ( 4 ) a remote leaf node (RL) that contains a processing unit 100 designated as those located in a processing node different from the local master. 202 is located and is not a distant hub.

[0029] As in Fig.As shown in section 4, the exemplary operation has at least three phases as above, with reference to Fig. As described in section 3, the operation consists of a request (or address) phase, a partial response (presp) phase, and a linked response (cresp) phase. These three phases preferably occur in the order listed above and do not overlap. The operation may additionally include a data phase, which may overlap with any of the request, partial response, and linked response phases.

[0030] Still referring to Fig. 4. The request phase begins when a local master 100a0c (i.e., the processing unit) is present. 100c of the processing node 202a0 ) a synchronized broadcast of a request, for example a read request, to each of the local hubs 100a0a , 100a0b , 100a0c and 100a0d in its processing node 202a0carries out this process. It should be noted that the list of local hubs includes the local hub. 100a0c This includes the local master. This internal transfer can be advantageously used to optimize the operation of the local hub. 100a0c with the local hubs 100a0a , 100a0b and 100a0d to synchronize so that the flow control restrictions can be met more easily.

[0031] In response to receiving the request, each local hub transmits 100 , which communicates via its lines 'A' or 'B' with a distant hub 100 is connected to the operation at its distant hub(s). 100 Thus, the local hub leads 100a0a The operation is not transmitted on its output line 'A', but is transmitted via its output line 'B' to a remote hub in the processing node 202al. The local hubs 100a0b , 100a0c and 100a0dThey transmit the operation via their respective output lines 'A' and 'B' to remote hubs in the processing nodes. 202b0 and 202bl, processing node 202c0 and 202cl or processing nodes 202d0 and 202dl. Every distant hub 100 The node that receives the operation, in turn sends the operation to each distant terminal node. 100 in its processing node 202 For example, the far node transmits 100b0a the operation on distant terminal nodes 100b0b , 100b0c and 100b0d In this way, the operation is efficiently distributed to all processing units. 100 in the data processing system 200 sent, using no more than three lines for transmission.

[0032] After the request phase, the partial response (presp) phase occurs. In the partial response phase, each distant terminal node evaluates the response. 100the operation and presents its partial response to the operation to its respective distant hub. 100 ready. For example, the distant terminal nodes transmit 100b0b , 100b0c and 100b0d their respective partial answers to the remote hub 100b0a . Every remote hub 100 In turn, it sends these partial answers as well as its own partial answer to each of the local hubs. 100a0a , 100a0b , 100a0c and 100a0d Local Hubs I00a0a, 100a0b , 100a0c and 100a0d They then send these partial answers, as well as their own partial answers, to each local hub. 100 in the processing node 202a0 It should be noted that the distribution of partial responses by the local hubs 100 in the processing node 202a0 For timing reasons, the self-transmission of its own partial response by each local hub 100 contains.

[0033] It is understood that the collection of partial responses as shown can be implemented in several different ways. For example, an individual partial response can be propagated from any other local hub, remote hub, and remote end node back to any local hub. Alternatively, for greater efficiency, it may be desirable to accumulate partial responses as they are propagated back to the local hubs. This ensures that the effect of each partial response is precisely reflected in the local hubs. 100 If the data is transferred back, it is preferable that the partial responses be accumulated non-destructively, for example using a logical OR function and an encoding in which no relevant information is lost when subjected to such a function (e.g., a "one-hot" encoding).

[0034] As further in Fig. Figure 4 shows how the response logic is compiled. 122 in each local hub100 in the processing node 202a0 the partial responses of the other processing units 100 , in order to obtain a linked response that represents the system-wide coherent response to the request. The local hubs 100a0a until 100a0d Then send the linked response to all processing units. 100 , which follow the same distribution paths used for the request phase. Thus, the linked response is first sent to remote hubs. 100 sent, which in turn sends the linked response to each distant terminal node 100 in its respective processing node 202 transferred. For example, the local hub transfers 100a0b the linked response to the remote hub 100b0a , which in turn sends the linked response to the distant terminal nodes 100b0b , 100b0c and 100b0d transfers.

[0035] As mentioned above, executing the operation may require an additional data phase. For example, if the operation is a read-type operation, such as a READ or RWITM operation, the remote end node I00b0d can load the requested memory block to the local master I00a0c via the lines connecting the remote end node I00b0d to the remote hub I00b0a, the remote hub I00b0a to the local hub I00a0b, and the local hub I00a0b to the local master. 100a0c connect. If, on the other hand, the operation is a write operation, such as a cache castout operation, where a modified block of memory is written back to system memory. 132 of the distant terminal node 100b0b When writing, the memory block is transferred via the lines that connect to the local master. 100a0c with the local hub 100a0b , the local hub 100a0b with the remote hub 100b0a and the distant hub 100b0a with the distant terminal node 100b0b connect.

[0036] Of course, that is in Fig. The scenario shown is merely an example of the multitude of possible operations that can be performed simultaneously in a multi-processor data processing system, such as the data processing system. 200 can occur.

[0037] As above with reference to Fig. As described in section 3, the coherence during the "handoff" of the coherence ownership of a memory block is monitored by a monitoring device. 304n to a requesting master 300 maintained in the possible presence of other masters who pass through the protective window 312a , the window extension 312b and the protective window 313 to compete for ownership of the same memory block. For example, the protective window must 312a and the window extension 312btogether have a sufficient duration to transfer coherence ownership of the requested memory block to a winning master (WM) 300 to protect in the presence of a competing requirement by a competing master (CM). To ensure that the protection window 312a and the window extension 312b have sufficient duration to transfer ownership of the requested memory block to the winning master 300 To protect against this is the latency of data exchange between the processing units. 100 according to Fig. 4 preferably restricted in such a way that the following conditions are met: A_lat ( CM_S ) < A_lat ( CM_VVM ) + C_lat ( VVM_S ) + ε , where A lat(CM S) is the address latency of a competing master (CM) to the monitoring device (S) 304nis, which possesses the coherence of the requested memory block, A lat(CM WM) is the address latency of a competing master (CM) to the "winning" master (WM) 300 , to which the coherence ownership is transferred by the monitoring body 304n C_lat(WM_S) is the latency of the linked response from the time the linked response is awarded by the winning master (WM). 300 is received until the time at which the linked response is received from the monitoring device (S) 304n is received, which has the requested memory block, and ε is the duration of the window extension. 312b .

[0038] If the preceding time constraint, applicable to any system of any topology, is not met, the request from the competing master can be received ( 1 ) by the winning master 300 , before the winning master 300takes over coherence ownership and the protective window 312b triggers, and ( 2 ) by the monitoring device 304n after the end of protective windows 312a and window extension 312b In such cases, neither the winning master delivers 300 and the monitoring device 304n a partial response to the concurrent request that prevents the concurrent master from taking coherence ownership of the memory block and reading incoherent data from memory. To avoid this coherence error, the window extension can be used. 312b However, it is programmable (e.g., by appropriately setting the configuration register (CR)). 123The window extension can be set to any desired length (ε) to compensate for latency variations or the drawbacks of a physical implementation where the time constraint might otherwise not be met, but which must be met to maintain coherence. Thus, by solving the above equation for ε, the ideal length of the window extension can be determined. 312b to be determined for each implementation.

[0039] Several observations can be made regarding the aforementioned time limitation. First, the address latency from the competing master to the owning monitoring facility must be considered. 304a There is no necessary lower limit, but there must be an upper limit. The upper limit is determined to ascertain the achievable worst-case latency, which is influenced by factors such as the maximum possible oscillator drift, the longest transmission lines, and the processing units. 100The connection must be configured with a maximum number of accumulated blocks and a guaranteed worst-case throughput. To ensure that the upper limit is respected, the connection structure must exhibit non-blocking behavior.

[0040] Secondly, the address latency from the competing master to the winning master 300 There is no necessary upper limit, but there must be a lower limit. The lower limit is determined by the achievable latency in the best possible case, which is achieved, among other things, by the absence of blocking and the shortest possible path between the processing units. 100 and the slowest oscillator drift is given for a given static configuration. Indeed, for a given operation, each winning master has 300and competing master only imposes a time limit on its respective requirement; however, it should be noted that in the operational process, each processing unit 100 It can be a winning master for some operations and a competing (and losing) master for other operations. Consequently, each processing unit has 100 effectively an upper limit and a lower limit for their address latency.

[0041] Thirdly, the latency of the linked response, from the time the linked response is generated until the time the linked response is received by the winning master, is 300 It is recognized that there is no necessary lower limit (the linked answer can be applied to the winning master). 300(arriving at any early time), but must have an upper limit. In contrast, the latency of the linked response is the time from when a linked response is generated until the linked response arrives at the monitoring device. 304n The number of operations received has a lower limit, but no necessary upper limit (although this can be arbitrarily imposed to limit the number of operations that can be performed simultaneously).

[0042] Fourth, there is no limitation on partial response latency. That is, since all terms of the time constraint listed above relate to the request / address latency and the latency of the linked response, the partial response latencies of the monitoring devices are not affected. 304 and the competing master's to the winning master's 300 no necessary upper or lower limits.

[0043] The links of the first and second layers, the processing units 100 Connecting elements can be implemented in various ways to preserve the topology that is in Fig. 2 is shown, and time constraints must be observed. In a preferred embodiment, each incoming and outgoing line of the first layer ('X', 'Y' and 'Z') and each incoming and outgoing line of the second layer ('A' and 'B') is implemented as a unidirectional 8-byte bus containing a number of different virtual channels or objects to transmit address, data, control and coherence information.

[0044] Now, referring to Fig. Figure 5 is a more detailed block representation of an example. L2 -Cache storage 110 as shown in one embodiment. As in Fig. 5 shown, contains the L2 -Cache memory 110 a cache storage arrangement 502 and a directory 508of the contents of the cache storage arrangement 502 Although not explicitly shown, the cache storage arrangement is... 502 preferably implemented with a single read port and a single write port to implement the cache memory arrangement 502 to reduce the required chip area.

[0045] Assuming that the cache storage arrangement 502 and the directory 508 Typically sentence-associative, storage locations in the system memory 132 specific congruence classes in the cache storage arrangement 502 mapped using predefined index bits in the (real) system memory addresses. The specific memory blocks that are located in the cache memory rows of the cache memory array. 502 They are stored in the cache directory. 508recorded, which contains a directory entry for each cache memory line. Although not explicitly in Fig. As shown in Figure 5, it is clear to an expert that every directory entry is stored in the cache directory. 508 contains various fields, for example a marker field that identifies the actual address of the memory block that is located in the corresponding cache memory row of the cache memory array. 502 is held, a state field that specifies the coherence state of the cache memory row, and a last used (LRU) field that specifies a replacement order for the cache memory row with respect to other cache memory rows in the same congruence class.

[0046] The L2 -Cache memory 110 contains several (e.g., 16 or 32) read claim (RC) control routines 512a until 512nfor independently and simultaneously handling load (LD) and store (ST) requests from the associated processor core 102 be received. L2 -Cache memory 110 It also contains several (e.g., 16 or 32) monitoring control routines 51Ia to 51Im to handle remote memory access requests from other processor cores. 102 as the associated processor core 102 to operate. Every monitoring and control routine 511 can independently and simultaneously handle a remote storage access request that is independent of the local connection. 514 is "detected". It goes without saying that handling memory access requests by the L2 -Cache memory 110 Replacing or invalidating memory blocks in the cache memory arrangements 502 may require it. Accordingly, the L2 -Cache memory 110 CO (castout) control routines 510a until510n , which involves removing and writing memory blocks from the cache memory array 502 administer.

[0047] The L2 -Cache memory 110 It also includes an RC queue. 520 and a CPI (Cast Push Intervention) queue 518 , each of which buffers data that is stored in the cache memory arrangement 502 inserted and removed from the RC queue. 520 contains a number of buffer entries, each individually assigned to a specific RC control routine. 512 correspond so that each RC control routine 512 The queue being processed only retrieves data from the designated buffer entry. Similarly, the CPI queue contains... 518 a number of buffer entries, each corresponding to one of the specific castout control routines 510 and monitoring and control routines 511correspond so that each CO control routine 510 and every monitoring device 511 , which are processed, only retrieve data from the respective designated CPI buffer entry.

[0048] Each RC control routine 512 is also one of several RC data (RCDAT) buffers 522 allocated to buffering a memory block that is part of the cache memory arrangement 502 is read and / or from the local connection 514 via the reload bus 523 is received. The RCDAT buffer 522 , which is assigned to each RC control routine512, is preferably constructed with connections and functionality corresponding to the memory access requirements of the associated RC control routine. 512 They can be operated. At least some of the RCDAT buffers. 522 have an associated storage data multiplexer M4, the data bytes at its inputs for buffering in the RCDAT buffer 522 in response to unspecified selection signals provided by the allocator (arbiter) 505 be generated.

[0049] The L2 -Cache memory 110 also includes a dispenser 505 , which is used to control the multiplexers M1 until M2 , to process local memory access requests from the connected processor core 102 can be received, and is set up for remote requests that rely on the local connection 514 Memory access requests are monitored. These include local load and save operations and remote read and write operations, and are processed in accordance with the allocation guidelines. 505 implemented decision policy (arbitration policy) to a shipping pipeline 506forwarded, where each request involves reading / loading and saving, taking the directory into account. 508 and the cache storage arrangement 502 is processed over a given number of cycles.

[0050] The L2 -Cache memory 110 Additionally, it offers support for atomic updates by the associated processor core(s). 102 by implementing the reservation logic 513 , which tracks reservations established by atomic load requests (e.g., load-and-reserve (LARX) requests) to ensure that corresponding atomic save requests (e.g., conditional save (STCX)) are executed successfully only if the reserved cache memory rows have not been modified since the reservations were established (i.e., if the reservation in question is still valid). In a typical embodiment, the reservation logic includes 513For each thread, a respective reservation address register that specifies the base address of a cache memory row for which a reservation has been set up, and a reservation marker that indicates whether the reservation for the specified cache memory row is still valid.

[0051] During operation, processor memory requests, which have a transaction type (ttype), real destination address and memory data, are handled by the connected processor core. 102 in a storage queue (STQ) 504 received by STQ 504 The storage data will be sent to the data multiplexer M4 via the data path 524 The memory type and destination address are transmitted to the multiplexer. M1 handed over. The multiplexer M1 It also receives processor load requests from the processor core as input. 102 and directory write requests from RC control routines 512In response to unsigned selection signals from the allocation system 505 The multiplexer selects which are generated. M1 one of his entered requests to be forwarded to the multiplexer M2 from which, in addition, a remote request is received as input from the local connection 514 via the remote request method 526 is received. The distributor 505 It plans local and remote storage access requests for processing and generates a sequence of selection signals based on the planning. 528 In response to the allocation 505 generated selection signals 528 The multiplexer selects M2 either those from the multiplexer M1 received local request or the one on the local connection 514 identified the remote request as the next memory access request to be processed.

[0052] A requirement that must be processed by the arbiter 505 selected by the multiplexer M2 in the shipping pipeline 506 placed. The shipping pipeline 506 is preferably implemented as a fixed-duration pipeline in which each of several possible overlapping requests A, B, C, etc., is processed in a predetermined number of clock cycles. For example, the shipping pipeline could be... 506 Process each request in four cycles.

[0053] During an initial processing cycle in the shipping pipeline 506 A one-cycle directory read operation is performed using the request address to determine if the request address is in the directory. 508 whether it occurs or is missing, and if the memory address occurs, the coherence state of the memory block in the directory 508to determine. The directory information, which includes a hit / miss display and the coherence state of the memory block, is determined by the directory. 508 in a subsequent cycle, for example in the fourth cycle, to the shipping pipeline 506 returned. As can be seen, this is generally done in a L2 -Cache memory 110 No action is taken in response to a failed remote storage access request; such remote storage requests are accordingly removed from the shipping pipeline. 506 discarded. In the event of a hit or miss on a local memory access request or a hit on a remote memory access request, the L2 -Cache memory 110 to handle the memory access request that occurs when requests are not fully processed by the processing unit. 100 can be operated, data exchange on the local connection514 via the structure controller 516 contains.

[0054] At a predetermined point during the processing of the storage access request in the shipping pipeline 506 The distributor sends 505 the request address via the address and control path 530 regarding the cache storage arrangement 502 to trigger a cache read operation of the memory block specified by the request address. In the exemplary embodiment, a cache read operation takes 2 Cycles. The one from the cache memory arrangement 502 The read memory block is accessed via the data path 542 to the error correction code (ECC) logic 544 The data transmitted checks the memory block for errors and corrects any detected errors. For processor load requests, the memory block also acts as a load data multiplexer. M3 via the data path 540to forward to the corresponding processor core 102 transmitted.

[0055] In the final cycle of processing a memory access request in the shipping pipeline 506 establishes the shipping pipeline 506 A shipping investigation is initiated. The shipping pipeline 506 For example, the shipping determination can be based on a number of criteria, including ( 1 ) the presence of an address collision between the request address and a previous request address that is currently being used by a castout control routine 510 , monitoring control routine 511 or RC control routine 512 is processed, ( 2 ) the directory information and ( 3 ) the availability of an RC control routine 512 (for a local request of the associated processor core) 102 ) or a monitoring and control routine 511(for a monitored request from a remote processor core) to process the memory access request. If the shipping pipeline 506 When a dispatch order is executed that the memory access request is to be dispatched, the memory access request is sent from the dispatch pipeline. 506 to an RC control routine 512 or, if applicable, a monitoring and control routine 511 sent. If the memory access request fails, the error is reported to the requester (e.g., the local or remote processor core). 102 ) is signaled by a retry response. The requester can then retry the failed memory access request if necessary.

[0056] During an RC control routine 512 The RC control routine is located when processing a local memory access request. 512in an occupied state and is unavailable to handle another request. While an RC control routine is running... 512 If the RC control routine is in an occupied state, it cannot be used. 512 Perform a directory write operation to update the relevant directory entry. 508 to update if necessary. Additionally, the RC control routine can 512 Perform a cache write operation to the relevant cache row of the cache array. 502 to update. A directory write operation and a cache write operation can be performed by the allocator. 505 be scheduled during an interval in which the shipping pipeline 506 It has not already processed other requests according to the fixed schedule of directory and cache write operations. When all operations for the given request have finished, the RC control routine returns.512 returns to an unoccupied state.

[0057] It is understood that the scheduling of non-scheduled operations such as directory writes and cache writes can affect the scheduling of other operations, including those processed according to a fixed schedule.

[0058] In Fig. Figure 6 shows a logical overview flowchart of a conventional process, in which an RC control routine is used. 512 one L2 -Cache storage 110 a memory access request from an associated processor core 102 via a connection operation. To promote a better understanding, additional information is provided on Fig. 7. Referenced, this is a time-lapse representation showing a specific operating scenario in which the L2 -Cache memory 110Before receiving the linked response for the connection operation request, a target cache memory line specified by the request is received via cache-to-cache intervention.

[0059] The process of Fig. 6 begins in block 600 in response to receiving a memory access request from a processor core 102 in the shipping pipeline 506 its associated L2 -Cache storage 110 The process then proceeds to the block. 602 , which illustrates that the shipping pipeline 506 an unused RC control routine 512 assigns to handle the request. In response to the assignment of the RC control routine. 512 The RC control routine is used to handle the memory access request. 512 into an occupied state (the occupied state of the RC control routine) 512 is at the reference mark 700 from Fig. 7 shown).

[0060] Assuming that the state of coherence, as defined by the directory 508 of the Master's L2 -Cache storage 110 The response returned indicates that the memory access request cannot be served without the RC control routine being executed. 512 When a copy of the target cache line is retrieved (as would be the case, for example, if a cache miss occurs), the process in the block is initiated. 602 assigned RC control routine 512 a connection operation by issuing a suitable request for the target cache memory line (e.g., READ or RWITM) on the connection structure (block 604 ).

[0061] Outputting the requirement for the connection structure is possible at the reference number. 702 from Fig. Figure 7 illustrates this. Depending on the implemented connection topology, the request can be received (and is likely to be received) at various different times by monitoring devices across the entire data processing system. 200 are distributed. The receipt of the request by one of the monitoring devices, which will serve as the data source for the target cache line, is specifically indicated at the reference sign. 704 in Fig. 7. In response to receiving the request, the monitoring device (in this example, a monitoring control routine) is activated. 511 one L2 -Cache storage 110 , which is the UPC), assume an occupied state (where the occupied state of the monitoring device is at the reference sign 706(as shown). While the monitoring device is in the occupied state, the monitoring device performs processing necessary to serve the request, as indicated by the reference numeral. 708 is specified. In this case, this processing includes providing the target cache memory line specified by the request to the master. L2 -Cache memory 110 through cache-to-cache intervention before receiving the linked response 720 by the monitoring device. After receiving the linked response 720 The monitoring device remains in an occupied state (thus protecting the master from retaining coherence ownership of the target cache row) for the duration of the window extension. 312b , as with the reference symbol 722 shown.

[0062] In Fig. 6 is monitored by the master- L2 -Cache memory 110after issuing the request in the block 604 both returning the requested data (e.g. from the monitoring system) L2 -Cache memory 110 , which is the HPC) as well as receiving the linked response (cresp) of the operation (blocks) 606 until 608 ). In response to the fact that the L2 -Cache memory 110 in the block 608 If it is determined that the linked response was received before the requested data, the process proceeds from Fig. 6 to the block 610 and the following blocks. For clarity, this timeline scenario is in Fig. 7 not explicitly shown. In response to the fact that the L2 -Cache memory 110 in the block 606 However, if the process determines that the requested data was received before the associated response of the operation, it proceeds to blocking. 630 and the following blocks, which are referred to below with additional reference to Fig. 7 will be described.

[0063] First in the block 610 and the following blocks are determined by the RC control routine 512 , whether the linked answer, which is used for the read operation in the block 608 The response received is "good", which means that the linked response indicates that the requested target cache memory row was sent to the requesting party. L2 -Cache memory 110 is delivered (block 610 In response to a finding in the block 610 If the linked answer is not the combined answer "good", the process returns to the block. 604 back, which indicates that the RC control routine 512 The request will be re-issued on the connection structure. This is in response to the RC control routine. 512 in the block 610 However, if the combined answer is determined to be "good", the process proceeds from the block. 610 to the block 612 .

[0064] block612 illustrates that the RC control routine 512 possibly a protective window 313 opens to transfer coherence ownership of the target cache row from the monitoring device to the requesting device. L2 -Cache memory 110 to protect. The process then iterates in the block. 614 , until the target cache memory data line is in the buffer in RCQ 520 is received, which is the RC control routine 512 This corresponds to the response to receiving the target cache data line in the RCQ. 520 placed L2 -Cache memory 110 the requested data in the RCDAT buffer 522 , which is the RC control routine 512 corresponds to (block 616 Additionally, the RC control routine performs 512 in the block 618 additional processing to handle the memory access request of the associated processor core 102to operate, for example by initiating the transfer of the requested data from the RCDAT 522 to the associated processor core 102 by issuing a cache write request to the shipping pipeline 506 , which involves transferring the target cache memory line from the buffer in the RCQ 520 to the cache storage arrangement 502 requests, and / or a directory write request that requests an update of the coherence state of the target cache memory line that is accessed through the directory 508 is specified. At the end of the RC control routine 512 The RC protective window will be installed after processing. 313 closed (finished) and the RC control routine 512 is released, causing the RC control routine to return to an unoccupied state (block 620 The proceedings then end. Fig. 6 in the block 622 , until the RC control routine 512is instructed to handle another memory access request.

[0065] In Fig. 6 will now be the processing that occurs in response to receiving requested data by the L2 -Cache memory 110 before the linked response is performed, with reference to block 630 and the following blocks are described. In response to receiving the target cache line in the RCQ 520 (as with the reference symbol) 710 from Fig. 7 shown) placed the L2 -Cache memory 110 the requested data in the RCDAT buffer 522 , which is the RC control routine 512 corresponds to (block 630 The RC control routine 512 Then monitors the receipt of the linked response (see e.g. Cresp). 712 from Fig. 7) for the requirement, as in the block 632 shown. As in Fig. As stated in point 7, in some connection topologies and / or operating scenarios, the interval between issuing the request and receiving the associated response may be significantly longer (e.g., three times longer) than the interval between issuing the request and receiving the target cache memory line in the RCDAT buffer. 522 The difference between the durations of these intervals, represented in Fig. 7 at the reference mark 724 , represents a time period in which the RC control routine 512 , which handles the memory access request, does not perform any useful work.

[0066] When finally in the block 632 from Fig. 6. It is determined that the linked response to the request has been received (see e.g. Cresp). 712 from Fig. 7) determines the RC control routine 512 , whether the linked answer is a combined answer “good” (Block 634In response to a finding in the block 634 The process reverses when the linked answer is not the "good" linked answer. Fig. 6 to the block 604 and the following blocks that were described. In response to a finding in the block 634 If the linked answer is a "good" linked answer, the process goes to the block. 636 Before. Block 636 shows that the RC control routine 512 possibly a protective window 716 opens to transfer the coherence ownership of the target cache row to the requesting L2 -Cache memory 110 to protect. The process then proceeds with the blocks. 618 until 622 further, which were described. As in Fig. 7 at the reference mark 718 specified after the RC protective window 716 closed and the RC control routine 512 will be released (as in the block 620(shown), is the Master- L2 -Cache memory 110 , which has received the target cache row, is able to serve as a data source for the target cache memory row if needed in response to a request from a subsequent master.

[0067] As noted above, in a conventional multiprocessor data processing system, a master requesting a target cache row does not begin processing that row until it receives a linked response confirming its acquisition of coherence ownership. This occurs even in operational scenarios where the master receives the target cache row long before the coherence message confirming ownership, resulting in a significant period of unproductive time. 724 results in an RC control routine 512a memory access request is actively served. However, the present application recognizes that this unproductive period can be reduced or eliminated in operating scenarios where the system-wide coherence response to a master request can be known during data provisioning (and thus the latency with which a given cache memory row can be obtained by successive masters can be reduced). The present application also recognizes that while the system-wide coherence response cannot be determined a priori in all operating scenarios, it can be known before the associated response is received by a monitoring device, which is an HPC that holds a target cache memory row in a modified state (e.g.,the MESI state “M”), because this monitoring device (which holds the only unique copy of the target cache line) is responsible for granting or denying requests for the target cache line and is thus decisive for the associated response. Accordingly, in preferred embodiments of the present invention, an HPC snooper, if capable of doing so, provides an early indication of the associated response to a requesting master in conjunction with the target cache line.

[0068] In Fig. 8 is an example data block (data tenure) 800 on the connection structure of the data processing system of Fig. Figure 2 illustrates how a data status field can be used to provide a requesting master with an early indication of the system-wide coherence response for a request. As shown, the example data block contains 800at least two fields, namely one data field 804 and a data status (Dstat) field 802 , which can be transmitted in one or more clock cycles on the system connection. The data field 804 can transmit one or more cache memory data lines (including a target cache memory line) from a monitoring device to a requesting master. The data status field 802 , which in conjunction with the data field 804 The information transmitted can be used by the monitoring device, which serves as the data source of the target cache row, to provide a potentially early indication of whether the requesting master is satisfied with a linked response to its request for the target cache row. In some embodiments, the indication may include an indication of a coherence state.

[0069] In Fig. Figure 9 shows a logical overview flowchart of an exemplary process with which an RC control routine can be implemented. 512 one L2 -Cache storage 110 a memory access request from an associated processor core 102 via a connection operation. For better understanding, reference is also made to Fig. 10, which is an exemplary timeline showing a specific operational scenario in which the L2 -Cache memory 110 Before receiving the linked response for the connection operation request, a cache-to-cache intervention receives a target cache row specified by the request.

[0070] The process of Fig. 9 begins in block 900 in response to receiving a memory access request from a processor core 102 in the shipping pipeline 506 its associated L2-Cache storage 110 The process then continues to the block. 902 , which illustrates that the shipping pipeline 506 To handle the request, an unassigned RC control routine is required. 512 assigns. In response to the assignment of the RC control routine. 512 The RC control routine is used to handle the memory access request. 512 into an occupied state (the occupied state of the RC control routine) 512 , as with the reference figure 1000 from Fig. 10 shown).

[0071] Assuming that the state of coherence, as defined by the directory 508 of the Master's L2 -Cache storage 110 The response returned indicates that the memory access request cannot be served without the RC control routine being executed. 512When a copy of the target cache line is retrieved (as would be the case, for example, if a cache miss occurs), the RC control routine is initiated. 512 , which are in the block 902 was assigned a connection operation by issuing a suitable request for the target cache memory line (e.g., READ or RWITM) on the connection structure (block 904 ).

[0072] Outputting the request on the connection structure is at the reference number. 1002 from Fig. Figure 10 illustrates this. Depending on the implemented connection topology, the request can be received (and is likely to be received) at various different times by monitoring devices across the entire data processing system. 200 are distributed. The receipt of the request by one of the monitoring devices, which serves as the data source for the target cache line, is specifically indicated at the reference sign.1004 in Fig. 10 is specified. In response to receiving the request, the monitoring device (in this example, a monitoring control routine) takes 511 one L2 -Cache storage 110 , which is capable of resolving the system-wide coherence response in response to the request (e.g., a UPC- L2 -Cache memory 110 in a suitable state of coherence)) an occupancy state (where the occupancy state of the monitoring device is indicated at the reference sign 1006 from Fig. (as shown in Figure 10). While the monitoring device is in the occupied state, it performs processing necessary to serve the request, as indicated by the reference numeral. 1008 specified. In this case, this processing involves providing the target cache memory line specified by the request to the master. L2 -Cache memory 110by cache-to-cache intervention before the monitoring device at the reference point receives the linked response 1020 After receiving the linked response, the monitoring device remains in a busy state (thus protecting the master from maintaining the coherence of the target cache line) for the duration of the window extension. 312b as at the reference mark 1022 shown.

[0073] In Fig. 9 is monitored by the L2 -Cache memory 110 after issuing the request in the block 904 simultaneously returning the requested data (e.g., from the monitoring system) L2 -Cache memory 110 , which is the HPC) and receiving the linked response (cresp) of the operation (blocks) 906 until 908 ). In response to the fact that the L2 -Cache memory 110 in the block 908If it is determined that the linked response was received before the requested data, the process proceeds from Fig. 9 to the block 910 and the following blocks. For clarity, this timeline scenario is in Fig. 10 not explicitly shown. In response to the fact that the L2 -Cache memory 110 in the block 906 However, if the process determines that the requested data was received before the associated response of the operation, it proceeds to blocking. 930 and the following blocks, which are referred to below with additional reference to Fig. 10 will be described.

[0074] First in the block 910 and in the following blocks the RC control routine determines 512 , whether the linked answer, which is for the operation Read in block 908 The response received is "good", which means that the linked response indicates that the requested target cache memory row was sent to the requesting party. L2-Cache memory 110 is delivered (block 910 In response to a finding in the block 910 If the linked answer is not the combined answer "good", the process returns to the block. 904 back, which indicates that the RC control routine 512 The request is re-outputted on the connection structure. This is in response to the RC control routine. 512 in the block 910 However, if the combined answer is determined to be "good", the process moves from the block. 910 to the block 912 .

[0075] block 612 illustrates that the RC control routine 512 may have a protective window 313 opens to transfer the coherence ownership of the target cache row from the monitoring device to the requesting device. L2 -Cache memory 110 to protect. The process then iterates in the block. 914, until the target cache memory data line is in the buffer in RCQ 520 is received, which is the RC control routine 512 This corresponds to the response to receiving the target cache memory data line in the RCQ. 520 placed L2 -Cache memory 110 the requested data in the RCDAT buffer 522 , which is the RC control routine 512 corresponds to (block 916 Furthermore, the RC control routine performs 512 in the block 918 additional processing to handle the memory access request of the associated processor core 102 to operate, for example by forwarding the requested data from the RCDAT 522 to the processor core 102 , by issuing a cache write request, transferring the target cache row from the buffer in RCQ 520 to the cache storage arrangement 502 demands to the shipping pipeline 506and / or by issuing a directory write request that requests an update of the coherence state of the target cache line traversed by the directory 508 is specified, to the shipping pipeline 506 At the end of the RC control routine 512 The RC protective window will be installed after processing. 313 closed (finished), and the RC control routine 512 is released, causing the RC control routine to return to an unoccupied state (block 920 The proceedings then end. Fig. 9 in the block 922 , until the RC control routine 512 is assigned to serve another memory access request.

[0076] Continue in Fig. 9 will now process the block. 930 and the following blocks described that in response to receiving requested data by a L2 -Cache memory 110This is executed before the linked response is processed. It occurs in response to receiving a data block. 800 in the RCQ 520 , which contains the target cache memory line (as indicated by the reference character 1010 from Fig. 10 shown), placed the L2 -Cache memory 110 the target cache memory line of the requested data in the RCDAT buffer 522 , which is the RC control routine 512 corresponds to (block 930 ). In the block 932 The RC control routine determines 512 Furthermore, whether the data status field 802 of the data block 800 provides an early indication of a linked response "good", which allows the RC control routine to immediately process the target cache line. 512 This enables it. If this is not the case, the process exits the block. 932 to the block 934 and the following blocks. If the data status field 802However, an early indication of a linked response "good" provides immediate processing of the target cache line by the RC control routine. 512 This enables the process to proceed from the block 932 to the block 940 and the following blocks.

[0077] In the block 934 and the following blocks are monitored by the RC control routine 512 Receiving the linked response for the request (block) 934 In response to a finding in the block 934 from Fig. 9. The RC control routine determines that the linked response to the request has been received. 512 , whether the linked answer is a linked answer “good” (Block 936 In response to a finding in the block 936 , that the linked answer is not a linked answer "good", the process reverses from Fig. 9 to the block 904and back to the following blocks that were described. In response to a finding in the block 936 However, if the linked answer is deemed to be a "good" linked answer, the process proceeds to blocking. 938 . Block 938 shows that the RC control routine 512 A protective window may open to prevent the transfer of coherence ownership of the target cache row to the requesting server. L2 -Cache memory 110 to protect. The process then moves to the blocks. 918 until 922 , which were described.

[0078] When referring to block 940 and the following blocks (which represent the operational scenario that is in Fig. (as shown in section 10) the process reacts to the fact that the data status field 802 of the data block 800provides an early indication that the linked answer is a good linked answer, to the block 940 . Block 940 shows that the RC control routine 512 a protective window 1014 opens to transfer the coherence ownership of the target cache row from the monitoring L2 -Cache memory 110 to the requesting party L2 -Cache memory 110 to protect. In Fig. 10. It should be noted that the opening and closing of the protective window 1014 asynchronously to receiving the linked response to the request at the reference point 1018 This was done. As on the block 942 from Fig. 9 and at the reference mark 1012 from Fig. As shown in 10, the RC control routine is executed. 512 Furthermore, additional processing is performed to handle the memory access request of the associated processor core. 102to operate, for example by retrieving the requested data from the RCDAT buffer 522 to the processor core 102 returned by a cache write request that involves transferring the target cache row from the buffer in RCQ 520 to the cache storage arrangement 502 demands to the shipping pipeline 506 is issued and / or a directory write request that updates the coherence state of the directory 508 The specified target cache line requests the shipment pipeline. 506 is issued. It should be different from Fig. 7. Note that the presence of the linked response in the data block 800 enables the RC control routine 512Between receiving the target cache line and receiving the associated response, it performs useful work (and possibly all of its processing). At the end of the processing, which is carried out by the RC control routine... 513 When executed, the RC control routine closes. 512 their protective window 1014 (Block 944 ), which means that the master- L2 -Cache memory 110 no longer provides repeat partial responses to other competing masters requesting the target cache line, and that the master- L2 -Cache memory 110 thus able to serve as a data source for the target cache row, if necessary, in response to a request from a subsequent master (as indicated by the reference sign). 1016(specified). The coherence protocol is preferably designed to favor partial responses from cache memories that have received write ownership of a target cache memory row relative to those from previous monitoring devices, thus enabling the master- L2 -Cache memory 110 can serve as a data source even in the presence of one or more prior monitoring devices that continue to provide repeat partial responses while in the occupied state, as indicated by the reference sign 1006 depicted. It should be noted that the time period 1016 , in which the Master- L2 -Cache memory 110 can serve as a data source before (and in many cases long before) receiving the linked response 1018 through the Master's L2 -Cache memory 110can begin. Therefore, the minimum handover time at which a cache memory row can be obtained between vertical cache memory hierarchies is no longer determined by the interval between request and associated response as in Fig. 7 is defined, but instead defined by the interval between request and data as in Fig. 10 shown.

[0079] After the block 944 The RC control routine is waiting 512 upon receiving the linked response to their request, as in the block 946 from Fig. 9 and at the reference number 1018 from Fig. Figure 10 shows that in response to the linked response, the marker assigned to the link operation is cleared, and the RC control routine... 512 is released, thereby activating the RC control routine 512 returns to a state unoccupied (block) 948), from which it can be reassigned. The process then ends. Fig. 9 in the block 922 , until the RC control routine 512 is allocated to serve another memory access request.

[0080] As described, a multiprocessor data processing system in at least one embodiment includes multiple vertical cache hierarchies supporting multiple processor cores, a system memory, and a system connection linked to the system memory and the multiple vertical cache hierarchies. A first cache in a first vertical cache hierarchy issues a request for a target cache row on the system connection. In response to the request, and before receiving a system-wide coherence response for the request, the first cache receives the target cache row and an early indication of the system-wide coherence response for the request from a second cache in a second vertical cache hierarchy via cache-to-cache intervention.In response to the early notification of the system-wide coherence response and prior to receiving the system-wide coherence response, the first cache initiates processing to install the target cache line into the first cache. In one embodiment, the first cache places the target cache line into a third cache in a third vertical cache hierarchy prior to receiving the system-wide combined response.

[0081] Furthermore, it is clear from the present application that transmitting an early indication of the system-wide coherence response in conjunction with an intervened cache memory line also facilitates a fast transfer of coherence ownership of the intervened cache memory line in the case where multiple threads make atomic updates to the intervened cache memory line. For example, the exemplary multithreaded program of Fig. 11 considers, for example, three simultaneously executed hardware threads Ta, Tb and Tc from three different processor cores. 102 of the multiprocessor data processing system 200 is executed. As shown, each of the threads Ta, Tb, and Tc contains, in addition to other instructions generally indicated by ellipses, an atomic update sequence. 1100 The atomic update sequence 1100 In each of the threads Ta, Tb and Tc, a corresponding command is used to load and reserve (e.g. LARX). 1102a , 1102b , 1102c , which, when executed, generates a corresponding request LARX to the distributed shared storage system, which places a target cache memory row, mapped to effective address A, into the register r1 of the executing processor core 102 loads. The atomic update sequence 1000Each thread then contains one or more commands that change the value of the value in the register. r1 privately modify the held target cache line, with this instruction(s) being implemented in the example atomic update sequence by an ADD instruction. 1104a , 1104b , 1104c are shown, which represents the value of the register r1 incremented by 1. (The ADD command) 1104a , 1104b , 1104c This is followed by a conditional save command (e.g., STCX). 1106a , 1106b , 1106c , which replaces the target cache memory line assigned to address A with the modified value of the register r1 conditionally updated depending on whether any time has passed since the execution of the previous LARX command. 1102 An intermediate update was performed. In this example, the executing processor core sets 102 a field of a condition code register in the processor core 102return to zero when the STCX command is executed. 1106 Successfully updates the target cache memory line; otherwise, sets the condition code register field to a non-zero value. The example atomic update sequence 1000 Conditional branching in each thread ends with a specific command. 1108a , 1108b , 1108a (e.g. Branch Not Equal (BNE)), which is based on the value of the field in the condition code register in the processor core. 102 Conditionally branched. In particular, the BNE command causes 1108a , 1108b , 1108c , that during execution there is a return in the loop to the Load and Reserve command (e.g. LARX) 1102a , 1102b , 1102c is executed if the atomic update is unsuccessful, and continues with subsequent commands in the relevant thread if the atomic update is successful.

[0082] As further in Fig. 11 by arrows 1110 and 1112 The value specified by the STCX instruction is... 1106a The thread Ta is stored at the memory location in the distributed community memory corresponding to the effective address A, by the LARX instruction. 1102b the thread Tb loaded, and the value that is entered by the STCX command 1106b The value stored at this memory location of the distributed shared memory for thread Tb is accessed by the LARX command. 1102c of the thread Tc. Thus, it can be seen that the performance of the exemplary multithreaded program, which is in Fig. Figure 11 shows that the latency between saving an updated value to a target cache memory line in the distributed shared memory (by executing an STCX instruction) is improved. 1106) and by loading the updated value of the target cache memory line from the distributed shared memory (by executing a LARX instruction) according to the teachings of the present application.

[0083] In the Fig. 12 to Fig. Figure 13 shows a logical overview flowchart of an exemplary process with which an RC control routine can be implemented. 512 a lower-level cache (e.g., a L2 -Cache memory 110 ) the requirements of Load and Reserve (LARX) and Conditional Memory (STCX) of an associated processor core 102 according to one embodiment. In this embodiment, a single RC control routine is used. 512 allocated to serve both a load and reserve request and the subsequent conditional save request, thus combining the service of both requests into a single resource.

[0084] As in Fig. As shown by the same reference symbols in 12, this is the processing of an RC control routine. 512 , which is generated in response to receiving a request LARX (e.g., by executing a LARX command). 1102 ) is executed, essentially similarly to the one executed in response to other memory access requests, as described above with reference to Fig. 9 described. Accordingly, the steps described in Fig. 12 are executed and correspond to those previously described with reference to Fig. The processes described in section 9 will not be described again in detail here. However, it should be noted that the processing of the LARX request by an RC control routine 512 in the block 942 from Fig. 12. Setting up a reservation for the target address of the request LARX in the reservation logic 513 This may include... After the LARX request has been processed by an RC control routine...512 in the block 942 will the process of Fig. 12 via side connector A to Fig. 13 led

[0085] As in Fig. As shown in 13, the process branches and runs in parallel to a first path that leads to the block 1300 contains, and a second path that contains the blocks 1310 until 1318 It contains. These two paths merge and return to the block via side connector B. 948 from Fig. 12 returns only after the processing shown in both paths is complete.

[0086] First, the RC control routine waits. 512 in the block 1300 upon receiving the linked response (Cresp) of the request, which is based on the connection structure of the data processing system. 200 in the block 904 from Fig. 12 is output. In response to receiving the linked response, the first path of the process goes from Fig. 13 from the block1300 to the point where it intersects with the second path.

[0087] In this exemplary embodiment, where a requirement LARX and an associated requirement STCX are included in a single RC control routine 512 When merged, the RC control routine remains. 512 Occupied for a timeout interval while waiting to receive a matching STCX request specifying the same destination address as the LARX request to which it was assigned. Fig. Figure 13 illustrates this behavior in the blocks 1310 until 1312 , which represent a determination of whether a matching STCX request has been received from the associated processor core 102 is received (block 1310 ), before the timeout interval expires (block 1312 If the timeout interval expires before the shipping pipeline 506STCX receives a matching request, it is placed in the block 1312 a confirmatory investigation was carried out, and the second path, which leads into Fig. As shown in section 13, it leads directly from the block 1312 to the block 1318 . Block 1318 illustrates that the RC control routine 512 Its protective window for the target address of the cache memory line reserved by the LARX request closes. The process then proceeds to the junction point with the first path from the block. 1300 It's coming. In response to an investigation within the block. 1310 that the RC control routine 512 However, if a matching STCX request is received before the timeout interval expires, the process exits the block. 1310 to the block 1314 , which illustrates that the shipping pipeline 506 the STCX requirement in the RC control routine 512brings together those that already meet the matching requirement LARX.

[0088] The RC control routine 512 The STCX request is then served, for example by issuing a cache write request that updates the target cache row in the cache array. 502 demands to the shipping pipeline 506 and / or by issuing a directory write request that requests an update of the coherence state of the target cache line being accessed by the directory 508 is specified, to the shipping pipeline 506 and / or by resetting the reservation for the target cache row in the reservation logic 513 (Block 1316 The process then proceeds to the block. 1318 , which illustrates that the RC control routine 512Its protective window for the target cache line closes. The process then proceeds along the second path to the union point and returns to the block via side connector B. 948 from Fig. 12, as described. Thus, in the illustrated embodiment, the RC protective window closes in response to the RC control routine. 512 Its processing of a request STCX has ended, or in response to the fact that no request STCX is received within the timeout interval, and the RC control routine remains occupied until the subsequent receipt of the associated response for the link operation of the request LARX and the closing of the RC protection window.

[0089] In the Fig. 14 to Fig. 15 are time-lapse diagrams of exemplary operational scenarios under the process of Fig. 12 to Fig. 13 shown. In Fig. 14. An early indication of the linked response of a memory access request and a target cache memory row of a Load and Reserve (LARX) request are received before the associated linked response. The linked response, in turn, is received before processing a matching Conditional Store (STCX) request (and in some cases, before receiving the STCX request). Fig. 15. The timing of receiving the linked response differs from that in Fig. The time sequence shown in Figure 14 indicates that the linked response of the connection operation is received after processing a matching request STCX.

[0090] The Fig. 14 to Fig. 15 illustrate that the RC control routine 512 in response to the assignment of an RC control routine 512 to operate a request LARX at the reference sign 1402an occupancy state 1400 It accepts. In response to the LARX request, the RC control routine returns... 512 on the connection structure of the data processing system 200 a memory access request for the target cache memory line of the request LARX, as indicated by the reference character 1406 the Fig. 14 to Fig. 15 and in block 904 from Fig. 12 shown.

[0091] This storage access request is monitored by monitoring devices throughout the entire data processing system. 200 receive, including an operating monitoring device (e.g., a monitoring control routine). 511 one L2 -Cache storage 110 ), whose receipt of the memory access request in the Fig. 14 to Fig. Figure 15 illustrates this. In response to receiving the memory access request via the connection structure, the operating monitoring device outputs a data block.800 with a data field 804 to the requesting party L2 -Cache memory 110 back, which shows the target cache line and the data status field 802 contains an early indication of a linked response "good", which allows immediate processing of the target cache memory line by the RC control routine512 (as indicated by the reference sign 1410 from Fig. 14). (The operating monitoring device also performs further processing, which is shown in Fig. 10 is shown, but in the Fig. 14 to Fig. (15 is omitted to avoid obscuring other aspects of the process according to the invention.)

[0092] In response to receiving the early indication of a linked answer, good at their L2 -Cache memory 110 , opens the RC control routine 512 , which meets the LARX requirement, an RC protective window 1412(as in the block 942 from Fig. 12 shown). During the RC protective window 1412 represents the monitoring and control routine 511 the L2 -Cache storage 110 Retry partial responses are provided for all conflicting memory access requests that are monitored on the connection structure and target the reserved cache memory row. This occurs within the RC protection window. 1412 executes the RC control routine 512 their processing of the LARX request, as with the reference numeral 1420 the Fig. 14 to Fig. 15 and in block 942 from Fig. Shown in 12. After processing the LARX request, the RC control routine returns. 512 does not return to an unoccupied state, but instead remains in an occupied state during a subsequent wait window STCX 1422 , during which the RC control routine 512STCX is waiting for a matching request to be sent, as shown in the block. 1310 from Fig. Figure 13 shows that a request STCX specifying a destination address matching that of the preceding request LARX is received within the timeout interval (as shown at the reference sign). 1404 (as shown), the STCX request is processed in the RC control routine. 512 merged, and the RC control routine 512 Executes the processing to satisfy the STCX request, as indicated by the reference numeral. 1424 the Fig. 14 to Fig. 15 and in the blocks 1314 until 1316 from Fig. 13 shown.

[0093] The Fig. 14 to Fig. 15 illustrated that the RC control routine 512 until the linked reply is received later 1414 for the RC control routine 512Memory access request initiated on the connection structure and closing the RC protection window 1412 occupied state 1400 remains. Fig. 14 to Fig. 15 further illustrate that the RC protective window 1412 , during which the RC control routine 512 the coherence ownership of the reserved cache memory row through its L2 -Cache memory 110 protects, in response to the conclusion that the RC control routine 512 Their processing of the STCX request to update the target cache line has finished. Updating the target cache line in response to the STCX request is independent of when the associated response is received, and since the STCX request typically results in a hit in the L2 -Cache memory 110This leads to the target cache memory line being updated frequently before the associated response for the memory access request is received (as in Fig. 15 shown). The L2 -Cache memory 110 The entity that updated the target cache line can therefore serve as the source of the target cache line once the RC protection window is closed. 1412 is closed, which, as in Fig. 15 shown, well before receiving the linked reply 1414 the RC control routine 512 can be initiated by a memory access request.

[0094] In Fig. 16 is an alternative embodiment of the process of Fig. 13 shown. In this alternative embodiment, a L2 -Cache memory 110 another of his RC control routines 512 to, in order to meet a matching STCX request of the associated processor core 102to operate, instead of the STCX request in the same RC control routine 512 to integrate, which serves the preceding requirement LARX. As indicated by identical reference numerals, the procedure is of Fig. 16 essentially similar to the one that was in Fig. 13 is executed. Consequently, the steps that are in Fig. 16 shall be executed, which correspond to those previously executed with reference to Fig. The 13 items described are not described in detail here.

[0095] In the block 1310 from Fig. 16. The process responds to receiving a suitable STCX request from the associated processor core. 102 through a L2 -Cache memory 110 , while an RC control routine 512 (here referred to as RC1) remains occupied in the state initiated by the previous request LARX, from the block 1310 to the block 1600 . Block 1600illustrates how to determine if an RC control routine 512 the L2 -Cache storage 110 The item is in a state of being unassigned (unavailable). If this is not the case, the shipping pipeline discards it. 506 the block 1310 Received request STCX and sends a retry instruction to the (not shown) processor core. 102 back. The process of Fig. 16 then returns to the block 1310 back, as described. In response to a confirming finding in the block. 1600 indicates the shipping pipeline 506 the L2 -Cache storage 110 However, another RC control routine 512 (referred to here as RC2) to serve the STCX request, as shown in the block 1602 shown. The process then proceeds from Fig. 16 to the block 1318 , which was described.

[0096] In Fig. Figure 17 shows a logical overview flowchart of a process through which an RC control routine is executed. 512 (i.e. RC2) of L2 -Cache storage 110 a requirement conditional memory (STCX) of an associated processor core 102 in the embodiment of Fig. 16 served. The process of Fig. 17, for example, begins in block 1700 in response to the fact that the shipping pipeline 506 one L2 -Cache storage 110 the RC control routine RC2 assigns to fulfill a request STCX in the block 1602 from Fig. 16 to operate. The process then proceeds to the block. 1702 , which illustrates that the RC control routine RC2 a protective window 1014 opens to determine the coherence ownership of the target cache row through its L2 -Cache memory 110 to protect. Additionally, the RC control routine operates RC2 The STCX request, for example, is made by issuing a cache write request that updates the target cache row in the cache array. 502 demands to the shipping pipeline 506 and / or by issuing a directory write request that updates the coherence state of the directory 508 The specified target cache line requests the shipment pipeline. 506 and / or, among other things, by resetting the reservation for the target cache row in the reservation logic 513 (Block 1704 In at least some embodiments, the directory write request updates the coherence state to a coherence state that increases the probability that the target cache memory line is from the L2 -Cache memory 110 through a cache-to-cache intervention into another L2 -Cache memory 110enters a state of coherence that allows a storage operation to be terminated immediately. After the block 1704 The process goes to the block 1706 , which illustrates that the RC control routine RC2 Its protective window for the target cache line closes. After that, the process ends. Fig. 17 in the block 1708 .

[0097] In Fig. Figure 18 is a time-lapse representation of an exemplary operating scenario according to the [document / guidelines / etc.]. Fig. 12, Fig. 16 and Fig. The processes shown in Figure 17 illustrate how an early indication of the linked response of a memory access request is received in conjunction with the target cache memory row of a Load and Reserve (LARX) request prior to the associated linked response.

[0098] Fig. Figure 18 illustrates that the RC control routine RC1 in response to the assignment of an RC control routine RC1 to handle a request LARX at the reference sign 1802 an occupancy state 1800 occupies. In response to the LARX request, the RC control routine outputs RC1 a memory access request for the target cache memory line of the request LARX on the connection structure of the data processing system 200 as in the reference mark 1806 from Fig. 18 and in block 904 from Fig. 12 shown.

[0099] This storage access request is monitored by monitoring devices throughout the entire data processing system. 200 received, including an operating monitoring device (e.g., a monitoring control routine) 511 one L2 -Cache storage 110 ), where receiving the memory access request in Fig. 18 at the reference mark 1808This is shown. In response to receiving the memory access request via the connection structure, the operating monitoring device sends a notification to the requesting device. L2 -Cache memory 110 a data block 800 back, which is a data field 804 contains the target cache memory line and the data status field 802 contains an early indication of a linked "good" response, which allows the RC control routine to immediately process the target cache line. 512 enabled (as with the reference mark) 1810 from Fig. 18). (The operating monitoring device also performs further processing, which is shown in Fig. 10 is shown, but in Fig. (18 is omitted to avoid obscuring other aspects of the process according to the invention.)

[0100] In response to receiving the early indication of a linked answer, good at his L2-Cache memory 110 opens the RC control routine RC1 a RC1 -Protective window 1812 (as in block 940 from Fig. 12 shown), during the monitoring control routines 511 the L2 -Cache storage 110 Provide retry partial responses for conflicting memory access requests that target the reserved cache memory row and are monitored on the system connection. RC1 -protective window 1812 remains for a RC1 -LARX processing interval 1820 , during which the RC1 their processing of the LARX request, and one subsequent one RC1 -STCX waiting interval 1822 open, during which the L2 -Cache memory 110 Receiving an STCX request from the associated processor core 102It expects a target address that matches the one specified in the previous request, LARX. Upon completion of the RC1 -STCX waiting interval 1822 does that RC1 -protective window 1812 , thereby protecting the coherence ownership of the target cache row by the RC1 is completed. However, the RC1 It does not necessarily immediately return to an unoccupied state, but instead remains in an occupied state until the linked response is received later. 1814 for their storage access request and termination of the RC1 -STCX waiting interval 1822 .

[0101] In response to receiving the STCX request from the associated processor core 102 through the L2 Cache memory 110 (as with the reference mark 1804 (shown), indicates the shipping pipeline 506 another RC control routine RC2 to handle the STCX request (as in the block 1602 (shown). In response to assigning the RC control routine RC2 To handle the STCX request, the RC2 a RC2 -protective window 1830 , during which the RC2 the coherence ownership of the target cache row by its L2 -Cache memory 110 protects, as also in the block 1702 from Fig. 17 shown. During the RC2 -protective window 1830 leads the RC2 their processing of the STCX request to update the target cache line, as indicated by the reference character 1832 shown. RC2 closes her RC2 -protective window 1830 in response to processing for the STCX request as in the block 1706 from Fig. 17 shown.

[0102] It should be noted that the L2 -Cache memory 110, which updates the target cache row, can serve as the data source of the target cache row (e.g., through cache-to-cache intervention on another L2 -Cache memory 110 ), as soon as the RC2 -protective window 1832 is closed, which is before receiving the linked response. 1814 the RC control routine RC1 This can occur upon an initiated memory access request. It should also be noted that updating the target cache memory line is done by the RC control routine. RC2 in response to the STCX request, also independent of the time the associated response is received, and because the STCX request normally results in a hit in L2 -Cache memory 110 This can lead to the target cache memory line being updated before the associated response for the memory access request is received.

[0103] Fig. Figure 19 shows a block diagram of an exemplary development process. 1900 , which is used, for example, in the design of a semiconductor IC logic circuit, its simulation, testing, layout, and manufacturing. The development process 1900 Includes processes, machines, and / or mechanisms for processing assembly structures or units to generate logically or otherwise functionally equivalent representations of the assembly structures and / or units described above and shown here. The assembly structures generated by the development process 1900Machines that are processed and / or generated can be encoded on machine-readable transmission or storage media so that they contain data and / or instructions that, when executed or otherwise processed in a data processing system, produce a logically, structurally, mechanically, or otherwise functionally equivalent representation of hardware components, circuits, units, or systems. Machines include, but are not limited to, all machines used in an IC design process, such as the design, fabrication, or simulation of a circuit, component, unit, or system. Examples of machines include: lithography machines, machines and / or units for generating masks (e.g.,Electron beam writing equipment), computers or equipment for simulating design structures, any devices used in the manufacturing or testing process, or any machines for programming functionally equivalent representations of the design structures in any medium (e.g., a machine for programming a programmable gate array).

[0104] The development process 1900 This can vary depending on the type of representation being designed. For example, a development process might... 1900 to create an application-specific integrated circuit (ASIC) from a development process 1900 to design a standard component or a development process 1900to instantiate the design into a programmable array, for example a programmable gate array (PGA) or a field programmable gate array (FPGA), which are offered by Altera® Inc. or Xilinx® Inc.

[0105] Fig. Figure 19 illustrates several such organizational structures, including an input organizational structure. 1020 , preferably through a development process 1910 is processed. Regarding the structural design 1920 It could be a structural framework for logical simulation, which is implemented through the development process. 1910 It is generated and processed to create a logically equivalent functional representation of a hardware unit. The development structure 1920 It may also or alternatively contain data and / or program commands that are processed during the development process. 1910Create a functional representation of the physical structure of a hardware unit. Regardless of whether functional and / or structural design features are represented, the development structure can 1920 generated using electronic computer-aided design (ECAD), which is implemented, for example, by a core developer / designer. If the development structure 1920 encoded in a machine-readable data transmission, a gate array, or a storage medium, it can be implemented by one or more hardware and / or software modules during the development process. 1910 They can be retrieved and processed to simulate or otherwise functionally represent an electronic component, circuit, electronic or logic module, device, unit, or system such as those shown here. Thus, the development structure can be 1920These data structures may include files or other data structures, including human- and / or machine-readable source code, compiled structures, and computer-executable code structures, which, when processed by a design or simulation data processing system, functionally simulate or otherwise represent circuits or other levels of hardware logic design. Such data structures may include hardware description language (HDL) design entities or other data structures that are consistent with and / or compatible with lower-level HDL design languages ​​such as Verilog and VHDL and / or higher-level design languages ​​such as C or C++.

[0106] The development process 1910preferably uses and integrates hardware and / or software modules to synthesize, translate, or otherwise process a design / simulation functional equivalent of the components, circuits, units, or logic structures shown herein to generate a netlist 1980 to generate structures that may include organizational structures such as the organizational structure 1920 The netlist may contain 1980 For example, it can contain compiled or otherwise processed data structures that represent a list of lines, discrete components, logic gates, control circuits, I / O units, models, etc., describing the connections to other elements and circuits in the development of integrated circuits. The netlist 1980 can be synthesized using an iterative process where the netlist 1980Depending on development specifications and parameters for the unit, it is resynthesized once or several times. As with other types of structure described here, the netlist can 1980 The data can be recorded on a machine-readable storage medium or programmed into a programmable gate array. The medium can be a non-volatile storage medium such as a magnetic or optical disk drive, a programmable gate array, CompactFlash, or other flash memory. Additionally or alternatively, the medium can be system memory, cache memory, or buffer memory.

[0107] The development process 1910 It can include hardware and software modules for processing a variety of input data structure types, including netlists. 1980 , contained. Such data structure types can be found, for example, in library elements. 1930They are located and contain a set of commonly used elements, circuits, and units, including models, layouts, and symbolic representations for a given manufacturing technology (e.g., different technology nodes, 32 nm, 45 nm, 90 nm, etc.). The data structure types can also include development specifications. 1940 , characterization data 1950 , verification data 1960 , design rules 19190 and test data files 1985 This includes input test patterns, output test results, and other test information. The development process 1910It may also include, for example, common mechanical development processes such as stress analysis, thermal analysis, simulation of mechanical events, process simulation for operations such as casting, molding, and compression molding, etc. A person skilled in the field of mechanical design can assess the scope of possible mechanical development tools and applications that may be used in the development process. 1910 They can be used, recognizing the invention without deviating from its scope and spirit. The development process 1910 It may also include modules for performing common circuit design processes such as timing analysis, verification, checking design rules, placement and routing operations, etc.

[0108] The development process 1910 uses and integrates logical and physical development tools such as HDL compilers and simulation modeling tools to design the architecture 1920together with some or all of the supporting data structures shown, together with additional mechanical development or, if necessary, data to generate a second structure 1990 to process. The structural design 1990 It resides on a storage medium or in a programmable gate array in a data format used for exchanging data between mechanical units and structures (e.g., information stored in IGES, DXF, Parasolid XT, JT, DRG, or any other suitable format for storing or rendering such mechanical structures). Similar to the structure 1920 The structural structure 1990preferably one or more files, data structures, or other computer-encoded data or commands located on transmission or data storage media, which, when processed by an ECAD system, generate a logically or otherwise functionally equivalent form of one or more of the embodiments of the invention shown herein. In one embodiment, the development structure can 1990 exhibit a compiled, executable HDL simulation model that functionally simulates the units shown herein.

[0109] The development structure 1990 It can also use a data format used for exchanging layout data of integrated circuits and / or symbolic data formats (e.g., information in a GDSII (GDS2) format). GL1 , OASIS, in mapping files or another suitable format for storing such design data structures). The structure 1990 It can contain information such as symbolic data, mapping files, test data files, design content files, manufacturing data, layout parameters, wires, metal plates, vias, shapes, routing data through the production line, and any other data required by a manufacturer or other designer / developer to produce a unit or structure as described above and shown here. The assembly structure 1990 can then lead to a stage 1995 progress in which the structural design 1990 for example: is issued, is issued for production, is released for mask production, is sent to another development site, is returned to the customer, etc.

[0110] As described, in at least one embodiment, a multiprocessor data processing system includes multiple vertical cache memory hierarchies supporting multiple processor cores, system memory, and a system connection. In response to a load and reserve request from a first processor core, a first cache memory supporting the first processor core issues a memory access request on the system connection for a target cache memory row of the load and reserve request. In response to the memory access request, and prior to receiving a system-wide coherence response for the memory access request, the first cache memory receives the target cache memory row and an early indication of the system-wide coherence response for the memory access request from a second cache memory in a second vertical cache memory hierarchy via cache-to-cache intervention.In response to the early specification and prior to receiving the system-wide coherence response, the first cache initiates processing to update the target cache line in the first cache.

[0111] Although various embodiments have been specifically shown and described, it is clear to those skilled in the art that various modifications to the form and details can be made without departing from the spirit and scope of the appended claims, and all of these alternative implementations fall within the scope of the appended claims. For example, while aspects relating to a computer system that executes program code controlling the functions of the present invention have been described, it should be clear that the present invention can alternatively be implemented as a program product comprising a computer-readable storage unit that stores program code which can be processed by a data processing system. The computer-readable storage unit may comprise volatile or non-volatile memory, an optical or magnetic disk, or the like.When used in this context, a "storage unit" is specifically defined as containing only legally approved manufactured items and excluding signal media, transiently propagating signals, and energy per se.

[0112] The program product may, for example, contain data and / or instructions that, upon execution or other processing in a data processing system, are processed to generate a logically, structurally, or otherwise functionally equivalent representation (including a simulation model) of hardware components, circuits, units disclosed herein, or systems. Such data and / or instructions may contain hardware description language (HDL) design entities or other data structures compatible with lower-level HDL design languages ​​such as Verilog and VHDL and / or higher-level design languages ​​such as C and / or C++. Furthermore, the data and / or instructions may also use a data format used for exchanging layout data of integrated circuits and / or symbolic data formats (e.g., information in a GDSII (GDS2) format). GL1, OASIS, in mapping files or in another suitable format for storing such design data structures). QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 7774555

[0020]

Claims

[1] A method of data processing in a multi-processor data processing system comprising multiple vertical cache memory hierarchies supporting a plurality of processor cores, a system memory and a system connection connected to the system memory and the multiple vertical cache memory hierarchies, wherein the method comprises: In response to receiving a load-and-reserve request from a first processor core, output through a first cache memory in a first vertical cache memory hierarchy supporting the first processor core, on the system connection a memory access request for a target cache memory row of the load-and-reserve request; In response to the memory access request and prior to receiving a system-wide coherence response for the memory access request, the first cache memory receives the target cache memory row from a second cache memory in a second vertical cache memory hierarchy through cache-to-cache intervention, and provides an early indication of the system-wide coherence response for the memory access request; and In response to the early specification of the system-wide coherence response and prior to receiving the system-wide coherence response, the first cache initiates processing to update the target cache line in the first cache. [2] The method of claim 1, further comprising: Passed through the first cache of the target cache row to a third cache in a third cache hierarchy before receiving the system-wide linked response. [3] The method of claim 1, further comprising: in response to receiving the early indication of the system-wide coherence response; beginning through the first cache of a protection window, in which the first cache protects its capture of the coherence ownership of the target cache row; and Continue the protection window for at least one wait interval after processing the load-and-reserve request, during which the first cache awaits receiving a conditional save request from the first processor core. [4] Method according to claim 3, further comprising an asynchronous termination of the protection window as a consequence of receiving the system-wide coherence response through the first cache memory in response to performing the conditional store request processing. [5] Method according to claim 3, wherein: the protective window is a first protective window; and Furthermore, the procedure exhibits an asynchronous termination of the first protection window as a result of the first cache receiving the system-wide coherence response, in response to the initiation of a second protection window by the first cache in response to receiving the conditional save request. [6] The method of claim 3, further comprising: Assigning a read claim state machine by the first cache to manage the request in response to receiving a read-and-reserve request from the first processor core; and The first cache memory removes the assignment of the read claim state machine in response to the fact that the system-wide coherence response is received later and the protection window is closed later. [7] Method according to claim 1, wherein the first cache memory, which initiates processing to update the target cache memory line in the first cache memory, comprises the initiation processing of the first cache memory to update the target cache memory line in the first cache memory in response to receiving a conditional memory request from the first processor core. [8] Processing unit for a multi-processor data processing system, wherein the processing unit comprises: a processor core; Connection logic that is set up to connect the processing unit to a system connection of the multiprocessor data processing system; a first vertical cache memory hierarchy supporting the processor core, wherein the first vertical cache memory hierarchy has a first cache memory configured for: In response to receiving a load-and-reserve request, the processor core issues a memory access request on the system memory connection for a target cache memory row of the load-and-reserve request; In response to the memory access request and prior to receiving a system-wide coherence response for the memory access request, receiving the target cache memory row from a second cache memory in a second vertical cache memory hierarchy through cache-to-cache intervention and an early indication of the system-wide coherence response for the memory access request; and In response to the early specification of the system-wide coherence response and prior to receiving the system-wide coherence response, initiate processing to update the target cache memory line in the first cache memory. [9] Processing unit according to claim 8, wherein the first cache memory is further configured to store the target cache memory line in a third cache memory in a third vertical cache memory hierarchy before receiving the system-wide linked response. [10] Processing unit according to claim 8, wherein the first cache memory is further configured to: in response to receiving the early indication of the system-wide coherence response, initiate a protection window in which the first cache memory protects its acquisition of the coherence ownership of the target cache memory row; and continue the protection window for at least one wait interval after processing the load-and-reserve request, during which the first cache memory awaits receiving a conditional save request from the first processor core. [11] Processing unit according to claim 10, wherein the first cache memory is further configured to asynchronously terminate the protection window as a consequence of receiving the system-wide coherence response through the first cache memory, in response to the termination of processing the conditional store request. [12] Processing unit according to claim 10, wherein: the protective window is a first protective window; and The first cache is further configured to asynchronously terminate the first protection window as a consequence of the first cache receiving the system-wide coherence response in response to the first cache initiating a second protection window in response to receiving the conditional memory request. [13] Processing unit according to claim 10, wherein the first cache memory is further configured to: Assigning a read claim state machine to manage the request in response to receiving a load-and-reserve request from the first processor core; and Revoking the assignment of the read claim state machine in response to the fact that receiving the system-wide coherence response and exiting the protective window will occur later. [14] Processing unit according to claim 8, wherein the first cache memory is further configured to initiate processing to update the target cache memory line in the first cache memory in response to receiving a conditional storage request of the first processor core. [15] Multi-process data processing system which features: first, second and third processing units according to any one of claims 8 to 14; and the system connection connecting the first, second and third processing units. [16] Organizational structure materially embodied in a machine-readable storage unit for developing, manufacturing or testing an integrated circuit, wherein the organizational structure comprises: a processing unit for a multiprocessor data processing system, wherein the processing unit comprises: a processor core; Connection logic that is set up to connect the processing unit to a system connection of the multiprocessor data processing system; a first vertical cache memory hierarchy supporting the processor core, wherein the first vertical cache memory hierarchy has a first cache memory configured to: In response to receiving a load-and-reserve request from the processor core, issuing a memory access request for a target cache memory row of the load-and-reserve request on the system connection; In response to the memory access request and prior to receiving a system-wide coherence response for the memory access request, receiving from a second cache store in a second vertical cache store hierarchy by cache-to-cache intervention of the target cache store row and an early indication of the system-wide coherence response for the memory access request; and in response to the early indication of the system-wide coherence response and prior to receiving the system-wide coherence response, initiating the processing to update the target cache store row in the first cache store. [17] Structure according to claim 16, wherein the first cache memory is further configured to store the target cache memory line in a third cache memory in a third vertical cache memory hierarchy before receiving the system-wide linked response. [18] Organizational structure according to claim 16, wherein the first cache memory is further configured to: In response to receiving the early indication of the system-wide coherence response, a protective window begins in which the first cache store protects its acquisition of coherence ownership of the target cache store row; and Continue the protection window for at least one wait interval after processing the load-and-reserve request, during which the first cache awaits receiving a conditional memory request from the first processor core. [19] Structure according to claim 18, wherein the first cache memory is further configured to terminate the protection window asynchronously to receive the system-wide coherence response through the first cache memory in response to the processing of the conditional memory request. [20] Organizational structure according to claim 18, wherein: that the protective window is a first protective window; and The first cache is further configured to asynchronously terminate the first protection window as a consequence of receiving the system-wide coherence response by the first cache in response to the initiation of a second protection window by the first cache in response to receiving the conditional storage request.