Maintaining domain consistency state including domain state non-ownership (DSN) in a processor-based device

Abstract

Maintaining domain coherency state including domain state non-ownership (DSN) in a processor-based device is disclosed. In this regard, the processor-based device provides a plurality of processing elements (PEs) organized into a plurality of domains, each processing element including one or more PEs and a local ordering point circuit (LOP). The processor-based device supports a domain coherency state for a given domain with respect to a coherency granule cached by a PE. The domain coherency state includes a DSN domain coherency state indicating that the coherency granule is not cached in any domain with a shared modified state. In some embodiments, upon receiving a read access request for a coherency granule, a system ordering point circuit (SOP) determines that the coherency granule is cached in a domain of the plurality of domains with a DSN domain coherency state, and if necessary, can safely read the coherency granule from system memory to satisfy the read access.

Description

Technical Field

[0001] The technology disclosed herein relates to cache coherence protocols in processor-based devices, and more specifically, to maintaining domain consistency state. Background Technology

[0002] Processor-based devices may include multiple processing elements (PEs) (e.g., processor cores as a non-limiting example), each providing one or more local caches for storing frequently accessed data. Because the multiple PEs of a processor-based device can share memory resources such as system memory, multiple copies of shared data read from a given memory address can coexist in both system memory and the PEs' local caches. Therefore, to ensure that all PEs have a consistent view of the shared data, processor-based devices provide mechanisms for maintaining cache coherency. This allows for the avoidance of conflicts that may arise between PEs due to inconsistencies between local copies of shared data.

[0003] One such mechanism for ensuring a consistent view of shared data observed by PEs within a processor-based device is the System Order Point Circuit (SOP). In a processor-based device employing an SOP, a PE seeking a copy of a consistent particle (i.e., the smallest memory block maintained for its consistency, typically corresponding to a cache line) from system memory sends a read request for the consistent particle to the SOP. The SOP then sends snoops to other PEs within the processor-based device to determine if any of the other PEs possess a consistent copy of that consistent particle. The PE with a consistent copy of the consistent particle responds with the snoop response and provides a copy of its consistent particle to the requesting PE to satisfy the read request. The performance of a processor-based device using an SOP can be further improved by subdividing the PEs of the processor-based device into multiple domains, each with a local order point circuit (LOP), to which PEs within the domain send memory access requests. Using LOPs in conjunction with SOPs can make requests satisfyed faster, for example, when the requesting PE and the PE holding a consistent copy of the requested consistent particle are in the same domain.

[0004] However, a situation may arise where a first PE sends a snooping request seeking a copy of the consistent particle, but a second PE holding a consistent copy of the consistent particle cannot serve the snooping in a timely manner (e.g., because the second PE's snooping processing resources or the LOP of its domain are busy due to previous operations). The SOP cannot simply read the requested consistent particle from system memory on behalf of the first PE, because the SOP cannot determine whether the consistent particle in system memory is outdated (i.e., whether the cached copy of the consistent particle held by the PE has been modified but not written back to system memory). Therefore, either SOP 124 must continuously resend the snooping request until the second PE 104(0) serves the snooping request, or the second PE 104(0) can pause the snooping channel until the second PE 104(0) is able to serve the snooping request. Both approaches can result in unacceptably long wait times for accessing the consistent particle and may degrade the overall system performance of processor-based devices.

[0005] Therefore, a mechanism is desired to reduce the need for communication with non-responding domains. Summary of the Invention

[0006] The exemplary embodiments disclosed herein include maintaining a domain consistency state, including a domain state non-ownership (DSN), in a processor-based device. In this regard, in one exemplary embodiment, the processor-based device provides a plurality of processing elements (PEs) organized into multiple domains, each domain containing one or more PEs. Each domain includes a local sort point circuit (LOP) communicatively coupled to the PEs within the domain and also communicatively coupled to the system sort point circuit (SOP) of the processor-based device. The processor-based device is configured to support a domain consistency state that is a superset of regular cache consistency states indicating the domain-level consistency state of each consistency particle cached by a PE within a given domain. The domain consistency states supported by the processor-based device include a DSN domain consistency state indicating that a consistency particle is not cached in any domain with an owner (O) cache consistency state (i.e., the consistency particle is not cached in a shared modified state). In some embodiments, if the system sorting point circuit (SOP) of a processor-based device subsequently receives a request for a read access to a consistent particle, and determines in the DSN domain consistency state within one of multiple domains that the consistent particle is cached, the SOP can safely read the consistent particle from system memory to satisfy the read access if necessary (e.g., if a snoop response is not received from the LOP in a timely manner). In this way, read requests can be satisfied more quickly because the SOP does not have to wait for a snoop response from an unresponsive LOP, and it can ensure that the consistent particle maintained by the SOP in the DSN domain consistency state is not outdated in system memory.

[0007] In another exemplary embodiment, a processor-based device is provided. The processor-based device includes a plurality of PEs and a plurality of domains. Each domain includes one or more PEs from the plurality of PEs, and a LOP communicatively coupled to the one or more PEs. A first LOP of a first domain in the plurality of domains is configured to determine that a consistent particle cached as a first cache copy within the local cache of the first PE of the one or more PEs in the first domain is not cached in a cache-consistent state of 0 in any of the plurality of domains. The first LOP is also configured to update a domain consistency state indicator for the consistent particle to indicate that the consistent particle is cached in a DSN domain consistency state.

[0008] In another exemplary embodiment, a method is provided. The method includes determining, by a LOP (Level of Validation) of a processor-based device, that a consistent particle is not cached in a cached consistent state of 0 in any of a plurality of domains, the consistent particle being cached as a first cache copy in the local cache of a first PE (Performance Provider) of one or more PEs in a first domain, the LOP being located in the first domain of a plurality of domains, each domain comprising one or more PEs of a plurality of domains. The method further includes updating a domain consistent state indicator for the consistent particle to indicate that the consistent particle is cached in a DSN (Digital Service Provider) domain consistent state within the first domain.

[0009] In another exemplary embodiment, a non-transitory computer-readable medium is provided on which computer-executable instructions are stored. When executed by a processor device, the computer-executable instructions cause the processor device to: determine that the consistency particle is not cached in a 0-cache consistency state in any of a plurality of domains, the consistency particle being cached as a first cache copy in the local cache of a first PE in one or more PEs of a first domain among the plurality of domains. The computer-executable instructions further cause the processor device to update a domain consistency state indicator for the consistency particle to indicate that the consistency particle is cached in a DSN domain consistency state in the first domain.

[0010] Those skilled in the art will understand the scope of this disclosure and will implement additional embodiments of this disclosure after reading the following detailed description of preferred embodiments associated with the accompanying drawings. Attached Figure Description

[0011] The accompanying drawings, which are included in and form a part of this specification, illustrate several embodiments of the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0012] Figure 1This is a block diagram illustrating an exemplary processor-based device according to some embodiments, the device being configured to maintain a domain consistency state, including a domain state non-ownership (DSN) domain consistency state, in a processor-based device.

[0013] Figure 2 This is a consistency state transition diagram illustrating the transitions between domain consistency states and domain consistency states according to some embodiments;

[0014] Figures 3A to 3D This illustrates, according to some embodiments, methods for maintaining a domain consistency state, including a DSN domain consistency state. Figure 1 Exemplary communication flows between components and by Figure 1 The message flow graph of the operations performed by the components;

[0015] Figures 4A to 4C This illustrates, according to some embodiments, a method for transitioning from a Domain State Sharing (DSS) domain-consistent state to a Domain State Sharing (DSN) domain-consistent state. Figure 1 Exemplary communication flows between components and by Figure 1 The message flow graph of the operations performed by the components;

[0016] Figure 5 This illustrates, according to some embodiments, a method for transitioning from a DSS domain consistency state to a DSN domain consistency state based on receiving a DSO_NOW_CLEAN message. Figure 1 Exemplary communication flows between components and by Figure 1 The message flow graph of the operations performed by the components;

[0017] Figure 6 This illustrates, according to some embodiments, a method for transitioning from a Domain State Ownership (DSO) domain-consistent state to a Domain State Number (DSN) consistent state. Figure 1 Exemplary communication flows between components and by Figure 1 The message flow diagram of the components performing operations;

[0018] Figure 7 This illustrates, according to some embodiments, a method for transitioning from a Domain State Exclusive (DSE) domain-consistent state to a Domain State Non-Signal (DSN) domain-consistent state. Figure 1 Exemplary communication flow between components and by Figure 1 The message flow graph of the operations performed by the components;

[0019] Figure 8 This illustrates, according to some embodiments, a method for transitioning from a domain state invalid (DSI) domain consistency state to a DSN domain consistency state. Figure 1 Exemplary communication flow between components and by Figure 1 The message flow graph of the operations performed by the components;

[0020] Figure 9A and Figure 9B This is a flowchart illustrating exemplary operations for maintaining a domain consistency state (including a DSN domain consistency state) and for transitioning from a DSN domain consistency state to a DSE domain consistency state or a DSI domain consistency state, according to some embodiments.

[0021] Figure 10 This is a flowchart illustrating exemplary operations for transitioning from a DSS domain consistent state to a DSN domain consistent state according to some embodiments;

[0022] Figure 11 This is a flowchart illustrating additional exemplary operations for transitioning from a DSS domain consistent state to a DSN domain consistent state according to some embodiments;

[0023] Figure 12 This is a flowchart illustrating an exemplary operation based on receiving a DSO_NOW_CLEAN message, according to some embodiments, to transition from a DSS domain consistency state to a DSN domain consistency state.

[0024] Figure 13 This is a flowchart illustrating exemplary operations for transitioning from a DSO domain consistent state to a DSN domain consistent state according to some embodiments;

[0025] Figure 14 This is a flowchart illustrating exemplary operations for transitioning from a DSE domain consistency state to a DSN domain consistency state according to some embodiments;

[0026] Figure 15 This is a flowchart illustrating exemplary operations for transitioning from a DSI domain consistent state to a DSN domain consistent state according to some embodiments; and

[0027] Figure 16 These are exemplary processor-based devices according to some embodiments (e.g.) Figure 1 A block diagram of a processor-based device configured to maintain a domain consistency state, including the DSN domain consistency state. Detailed Implementation

[0028] The exemplary embodiments disclosed herein include maintaining a domain consistency state in a processor-based device, the domain consistency state including a domain state of non-ownership (DSN). In this regard, in one exemplary embodiment, the processor-based device provides a plurality of processing elements (PEs) organized into a plurality of domains, each domain containing one or more PEs. Each domain includes a local sort point circuit (LOP) communicatively coupled to the PEs within the domain and also communicatively coupled to the system sort point circuit (SOP) of the processor-based device. The processor-based device is configured to support a domain consistency state that is a superset of regular cache consistency states indicating the domain-level consistency state of each consistency particle cached by a PE within a given domain. The domain consistency states supported by the processor-based device include a DSN domain consistency state indicating that a consistency particle is not cached in any domain with an owned (O) cache consistency state (i.e., the consistency particle is not cached in a shared modified state). If the SOP receives a read access request for a consistent granule and determines that the consistent granule is cached in the DSN domain consistent state within one of the multiple domains, the SOP can safely read the consistent granule from system memory to satisfy the read access if necessary (e.g., if a snoop response is not received from the LOP in a timely manner). In this way, read requests can be satisfied more quickly because the SOP does not have to wait for a snoop response from an unresponsive LOP and can ensure that the consistent granule kept by the SOP in the DSN domain consistent state is not outdated in system memory.

[0029] in this regard, Figure 1 An exemplary processor-based device 100 is illustrated, which provides a plurality of processing elements (PEs) 102(0)-102(P), 104(0)-104(Z) for processing executable instructions. Each of the PEs 102(0)-102(P), 104(0)-104(Z) may include, for example, a single processor core, which includes a logical execution unit and associated cache and functional units. The PEs 102(0)-102(P), 104(0)-104(Z) are organized into a plurality of domains 106(0)-106(D), wherein each of the plurality of domains 106(0)-106(D) contains one or more of the plurality of PEs 102(0)-102(P), 104(0), 104(Z). Thus, in Figure 1 In the example, while domain 106(D) includes PE 104(0)-104(Z), domain 106(0) includes PE 102(0)-102(P).

[0030] Figure 1The processor-based devices 100, PE 102(0)-102(P) and 104(0)-104(Z), are interconnected with each other and connected to system memory 108 via interconnect bus 110. Figure 1 As shown, system memory 108 is subdivided into multiple coherence granules 112(0)-112(G), each coherence granule representing the smallest memory unit (e.g., 64 bytes, as a non-limiting example) maintained by processor-based device 100 for its cache coherence. Figure 1 In the example, PE 102(0) includes a local cache 114 and a cache controller circuit 118, the local cache 114 including cache lines 116(0)-116(C), and the cache controller circuit 118 embodying logic for controlling access to the local cache 114. It should be understood that, in addition to Figure 1 In addition to the local cache 114 shown, PE 102(0) may also include other caches. PE 102(0) uses the local cache 114 to locally store data loaded from system memory 108 for faster access. For example, as Figure 1 As shown, cache line 116(0) of local cache 114 stores a cache copy 120 of one of the consistency granules 112(0)-112(G) (e.g., consistency granule 112(0) as a non-limiting example). It should be understood that, although Figure 1 Not shown, but each of PEs 102(0)-102(P) and 104(0)-104(Z) includes a local cache and a cache controller circuit, which functionally correspond to the local cache 114 and cache controller circuit 118 of PE 102(0).

[0031] Each of the domains 106(0)-106(D) of the processor-based device 100 includes a corresponding local sort point circuit (LOP) 122(0)-122(D) configured to arbitrate cache coherence bus commands received from PEs 102(0)-102(P) and 104(0)-104(Z) within the domains 106(0)-106(D). The processor-based device 100 also includes a system sorting point circuit (SOP) 124 configured to communicate with LOPs 122(0)-122(D) to arbitrate cache coherence bus commands received from the master PEs 102(0)-102(P) and 104(0)-104(Z), to directly snoop and receive snoop responses from the snoop PEs 102(0)-102(P) and 104(0)-104(Z), and to send response messages to both the master PEs and the snoop PEs 102(0)-102(P) and 104(0)-104(Z). SOP124 can work in conjunction with a snooping filter (not shown) configured to monitor traffic on interconnect bus 110 to track the consistency status of cache lines such as cache lines 116(0)-116(C).

[0032] Figure 1 The processor-based device 100 and its constituent elements may include any one of known digital logic elements, semiconductor circuits, processing cores and / or memory structures, as well as other elements or combinations thereof. The embodiments described herein are not limited to any particular arrangement of elements, and the disclosed techniques can be readily extended to various structures and layouts on semiconductor sockets or packages. It should be understood that some embodiments of the processor-based device 100 may include, in addition to… Figure 1 Components other than those shown. For example, each of PE 102(0)-102(P) may also include one or more functional units, instruction caches, unified caches, memory controllers, interconnect buses and / or additional memory devices, cache and / or controller circuitry, for clarity. Figure 1 These have been omitted.

[0033] As described above, a situation may arise where a first PE (e.g., PE102(0) as a non-limiting example) sends a snooping request for a copy of a consistent particle (e.g., consistent particle 112(0)), but a second PE (e.g., PE104(0) as a non-limiting example) holding a consistent copy of consistent particle 112(0) is not available to serve the snooping in a timely manner. In this case, SOP 124 cannot simply read the requested consistent particle 112(0) from system memory 108 on behalf of the first PE 102(0) because SOP 124 cannot determine whether the consistent particle 112 in system memory 108 is outdated (i.e., the cached copy of consistent particle 112(0) has been modified by another of PEs 102(0)-102(P) and 104(0)-104(Z)). Therefore, either SOP 124 must continuously retransmit the snooping until the second PE 104(0) is available for snooping, or the second PE 104(0) can pause the snooping channel until it is available for snooping. Both of these approaches can result in unacceptably long wait times for accessing the consistency granularity 112(0) and may degrade the overall system performance of the processor-based device 100.

[0034] In this regard, the LOPs 122(0)-122(D) of the processor-based device 100 are configured to provide domain consistency status indicators 126(0)-126(N), 128(0)-128(N) to indicate the domain consistency status cached within the corresponding domains 106(0)-106(D) for consistency granularity. The domain consistency status indicators 126(0)-126(N), 128(0)-128(N) may be stored or embodied as part of a snooping filter (not shown), a directory (not shown), or any other suitable data structure or circuitry within the LOPs 122(0)-122(D). See below for details. Figure 2 In more detail, the domain consistency state is a superset of the regular cache consistency state and indicates the consistency state of the consistency granules at the domain level. Specifically, LOP 122(0)-122(D) is configured to support DSN domain consistency states, which indicate that the corresponding consistency granules are not cached in any of the domains 106(0)-106(D) in a cache consistency state of 0 (i.e., in a shared modification state).

[0035] When it is determined that a consistent particle (e.g., as a non-limiting example, consistent particle 112(0)) is not cached in O-cache consistent state within any domain of domains 106(0)-106(D), a LOP (such as LOP 122(0)) updates a domain consistent state indicator (e.g., as a non-limiting example, domain consistent state indicator 126(0)) to indicate that consistent particle 112(0) is cached in DSN domain consistent state within domain 106(0). Subsequently, if SOP 124 receives a request for read access to consistent particle 112(0), SOP 124 can determine (e.g., by receiving domain consistent state indicator 126(0) from LOP 122(0)) that consistent particle 112(0) is known to have DSN consistent state by domain 106(0). Then, SOP124 is able to read the coherent granules from system memory 108 to satisfy the read request (i.e., instead of having to wait for a snooping response from an unresponsive domain in domains 106(0)-106(D)).

[0036] To illustrate the exemplary domain consistency state and the transition between domain consistency states according to some embodiments, Figure 2 A consistency state transition diagram 200 is provided. As mentioned above, domain consistency states are a superset of regular cache consistency states and include Domain State Exclusive (DSE) domain consistency state 202, Domain State Shared (DSS) domain consistency state 204, Domain State Owned (DSO) domain consistency state 206, Domain State Invalid (DSI) domain consistency state 208, and Domain State Non-Owned (DSN) domain consistency state 210. A description of each domain consistency state is provided below in Table 1.

[0037] Table 1

[0038]

[0039]

[0040] The transitions between different domain consistency states are further described in Table 2 below:

[0041] Table 2

[0042]

[0043]

[0044]

[0045]

[0046] Figures 3A to 3D , Figures 4A to 4C as well as Figures 5 to 8 Message flow diagrams according to some embodiments are provided to illustrate the communication flows and operations for transitioning to and from DSN domain consistency state 210. Figures 3A-3D It shows the result of Figure 1 Exemplary communication flows and operations performed by the components for determining and updating consistency granules (e.g., Figure 1 The domain consistency state of the consistency particle 112(0)), and the domain consistency state used to transition from the DSN domain consistency state 210 to the DSE domain consistency state 202 or the DSI domain consistency state 208. Figures 4A-4C and Figure 5 Exemplary communication flows and operations for transitioning from DSS domain consistency state 204 to DSN domain consistency state 210 are shown respectively. Figures 6-8 Exemplary communication flows and operations are shown for transitioning from DSO domain consistency state 206, DSE domain consistency state 202, and DSI domain consistency state 208 to DSN domain consistency state 210, respectively.

[0047] First go to Figures 3A-3D According to some embodiments, message flow diagram 300 shows Figure 1 Exemplary communication flows between components and by Figure 1 The various components perform operations to maintain domain consistency, including the DSN domain consistency state 210. For clarity, in the description... Figures 3A-3D Referenced at the time Figure 1 and Figure 2 Components. For example... Figures 3A-3D As shown, message flow diagram 300 displays PE 102(0), LOP 122(0), SOP 124, LOP 122(D), PE 104(0), and system memory 108 (“SYSTEM MEM”) represented by vertical lines. Communication between these components is indicated by arrows under the headings, and operations performed by these components are indicated by boxes under the headings. It should be understood that this is not... Figures 3A-3D All operations shown can be performed by all embodiments, and / or can be performed in a different order than that shown herein. Figures 3A-3D Some of the operations shown are illustrated.

[0048] exist Figure 3AIn this process, the operation begins with LOP 122(0), which determines that a consistent particle 112(0) cached as a cache copy 120 in the local cache 114 of one or more PEs 102(0)-102(P) of domain 106(0) is not cached in a cached consistent state of 0 in any of the multiple domains 106(0)-106(D) (box 302). LOP 122(0) updates the domain consistent state indicator (e.g., domain consistent state indicator 126(0)) for consistent particle 112(0) to indicate that consistent particle 112 is cached in a DSN domain consistent state 210 within domain 106(0) (box 304). Subsequently, PE 104(0) sends a request 306 to LOP 122(D) for read access to consistency granule 112(0), and LOP 122 then forwards the request 308 to SOP 124 for read access to consistency granule 112(0). Then... Figure 3B Continue the operation.

[0049] Now for reference Figure 3B SOP 124 determines that the consistent particle 112(0) is cached in the DSN domain consistency state 210 within domain 106(0) (box 310). Because SOP 124 knows from the DSN domain consistency state 210 that the consistent particle 112(0) in system memory 108 is not outdated relative to its cached copy in domain 106(0), SOP 124 can read the consistent particle 112(0) from system memory 108 to satisfy a read access (e.g., in cases where some of the LOPs 122(0)-122(D) cannot respond to snooping in a timely manner). Therefore, as indicated by arrow 314, SOP 124 sends a request 312 to system memory 108 to read the consistent particle 112(0) and receives the consistent particle 112(0) in response to request 312. As indicated by arrow 316, SOP 124 then provides the consistent particle 112(0) to PE 104(0). Then... Figure 3C Continue the operation.

[0050] Now go to Figure 3CIn some embodiments, a PE (e.g., PE 102(0)) within domain 106(0) may send a request 318 to LOP 122(0) to update the cache consistency state of consistency particle 112(0) from S cache consistency to E cache consistency state. In such an embodiment, LOP 122(0) then forwards request 320 to SOP 124 to obtain permission to upgrade the domain consistency state of domain 106(0). After SOP 124 has completed snooping (if necessary) on other domains in the plurality of domains 106(0)-106(D) to instruct them to relinquish copies of consistency particle 112(0), and has determined that no other cache copies exist in the other domains of the plurality of domains 106(0)-106(C), SOP 124 sends a response 322 granting request 320. Upon receiving response 322, in these embodiments, LOP 122(0) updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112 is cached in domain 106(0) with DSE domain consistency state 202 (box 324). Then Figure 3D Continue the operation. Now refer to... Figure 3D Some embodiments may provide that LOP 122(0) determines that the first cache copy 120 of the consistency particle 112(0) held by PE 102(0) is the last remaining copy of the consistency particle 112(0) within domain 106(0) (box 326). Subsequently, PE 102(0) evicts the first cache copy 120 (box 328). After determining that PE 102(0) has evictred the first cache copy 120, LOP 122(0) updates the domain consistency status indicator 126(0) for the consistency particle 112(0) to indicate that the consistency particle 112(0) is cached in DSI domain consistency state 208 (box 330).

[0051] To illustrate the process of transitioning from DSS domain consistency state 204 to DSN domain consistency state 210 according to some embodiments Figure 1 Exemplary communication flow between components and by Figure 1 The operations performed by the components Figures 4A-4C A message flow graph 400 is provided. For clarity, in the description... Figures 4A-4C Referenced at the time Figure 1 and Figure 2 Components. Figures 4A-4C Message flow diagram 400 shows PE 102(0), PE 102(P), SOP 124, LOP 122(D), and PE 104(0) represented by vertical lines. Communication between these components is indicated by arrows under the headers, and operations performed by these components are indicated by boxes under the headers. It should be understood that this is not... Figures 4A-4C All operations shown can be performed by all embodiments, and / or can be performed in a different order than that shown herein. Figures 4A-4C Some of the operations shown are illustrated.

[0052] Operation in Figure 4A Starting at LOP122(0), LOP122(0) updates the domain consistency status indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) is cached in domain 106(0) with DSS domain consistency status 204 (box 402). PE 102(P) in domain 106(0) sends a request 404 to LOP122(0) for read access to consistency granule 112(0), and LOP122(0) then forwards the request 406 to SOP124. After determining that domains 106(0) and 106(D) need to be snooped, SOP124 sends snooping requests 408 and 410 to LOP122(0) and 122(D), respectively. Figure 4A In the example, LOP 122(0) already knows that domain 106(0) does not have the requested data (because it forwards request 406 to SOP 124), so LOP 122(0) provides SOP 124 with a sniffer response 412 to indicate that its state is unmodified in the absence of data. LOP 122(D) forwards sniffer 414 to PE104(0), and PE104(0) sends a sniffer response 416 to LOP 122(D) to indicate that its state is unmodified and that it can satisfy the read request. LOP 122(D) then provides SOP 124 with a sniffer response 418 to indicate that its domain state is unmodified.

[0053] SOP 124 responds to request 404, aggregates the sent snoop responses 412 and 418 (referred to herein as "one or more snoop responses 412, 418"), and sends response 420 to LOP 122(0) in requesting domain 106(0) to indicate that none of the domains 106(0)-106(D) holds consistent particle 112(0) in modified state (or one of the domains 106(0)-106(D) knows that consistent particle 112(0) is held in DSN domain consistent state). LOP 122(0) is now able to transition to DSN domain consistent state. LOP 122(0) forwards response 422 to request PE 102(P), which will receive an unmodified copy of consistent particle 112(0). Then Figure 4B Continue the operation.

[0054] Now for reference Figure 4BIn some embodiments, based on one or more snoop responses 412, 418, SOP 124 can determine that none of the plurality of domains 106(0)-106(D) has a cache consistency granularity 112(0) with DSO domain consistency state 206 (block 424). Alternatively, based on one or more snoop responses 412, 418, SOP 124 can determine that at least one of the plurality of domains 106(0)-106(D) has a cache consistency granularity 112(0) with DSN domain consistency state 210 (block 426). Then Figure 4C Continue the operation.

[0055] Turn now Figure 4C If SOP 124 determines that none of the domains 106(0)-106(D) caches consistency granule 112(0) in DSO domain consistency state 206, or at least one of the domains 106(0)-106(D) caches consistency granule 112(0) in DSN domain consistency state 210, SOP 124 sends instruction 428 to LOP 122(0) for this purpose. Based on instruction 428, LOP 122(0) can safely conclude that DSN domain consistency state 210 can be applied to consistency granule 112(0) of domain 106(0). Therefore, LOP 122(0) updates the domain consistency state indicator 126(0) from DSS domain consistency state 204 to DSN domain consistency state 210 (box 430). It should be understood that in some embodiments, LOP 122(0) itself may observe one or more snoop responses 412, 418 and may determine, based on its own observation, that no domain among the plurality of domains 106(0)-106(D) caches the consistency granule 112(0) in DSO domain consistency state 206 or at least one domain among the plurality of domains 106(0)-106(D) caches the consistency granule 112(0) in DSN domain consistency state 210.

[0056] Figure 5 A message flow diagram 500 is provided, illustrating, according to some embodiments, a transition from DSS domain consistency state 204 to DSN domain consistency state 210 based on the receipt of a DSO_NOW_CLEAN message. Figure 1 Exemplary communication flow between components and by Figure 1 The operations performed by the components. For clarity, in the description Figure 5 Referenced at the time Figure 1 and Figure 2 Components. For example... Figure 5As shown, message flow diagram 500 illustrates LOP 122(0), SOP 124, and LOP 122(D) represented by vertical lines. Communication between these elements is indicated by arrows under the headers, and the operations performed by these elements are indicated by boxes under the headers. It should be understood that this is not... Figure 5 All operations shown can be performed by all embodiments, and / or can be performed in a different order than that shown herein. Figure 5 Some of the operations shown are illustrated.

[0057] exist Figure 5 In this process, the operation begins at LOP 122(0), where LOP 122(0) updates the domain consistency state indicator 126(0) for consistency particle 112(0) to indicate that consistency particle 112(0) is cached in domain 106(0) with DSS domain consistency state 204 (box 502). LOP 122(0) then receives a DSO_NOW_CLEAN message 504 from LOP 122(D) of domain 106(D) via SOP 124, which indicates that consistency particle 112 (previously cached in domain 106(D) with DSO domain consistency state 206) has transitioned to DSN domain consistency state 210. LOP 122(D) then updates the domain consistency state indicator 126(0) from DSS domain consistency state 204 to DSN domain consistency state 210 (box 506).

[0058] To illustrate the process of transitioning from DSO domain consistency state 206 to DSN domain consistency state 210 according to some embodiments Figure 1 Exemplary communication flow between components and by Figure 1 The operations performed by the components Figure 6 A message flow graph 600 is provided. For clarity, in the description... Figure 6 Referenced at the time Figure 1 and Figure 2 Components. Figure 6 Message flow diagram 600 shows LOP 122(0), system memory 108 (“SYSTEM MEM”), and LOP 122(D) represented by vertical lines. Communication between these elements is indicated by arrows under the headings, and operations performed by these elements are indicated by boxes under the headings. It should be understood that this is not... Figure 6 All operations shown can be performed by all embodiments, and / or can be performed in a different order than that shown herein. Figure 6 Some of the operations shown are illustrated.

[0059] Figure 6The operation begins with LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency particle 112(0) to indicate that consistency particle 112(0) is cached in domain 106(0) in DSO domain consistency state 206 (box 602). LOP 122(0) then writes a first cache copy 120 to consistency particle 112(0) in system memory 108. Because LOP 122(0) knows that the first cache copy 120 is in O cache consistency state (i.e., shared modification) and has been written to system memory 108, LOP 122(0) can determine that no other domain among the multiple domains 106(0)-106(D) holds a copy of consistency particle 112(0) in O consistency state. Therefore, LOP 122(0) updates the domain consistency state indicator 126(0) from DSO domain consistency state 206 to DSN domain consistency state 210 (box 604). In some embodiments, LOP 122(0) may also transmit a DSO_NOW_CLEAN message 606 to other LOPs, such as LOP 122(D), indicating that consistency particle 112(0) has transitioned to DSN domain consistency state 210. In response to receiving the DSO_NOW_CLEAN message 606, LOP 122(D) may, in turn, transition to DSN domain consistency state 210 for consistency particle 112(0). It should be understood that in some embodiments, Figure 1 When SOP 124 accepts a writeback from one of the domains 106(0)-106(D) held in the consistency granule 112(0) in the consistency state 206 of the DSO domain, it may send a DSO_NOW_CLEAN message to LOP 122(0)-122(D).

[0060] Figure 7 A message flow diagram 700 according to some embodiments is provided, which illustrates the process for transitioning from DSE domain consistency state 202 to DSN domain consistency state 210. Figure 1 Exemplary communication flow between components and by Figure 1 The operations performed by the components. For clarity, in the description Figure 7 Referenced at the time Figure 1 and Figure 2 Components. In Figure 7 In the message flow diagram 700, PE 102(0), LOP 122(0), and PE 104(0) are represented by vertical lines. Communication between these elements is indicated by arrows under the headers, and the operations performed by these elements are indicated by boxes under the headers. It should be understood that this is not... Figure 7All operations shown can be performed by all embodiments, and / or can be performed in a different order than that shown herein. Figure 7 Some of the operations shown are illustrated.

[0061] Figure 7 The operation begins at LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) is cached within domain 106(0) in DSE domain consistency state 202 (box 702). PE 102(0) then provides a shared copy 704 of consistency granule 112(0) to PE 104(0) within domain 106(D). Figure 7 In the example, PE 102(0) will send a snoop response (not shown) to indicate that it will provide shared copy 704 to PE 104(0). In this example, LOP 122(0) infers from the snoop response that the first cache copy 120 held by PE 102(0) has not been modified. Therefore, LOP 122(0) updates the domain consistency state indicator 126(0) from DSE domain consistency state 202 to DSN domain consistency state 210 (box 706).

[0062] To illustrate the method for transitioning from DSI domain consistency state 208 to DSN domain consistency state 210 according to some embodiments Figure 1 Exemplary communication flow between components and by Figure 1 The operations performed by the components Figure 8 A message flow graph 800 is provided. For clarity, in the description... Figure 8 Referenced at the time Figure 1 and Figure 2 Components. Figure 8 The message flow diagram 800 shows PE 102(0), LOP 122(0), and SOP 124 represented by vertical lines. Communication between these components is indicated by arrows under the headers, and the operations performed by these components are indicated by boxes under the headers. It should be understood that this is not... Figure 8 All operations shown can be performed by all embodiments, and / or can be performed in a different order than that shown herein. Figure 8 Some of the operations shown are shown.

[0063] exist Figure 8In this process, the operation begins at LOP 122(0), where LOP 122(0) updates the domain consistency state indicator 126(0) for consistency particle 112(0) to indicate that consistency particle 112(0) has a DSI domain consistency state 208 within domain 106(0) (box 802). LOP 122(0) then receives a request 804 from the first PE 102(0) for consistency particle 112(0) and forwards the corresponding request 806 to SOP 124. LOP 122(0) then receives a response 808 from SOP 124 indicating that no domain in domains 106(0)-106(D) holds a modified copy of consistency particle 112(0) (i.e., no domain 106(0)-106(D) holds a copy of consistency particle 112(0) in the O or M cache consistency state). LOP 122(0) then updates the domain consistency status indicator 126(0) for consistency granules 112(0) by updating the domain consistency status indicator 126(0) from DSI domain consistency status 208 to DSN domain consistency status 210 (box 810).

[0064] Figure 9A and Figure 9B A flowchart 900 according to some embodiments is provided, illustrating exemplary operations for maintaining a domain consistency state (including DSN domain consistency state 210) and for transitioning from DSN domain consistency state 210 to DSE domain consistency state 202 or DSI domain consistency state 208. For clarity, in the description Figure 9A and Figure 9B Referenced at the time Figure 1 , Figure 2 and Figures 3A-3D Components. Figure 9A and Figure 9B Description Reference Figure 1 LOP 122 is called "first LOP 122(0)", and Figure 1 The field 106(0) is called "the first field 106(0)". Figure 1 PE 102(0) is referred to as "first PE 102(0)" and will Figure 1 The cache copy 120 is referred to as "first cache copy 120".

[0065] exist Figure 9A In the middle, the operation begins Figure 1The first LOP122(0) of the first domain 106(0) of the multiple domains 106(0)-106(D), each domain including one or more PEs 102(0)-102(P) among multiple PEs 102(0)-102(P) and 104(0)-104(Z), determines that the consistency particle 112(0) is not cached in a cache consistency state of O in any domain of the multiple domains 106(0)-106(D), and the consistency particle 112(0) is cached as a first cache copy 120 (box 902) in the local cache 114 of the first PE 102(0) of one or more PEs 102(0)-102(P) of the first domain 106(0). The first LOP 122(0) then updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) is cached in the first domain 106(0) with DSN domain consistency state 210 (box 904). The following is about... Figures 10-15 The operations for performing the functions described in blocks 902 and 904 of some embodiments will be discussed in more detail in some embodiments.

[0066] SOP 124 then proceeds from the second field (e.g.) Figure 1 Domain 106(1) receives a request 308 (box 906) for read access to consistency particle 112(0). SOP 124 determines that consistency particle 112(0) is cached in DSN domain consistency state 210 within the first domain 106(0) (box 908). SOP 124 then reads consistency particle 112(0) from system memory 108 to satisfy the read access (i.e., instead of waiting for a snooping response from a non-responsive domain) (box 910). Then, the processing in some embodiments is... Figure 9B Continue at frame 912.

[0067] Now for reference Figure 9B According to some embodiments, a first LOP 122(0) may receive from a first PE 102(0) a request 318 (box 912) to update the cache consistency state of a first cache copy 120 of consistency granule 112(0) from S cache consistency state to E cache consistency state. Then, the first LOP 122(0) updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) is cached in the first domain 106(0) with DSE domain consistency state 202 (i.e., after LOP 122(0) forwards request 320 to SOP 124 and subsequently receives a response 322 granting request 320) (box 914).

[0068] Some embodiments may provide that a first LOP 122(0) determines that a first cached copy 120 of a consistency particle 112(0) held by a first PE 102(0) is the last remaining copy of the consistency particle (112(0)) within a first domain 106(0) (box 916). The first LOP 122(0) then determines that the first PE 102(0) has evicted the first cached copy 120 (box 918). In response, the first LOP 122(0) updates the domain consistency status indicator 126(0) for the consistency particle 112(0) to indicate that the consistency particle 112(0) is cached in DSI domain consistency state 208 (box 920).

[0069] To illustrate exemplary operations for transitioning from DSS domain consistency state 204 to DSN domain consistency state 210 according to some embodiments, Figure 10 Flowchart 1000 is provided. For clarity, in the description... Figure 10 Referenced at the time Figure 1 , Figure 2 and Figures 4A-4C The components. Below. Figure 10 The description will Figure 1 LOP 122(0) is referred to as "first LOP 122(0)". Figure 1 The field 106(0) in the text is called "the first field 106(0)", and the field 106(0) in the text is called "the first field 106(0)". Figure 1 PE 102(P) in the text is referred to as "second PE 102(P)".

[0070] Figure 10 The operation begins at the first LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) is cached in the first domain 106(0) with DSS domain consistency state 204 (box 1002). (See above regarding...) Figure 9A As described in box 902, the first LOP 122(0) determines that the consistency granule 112(0) is not cached in a 0 cache consistency state in any of the multiple domains 106(0)-106(D) (box 1004). Figure 10 In the example, the operation of block 1004 for determining that consistency particle 112(0) is not cached in O cache consistency state in any of the domains 106(0)-106(D) includes a first LOP 122(0) determining that no domain in the plurality of domains 106(0)-106(D) caches consistency particle 112(0) in DSO domain consistency state 206 (block 1006).

[0071] In some embodiments, the operation of block 1006 is used to determine, based on one or more snoop responses 412, 418 that can be observed based on LOP 122(0), that no domain among the plurality of domains 106(0)-106(D) caches the consistency granule 112(0) in DSO domain consistency state 206. One or more snoop responses 412, 418 are sent for a request 404 for read access to the consistency granule, which is sent by a second PE 102(P) among the plurality of PEs 102(0)-102(P), 104(0)-104(P). Some embodiments may provide SOP 124 to observe one or more snoop responses 412, 418 sent in response to request 404, and based on one or more snoop responses 412, 418, determine that no domain among the plurality of domains 106(0)-106(D) 106 caches the consistency granule 112(0) in DSO domain consistency state 206. SOP 124 can then send instruction 428 to the first LOP 122(0), which can be determined based on instruction 428.

[0072] As mentioned above Figure 9A As described in box 904, the first LOP 122(0) then updates the domain consistency status indicator 126(0) for consistency granularity 112(0) (box 1008). Figure 10 In the example, the operation of box 1010 for updating the domain consistency status indicator 126(0) includes updating the domain consistency status indicator 126(0) from DSS domain consistency status 204 to DSN domain consistency status 210 (box 1010).

[0073] Figure 11 A flowchart 1100 is provided, illustrating additional exemplary operations according to some embodiments for transitioning from DSS domain consistency state 204 to DSN domain consistency state 210. For clarity, in the description... Figure 11 Referenced at the time Figure 1 , Figure 2 and Figures 4A-4C The components. Below. Figure 11 The description will Figure 1 LOP 122(0) is referred to as "first LOP 122(0)". Figure 1 The field 106(0) in the text is called "the first field 106(0)", and the field 106(0) in the text is called "the first field 106(0)". Figure 1 PE 102(P) in the text is referred to as "second PE 102(P)".

[0074] exist Figure 11In this process, the operation begins at the first LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) is cached in the first domain 106(0) with DSS domain consistency state 204 (box 1102). As mentioned above regarding... Figure 9A As described in box 902, the first LOP 122(0) determines that the consistency granule 112(0) is not cached in O cache consistency state in any of the multiple domains 106(0)-106(D) (box 1104). Figure 11 In the example, the operation of box 1104 for determining that consistency particle 112(0) is not cached in any domain 106(0)-106(D) with a cached consistency state of O includes: the first LOP 122(0) determines that at least one of the multiple domains 106(0)-106(D) is cached in DSN domain consistency state 210 with a cached consistency particle 112(0) (box 1106).

[0075] According to some embodiments, the operation of block 1106 for determining that at least one of a plurality of domains 106(0)-106(D) caches a consistency granule 112(0) in DSN domain consistency state 210 can be based on LOP 122(0) observing one or more snoop responses 412, 418 sent in response to a request 404 for read access to consistency granule 112(0), the request 404 being sent by a second PE 102(P) of a plurality of PEs 102(0)-102(P), 104(0)-104(P). In some embodiments, SOP 124 can observe one or more snoop responses 412, 418 sent in response to request 404, and based on one or more snoop responses 412, 418, determine that at least one of the plurality of domains 106(0)-106(D) caches a consistency granule 112(0) in DSN domain consistency state 210. SOP 124 can then send instruction 428 to the first LOP 122(0), which can then make a determination based on instruction 428.

[0076] As mentioned above Figure 9A As described in box 904, the first LOP 122(0) then updates the domain consistency status indicator 126(0) for consistency granularity 112(0) (box 1108). Figure 11 In the example, the operation of box 1110 for updating the domain consistency status indicator 126(0) includes updating the domain consistency status indicator 126(0) from DSS domain consistency status 204 to DSN domain consistency status 210 (box 1110).

[0077] To illustrate exemplary operations based on receiving a DSO_NOW_CLEAN message 504, according to some embodiments, for transitioning from DSS domain consistency state 204 to DSN domain consistency state 210, Figure 12 Flowchart 1200 is provided. For clarity, in the description... Figure 12 Referenced at the time Figure 1 , Figure 2 and Figure 5 The components. Below. Figure 12 The description will Figure 1 LOP 122(0) is referred to as "first LOP 122(0)". Figure 1 The field 106(0) in the text is called "the first field 106(0)", and the field 106(0) in the text is called "the first field 106(0)". Figure 1 The field 106(D) in the code is called "the third field 106(D)".

[0078] Figure 12 The operation begins at the first LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) is cached in the first domain 106(0) with DSS domain consistency state 204 (box 1202). (See above regarding...) Figure 9A As described in box 902, the first LOP 122(0) determines that the consistency granule 112(0) is not cached in O cache consistency state in any of the multiple domains 106(0)-106(D) (box 1204). Figure 12 In the example, the operation of box 1204 for determining that the consistency particle 112(0) is not cached with an O-cached consistency state in any domain of any domain 106(0)-106(D) includes LOP 122(0) receiving a DSO_NOW_CLEAN message 504 from a third domain 106(D) of the plurality of domains 106(0)-106(D), which indicates that the consistency particle 112(0) with DSO domain consistency state 206 in the third domain 106(D) has transitioned to DSN domain consistency state 210 (box 1206). It should be understood that in some embodiments, LOP 122(0) may receive DSO_NOW_CLEAN message 504 directly from third domain 106(D), while in some embodiments, LOP 122(0) may receive DSO_NOW_CLEAN message 504 indirectly from third domain 106(D) via SOP 124 (e.g., third domain 106(D) may transmit DSO_NOW_CLEAN message 504 to SOP 124, and SOP 124 may then forward it to LOP 122(0)).

[0079] As mentioned above Figure 9AAs described in box 904, the first LOP 122(0) then updates the domain consistency status indicator 126(0) for consistency granularity 112(0) (box 1208). Figure 12 In the example, the operation of box 1208 for updating the domain consistency status indicator 126(0) includes updating the domain consistency status indicator 126(0) from DSS domain consistency status 204 to DSN domain consistency status 210 (box 1210).

[0080] Figure 13 A flowchart 1300 according to some embodiments is provided, illustrating exemplary operations for transitioning from DSO domain consistency state 206 to DSN domain consistency state 210. For clarity, in the description Figure 13 Referenced at the time Figure 1 , Figure 2 and Figure 6 The components. Below. Figure 13 The description will Figure 1 LOP 122(0) is referred to as "first LOP 122(0)". Figure 1 The cache copy 120 in the middle is called "first cache copy 120", and will Figure 1 The field 106(0) in the code is called "the first field 106(0)".

[0081] exist Figure 13 In this process, the operation begins at the first LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112(0) within the first domain 106(0) is cached in DSO domain consistency state 206 (box 1302). As mentioned above regarding... Figure 9A As described in box 902, the first LOP 122(0) determines that the consistency granule 112(0) is not cached in a 0 cache consistency state in any of the multiple domains 106(0)-106(D) (box 1304). Figure 13 In the example, the operation of box 1304 is used to determine that the consistency particle 112(0) is not cached in any domain of domains 106(0)-106(D) with a cache consistency state of O, and the first LOP 122(0) writes the first cache copy 120 to the consistency particle 112(0) in system memory 108 (box 1308).

[0082] As mentioned above Figure 9A As described in box 904, the first LOP 122(0) then updates the domain consistency status indicator 126(0) for consistency granularity 112(0) (box 1308). Figure 13In the example, the operation of block 1308 for updating the domain consistency state indicator 126(0) includes updating the domain consistency state indicator 126(0) from DSO domain consistency state 206 to DSN domain consistency state 210 (block 1310). In some embodiments, the first LOP 122(0) may also transmit a DSO_NOW_CLEAN message 606 to other LOPs among the plurality of LOPs 122(0)-122(L), which indicates that consistency granule 112(0) has transitioned to DSN domain consistency state 210 (block 1312). It should be understood that the operation of block 1312 for transmitting DSO_NOW_CLEAN message 606 may include the first LOP 122(0) transmitting DSO_NOW_CLEAN message 606 directly to other LOPs, or may include the first LOP 122(0) transmitting DSO_NOW_CLEAN message 606 to SOP 124, and SOP 124 transmitting DSO_NOW_CLEAN message 606 to other LOPs among the plurality of LOPs 122(0)-122(L). In some embodiments, in response to SOP 124 observing that the first LOP 122(0) has cleared the consistency particle 112(0) to system memory 108 (i.e., while the first LOP 122(0) retains a copy of its consistency particle 112(0), the system memory 108 is updated with data stored in the copy of the consistency particle 112(0) of the first LOP 122(0)), SOP 124 may transmit the DSO_NOW_CLEAN message 606 to other LOPs among the plurality of LOPs 122(0)-122(L).

[0083] To illustrate exemplary operations for transitioning from DSE domain consistency state 202 to DSN domain consistency state 210 according to some embodiments, Figure 14 Flowchart 1400 is provided. For clarity, in the description... Figure 14 Referenced at the time Figure 1 , Figure 2 and Figure 7 The components. Below. Figure 14 The description will Figure 1 LOP 122(0) is called "first LOP 122(0)" and will Figure 1 The field 106(0) in the text is called "the first field 106(0)". Figure 1 PE 102(0) is referred to as "first PE 102(0)" and will Figure 1 The field 106(D) in the code is called "the third field 106(D)".

[0084] Figure 14The operation begins at the first LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112 is cached in the first domain 106(0) with DSE domain consistency state 202 (box 1402). (See above regarding...) Figure 9A As described in box 902, the first LOP 122(0) determines that the consistency granule 112(0) is not cached in a cached consistency state of 0 in any of the multiple domains 106(0)-106(D) (box 1404). Figure 14 In the example, the operation of box 1404 for determining that the consistency particle 112(0) is not cached with a cached consistency state in any of the domains 106(0)-106(D) includes the first LOP 122(0) determining that the first PE 102(0) holds an unmodified copy of the consistency particle 112(0) and that a shared copy 704 (box 1406) of the consistency particle 112(0) has been provided to a third domain 106(D) among the multiple domains 106(0)-106(D).

[0085] As mentioned above Figure 9A As described in box 904, the first LOP 122(0) then updates the domain consistency status indicator 126(0) for consistency granularity 112(0) (box 1408). Figure 14 In the example, the operation of box 1408 for updating the domain consistency status indicator 126(0) includes updating the domain consistency status indicator 126(0) from DSE domain consistency status 202 to DSN domain consistency status 210 (box 1410).

[0086] Figure 15 A flowchart 1500 according to some embodiments is provided, illustrating exemplary operations for transitioning from DSI domain consistency state 208 to DSN domain consistency state 210. For clarity, in the description Figure 15 Referenced at the time Figure 1 , Figure 2 and Figure 8 The components. Below. Figure 15 The description will Figure 1 LOP 122(0) is referred to as "first LOP 122(0)". Figure 1 PE102(0) in the above is called "first PE 102(0)".

[0087] In some embodiments, Figure 15The operation begins at the first LOP 122(0), which updates the domain consistency state indicator 126(0) for consistency granule 112(0) to indicate that consistency granule 112 has a DSI domain consistency state 208 (box 1502) within the first domain 106(0). (See above regarding...) Figure 9A As described in box 902, the first LOP 122(0) determines that the consistency granule 112(0) is not cached in a cached consistency state of 0 in any of the multiple domains 106(0)-106(D) (box 1504). Figure 15 In the example, the operation of block 1504 for determining that consistency particle 112(0) is not cached in O cache consistency state in any of the domains 106(0)-106(D) includes, in response to a request 804 by the first PE 102(0) for consistency particle 112(0), the first LOP 122(0) determines that there is no modified copy of consistency particle 112(0) in any of the multiple domains 106(0)-106(D) (block 1506).

[0088] As mentioned above Figure 9A As described in box 904, the first LOP 122(0) then updates the domain consistency status indicator 126(0) for consistency granularity 112(0) (box 1508). Figure 15 In the example, the operation of box 1308 for updating the domain consistency status indicator 126(0) includes updating the domain consistency status indicator 126(0) from DSI domain consistency status 208 to DSN domain consistency status 210 (box 1510).

[0089] Figure 16 It is an exemplary processor-based device 1600 (e.g. Figure 1 A block diagram of a processor-based device 1600 (DSN-based device 1600) that maintains a domain consistency state including a DSN domain consistency state. The processor-based device 1600 may be one or more circuits included in an electronic board, such as a printed circuit board (PCB), server, personal computer, desktop computer, laptop computer, personal digital assistant (PDA), computing board, mobile device, or any other device, and may represent, for example, a server or a user's computer. In this example, the processor-based device 1600 includes a processor 1602. The processor 1602 represents one or more general-purpose processing circuits, such as a microprocessor, central processing unit, etc., and may be integrated with... Figure 1The PEs 102(0)-102(P) correspond to this. Processor 1602 is configured to execute the processing logic in the instructions to perform the operations and steps discussed herein. In this example, processor 1602 includes instruction cache 1604, a temporary, fast-access memory for storing instructions, and instruction processing circuitry 1610. Instructions fetched or prefetched from memory (e.g., from system memory 1608 via system bus 1606) are stored in instruction cache 1604. Instruction processing circuitry 1610 is configured to process the instructions fetched into instruction cache 1604 and process the instructions for execution.

[0090] Processor 1602 and system memory 1608 are coupled to system bus 1606 and can interconnect with peripheral devices included in processor-based device 1600. Processor 1602 is known to communicate with these other devices by exchanging address, control, and data information on system bus 1606. For example, as an example of a peripheral device, processor 1602 can transmit bus transaction requests to memory controller 1612 in system memory 1608. Although Figure 16 Not shown, but multiple system buses 1606 may be provided, each forming a different architecture. In this example, memory controller 1612 is configured to provide memory access requests to memory array 1614 in system memory 1608. Memory array 1614 includes an array of storage bit cells for storing data. As a non-limiting example, system memory 1608 may be read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), etc., and static memory (e.g., flash memory, static random access memory (SRAM), etc.).

[0091] Other devices can be connected to system bus 1606. For example... Figure 16 As shown, as an example, these devices may include system memory 1608, one or more input devices 1616, one or more output devices 1618, modem 1624, and one or more display controllers 1620. The input devices 1616(s) may include any type of input device, including but not limited to input keys, switches, voice processors, etc. The output devices 1618 may include any type of output device, including but not limited to audio, video, other visual indicators, etc. The modem 1624 may be any device configured to allow data exchange with network 1626. Network 1626 may be any type of network, including but not limited to wired or wireless networks, private or public networks, local area networks (LANs), wireless local area networks (WLANs), wide area networks (WANs), and Bluetooth. TMNetworks and the Internet. Modem 1624 can be configured to support any type of communication protocol required. Processor 1602 can also be configured to access multiple display controllers 1620 via system bus 1606 to control information sent to one or more displays 1622. The multiple displays 1622 can include any type of display, including but not limited to cathode ray tube (CRT), liquid crystal display (LCD), plasma display, etc.

[0092] Figure 16 The processor-based device 1600 may include a set of instructions 1628, which are executed by the processor 1602 for any application as desired by the instructions. As an example of a non-transitory computer-readable medium 1630, the instructions 1628 may be stored in system memory 1608, the processor 1602, and / or the instruction cache 1604. The instructions 1628 may also reside wholly or at least partially in system memory 1608 and / or the processor 1602 during their execution. The instructions 1628 may also be transmitted or received on a network 1626 via a modem 1624, such that the network 1626 includes the computer-readable medium 1630.

[0093] Although computer-readable medium 1630 is shown as a single medium in the exemplary embodiments, the term "computer-readable medium" should be understood to include a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) storing one or more sets of instructions 1628. The term "computer-readable medium" should also be understood to include any medium capable of storing, encoding, or carrying a set of instructions for execution by a processing device and causing the processing device to perform any one or more methods of the embodiments disclosed herein. Therefore, the term "computer-readable medium" should include, but is not limited to, solid-state storage, optical media, and magnetic media.

[0094] The embodiments disclosed herein include various steps. These steps may be formed by hardware elements or embodied in machine-executable instructions that can be used by a general-purpose or special-purpose processor programmed with those instructions to perform the steps. Alternatively, these steps may be performed by a combination of hardware and software processes.

[0095] The embodiments disclosed herein may be provided as a computer program product or software process, which may include a machine-readable medium (or computer-readable medium) having instructions stored thereon that can be used to program a computer system (or other electronic device) to perform processes according to the various embodiments disclosed herein. A machine-readable medium includes any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form. For example, machine-readable media include machine-readable storage media (e.g., ROM, random access memory (“RAM”), disk storage media, optical storage media, flash memory devices, etc.).

[0096] Unless otherwise specifically stated and apparent from the foregoing discussion, it is understood that throughout the description, discussions using terms such as “processing,” “computing,” “determining,” and “displaying” refer to actions and processes of computer systems or similar electronic computing devices that manipulate data and memory in computer system registers, represented as physical (electronic) quantities, and convert them into other data or other such information storage, transmission, or display devices similarly represented as physical quantities in computer system memory or registers.

[0097] The algorithms and displays presented herein are not inherently associated with any particular computer or other device. Various systems can be used with the programs taught herein, or it may prove convenient to construct more specialized devices to perform the required method steps. The required structures for various such systems can be seen from the description above. Furthermore, the embodiments described herein are not described with reference to any particular programming language. It will be appreciated that the teachings of the embodiments described herein can be implemented using various programming languages.

[0098] Those skilled in the art will further understand that the various illustrative logic blocks, modules, circuits, and algorithms described in connection with the embodiments disclosed herein can be implemented as electronic hardware, stored in memory or another computer-readable medium and executed by a processor or other processing device, or a combination of both. As an example, components of the distributed antenna system described herein can be used in any circuit, hardware element, integrated circuit (IC), or IC chip. The memory disclosed herein can be of any type and size and can be configured to store any type of information as required. To clearly illustrate this interchangeability, various illustrative components, blocks, modules, circuits, and steps have been described above in general terms of their functionality. How this functionality is implemented depends on the specific application, design choices, and / or design constraints imposed on the entire system. Those skilled in the art can implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this embodiment.

[0099] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented or executed using a processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, which may include hardware components, discrete hardware components, or any combination thereof designed to perform the functions described herein. Furthermore, the controller can be a processor. The processor can be a microprocessor, but alternatively, the processor can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration).

[0100] The embodiments disclosed herein can be embodied in hardware and instructions stored in the hardware, and can reside in, for example, RAM, flash memory, ROM, electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CD-ROMs, or any other form of computer-readable medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read information from and write information to the storage medium. Alternatively, the storage medium can be integrated into the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in a remote station. Alternatively, the processor and storage medium can reside as discrete components in a remote station, base station, or server.

[0101] It should also be noted that the operational steps described in any exemplary embodiment herein are described as being for the purpose of example and discussion. The described operations can be performed in many different sequences than those shown. Furthermore, the operations described in a single operational step can actually be performed in multiple different steps. Additionally, one or more operational steps discussed in the exemplary embodiments can be combined. Those skilled in the art will also understand that information and signals can be represented using any of a variety of techniques and technologies. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the foregoing description can be represented by voltage, current, electromagnetic waves, magnetic fields or particles, light fields or particles, or any combination thereof.

[0102] Unless otherwise expressly stated, it is by no means intended to interpret any method set forth herein as requiring its steps to be performed in a particular order. Therefore, if a method claim does not actually enumerate the order in which its steps are followed, or if the claims or description do not otherwise specify that the steps will be restricted to a particular order, it is by no means intended to infer any particular order.

[0103] Various modifications and variations can be made by those skilled in the art without departing from the spirit or scope of the invention. Since modifications, combinations, sub-combinations, and variations of the disclosed embodiments in conjunction with the spirit and essence of the invention may occur to those skilled in the art, the invention should be construed as including all contents within the scope of the appended claims and their equivalents.

Claims

1. A processor-based device, comprising: System sorting point circuit SOP; System memory; Multiple processing elements PE; as well as Multiple domains, each domain including: One or more of the plurality of PEs; and A local sorting point circuit (LOP) communicatively coupled to the one or more PEs and the SOP; in: The first LOP of the first domain in the plurality of domains is configured as follows: Determining that the consistent particle is not cached in any of the plurality of domains in a cached consistent state with 0 cache values, the consistent particle is cached as a first cache copy in the local cache of the first PE in the first domain of the one or more PEs; and Update the domain consistency status indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state without owning the DSN domain consistency status.

2. The processor-based device of claim 1, wherein the SOP is configured as follows: Receive a request for read access to the consistency granules from the second of the plurality of domains; Determine that the consistency granules are cached within the first domain in the DSN domain consistency state; and The consistency granules are read from the system memory to satisfy the read access.

3. The processor-based device according to claim 1, wherein: The first LOP is also configured to: update the domain consistency state indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state sharing DSS domain consistency state; The first LOP is configured to determine that the consistency granules are not cached in any of the plurality of domains in the state of having a DSO domain consistency state by being configured to determine that no domain among the plurality of domains has a DSO domain consistency state cached the consistency granules in the domain state. and The first LOP is configured to update the domain consistency status indicator for the consistency granules by updating the domain consistency status indicator from the DSS domain consistency status to the DSN domain consistency status.

4. The processor-based device of claim 3, wherein the first LOP is configured to: determine, based on one or more snoop responses, that no domain among the plurality of domains caches the consistency granules in a DSO domain consistency state, the one or more snoop responses being sent in response to a request for read access to the consistency granules and observed by the first LOP, the request being sent by a second PE among the plurality of PEs.

5. The processor-based device according to claim 3, wherein: The SOP is configured as follows: Observe one or more snooping responses, which are sent in response to a request for read access to the consistency granules, the request being sent by a second PE among the plurality of PEs; and Based on the one or more snooping responses, it is determined that none of the plurality of domains caches the consistency granules in a DSO domain consistency state; as well as Send an indication to the first LOP that no domain among the plurality of domains caches the consistency granules in the DSO domain consistency state; and The first LOP is configured to: determine, based on the indication, that no domain among the plurality of domains caches the consistency granules in a DSO domain consistency state.

6. The processor-based device according to claim 1, wherein: The first LOP is also configured to: update the domain consistency state indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state sharing DSS domain consistency state; The first LOP is configured to determine that the consistency granules are not cached in the O cache consistency state in any of the plurality of domains by being configured to determine that at least one of the plurality of domains caches the consistency granules in the DSN domain consistency state; and The first LOP is configured to update the domain consistency status indicator for the consistency granules by updating the domain consistency status indicator from the DSS domain consistency status to the DSN domain consistency status.

7. The processor-based device of claim 6, wherein the first LOP is configured to: determine, based on one or more snoop responses, that no domain among the plurality of domains caches the consistency granules in a DSN domain consistency state, the one or more snoop responses being sent in response to a request for read access to the consistency granules and observed by the first LOP, the request being sent by a second PE among the plurality of PEs.

8. The processor-based device according to claim 6, wherein: The SOP is configured as follows: Observe one or more snooping responses, which are sent in response to a request for read access to the consistency granules, the request being sent by a second PE among the plurality of PEs; and Based on the one or more snooping responses, at least one of the plurality of domains is determined to cache the consistency granules with the DSN domain consistency state; as well as Send an indication to the first LOP of at least one of the plurality of domains to cache the consistency granules in the consistency state of the DSN domain; and The first LOP is configured to: determine, based on the indication, that no domain among the plurality of domains has a DSO domain consistency state cache of the consistency granules.

9. The processor-based device according to claim 1, wherein: The first LOP is also configured to: update the domain consistency state indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state sharing DSS domain consistency state; The first LOP is configured to receive a DSO_NOW_CLEAN message from a third of the plurality of domains to determine that the consistency particle is not cached in any of the plurality of domains in the state of having a cached state of having 0, wherein the DSO_NOW_CLEAN message indicates that the consistency particle having a domain state of having a DSO domain consistency state in the third domain has transitioned to the DSN domain consistency state. and The first LOP is configured to update the domain consistency status indicator for the consistency granules by updating the domain consistency status indicator from the DSS domain consistency status to the DSN domain consistency status.

10. The processor-based device according to claim 1, wherein: The first LOP is also configured to: update the domain consistency state indicator for the consistency granule to indicate that the consistency granule has the DSO domain consistency state cached in the first domain. The first LOP is configured to determine that the consistency granule is not cached in the O cache consistency state in any of the plurality of domains by being configured to write the first cache copy to the consistency granule in the system memory; and The first LOP is configured to update the domain consistency status indicator for the consistency granularity by updating the domain consistency status indicator from the DSO domain consistency status to the DSN domain consistency status.

11. The processor-based device of claim 10, wherein: The first LOP is also configured to send a DSO_NOW_CLEAN message to other LOPs, the DSO_NOW_CLEAN message indicating that the consistency granule has transitioned to the DSN domain consistency state.

12. The processor-based device according to claim 1, wherein: The first LOP is also configured to: update the domain consistency state indicator for the consistency granule to indicate that the consistency granule has exclusive domain state DSE domain consistency state cached within the first domain; The first LOP is configured to determine that the consistent particle is not cached in the O cached consistent state in any of the plurality of domains by being configured to determine that the first PE holds an unmodified copy of the consistent particle and has provided a shared copy of the consistent particle to a third domain among the plurality of domains; and The first LOP is configured to update the domain consistency status indicator for the consistency granularity by updating the domain consistency status indicator from the DSE domain consistency status to the DSN domain consistency status.

13. The processor-based device according to claim 1, wherein: The first LOP is also configured to: update the domain consistency state indicator for the consistency granule to indicate that the consistency granule has a domain state invalid DSI domain consistency state within the first domain; The first LOP is configured to determine that the consistency particle is not cached in the O cache consistency state in any of the plurality of domains by being configured to determine, in response to a request from the first PE for the consistency particle, that no domain in the plurality of domains caches a modified copy of the consistency particle. and The first LOP is configured to update the domain consistency status indicator for the consistency granularity by updating the domain consistency status indicator from the DSI domain consistency status to the DSN domain consistency status.

14. The processor-based device of claim 1, wherein the first LOP is further configured to: Receive a request from the first PE to update the cache consistency state of the first cache copy for the consistency granules from a shared S cache consistency state to an exclusive E cache consistency state; and Update the domain consistency state indicator for the consistency granule to indicate that the consistency granule has exclusive domain state DSE domain consistency state cached in the first domain.

15. The processor-based device of claim 1, wherein the first LOP is further configured to: It is determined that the first cache copy of the consistency granule held by the first PE is the last remaining copy of the consistency granule in the first domain; It has been determined that the first PE has evicted the first cache copy; as well as Update the domain consistency state indicator for the consistency granule to indicate that the consistency granule is cached in the first domain with a domain-state invalid DSI domain consistency state.

16. A method comprising: The first local sort point circuit (LOP) of the processor-based device determines that the consistency particle is not cached in any of the multiple domains in a cache-coherent state with 0 caches. The first LOP is located in a first domain of the multiple domains, each of the multiple domains including one or more processing elements (PEs) of a plurality of processing elements (PEs). The consistency particle is cached as a first cache copy in the local cache of the first PE of the one or more PEs in the first domain. as well as Update the domain consistency status indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state without owning the DSN domain consistency status.

17. The method of claim 16, further comprising: The system sorting point circuit (SOP) of the processor-based device receives a request for read access to the consistency granules from a second of the plurality of domains. The SOP determines that the consistency granules are cached in the first domain in the DSN domain consistency state; as well as The consistency granules are read from system memory to satisfy the read access.

18. The method of claim 16, wherein: The method further includes: updating the domain consistency state indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state sharing DSS domain consistency state; Determining that no domain among the plurality of domains has a DSO domain consistency state cached in the consistency granules is configured to determine that the consistency granules are not cached in the state of having a DSO cache consistency state in any of the plurality of domains; and The domain consistency status indicator is configured to update the domain consistency status indicator for the consistency granularity by updating the domain consistency status indicator from the DSS domain consistency status to the DSN domain consistency status.

19. The method of claim 18, wherein the first LOP determines, based on one or more snooping responses, that no domain among the plurality of domains caches the consistency granules in a DSO domain consistency state, the one or more snooping responses being sent in response to a request for read access to the consistency granules and observed by the first LOP, the request being sent by a second PE among the plurality of PEs.

20. The method of claim 18, further comprising: One or more snoop responses are observed by the SOP, the one or more snoop responses being sent in response to a request for read access to the consistency granules, the request being sent by a second PE among the plurality of PEs; and Based on the one or more snoop responses, the SOP determines that none of the plurality of domains caches the consistency granules in a DSO domain consistency state; and The SOP sends an indication to the first LOP that none of the plurality of domains caches the consistency granules in the DSO domain consistency state; Specifically, the first LOP determines, based on the indication, that no domain among the plurality of domains caches the consistency granules in a DSO domain consistency state.

21. The method of claim 16, wherein: The method further includes: updating the domain consistency state indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state sharing DSS domain consistency state; Determining that at least one of the plurality of domains caches the consistency granules in the DSN domain consistency state is configured to determine that the consistency granules are not cached in the O cache consistency state in any of the plurality of domains; and The domain consistency status indicator is configured to update the domain consistency status indicator for the consistency granularity by updating the domain consistency status indicator from the DSS domain consistency status to the DSN domain consistency status.

22. The method of claim 21, wherein the first LOP determines, based on one or more snooping responses, that no domain among the plurality of domains caches the consistency granules in a DSN domain consistency state, the one or more snooping responses being sent in response to a request for read access to the consistency granules and observed by the first LOP, the request being sent by a second PE among the plurality of PEs.

23. The method of claim 21, further comprising: One or more snoop responses are observed by the SOP, the one or more snoop responses being sent in response to a request for read access to the consistency granules, the request being sent by a second PE among the plurality of PEs; and Based on the one or more snoop responses, the SOP determines at least one of the plurality of domains to cache the consistency granules in the DSN domain consistency state; and The SOP sends an indication to the first LOP that at least one of the plurality of domains is cached with the consistency granules of the DSN domain consistency state; The first LOP determines, based on the indication, that none of the plurality of domains has a DSO domain consistency state cache of the consistency granules in a domain state.

24. The method of claim 16, wherein: The method further includes: updating the domain consistency state indicator for the consistency granule to indicate that the consistency granule is cached within the first domain as a domain state-shared DSS domain consistency state. To receive a DSO_NOW_CLEAN message from a third domain among the plurality of domains, the configuration is to determine that the consistency particle is not cached in the O cache consistency state in any of the plurality of domains, the DSO_NOW_CLEAN message indicating that the consistency particle having a domain state with DSO domain consistency state in the third domain has transitioned to the DSN domain consistency state; and The domain consistency status indicator is configured to update the domain consistency status indicator for the consistency granularity by updating the domain consistency status indicator from the DSS domain consistency status to the DSN domain consistency status.

25. The method according to claim 16, wherein: The method further includes: updating the domain consistency state indicator for the consistency granule to indicate that the consistency granule has a DSO domain consistency state cached in the first domain. Writing the first cache copy to the consistency granule in system memory is configured to determine that the consistency granule is not cached in the state of having 0 cache consistency in any of the plurality of domains; and The domain consistency status indicator is updated from the DSO domain consistency status to the DSN domain consistency status to be configured to update the domain consistency status indicator for the consistency granularity.

26. The method of claim 25, wherein the method further comprises: Send a DSO_NOW_CLEAN message to other LOPs, the DSO_NOW_CLEAN message indicating that the consistency granule has transitioned to the DSN domain consistency state.

27. The method of claim 16, wherein: The method further includes: updating the domain consistency state indicator for the consistency granule to indicate that the consistency granule exclusively caching the DSE domain consistency state within the first domain. The determination that the first PE holds an unmodified copy of the consistent particle and has provided a shared copy of the consistent particle to a third domain among the plurality of domains is configured to determine that the consistent particle is not cached in the state of having 0 cache consistency in any of the plurality of domains; and The domain consistency status indicator is updated from the DSE domain consistency status to the DSN domain consistency status to be configured to update the domain consistency status indicator for the consistency granularity.

28. The method of claim 16, wherein: The method further includes: updating the domain consistency state indicator for the consistency particle to indicate that the consistency particle has a domain state invalid DSI domain consistency state in the first domain; In response to a request from the first PE regarding the consistent granule, determining that no domain among the plurality of domains caches a modified copy of the consistent granule, is configured to determine that the consistent granule is not cached in the state of having 0 cache consistency in any of the plurality of domains; and The domain consistency status indicator is configured to update the domain consistency status indicator for the consistency granularity by updating the domain consistency status indicator from the DSI domain consistency status to the DSN domain consistency status.

29. The method of claim 16, wherein the method further comprises: Receive a request from the first PE to update the cache consistency state of the first cache copy for the consistency granules from a shared S cache consistency state to an exclusive E cache consistency state. as well as Update the domain consistency state indicator for the consistency granule to indicate that the consistency granule has exclusive domain state DSE domain consistency state cached in the first domain.

30. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed, cause a processor device to: It is determined that the consistent particle is not cached in any domain of multiple domains in a cached consistent state with 0 caches, and the consistent particle is cached as a first cache copy in the local cache of a first PE in one or more processing elements (PEs) of a first domain; and Update the domain consistency status indicator for the consistency granule to indicate that the consistency granule is cached in the first domain as a domain state without owning the DSN domain consistency status.

Images