Method, system and apparatus for enforcing cache consistency

Abstract

Example embodiments relate to a cache coherence protocol as applied to a range of memory blocks. A directory may track exclusive ownership of a range of memory blocks in a default sharing state. The directory may be associated with a first processor in a set of processors. When a request is received from a second processor in the set of processors to read one or more memory blocks not present in the directory, the one or more blocks may be transferred to the second processor in the default sharing state. Blocks not present in the directory may not be tracked in the directory.

Description

Technical Field

[0001] The present disclosure generally relates to the application of a default shared-state cache coherence protocol. Background Art

[0002] In a multiprocessor shared memory system, data can be shared and accessed between multiple processors. Multiple processors employ cache coherence. Cache coherence is employed to ensure that changes to shared data, or changes to copies of shared data, are propagated throughout the system so that each processor accesses uniform or consistent data. To this end, multiprocessor shared memory systems use hardware or software-based protocols and mechanisms to enforce cache coherence to arbitrate access to shared memory and enable processors to read and / or write the same data. Summary of the invention

[0003] According to one aspect of the present disclosure, a method for enforcing cache consistency is provided, comprising: tracking, by a directory associated with a first processor in a group of processors, exclusive ownership of a range of blocks of a memory region when a memory region is in a default shared state; in response to receiving a request from a second processor in the group of processors to read one or more blocks of memory that do not exist in the directory, transferring the one or more blocks in the default shared state to the second processor in a shared state without tracking the non-existent blocks in the directory, and tracking the one or more blocks of the memory region in a cache of the directory when the memory region is in a default invalid state, wherein the one or more blocks of memory are marked as shared in the directory cache; and in response to receiving a request from a second processor in the group of processors to read one or more blocks of memory that do not exist in the directory, tracking the one or more blocks that do not exist in the directory and transferring the one or more non-existent blocks to the second processor, and upon determining that a threshold is met regarding the memory region in the default invalid state, changing the state of the memory region to the default shared state. According to another aspect of the present disclosure, a system for enforcing cache coherence is provided, comprising: a memory, the memory comprising a first memory block range in a default shared state and a different second memory block range in a default invalid state; a directory, the directory being used to track exclusive ownership of the first memory block range; and a node controller associated with the memory, wherein the node controller, in response to receiving a request from a processor to read one or more blocks of memory in the first range, applies a shared state protocol to: determine whether the one or more blocks of memory in the first range are tracked in the directory; and in response to determining that the one or more blocks of memory do not exist in the directory, transmit the one or more blocks of memory to the processor without being in the directory The invention relates to a method for tracking non-existent blocks in a directory, and wherein the node controller, in response to receiving a request from a second processor to read one or more blocks of memory in a second range, applies an invalid state protocol to: determine whether the one or more blocks of memory in the second range are tracked in the directory; in response to determining that the one or more blocks of memory in the second range do not exist in the directory, track the one or more blocks of memory in the second range in the directory, wherein the one or more blocks of memory are marked as shared in a cache of the directory; and transmit the one or more shared blocks to the processor, wherein the memory further includes a third range of memory blocks in a dynamic state, wherein the register can flip the state of the memory blocks in the third range in response to a threshold.

[0004] According to another aspect of the present disclosure, a device for enforcing cache consistency is provided, comprising: a node controller, the node controller is associated with a first processor and a memory in a group of processors, and is operable to enforce cache consistency of blocks of the memory, wherein the node controller, when in a default shared state: tracks exclusive ownership of a subset of blocks of the memory in a directory of the node controller; and in response to receiving a request from a second processor in the group of processors to read one or more blocks of the memory that do not exist in the directory, transfers the one or more blocks to the second processor in a shared state without tracking the non-existent blocks in the directory, and wherein the node controller, when in a default invalid state: tracks the one or more blocks of the memory in a cache of the directory, wherein the one or more blocks of the memory are marked as shared in the directory cache; and in response to receiving a request from a second processor in the group of processors to read the one or more blocks of the memory that do not exist in the directory, tracks the one or more blocks that do not exist in the directory, and transfers the one or more blocks that do not exist to the second processor, wherein the memory comprises a plurality of memory regions, and further comprises a register for determining a protocol to be applied to a memory region in the plurality of memory regions, wherein the register changes the state of the particular memory region to a default shared state when determining that a threshold is satisfied with respect to the particular memory region in the default invalid state. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Certain examples are described in the following detailed description with reference to the accompanying drawings, in which:

[0006] Figure 1 is an example block diagram of a cache-coherent computing system formed by various interconnected computing components.

[0007] FIG. 2a is a sequence diagram illustrating an example cache coherence protocol.

[0008] FIG. 2b is a sequence diagram further illustrating an example cache coherence protocol.

[0009] Figure 3 is a block diagram illustrating an example node controller for enforcing one or more cache coherence protocols.

[0010] Figure 4 is a block diagram illustrating an example system for enforcing one or more cache coherence protocols.

[0011] Figure 5 is a flow chart illustrating a method for implementing various cache coherence protocols.

[0012] Figure 6 is a flow chart illustrating a method for switching a memory region from a default shared state protocol to a default invalid state protocol.

[0013] Figure 7 is a circuit diagram illustrating switching a protocol applied to a dynamic memory region from a default shared state protocol to a default inactive state protocol using a scoreboard. DETAILED DESCRIPTION

[0014] In the foregoing description, many details are set forth to enable an understanding of the subject matter disclosed herein. However, embodiments may be practiced without some or all of these details. Other embodiments may include modifications and variations of the details discussed above. The appended claims are intended to cover such modifications and variations.

[0015] The present disclosure provides efficient management and enforcement of cache coherence using a partial directory protocol to ensure that processors sharing a memory can access uniform or consistent data. Cache coherence can be enforced using hardware or software-based mechanisms and protocols. For example, a hardware-based approach uses hardware components to enforce cache coherence in a multi-processor shared memory system. As known to those skilled in the art, these methods may include directory-based or snooping protocols.

[0016] Among other things, directory-based protocols use a consistency directory to track state and ownership information for memory blocks in shared memory, such as whether a shared memory block is shared with a processor and, if so, with which processors. When a request is made to process a consistency operation, such as a read or write to a memory block of shared memory, the state and ownership information of the directory is used to consistently process the request. This can include managing various communications between the processor and other computing components, such as to share state and ownership information and implement operations. Snoop-based protocols enforce cache consistency by broadcasting messages to processors that share the memory, indicating when a consistency operation on the shared memory is requested or processed. For example, these messages may cause the processor to update or invalidate its copy of the shared data in the shared memory. In this way, the processor can avoid processing inconsistent copies of data.

[0017] Figure 1An exemplary embodiment of a cache coherent computing system 100 formed by various interconnected computing components is illustrated. In some embodiments, the system 100 is a multi-processor system that includes a shared memory 110 accessible by some or all of the processors 102-108. As described in further detail below, each processor 102-108 may be associated with a corresponding memory block range (also referred to herein as a memory block region) 112-118 (collectively referred to herein as "memory 110"). Each processor may further correspond to a node controller 122-128, respectively. Each node controller 122-128 may be associated with a corresponding memory block range A 112-D 118, respectively, so that the directory of the node controller can track a subset of blocks of the memory block range associated with the node controller. In this example, for the sake of clarity and brevity, each node controller is associated with a single corresponding memory block range and a single processor. However, in other examples, the node controller may be associated with multiple memory block ranges and / or multiple processors.

[0018] Each of the processors 102-108 may be or refer to one or more processing resources, nodes, or units configured to execute instructions. For example, each of the processors 102-108 may be or refer to one or more central processing units (CPUs), dual-core or multi-core processors or processor sockets including two or more CPUs, a single core of a processor, a computing cluster, a cloud server, etc. In some embodiments, two or more of the processors 102-108 (e.g., processor 102 and processor 104) may be communicatively coupled using a point-to-point interconnect or bus.

[0019] In addition, each of the processors 102-108 may include or be associated with one or more processor caches 132-138 for storing copies of data stored in the shared system memory, thereby enabling the processor to access the data more quickly. It should be understood that in some embodiments, multiple processors or cores may share a single cache. It should also be understood that each cache may be physically arranged on the same or separate chip or component as its corresponding processor. Since a processor accesses data from its cache faster than from system memory, caches 132-138 may be used to store copies of data stored in the system memory, which data, for example, is frequently or immediately accessed or is more likely to be frequently or immediately accessed, and / or can be accessed by the processor with greater efficiency.

[0020] like Figure 1As shown, processors 102-108 are communicatively coupled to memory block ranges A 112-D 118, respectively. In some examples, memory block ranges 112-118 form a shared system memory that can be accessed by all or some of processors 102-108. It should be understood that while in some embodiments, each of processors 102-108 may be communicatively coupled to one or more of memory block ranges 112-118 and / or access data therein, each of processors 102-108 may be directly associated with a corresponding one of memory block ranges 112-118. This correspondence or association between each of processors 102-108 and one of memory block ranges 112-118 may result from a direct attachment or connection therebetween. For example, processor 102 may be directly attached to memory block range 112a, as shown in FIG. Figure 1 As shown. In some embodiments, a memory block range (e.g., memory block range 112a) can be considered "local" to its corresponding (e.g., directly connected) processor (e.g., processor 102) and can be referred to as "local memory" relative to its corresponding processor. On the other hand, with respect to other processors (e.g., processor 104) other than its local processor, a memory block range (e.g., memory block range 112a) can be considered "remote" and can be referred to as "remote memory".

[0021] The processors 102-108 (and / or memory controllers of the processors 102-108) may communicate with each other via one or more memory channels and / or buses (such as a peripheral component interconnect (PCI) bus, an industry standard architecture (ISA) bus, a PCI Express (PCIe) bus) and high-performance links (such as A direct media interface (DMI) system, QuickPath Interconnect, HyperTransport, double data rate (DDR), SATA, SCSI or Fibre Channel bus, etc.) is communicatively coupled (e.g., directly connected) to its corresponding range of memory blocks 112-118.

[0022] Each of the memory block ranges 112-118 may include or be composed of any number of memory devices, which may be or include volatile memory devices (e.g., random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM)) and / or non-volatile memory devices (e.g., non-volatile RAM (NVRAM), double data rate 4 synchronous dynamic RAM (DDR4 SDRAM)). Other types of memory devices that may be used include read-only memory (ROM) (e.g., mask ROM, programmable ROM (PROM), erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM)), flash memory, memristor devices, and the like.

[0023] In addition, each of the memory block ranges 112-118 can be used to store software such as an operating system (OS), a hypervisor, and other applications. The software stored on the memory 110 consists of processes and / or threads that can be executed simultaneously and share resources such as memory (e.g., memory 110) and processors (e.g., processors 102-108). The processes and / or threads, when executed, can cause requests and responses to be transmitted between processors 102-108 (and / or node controllers 122-128). As described in further detail below, in some embodiments, the software stored in the memory 110 can be used to provide cache coherence (e.g., in a software-managed cache coherence implementation).

[0024] As described above, memory block ranges 112-118 may form a shared system memory 110. Some or all of processors 102-108 may access data stored in system memory 110. For example, data stored in a memory block range (e.g., memory block range 112) that is shared and therefore part of system memory 110 may be accessed by processors other than the processor local to memory block range 112 (e.g., processor 104). It should be understood that permissions may be used to control access to all or part of system memory 110 and / or access by all or some of processors 102-108. It should be understood that for the purpose of simplicity, unless otherwise indicated, references herein to any or all of memory 110 shall refer to being shared and forming part of system memory 110, but in some embodiments, memory block ranges 112-118 may include non-shared areas that are accessed only by their respective local processors.

[0025] In some embodiments, access to and / or management of (e.g., cache coherent) memory 110 may be provided or handled by node controllers 122-128. In this regard, it should be understood that processors (e.g., processors 102-108) and their corresponding memory block ranges (e.g., memory block ranges 112-118) are considered to be local to the node controllers associated with them (e.g., directly connected or most closely connected). Node controllers 122-128 are therefore configured to provide and / or perform various functions on behalf of, for, or associated with their local or corresponding computing resources (e.g., memory, processors). Such functions may include or relate to cache coherence, routing, load balancing, fault recovery, etc. In some embodiments, some or all of the functions of each of the node controllers 122-128 described herein may be implemented in one or more of the processors 102-108.

[0026] The node controllers 122-128 (also referred to as "hubs" or "interconnect agents") are computing devices or components that may be or include a general purpose processor (e.g., a microprocessor, a conventional processor, a controller, a microcontroller, a state machine, a sequencer), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein, etc. It should be understood that the computing system 100 may include any number of node controllers. As described in further detail below, the node controllers 122-128 may provide, participate in, or otherwise facilitate cache coherence (e.g., enforce a cache coherence protocol) using hardware-based and software-based approaches.

[0027] although Figure 1 Not shown, each of the node controllers 122-128 may include one or more memories or memory devices, such as ROM, RAM, EEPROM, flash memory, registers, or any other form of storage medium or element known to those skilled in the art. In some embodiments, cache coherence may be directory-based, such that the memories or storage devices of the node controllers 122-128 may be used to store information such as status and ownership associated with memory blocks in their local memory. However, it should be understood that, as described herein, status and ownership information may be additionally or alternatively stored in the memory 110.

[0028] like Figure 1As shown in the exemplary diagram, each node controller 122-128 is communicatively coupled and / or associated with a processor 102-108, respectively, and a memory block range 112-118, respectively. It should be understood that each of the node controllers 122-128 can be communicatively coupled and / or associated with any number of local processors and / or memories. In some embodiments, the node controllers 122-128 are communicatively coupled and / or connected to their respective local processors via an interconnect such as a UPI link.

[0029] Each of the node controllers 122-128 can be configured to provide cache consistency so that changes to data stored in the shared memory can be propagated throughout the system 100 to ensure that the processors 102-108 can access a unified version of the shared data. The node controllers 122-128 can provide cache consistency by implementing a cache consistency scheme or protocol based on a directory or snooping. In this regard, in some embodiments, each of the node controllers 122-128 may include a snooping or request accelerator and / or a consistency controller configured to store information for cache consistency functions and perform cache consistency functions. The node controller (and / or its request accelerator and consistency controller) may be or include computer hardware (e.g., a sequencer, a state machine), such as a circuit including logic and a memory device or storage element. The logic is configured to implement or execute one or more of the functions, processes, and / or methods described herein, including those functions, processes, and / or methods for providing or managing cache consistency. The memory device or storage element may include a cache and may be used to store a directory.

[0030] In order to provide directory-based cache consistency, the node controller 122-128 uses a consistency directory 142-148 to track and / or store the state and ownership of all or a subset of memory blocks in its local memory. Although the consistency directory 142-148 is illustrated as being stored in the node controller 122-128 for clarity and brevity, the consistency directory can be stored in other components of the system 100, such as on the memory blocks 112-118, or stored remotely to be accessed, for example, over a network. By storing the directory in the node controller 122-128, in some cases the node controller can efficiently access state and ownership information without having to obtain the information from the directory within the memory block range 112-118. In some embodiments, the node controller 122-128 may include a consistency directory cache, which can be used to store copies of consistency directory entries from directories within the memory block 112-118 range. As with storing the consistency directory in the node controller 122-128, the use of the consistency directory cache allows the node controller 122-128 to access state and ownership information more efficiently.

[0031] In order to provide cache coherence based on snooping, the node controller 122-128 can process the snooping message. As known to those skilled in the art, the snooping message can be broadcast to the node controller having a copy of the memory block to which a read or write request has been made. In some embodiments, the node controller 122-128 can store information or counters of the number of memory blocks in the memory area shared (e.g., active) or not shared (e.g., invalid) with the processor or processor cache. In the case where the memory block to be accessed is shared, this information can be used to determine whether it is necessary to broadcast the snooping request to other node controllers or processors.

[0032] As described above, the directory can track ownership of blocks in a range of memory blocks to ensure cache coherence. In some cases, the directory can track which processor has exclusive ownership of a memory block, and / or can track shared ownership of a memory block (i.e., a memory block that is not exclusively owned by any single processor or otherwise cached in the system). A processor can be considered to have exclusive ownership of a memory block if it has exclusive read and write capabilities to the owned memory block. In some example embodiments, only one processor may have exclusive ownership of a memory block in the system at a given time.

[0033] In the case where the processor has non-exclusive read capability to the memory block, the processor may be considered to have shared ownership of the memory block. According to the cache coherence protocol, the processor may cache a copy of the shared memory block in a cache associated with the processor or otherwise local to the processor. In some example embodiments, any number of processors 102-108 may have cached shared copies of the memory block at the same time.

[0034] In some example systems, the directory may track only a subset of the total cache coherent memory space of the memory. A directory that tracks only a subset of the total cache coherent memory space may be referred to herein as a "partial directory." In a system that implements a partial directory, a protocol for handling memory blocks that are not tracked by the partial directory may be implemented. An example protocol (referred to herein as a default invalid state protocol) may include the partial directory invalidating and / or flushing memory blocks in each processor cache when the directory discards the memory block (i.e., the directory no longer tracks the memory block).

[0035] As an illustrative example of the default invalid state protocol, the processor 102 may request shared ownership of the first memory block, for example, a read request for a memory block in the memory block range B 114. The partial directory (e.g., directory 144) of the node controller 124 may determine that the requested first memory block is not in the partial directory's cache or is otherwise not tracked by the partial directory, whereupon the partial directory may add the first memory block to its cache (not shown). The partial directory may further track the requested first memory block when the requested first memory block is in a shared state with the processor 102 having a copy.

[0036] The processor 106 may request shared ownership of a second memory block in memory block range B 114 that is different from the first memory block. The partial directory may determine that the requested second memory block is not in the partial directory's cache, or is otherwise not tracked by the partial directory, whereupon the partial directory may add the second memory block to its cache. However, the partial directory may determine that there is not enough space in the partial directory to track both the first memory block and the second memory block, and therefore may evict the first memory block to make room. The partial directory may further snoop to invalidate the first memory block in the cache of the processor 102.

[0037] Therefore, the default invalid state protocol introduces some tradeoffs. Partial directories may discard memory blocks and invalidate memory blocks in processor caches that currently have a copy of the memory block and may still be using the memory block. Therefore, in order for the processor 102 to reclaim the memory block, the processor 102 may request access to the memory block again. This may increase latency and increase traffic in the network, resulting in lower system performance relative to the proposed alternative protocols discussed herein.

[0038] Another example protocol to be implemented with a partial directory (as will be further described below and referred to herein as a default shared state protocol) may include a partial directory treating a memory block that is not currently tracked by the directory as being in a shared state. In other words, a processor requesting to read a memory block that is not tracked by the directory will receive a shared copy of the data. Although the memory block is cached by the requesting processor, the directory may not track the memory block because the processor only keeps sharing, rather than exclusive ownership. A node controller that is considered to be in a default invalid state can implement a "default invalid protocol", also referred to herein as a "default invalid state protocol". Similarly, a node controller that is considered to be in a default shared state can implement a "default shared protocol", also referred to herein as a "default shared state protocol". In an example embodiment, registers 152-158 can be implemented at node controllers 122-128, respectively, to indicate the state of the node controller. Specifically, registers 152-158 can specify the protocol that the corresponding node controller 122-128 is to apply to a specific memory area (also referred to herein as a memory block range, for example, a memory block range 112-118).

[0039] The node controllers 122-128 may be communicatively coupled and / or interconnected to each other via a structure 101 (or "structure interconnect"), and the processors 102-108 may communicate with each other via the structure. The structure 101 may include one or more direct and / or switched interconnects. For example, in some embodiments, the structure 101 may include direct interconnections between the node controllers 122-128 (e.g., to minimize latency). Thus, the structure 101 may be used to transmit data and / or messages between or among one or more of the node controllers 122-128 and / or the processors 102-108. Such communications may include, for example, memory access requests (e.g., reading from or writing to a memory block or cache block), state change requests (e.g., invalidating), confirmations, and other messages known to those skilled in the art and as needed by the applicable cache coherence protocol.

[0040] It should be understood that the components of system 100 (e.g., node controllers 122-128, processors 102-108, and memory 110) may be physically housed together or separately. Figure 1 The exemplary components of the system 100 shown may be provided in a single chassis or in multiple chassis. As described above, the system 100 may include Figure 1Any number of node controllers, processors and / or memories other than those shown may be housed together or separately. In addition, in some embodiments, system 100 may include additional systems or subsystems formed by node controllers, processors and / or memories and communicatively coupled to structure 101, through which communication with node controllers 122-128 and / or processors 102-108 may be achieved.

[0041] Figure 2A and 2B 2 are sequence diagrams 200A and 200B that illustrate example cache coherence protocols, respectively. Specifically, sequence diagrams 200A and 200B illustrate example implementations of a default shared state protocol. Figure 1 The system 100 of FIG. 100 describes the execution of sequences 200A and 200B, respectively, but this is for illustrative purposes, and other suitable components for execution may be utilized. For illustrative purposes, Figure 2A and Figure 2B The processor 102, the processor 104 and the node controller 126 may be Figure 1 In some examples, the steps of sequences 200A and 200B may be performed substantially simultaneously or in parallel. Figure 2A and Figure 2B In some examples, sequences 200A and 200B may include Figure 2A and Figure 2B More or fewer steps may be shown in , and in some examples, some of the steps may be ongoing and / or may be repeated.

[0042] Go to Figure 2A , a processor (e.g., processor 102) may request to read an address of memory 110, or more specifically, request to read multiple memory blocks at a specific address within the memory. In some examples, processor 102 may first check its local cache, in this case cache 132, for the requested address. If the address is not present in the local cache (i.e., cache 132), processor 102 may transmit read request 202 to a node controller associated with the memory block range where the address is located—in this example, node controller 126.

[0043] Upon receiving a read request, the node controller 126 may check its local directory (i.e., directory 146) to determine whether the directory 146 is tracking a memory block associated with the address. If it is determined that the address is not tracked by the directory (e.g., the directory address does not exist in the cache of the directory 146), the node controller 126 may assume that - in the default shared state - the requested address is shared. Therefore, the directory 146 is not updated to track the requested address, as is the case in the default invalid state protocol. Instead, the memory block is transferred 204 to the requesting processor 102 in a shared state.

[0044] Different processors (e.g., processor 104) may request to read different addresses of memory 110. Processor 104 may first check its local cache, in this case cache 134, for the requested address. If the address is not present in the local cache (i.e., cache 134), processor 104 may transmit a read request 206 to node controller 126 to access a different address.

[0045] When receiving a read request, the node controller 126 can check its local directory (i.e., directory 146) to determine whether the directory 146 is tracking a memory block associated with a different address. If it is determined that the different address is not tracked by the directory (e.g., the directory address does not exist in the cache of the directory 146), the node controller 126 can assume that the requested address is shared. Therefore, the directory 146 is not updated to track the requested different address, as in the case of the default invalid state protocol. Instead, the memory block is transferred 208 to the requesting processor 104 in a shared state. By not tracking the addresses of these read requests in the example partial directory 146, the directory 146 will not run out of space, as may be the case in the default invalid state protocol. Therefore, the directory 146 in the default shared state protocol may not evict as many memory blocks as the directory 146 in the default invalid state protocol to create space for tracking additional addresses, and the directory 146 will not invalidate the memory blocks in the processor cache as frequently as the directory 146 in the default invalid state protocol. Because the directory does not bear the burden of tracking shared ownership in the default shared state protocol, the directory can have more space for tracking addresses of exclusive ownership.

[0046] Go to Figure 2B, a processor (e.g., processor 102) may request exclusive ownership of an address in memory 110—e.g., processor 102 may request to write to the address. In some examples, processor 102 may first check cache 132 for the requested address. If the address is not present in cache 132, processor 102 may transmit a request 212 for ownership of the address to a node controller associated with the memory block range where the address is located—in this example, node controller 126.

[0047] Upon receiving the ownership request, the node controller 126 may check its local directory (i.e., directory 146) to determine whether the memory block associated with the address is being tracked by the directory 146. If it is determined that the address is not being tracked by the directory (e.g., the directory address does not exist in the cache of the directory 146), the node controller 126 may assume that - in the default sharing state - the requested address is shared.

[0048] However, because processor 102 is requesting exclusive ownership, in this example instance, node controller 126 may invalidate the address at the cache of each processor (e.g., caches 132-138 of processors 102-108) that shares memory 110. In this illustrated example, for the sake of clarity and brevity, node controller 126 is shown as invalidating the address at processor 104 at 214. Because directory 146 is responding to a request for exclusive access to an address, in this example instance, directory 146 may keep track of the requested address.

[0049] Node controller 126 may receive a response from each processor (processor 104 in the illustrated example) confirming invalidation of the address 216. Upon receiving the confirmation, node controller 126 may transmit 218 exclusive ownership of the requested address to processor 102. Although the default shared state protocol may be more efficient in responding to read requests, the default invalid state protocol may be more efficient than the default shared state protocol in situations where exclusive ownership of an address is requested.

[0050] Therefore, for applications that primarily read data, such as large in-memory databases that primarily search memory, the default shared-state protocol may be more efficient than the default invalid-state protocol.

[0051] Figure 33 is a block diagram 300 illustrating an example node controller 302 for enforcing one or more cache coherence protocols. Node controller 302 may be any number of computing devices or components that may be or include a general purpose processor (e.g., a microprocessor, a conventional processor, a controller, a microcontroller, a state machine, a sequencer), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein, etc. Node controller 302 may be implemented, for example, in computing system 100, such as any of node controllers 122-128.

[0052] The node controller 302 may include a directory 310 for tracking ownership of blocks in a memory block range of shared memory to ensure cache coherence. In some cases, the directory may track which processor has exclusive ownership of a memory block, and / or may track shared ownership of a memory block (i.e., a memory block that is not exclusively owned by any single processor or otherwise cached in the system). The directory 310 may be, for example, any one of the directories 142-148, and may track corresponding memory block ranges 112-118 of the memory 110. Although the directory 310 is illustrated as being local to the node controller 302, the directory 310 may be part of or otherwise stored on the same or different hardware as the node controller 302, and may be accessed remotely by the node controller 302, for example, via a network.

[0053] The directory 310 may include any number of directory entries 362 for tracking exclusive memory blocks, or in some cases shared memory blocks. In some examples, the directory 310 may be a partial directory, where the directory 310 may only track a subset of the total cache coherent memory space, as determined by a protocol enforced by the node controller 302. Specifically, the node controller 302 may enforce a protocol that determines which blocks of the memory directory 310 are tracked, such as a default invalidation protocol, a default sharing protocol, etc.

[0054] The node controller 302 may further include a non-transitory computer-readable medium 330 that, when executed by a processor (e.g., processor 320), may perform a cache coherence protocol. The non-transitory computer-readable medium 330 may be implemented in a single device as shown, or may be distributed among multiple devices. Likewise, the processor 320 may represent any number of physical processing resources capable of executing the instructions stored by the computer-readable medium 330, which resources may be part of the same hardware—or different hardware communicating over a network. The processor 320 may be, for example, Figure 1Any combination of the processors 102 - 108 shown in the example system 100 .

[0055] As used herein, "computer-readable medium" can be any electronic, magnetic, optical or other physical storage device for containing or storing information such as executable instructions, data, etc. For example, any computer-readable medium described herein can be any one or a combination of RAM, EEPROM, volatile memory, non-volatile memory, flash memory, storage drive (e.g., HDD, SSD), any type of storage disk (e.g., compressed disk, DVD, etc.), etc. Further, any computer-readable medium described herein can be non-transitory. In the examples described herein, one or more computer-readable media can be part of an article (or product). An article or product can refer to any manufactured single component or multiple components.

[0056] Processor 320 may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, and / or other hardware device suitable for fetching and executing instructions stored in computer-readable medium 330. Processor 320 may fetch, decode, and execute program instructions 332-334 and / or other instructions. Instead of or in addition to fetching and executing instructions, processor 320 may include at least one electronic circuit including a plurality of electronic components for performing the functions of instructions 332-334 and / or other instructions.

[0057] In this example, the instructions 332-334 of the non-transitory computer readable medium 330, when executed, can enforce a default sharing protocol. In an example, the default sharing protocol can cause the directory 310 to track exclusive ownership of blocks of shared memory, and any memory blocks not tracked by the directory 310 can be considered "shared memory."

[0058] Specifically, block 332 includes instructions for directory 310 to track exclusive ownership of a subset of memory blocks. Thus, directory 310 can track memory blocks requested by a processor, for example, when the requesting processor is granted exclusive ownership of the requested memory blocks (e.g., in response to a write request by the processor).

[0059] Block 334 includes instructions for the directory 310 to transfer the requested memory block to the requesting processor in response to the read request. Specifically, the requested memory block is transferred in a shared state without tracking the address at the directory 310. Thus, for example, in response to or in anticipation of transferring the requested memory block to the processor in a shared state, a directory entry 362 will not be created at the directory 310. Thus, the node controller 302 can enforce a default shared state protocol such that the directory 310 acts as a partial directory, thereby tracking exclusive ownership of memory blocks and not tracking any memory blocks transferred to the requesting processor in a shared state.

[0060] Figure 4 is a block diagram illustrating an example system for enforcing one or more cache coherence protocols. System 400 may include a node controller 402 that communicates with Figure 3 Node controller 302 may be any number of computing devices or components, which may be or include a general purpose processor (e.g., a microprocessor, a conventional processor, a controller, a microcontroller, a state machine, a sequencer), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein, etc. Node controller 402 may be implemented, for example, in computing system 100, such as any of node controllers 122-128.

[0061] The node controller 402 may communicate with the directory 440 over a network to track ownership of memory blocks, thereby ensuring cache coherence. The node controller 402 may further communicate with the shared memory 410, for example, over a network. Although the node controller 402 is illustrated as having two memory block ranges (otherwise referred to herein as memory regions), 412 and 414, respectively, the memory 410 may include any number of memory block ranges. In an example embodiment, the directory 440 may be associated with a specific memory block range (e.g., memory block range A 412) of the memory 410, such that the directory 440 tracks at least a subset of the memory blocks in the associated memory block range.

[0062] Memory 410 can be shared because it can be accessed by other nodes of the network. In an example, the other nodes of the network can include other node controllers, other directories, other processors, or any combination thereof. These other directories of the network can be associated with different memory block ranges of memory 410 than directory 440. In addition, in some examples, other node controllers other than node controller 402 can implement a cache coherence protocol different from node controller 402. In an example, other processors of the network can request exclusive or shared access to memory blocks of shared memory 410.

[0063] The node controller 402 may include a non-transitory computer readable medium 430 that may include Figure 3 The computer readable medium 430 may include instructions 432 and 434, which when executed may implement another example default shared state protocol for cache coherence. The instructions of the computer readable medium 430 may be executed by the processor 420, which may include instructions similar to those described above. Figure 3 The architecture of processor 320 is similar to the architecture.

[0064] In an example, memory block range A 412 of memory 410 may be tracked by directory 440 according to a default shared state protocol. In some examples, memory block range B 414 may be tracked by the same or a different directory according to a different protocol (e.g., a default invalid state protocol). In response to receiving a read request from a processor to read any number of memory blocks in memory block range A 412, instructions 432, when executed, may determine whether the requested memory block is tracked in directory 440.

[0065] It may be determined that the memory blocks in the requested range A do not exist, or are otherwise tracked by the directory 440. In this case, the instructions 434, when executed, may cause the directory 440 to transfer the requested memory blocks to the requesting processor. In an example, the requested memory blocks may be transferred to the requesting processor without tracking the non-existent blocks in the directory 440. Thus, the memory blocks within the memory block range A 412 of the memory 410 may be tracked according to a default shared state protocol, which in this example may be a more efficient protocol for use of this particular memory range than the default invalid state protocol.

[0066] Applying different cache coherence protocols to different memory regions enables the flexibility to apply the most efficient protocol based on how a particular memory region is used. For example, for applications that primarily read data, such as a large in-memory database that is primarily used to search memory, the default shared state protocol can be more efficient than the default invalid state protocol for the reasons described above. Conversely, for applications that perform high-frequency memory writes, the default invalid state protocol can be more efficient than the default shared state protocol. Thus, different applications can be associated with different memory ranges of a common shared memory with different protocols associated with different memory ranges.

[0067] In some examples, the node controller 402 may include a register 452 that can track the state of one or more memory block ranges. In this example, the node controller 402 may check the register 452 to determine which cache consistency protocol to execute. As shown, the register 452 may be local to the node controller 402, or may be remote from the node controller 402 and may be accessed by the node controller 402 via a network. The register 452 may be hardware, or a combination of hardware and software. In some examples, the register 452 may be an electronic circuit including both a base address register and a boundary register, so that a series of base address registers and boundary registers may specify the address boundaries of each memory block range in the memory 410. In an example, the BIOS may provide a variable that controls which partial directory mode is enabled and how much memory is present in each memory block area (e.g., memory block range A 412 and memory block range B 414). The amount of memory in each area may be controlled by the setting of the base address register and the boundary register of the hardware. Each base address register and the boundary register may further include a directory mode register bit to determine which directory mode may be used to process a request.

[0068] In some examples, the protocol applied to a particular memory block range can be dynamic, such that a first protocol can be applied to the memory block range at a first time, and a different protocol can be applied to the memory block range at a second time different from the first time. In other words, the memory block range can be dynamic, such that the protocol applied to the memory block range can be automatically changed, or otherwise determined by various indicators as will be further described herein. Register 452 can, for example, flip a state, that is, in response to a threshold, instructing node controller 402 to apply a different protocol to a specific memory block range associated with the register. In some examples, any number of memory block ranges of memory 410 can be dynamic with respect to the protocol applied to the memory block range. In some examples, some memory block ranges of memory 410 are static, such that the protocols applied to them are fixed and do not change dynamically.

[0069] With respect to dynamic memory block ranges, the protocol applied to the dynamic memory block ranges can be determined by any number of metrics. For example, network protocol layer traffic generated by an application utilizing a particular memory block range can be analyzed. Additionally, statistics can be collected when a particular cache coherence protocol is run with respect to a particular memory block range.

[0070] For example, the count of directory entries 462 in the cache of directory 440 can be recorded. If the threshold count of shared memory block entries persists in the cache, register 452 can change the state from the default invalid state protocol to the default shared state protocol. Similarly, if the threshold count of exclusively owned memory block entries persists, register 452 can change the state from the default shared state protocol to the default invalid state protocol. In another example, the ratio of shared memory block entries to exclusively owned memory block entries on the cache of directory 440 can be tracked (or vice versa). In the case where the threshold ratio persists, register 452 can change the state.

[0071] In another example, the number of directory evictions caused by shared read requests can be tracked. A threshold number of evictions within a predetermined time period can indicate that directory 440 is thrashing and can trigger register 452 to flip to a default shared state protocol. In yet another example, the number of invalid snoops can be tracked, e.g., Figure 2B 214. A threshold number of snoops within a predetermined time period may trigger register 452 to flip to a default invalid state protocol.

[0072] When memory 410 is partitioned and / or otherwise allocated, a virtual machine manager (VMM) may allocate memory block regions to applications as indicated by the applications or otherwise preferred. In some examples, in the absence of sufficient memory to allocate preferred memory to applications, an operating system (OS) may allocate memory to applications according to non-preferred protocols. In some examples, the number of times non-preferred memory is allocated to applications within a predetermined time period may be tracked. When a threshold is reached, the protocol applied to the dynamic memory block region may be adjusted.

[0073] Thus, multiple indicators may be tracked to determine the appropriate protocol to use for a particular memory block region at a given time. Logic for switching between protocols may be utilized, and the logic may rely on the above indicators. For example, register 452 may include programmable threshold settings and performance counters to track any number of indicators. In an example, software may be used to program the threshold settings, and may dictate to the hardware when to flip register 452.

[0074] Figure 5is a flow chart 500 illustrating a method for implementing various cache coherence protocols. Figure 6 is a flowchart 600 illustrating a method for switching a memory region from a default shared state protocol to a default invalid state protocol. Figure 1 System 100 or Figure 4 The execution of method 500 and method 600 is described with reference to system 400, but this is for illustrative purposes, and other suitable components for executing method 500 and method 600 may be utilized. Method 500 and method 600 may be implemented in the form of executable instructions stored on a machine-readable storage medium and / or in the form of electronic circuits (e.g., hardware). In some examples, the method may be executed substantially simultaneously or in parallel with the execution of the method. Figure 5 and Figure 6 The steps of method 500 and method 600 are performed in different orders as shown. In some examples, method 500 and method 600 may include Figure 5 and Figure 6 In some examples, some steps of method 500 and method 600 may be performed at a specific time and / or may be repeated.

[0075] At block 502, a node controller (e.g., Figure 4 The node controller 402 of FIG. 400 may receive requests for any number of memory blocks. The request may be initiated by a processor (e.g., Figure 1 At block 504, it may be determined whether the request is a request for exclusive ownership of the memory block or a request for shared access to the memory block, such as a read request.

[0076] Determining that the request is for exclusive ownership may be followed by enabling the processor (e.g., Figure 1 The memory 110 or Figure 4 The memory block in each cache of the memory 410 of the request is invalidated, i.e., snooped. The snoop can be performed by a directory (e.g., Figure 1 At block 508, the directory may grant exclusive access to the requested memory block. The directory may further track owned blocks.

[0077] After determining that the request is for shared access to a memory block, it may be determined which protocol to apply to the requested memory block. Specifically, at block 510, it may be determined whether a default shared state protocol is to be applied. As described above, the determination may be made by the node controller checking the state information of the corresponding registry.

[0078] After a positive determination, the node controller may instruct the associated directory to transfer the requested memory block to the requesting processor at block 512. As described above, this transfer may occur without tracking the transaction in the directory.

[0079] After determining at block 510 that the default shared state should not be applied (or in other words, the default invalid state protocol is applied), at block 514 it is determined whether the directory corresponding to the requested memory block has sufficient space in its cache to track the transaction.

[0080] A positive determination at block 514 may result in the directory tracking the memory block in a shared state at block 518. Conversely, the directory associated with the requested memory block may evict older entries from its cache to make room to track the latest request at block 516. In an example implementation, for consistency purposes, the directory may invalidate the memory block associated with the older entry in each other processor sharing the memory before eviction.

[0081] As described above, some memory block regions may be dynamic, so that the protocol applied to the memory block region may be changed based on various indicators. Specifically, the memory block region may be switched from a default invalid state protocol to a default shared state protocol. The memory block region may also be switched from a default shared state protocol to a default invalid state protocol. In some examples, the mechanism described below may be implemented to maintain consistency during the switch from the default shared state protocol to the default invalid state protocol, because the directory does not track the shared state in the default shared state.

[0082] An example mechanism may include flushing each line associated with the transition memory region. However, if the memory region is large, this may take a significant amount of time. Too many switches between protocols may result in excessive flushing, resulting in degraded system performance.

[0083] Another example mechanism may include an implementation of a scoreboard. For example, each address tracked by the directory includes a corresponding bit on the scoreboard. When it is determined that a memory block region is switched from a default shared state protocol to a default invalid state protocol (e.g., due to satisfying any of the above thresholds), the scoreboard may be initialized. The scoreboard at initialization may indicate that each address in the memory block region to be switched is shared. As addressing is accessed, the corresponding scoreboard bit may be de-asserted. When accessing an address corresponding to a de-asserted bit, the directory state of the directory cache and / or register may be queried. Once all bits in the scoreboard are de-asserted, the scoreboard may be freely re-initialized. Therefore, in some example implementations, the scoreboard may be utilized by multiple memory regions.

[0084] Figure 6604, it can be determined whether the requested memory block has an address within the scoreboard range (i.e., within the dynamic memory region to be switched). A negative determination can be followed by a determination at block 606 whether the address falls within a memory block region having a default shared state protocol applied.

[0085] The positive determination at box 606 is followed by the assumption that the default shared state protocol will apply unless the relevant directory indicates otherwise. The negative determination at box 606 is followed by the assumption that the default invalid state protocol will apply unless the relevant directory indicates otherwise. The request is then processed at box 608 according to the assumed protocol.

[0086] The affirmative determination at block 604 is followed by the node controller checking the scoreboard bit corresponding to the address at block 610. A determination is made at block 612 whether the bit is set, i.e., asserted. If the bit is deasserted, then unless the directory indicates otherwise, a default invalid state protocol is assumed, and the request is processed according to the assumed protocol at block 608. Conversely, if it is determined at block 612 that the bit is set, then at block 614 the bit is cleared.

[0087] Then it can be determined at frame 616 whether each bit in the score has been cleared, that is, deasserted. The positive determination at frame 616 is followed by setting the scoreboard to inactivity at frame 618, because the memory block area has completed the switch from the default shared state protocol to the default invalid state protocol. Unless the directory shows otherwise, the negative determination at frame 616 is followed by assuming that the state is shared. The request is then processed at frame 608 according to the assumed protocol.

[0088] Figure 7 is a circuit diagram 700 further illustrating switching the protocol applied to a dynamic memory region from a default shared state protocol to a default inactive state protocol using a scoreboard. The example circuit may be utilized where the shared memory includes both a static default shared state protocol memory region and a dynamic default shared state memory region.

[0089] Circuit diagram 700 includes a first base address and limit register pair, 702 and 704, respectively, which together define the upper and lower boundaries of the static memory block region. Although only one pair of base address registers 702 and limit registers 704 are illustrated with respect to the static region for the sake of clarity and brevity, multiple base address and limit register pairs can be used to define the boundaries of multiple static memory block regions.

[0090] In the event that the incoming address 701 is determined to be within the range of the base register 702 and the bounds register 704, then the protocol applied to the requested address is determined by the default state of the static memory block region unless otherwise indicated by the directory 740. For example, if the address is within the default shared state memory block range determined by the base register 702 and the bounds register 704, then the default shared state protocol is applied unless otherwise indicated by the directory 740. If otherwise indicated by the directory 740, then the protocol applied is determined by the protocol indicated by the directory 740.

[0091] The circuit diagram 700 further includes a second base address and boundary register pair, 706 and 708, respectively. The second base address register 706 and boundary register 708 pair can together define the upper and lower boundaries of the dynamic memory block region. The second base address register 706 and boundary register 708 pair can be associated with a scoreboard 710. Although for the sake of clarity and brevity, only one pair of base address registers 706 and boundary registers 708 and one scoreboard 710 are shown with respect to the dynamic region, multiple base address and boundary register pairs and scoreboards can be used to define the boundaries of multiple dynamic memory block regions.

[0092] If the incoming address falls within the range of the base and limit registers and does not otherwise exist in the directory 740, the protocol applied may be determined by bit 710a on the scoreboard 710 corresponding to the address 701. If bit 710a is set or otherwise asserted, the default shared state protocol will be applied. Conversely, if bit 710a is cleared or otherwise de-asserted, the default invalid state protocol will be applied.

[0093] When the address is determined to fall within the region of the scoreboard 710 (as determined by the base register 706 and the limit register 708), the bit is cleared or otherwise de-asserted. Once each bit is cleared, the scoreboard may become inactive because the dynamic memory block region has been successfully changed from the default shared state region to the default invalid state region.

[0094] In the foregoing description, many details are set forth to enable an understanding of the subject matter disclosed herein. However, embodiments may be practiced without some or all of these details. Other embodiments may include modifications and variations of the details discussed above. The appended claims are intended to cover such modifications and variations.

Claims

1. A method for enforcing cache consistency, comprising: When the memory region is in the default shared state, tracking, by a directory associated with a first processor in a group of processors, exclusive ownership of a range of blocks of the memory region; In response to receiving a request from a second processor in the group of processors to read one or more blocks of the memory area that are not present in the directory, transferring the one or more blocks in the default shared state to the second processor in the shared state without tracking the one or more blocks to be read in the directory, and When the memory area is in the default exclusive state, tracking one or more blocks of the memory region in a directory of the cache, wherein the one or more blocks of the memory region are marked as shared in the directory of the cache; as well as In response to receiving a request from the second processor in the group of processors to read one or more blocks of the memory area that are not present in the directory, marking the one or more blocks read by the second processor as shared in the directory and transmitting the one or more blocks to be read to the second processor, and When it is determined that a threshold is satisfied with respect to the memory region in the default exclusive state, the state of the memory region is changed to the default shared state.

2. The method of claim 1, wherein: In response to receiving a request from the second processor to acquire ownership of one or more blocks of the memory region in the default shared state, invalidating the one or more blocks of the memory region for which ownership is to be acquired in each cache of the group of processors.

3. The method of claim 1, wherein: In response to receiving a request from the second processor to read one or more blocks of the memory area in a default exclusive state, in response to determining that there is insufficient space in the cache's directory to track the state of the one or more blocks of the memory region, evicting one or more different blocks of the memory region from the cache's directory; as well as One or more blocks of the memory region in the default exclusive state are tracked in a directory of the cache, wherein the one or more blocks to be read are marked as shared in the directory.

4. A system for enforcing cache consistency, comprising: a memory comprising a first range of memory blocks in a default shared state and a second, different range of memory blocks in a default exclusive state; a directory for tracking exclusive ownership of the first memory block range and for tracking sharing status of the second memory block range; as well as A node controller associated with the memory, wherein the node controller In response to receiving a request from a processor to read one or more blocks of the memory in the first range, applying a default shared state protocol to: determining whether one or more blocks of the memory in the first range are tracked in the directory; and In response to determining that one or more blocks of the memory are not present in the directory, transmitting the one or more blocks to be read to the processor without tracking the one or more blocks to be read in the directory, and wherein the node controller, in response to receiving a request from a second processor to read one or more blocks of the memory in the second range, applies a default exclusive state protocol to: determining whether one or more blocks of the memory in the second range are tracked in the directory; In response to determining that the one or more blocks of the memory in the second range are not present in the directory, tracking the one or more blocks to be read in the second range in the directory, wherein the one or more blocks to be read are marked as shared in the cached directory; and transferring the one or more blocks to be read to the second processor, The memory further comprises a third memory block range in a dynamic state, wherein the node controller flips the state of the third memory block range between a default shared state and a default exclusive state in response to a threshold.

5. The system of claim 4, further comprising a register for tracking the status of each memory block in the memory, wherein The node controller determines a status of a memory block in the memory based on the register.

6. The system of claim 4, wherein: The states and area sizes of the first memory block range and the second memory block range are fixed.

7. An apparatus for enforcing cache consistency, comprising: a node controller associated with a first processor of the set of processors and the memory and operable to enforce cache coherence of the blocks of the memory, Wherein, when the memory area of ​​the node controller is in a default sharing state: Tracking exclusive ownership of a subset of the memory region's blocks in a directory of the node controller; and In response to receiving a request from a second processor in the group of processors to read one or more blocks of the memory region that are not present in the directory, transferring the one or more blocks to the second processor in a shared state without tracking the one or more blocks to be read in the directory, and Wherein, when the memory area is in a default exclusive state, the node controller: Tracking one or more blocks of the memory region in a directory of the cache, wherein the one or more blocks of the memory region are marked as shared in the directory of the cache; and In response to receiving a request from the second processor in the group of processors to read one or more blocks of the memory area that are not present in the directory, marking the one or more blocks read by the second processor as shared in the directory and transmitting the one or more blocks to be read to the second processor, The memory includes multiple memory areas, and further includes a register for determining a protocol to be applied to a memory area among the multiple memory areas, wherein the node controller changes the state of the specific memory area to the default shared state when determining that a threshold is met for the specific memory area in the default exclusive state.

8. The device according to claim 7, wherein: The node controller, in response to receiving a request from the second processor to acquire ownership of one or more blocks of the memory region when the memory region is in the default sharing state: invalidating one or more blocks of the memory region to be owned in each cache of the group of processors; as well as The status of one or more blocks of the memory area to be owned in the directory is updated to be exclusively owned by the second processor.

9. The device according to claim 7, wherein: the node controller, in response to receiving the read request from the second processor when the memory region is in the default exclusive state, in response to determining that there is insufficient space in the cache's directory to track the state of the one or more blocks of the memory region, evicting one or more different blocks of the memory region from the cache's directory; as well as The one or more blocks to be read are tracked in a directory of the cache, wherein the one or more blocks to be read are marked as shared in the directory.

10. The device according to claim 7, wherein: The protocol to be applied determined by the register is one of the default shared state or the default exclusive state.

11. The device according to claim 7, wherein: The threshold comprises a percentage of memory blocks of the particular memory region tracked by the directory and marked as shared to memory blocks of the particular memory region that are exclusive.

12. The device of claim 7, wherein: The threshold comprises a number of entries in the directory evicted due to read requests for memory blocks associated with the particular memory region within a predetermined time period.

13. The device of claim 7, wherein: The threshold comprises a percentage of read requests received by the directory for memory blocks associated with the particular memory region to a number of ownership requests received by the directory for memory blocks associated with the particular memory region within a predetermined time period.

14. The device of claim 7, wherein: The node controller changes the state of the specific memory region to the default exclusive state upon determining that a threshold frequency of invalid snoops issued by the directory has been met for the specific memory region in the default shared state.

Images