Opportunistic cleaning of coherency directories to reduce cache coherency overhead

Abstract

A cache-consistent computer system node includes a network-on-chip, a plurality of computing elements in communication with the network-on-chip, a coherency directory including a plurality of addresses for caches of the computing elements, a plurality of coherency states for the addresses, and a plurality of trace vectors for the addresses, and a coherency directory controller configured to send the probe to the computing element during an idle period of the network-on-chip. The probe is configured to query whether an address of the plurality of addresses stored in the coherency directory is in a cache of the computing element, and the coherency directory is configured to, in response to a confirmation indicating that the address is not in the cache, clear the coherency directory to remove the address.

Description

Technical Field

[0001] The present application relates to systems and methods for clearing a consistency directory of a computer system. Background Art

[0002] A computer node includes multiple computing elements, such as cores and / or accelerators, that cache data. In some cache-coherent nodes, a consistency directory is used to track all addresses of cached data in the node and identify which computing elements are caching the cached lines. The consistency directory is used to send observation requests to obtain the current copy of the data and invalidate the cached copy in response to the computing element modifying the cached data. In addition, in some cache-coherent nodes, when the computing elements want to evict cached lines or change their state to keep the entire node up to date, they update the consistency directory. This is done by evicting existing lines from the directory and removing all copies of data associated with it from the node. In this scenario, having the latest information helps to evict old or unneeded lines to make room for new lines. In high-performance computing (HPC) applications and general data center systems with a small number of cores, eviction clears the directory and keeps the consistency state up to date.

[0003] However, it is becoming increasingly common for cores / accelerators to employ silent evictions, which discard clean or unmodified copies of data from the cache without notifying the coherence directory to avoid unnecessary core compute cycles (i.e., messages that would notify the directory) and reduce message traffic on the on-chip network. This causes the coherence directory to become stale, resulting in inefficient evictions, unnecessary invalidate or observe messages being issued, and performance degradation. Additionally, silent evictions also cause designers to have to significantly over-provision the coherence directory to account for the presence of stale lines. Without this over-provisioning and with silent evictions, all lines appear to be being used concurrently because the coherence directory is not cleared when a cache line is evicted. As the number of cores or accelerator caches on a node increases, the growth of the coherence directory (through the additional number of lines and the increase in the size of the tracking bit vector) can lead to serious area and power issues.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not constitute the prior art. Summary of the invention

[0005] The present disclosure relates to various embodiments of a cache-coherent computer system node. In one embodiment, the cache-coherent computer system node includes a network on chip, a plurality of computing elements in communication with the network on chip, a coherence directory including a plurality of addresses for caches of the computing elements, a plurality of coherence states of the addresses, and a plurality of tracking vectors of the addresses, and a coherence directory controller configured to send probes to the computing elements during idle cycles of the network on chip. The probe is configured to query whether an address stored in the coherence directory is in the cache of the computing element, and the coherence directory is configured to clear the coherence directory to remove the address in response to a confirmation indicating that the address is not in the cache.

[0006] Computing elements can be homogeneous and include multiple cores.

[0007] Computational elements can be homogeneous and include multiple accelerators.

[0008] Compute elements can be heterogeneous and include a combination of cores and accelerators.

[0009] The coherency directory controller may be configured to send probes when a computing element is busy.

[0010] The coherence directory controller may be configured not to send probes in response to commands from a computing element.

[0011] Probing can be incorporated into standard conformance protocols.

[0012] The standard consistency protocol may be the MESI protocol.

[0013] The coherence directory controller may be configured to send probes using the existing probe / observe channel of the standard coherence protocol.

[0014] A cache-associated computer system node may also include a main memory coupled to the cache.

[0015] The cache coherent computer system node may also include an interconnect, and the coherent directory controller may be configured to send the probe over the interconnect.

[0016] The coherence directory controller may be configured to send probes intermittently.

[0017] The present disclosure also relates to various embodiments of a method for clearing a consistency directory in a node of a computer system, the computer system comprising an on-chip network and a plurality of cores and / or a plurality of accelerators connected to the on-chip network. In one embodiment, the method comprises sending a probe from a consistency directory controller to a cache of the core and / or accelerator during an idle period of the on-chip network. The probe queries whether an address stored in the consistency directory is in the cache. The method also comprises receiving a confirmation in response to the probe, and in response to the confirmation indicating that the address is not in the cache, clearing the address from the consistency directory.

[0018] The core and / or accelerator may be busy during the time the probe is sent.

[0019] The sending of the probe may not be in response to a command from the core and / or accelerator.

[0020] Probing can be incorporated into standard conformance protocols.

[0021] The standard consistency protocol may be the MESI protocol.

[0022] The sending of probes may utilize the existing probe / observation channels of the standard conformance protocol.

[0023] The sending of the probe may include sending the probe over the interconnect.

[0024] The sending of the probe may include sending the probe intermittently.

[0025] This summary is provided to introduce some features and concepts of embodiments of the present disclosure, which are further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. One or more of the described features can be combined with one or more other described features to provide a feasible system or method. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The features and advantages of the embodiments of the present disclosure will be better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings.The drawings are not necessarily drawn to scale.

[0027] Figure 1 is a schematic diagram of a computer system according to one embodiment of the present disclosure, the computer system including a plurality of computing elements configured to utilize a cache, a coherency directory listing addresses of the cache, and a network on chip (NOC) connected to the computing elements; and

[0028] Figure 2 is a flow chart illustrating the tasks of a method of clearing a consistency directory according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The present disclosure relates to various embodiments of systems and methods for clearing a consistency directory of a node in a computer system, the computer system including a network on chip and multiple computing elements, such as a core, an accelerator, or a combination of a core and an accelerator. The systems and methods of the present disclosure utilize messages (i.e., probes) periodically sent from a consistency directory controller to a cache of a computing element during an idle period of a network on chip (NOC). In response to a message that determines that an address in a consistency directory is no longer in the cache of a computing element, the consistency directory is configured to clear itself by removing unused addresses. In this way, the systems and methods of the present disclosure are configured to clear the consistency directory without participation (e.g., request) from a computing element (e.g., a core and / or an accelerator). Clearing the consistency directory during NOC idle periods is useful in high-performance computing (HPC) applications, in which there are computing periods in which cores or accelerators are busy but NOCs are idle. In these idle period devices, opportunistic clearing can be performed in the background.

[0030] Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings, in which the same reference numerals always refer to the same elements. However, the present invention may be implemented in a variety of different forms and should not be construed as being limited to the embodiments shown here. On the contrary, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the present invention to those skilled in the art. Therefore, for those of ordinary skill in the art, processes, elements and techniques that are not necessary for a complete understanding of the aspects and features of the present invention may not be described. Unless otherwise stated, in all drawings and written descriptions, the same reference numerals represent the same elements, and therefore, their descriptions will not be repeated.

[0031] In the accompanying drawings, for the sake of clarity, the relative sizes of elements, layers and regions may be enlarged and / or simplified. For ease of explanation, spatially relative terms, such as "below", "below", "below", "below", "above", "above", etc., may be used herein to describe the relationship between an element or feature and another element or feature shown in the figure. It will be understood that, in addition to the orientations depicted in the accompanying drawings, spatially relative terms are intended to cover different orientations of the device in use or operation. For example, if the device in the figure is turned over, the elements described as "below" or "below" or "below" of other elements or features will be oriented "above" other elements or features. Therefore, the example terms "below" and "below" can include both above and below orientations. The device can be oriented in other ways (e.g., rotated 90 degrees or in other directions), and the spatially relative descriptors used herein should be interpreted accordingly.

[0032] It should be understood that although the terms "first", "second", "third", etc., can be used herein to describe a variety of elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, without departing from the spirit and scope of the present invention, the first element, component, region, layer or part described below can be referred to as a second element, component, region, layer or part.

[0033] The terms used herein are to describe specific embodiments, rather than to limit the present invention. As used herein, the singular forms "one" and "an" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that the terms "include", "include ... ", "comprise" and "comprises ... " when used in this specification, specify the presence of features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their combinations. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items. When before an element list, an expression such as "at least one" modifies the entire element list without modifying the individual elements of the list.

[0034] As used herein, the terms "substantially," "approximately," and similar terms are used as terms of approximation, rather than terms of degree, and are intended to account for the inherent variations in measurements or calculations that one of ordinary skill in the art would recognize. Additionally, when describing embodiments of the present invention, the use of "may" refers to "one or more embodiments of the present invention." As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilizing," "utilizing," and "utilized," respectively. Additionally, the term "exemplary" is intended to refer to an example or illustration.

[0035] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly formal sense, unless explicitly defined herein.

[0036] For the purpose of this disclosure, expressions such as "at least one" when preceding a list of elements modify the entire list of elements, without modifying a single element in the list. For example, "at least one of X, Y, and Z," "at least one of X, Y, or Z," and "at least one selected from the group consisting of X, Y, and Z" can be interpreted as any combination of only X, only Y, only Z, two or more of X, Y, and Z, such as XYZ, XYY, YZ, and ZZ, or any variant thereof. Similarly, expressions such as "at least one of A and B" can include A, B, or A and B. As used herein, "or" generally means "and / or," and the term "and / or" includes any and all combinations of one or more associated listed items. For example, expressions such as "A and / or B" can include A, B, or A and B.

[0037] Figure 1 1 is a schematic diagram of a node 100 of a computer system including a network on chip (NOC) 101 and a plurality of computing elements 102 connected to the NOC 101. In the illustrated embodiment, the computing elements 102 include cores 103 (e.g., cores C0, C1, C2, and C3) and accelerators 104 (e.g., accelerators A0, A1, A2, and A3). In one or more embodiments, the computing elements 102 of the node 101 can be homogeneous (e.g., only cores or only accelerators) or heterogeneous (e.g., a mix of cores and accelerators).

[0038] In the illustrated embodiment, the computing elements 102 (e.g., cores 103 and accelerators 104) include caches 105, 106, respectively (e.g., high-speed data storage layers stored in random access memory (RAM)), which store subsets of the blocks stored in main memory 107. In addition, in the illustrated embodiment, the system 100 includes a coherence directory 108, which contains a list of data addresses of cached data, a corresponding list of coherence states of the cached data (e.g., the cached line has been modified from the value in main memory 107 (M); the cached line matches the value in main memory 107 and is dedicated to one core or one accelerator (E); the cached line matches the value in main memory 107 and is shared between two or more cores and / or accelerators (S); or the cached line is invalid according to the MESI protocol (I)), and a list of corresponding tracking bit vectors indicating which core(s) 103 and / or accelerator 104 are caching the data address. For example, in the illustrated embodiment, the coherence directory 108 lists an address “X” and an address “Y”, each of which is cached by cores C0, C1, C2, and C3 and accelerators A0, A1, A2, and A3, and has a coherence state “S” because both address “X” and address “Y” are shared among multiple C0, C1, C2, and C3 and accelerators A0, A1, A2, and A3. Additionally, in the illustrated embodiment, the coherence directory 108 lists an address “Z” that is only cached by core C0, and has a coherence state “E” because address “Z” is only cached by a single computing element, core C0. However, as Figure 1 As shown, the cache at address "Y" is not utilized by any computing element 102 (eg, cores C0, C1, C2, and C3 or accelerators A0, A1, A2, and A3).

[0039] In the illustrated embodiment, the coherence directory controller 109 is configured to intermittently (e.g., periodically) send commands or messages 110 (e.g., "Check_presence" probes) to the caches 105, 106 of the core(s) 103 and / or the accelerator(s) 104 via the on-node interconnect 111 to determine whether a particular address in the coherence directory 108 is in any of the caches 105, 106 of the core(s) 103 and / or the accelerator(s) 104. In addition, in one or more embodiments, the coherence directory controller 109 is configured to send the commands 110 (e.g., probes) during idle periods of the NOC 101 (e.g., the coherence directory controller 109 is configured to send the commands 110 when the NOC 101 is idle). In one or more embodiments, the messages or commands 110 (e.g., probes) may be incorporated into any standard coherence protocol (e.g., the MSI protocol, the MESI protocol, or the MOSI protocol), and it may utilize existing probe / observation channels. Probe 110 looks at core(s) 103 and / or accelerator(s) 104 like any other probe from NOC 101 .

[0040] For example, in the illustrated embodiment, the coherence directory controller 109 is configured to send a message 110 (“Check_presence Y”) over the on-node interconnect 111 to determine whether an address “Y” in the coherence directory is in any of the caches 105, 106 of the core(s) 103 and / or the accelerator(s) 104. In response to a confirmation message 112 indicating that the address “Y” is not resident in any of the caches 105, 106 of the core(s) 103 and / or the accelerator(s) 104, the coherence directory 108 is configured to delete (e.g., scrub) the address entry “Y” from the coherence directory 108. In this manner, the coherence directory 108 is configured to opportunistically clean itself up in the background during idle periods of the NOC 101 without receiving requests, hints, or other inputs from the core(s) 103 and / or the accelerator(s) 104 (e.g., the probe 110 provides enhanced functionality for the coherence directory controller 109 so that it can initiate transactions on its own). Purging the coherence directory 108 without first receiving a request, hint, or other input from the core(s) 103 and / or accelerator(s) 104 enables the core(s) 103 and / or accelerator(s) 104 to continue to perform computations and access caches without being interrupted to update the coherence directory 108. Purging the coherence directory 108 during idle periods of the NOC 101 is particularly useful in high performance computing (HPC) applications where there are computational periods where the core(s) 103 and / or accelerator(s) 104 are busy but the NOC 101 is idle. Purging the coherence directory 108 without first receiving a request, hint, or other input from the core(s) 103 and / or accelerator(s) 104 also does not result in any deadlock or livelock in the system 100.

[0041] Figure 2 2 is a flow chart illustrating the tasks of a method 200 for clearing a coherence directory of a node in a computer system. The node of the computer system includes a network on a chip (NOC), a plurality of computing elements (e.g., cores, accelerometers, or a combination of cores and accelerometers) connected to the NOC, and a main memory. The computing elements include cached data from the main memory.

[0042] In the illustrated embodiment, method 200 includes a task 210 of sending a command or message (e.g., a probe) during an idle cycle of the NOC (i.e., when the NOC is idle) to determine whether a particular address in the coherency directory is in any cache of the core(s) and / or accelerator(s). The message (e.g., the probe) may be transmitted to the cache of the core(s) and / or accelerator(s) via an on-node interconnect. In one or more embodiments, the probe transmitted in task 210 may be incorporated into any standard coherency protocol (e.g., the MSI protocol, the MESI protocol, the MOSI protocol, or the MOESI protocol), and it may utilize an existing probe / observation channel.

[0043] In the illustrated embodiment, method 200 includes a task 220 of receiving an acknowledgement indicating whether the address transmitted in task 210 is in any cache of the core(s) and / or accelerator(s).

[0044] In the illustrated embodiment, in response to the acknowledgement of task 220 indicating that the address transmitted in task 210 is not in any cache of the core and / or accelerator, method 200 includes a task 230 of flushing the coherence directory to delete (eg, erase) the address entry from the coherence directory.

[0045] In this manner, the method 200 includes the coherence directory opportunistically clearing itself in the background during idle periods of the NOC without receiving requests, prompts, or other inputs from the core(s) and / or accelerator(s), which enables the core(s) and / or accelerator(s) to continue to perform computations and access caches without being interrupted to update the coherence directory, which is particularly useful in high performance computing (HPC) applications where there are computation periods when the core(s) and / or accelerator(s) are busy but the NOC is idle. Thus, the method 200 of clearing the coherence directory does not cause any deadlock or livelock in the computer system.

[0046] Although the present invention has been described in detail with specific reference to exemplary embodiments of the present invention, the exemplary embodiments described herein are not intended to be exhaustive or to limit the scope of the present invention to the exact forms disclosed. It will be understood by those skilled in the art and technology to which the present invention belongs that changes and variations may be made to the described structures and assemblies and methods of operation without intentionally departing from the principles, spirit and scope of the present invention as set forth in the above embodiments.

Claims

1. A cache-coherent computer system node, comprising: Network on chip; a plurality of computing elements communicatively coupled to a network on chip in a fully cache-coherent manner; a coherence directory comprising a plurality of addresses of caches for the plurality of computing elements, a plurality of coherence states for the plurality of addresses, and a plurality of tracking vectors for the plurality of addresses; as well as a coherence directory controller configured to send probes to the plurality of computing elements during idle periods of the on-chip network, wherein the probing is configured to query whether an address of the plurality of addresses stored in the coherence directory is in a cache of the plurality of computing elements, and Wherein the coherence directory is configured to, in response to a confirmation indicating that the address is not in the cache, flush the coherence directory to remove the address.

2. The cache-coherent computer system node of claim 1, wherein: The plurality of computing elements are homogeneous and include a plurality of cores.

3. The cache coherent computer system node of claim 1, wherein the plurality of computing elements are homogeneous and include a plurality of accelerators.

4. The cache coherent computer system node of claim 1, wherein the plurality of computing elements are heterogeneous and include a combination of a plurality of cores and a plurality of accelerators.

5. The cache coherent computer system node of claim 1, wherein the coherent directory controller is configured to send the probe when the plurality of computing elements are busy.

6. The cache coherent computer system node of claim 1, wherein the coherent directory controller is configured to send the probe not in response to a command from the plurality of computing elements.

7. The cache-coherent computer system node of claim 1, wherein: The detection is incorporated into the standard conformance protocol.

8. The cache coherent computer system node of claim 7, wherein the standard coherence protocol is the MESI protocol.

9. The cache-coherent computer system node of claim 7, wherein the coherence directory controller is configured to send the probes using an existing probe / observe channel of a standard coherence protocol.

10. The cache coherent computer system node of claim 1, further comprising a main memory coupled to the cache.

11. The cache coherent computer system node of claim 1 , further comprising an interconnect, and wherein the coherent directory controller is configured to send the probe over the interconnect.

12. The cache coherent computer system node of claim 1, wherein the coherent directory controller is configured to send the probes intermittently.

13. A method for clearing a consistency directory in a node of a computer system, the computer system comprising a network on chip and a plurality of cores and / or a plurality of accelerators connected to the network on chip, the method comprising: sending a probe from a coherent directory controller to a cache of the plurality of cores and / or the plurality of accelerators during an idle period of the network on chip, the probe querying whether an address stored in the coherent directory is in the cache; receiving an acknowledgement in response to the probe; as well as In response to a confirmation indicating that the address is not in the cache, the address is cleared from the coherency directory.

14. The method according to claim 13, wherein: During the sending of the probe, the plurality of cores and / or the plurality of accelerators are busy.

15. The method of claim 13, wherein the sending of the probe is not in response to a command from the plurality of cores and / or the plurality of accelerators.

16. The method of claim 13, wherein the probing is incorporated into a standard conformance protocol. The method of claim 16 , wherein the standard conformance protocol is the MESI protocol.

18. The method of claim 16, wherein the sending of the probe utilizes an existing probe / observation channel of a standard conformance protocol.

19. The method of claim 13, wherein sending the probe comprises sending the probe over an interconnect.

20. The method of claim 13, wherein sending the probe comprises sending the probe intermittently.

Images