Two-level cache consistency protocol based on many-core computing architecture

Abstract

The invention belongs to the technical field of extensible computer architecture, and particularly relates to a two-level cache coherence protocol based on a many-core computing architecture. Comprising a many-core computing architecture, the many-core computing architecture comprises a plurality of computing clusters, each computing cluster is formed by interconnection of a plurality of core grains and an I / O and memory controller, and the I / O and memory controller is interconnected with at least two memory DDRs; each core particle is formed by interconnection of a plurality of computing cores and a shared cache LLC; each calculation core is provided with a private cache, and when the data is not in the private cache, the memory access request continues to go to the next-level shared cache LLC or the memory DDR until the latest data is obtained; the shared cache LLC can access the memory DDR only after the shared cache LLC is in butt joint with the memory controller through high-speed interconnection and I / O; and the two-level cache consistency protocol directory is composed of a first-level directory and a second-level directory. According to the invention, the delay of re-reading from the storage system is omitted, the performance is further improved, and the congestion at the storage position is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of scalable computer architecture, and in particular relates to a two-level cache consistency protocol based on a many-core computing architecture. Background Art

[0002] Up to now, the many-core computing architecture is led by AMD, an American company, and the number of integrated computing cores can be as high as 128. The general cache consistency protocol directory adopts a single write invalidate write strategy to maintain the protocol. For some sharing modes, the communication cost is high. Therefore, the present invention proposes a two-level cache consistency protocol based on the many-core computing architecture, which can support two modes at the same time, allowing users to make choices according to their needs. Summary of the invention

[0003] The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art. The present invention provides a two-level cache consistency protocol based on a many-core computing architecture to solve the problems in the above-mentioned background technology.

[0004] To solve the above technical problems, the present invention provides a two-level cache consistency protocol based on a many-core computing architecture, which can simultaneously support two different cache strategies; the two-level cache consistency protocol includes:

[0005] Many-core computing architecture, including multiple computing clusters, each computing cluster is composed of multiple cores interconnected with an I / O and memory controller, and an I / O and memory controller is interconnected with at least two memory DDRs; each core is composed of multiple computing cores interconnected with a shared cache LLC; each computing core has its own private cache, when the data is not in the private cache, the memory access request will continue to the next level of shared cache LLC or memory DDR until the latest data is obtained; the shared cache LLC can access the memory DDR only after it is connected to the I / O and memory controller through high-speed interconnection;

[0006] The second-level cache consistency protocol directory consists of the first-level directory and the second-level directory; the first-level directory is built into each core particle and is used to manage the sharing status of data between computing cores stored in the same core particle; the second-level directory is built into the memory and I / O controller and is used to manage the sharing permissions of data between multiple core particles of the computing cluster to which it belongs.

[0007] Preferably, the structure of the first-level directory includes sharing permissions, sharing status and sharer vector bits.

[0008] Preferably, the two different cache strategies include a private cache strategy and a non-private cache strategy; wherein the private cache strategy is used to cache data blocks in a private cache and a shared cache LLC, and the non-private cache strategy can skip the private cache and only cache the data blocks at this serialization point in the shared cache LLC.

[0009] Preferably, the strategy for whether data needs private cache or not depends on what type of instruction the computing core uses to access memory; the instruction that starts the non-private cache strategy is called a non-private cache instruction; the non-private cache instruction includes the following two types: using the atomic instruction that already exists in the instruction set as the key to start the non-private cache strategy, all related cache blocks accessed by the atomic instruction will immediately write the latest data back to the shared cache LLC after the data is written; by extending the instruction set, adding some new memory write instructions specifically for one-time data writing scenarios, and the programmer or compiler starts the non-private cache strategy according to the needs of the scenario to achieve performance optimization.

[0010] Preferably, when there are multiple computing cores in the core particle reading and sharing, the first-level directory only needs to issue a read transaction request to the second-level directory when the first read miss occurs, and the subsequent read miss finds the accessed data block from the shared cache LLC to which it belongs, that is, the second-level directory will configure sharing permissions for the first-level directory in the shared cache LLC according to the current sharing situation between the core particles, and the sharing permissions include write permission M and read permission S.

[0011] Preferably, the non-private cache strategy is supported by extending the finite state machine of the standard MESI.

[0012] Preferably, the shared cache blocks in the last level cache of the many-core computing architecture can switch freely between private cache and non-private cache strategies according to the needs of performance optimization.

[0013] Preferably, when the first non-private cache instruction accesses the memory architecture, if it hits in the private cache, when the computing core writes back the data block to the shared cache LLC, it must first be merged with the data block in the private cache, and all copies in the private cache architecture must be invalidated along the way; if it does not hit in the private cache, the data is directly transferred from the shared cache LLC, so the computing core does not need to perform an operation to invalidate the private cache copy when writing back; when the first non-private cache instruction arrives at the shared cache LLC, it is found that the cache block is shared by other cores. At this time, the shared cache LLC needs to invalidate the shared data block through write-invalidate and call it back to the shared cache LLC, and then it can enter the UM's authority and respond to the request of the non-private cache instruction.

[0014] Compared with the prior art, the present invention has the following beneficial effects:

[0015] The present invention proposes a two-level cache consistency protocol that can simultaneously support two private cache strategies. In the design of a multi-core computing architecture interconnected by point-to-point interconnection, based on the simplicity of implementation, its cache consistency protocol adopts a write-invalidate write strategy to handle dirty blocks. The spirit of the write-invalidate strategy is to keep the shared data in the current private cache for as long as possible, until other computing cores issue a read or write request, then downgrade or invalidate the cache block and then reply the cache block to the requesting computing core. For a cache block with only one-time data write, if the data can skip the private cache and directly write back to the storage as a serialization point - that is, this type of data block does not need to be temporarily stored in the private cache at all - the invalidation step in the write-invalidate that requires four hops in the whole process can be avoided when a computing core makes a memory access next time to improve performance; the present invention calls it a non-private cache strategy. The present invention shows how to support two different cache strategies at the same time by extending the two-level cache consistency protocol; by analogy, this extension principle is also applicable to other cache consistency protocols. When the non-private cache strategy is used, a series of memory accesses to the same address will benefit from shorter data read latency; since data reads are usually on the critical execution path, this shorter latency can improve the performance of parallel program execution. In addition, when the sharing mode is migratory and multiple write requests to the same cache block arrive at the directory together, this strategy provides an opportunity to forward the data directly to the next one, omitting the latency of re-reading from the storage system, further improving performance and reducing congestion at the storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a block diagram of the many-core computing architecture of the present invention.

[0017] Figure 2 A schematic diagram showing that the present invention can achieve performance optimization.

[0018] Figure 3 It is a conversion diagram between the sharing status and sharing permissions of each level directory of the present invention.

[0019] In the figure: 100-computing cluster; 101-chip; 102-computing core; 103-private cache; 104-last level cache (LLC) in the chip; 105-high-speed interconnect; 106-I / O and memory controller; 107-memory; 108-first-level directory; 109-second-level directory; 300-status of data block not currently being requisitioned by any computing core; 301-status of no computing core sharing under write permission; 302-status of no computing core sharing under read permission; 303-status of not being requisitioned under non-private cache policy. DETAILED DESCRIPTION

[0020] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0021] For simplicity of implementation, the execution strategy in the second-level cache consistency protocol is to handle dirty blocks between and within cores in the only way of write-invalidate. The spirit of write-invalidate is to keep shared data in the current private cache for as long as possible until other computing cores issue read or write requests. The directory will then downgrade or invalidate the cache block copies outside according to the protocol, and then reply to the requester with the latest cache block.

[0022] For cache blocks that are only written once, if they can skip the private cache and write directly to the storage location that serves as the serialization point (such as the directory that executes the cache protocol) - that is, this type of data block does not need to be temporarily stored in the private cache at all - the next time other computing cores access the memory, they can avoid the invalidation step in the write-invalidate process that requires four hops to improve performance; this processing method is called a non-private cache strategy.

[0023] The present invention supports two private cache strategies simultaneously by extending a two-level cache consistency protocol design; the first strategy allows data blocks to be cached in private and shared cache architectures, and the second strategy skips the private cache architecture and only caches data blocks in the shared cache, which is a sequential point. The description of the present invention is based on the two-level cache consistency protocol of MESI, but its principle can be extended to any cache consistency protocol.

[0024] When the non-private cache strategy is used, a series of memory accesses to the same address will benefit from shorter data read latency; since data reads are usually on the critical execution path, this shorter latency can improve the performance of parallel program execution. In addition, when the sharing mode is migratory and multiple write requests to the same cache block arrive at the directory together, this strategy provides an opportunity to forward the data directly to the next one, omitting the latency of re-reading from the storage system, further improving performance and reducing congestion at the storage.

[0025] like Figure 1As shown, the embodiment of the present invention specifically describes a multi-core computing architecture. Each computing cluster 100 includes four or more cores 101. Each core 101 can have multiple computing cores 102, and each computing core 102 is connected to a private cache 103 architecture. The last level cache in the core 101, namely the shared cache LLC 104, is shared by all computing cores 102 in the core 101; it can be regarded as an extension of the memory DDR 107; when the computing core 107 performs memory access, whether it is for private data or shared data, it cannot be found in its own private cache 103, that is, it searches LLC first, and only after the search fails will it issue a memory access request. LLC can access memory only after connecting to the I / O and memory controller 106 through the high-speed interconnect 105. There is a second-level directory 109 in the I / O and memory controller 106. The second-level directory 109 manages all shared address spaces allocated to the computing cluster 100 to which it belongs, while the first-level directory 108 manages the shared data blocks stored in the chiplet LLC to which it belongs.

[0026] In an embodiment of the present invention, the strategy for whether data needs private caching or not depends on what instruction type the computing core uses to access memory. The present invention refers to this type of instruction used to start the non-private cache strategy as a non-private cache instruction; there are two feasible solutions: First, use the atomic instructions that already exist in the instruction set as the key to start the non-private cache strategy, and all related cache blocks accessed by the atomic instructions will immediately write the latest data back to the serialization point-shared cache LLC after the data is written; Second, by extending the instruction set, add some new memory write instructions specifically for one-time data writing scenarios, and the programmer or compiler will start the non-private cache strategy according to the needs of the scenario to achieve performance optimization. These two solutions do not conflict with each other and can be used at the same time; no matter which solution is used, it requires hardware support-computing core and LLC. The computing core end is responsible for identifying the timing of the start, which can be done through the encoding of the instruction; since LLC is the first serialization point when all shared information is exchanged, LLC needs to extend the original cache consistency protocol to record the state of the data block in the non-private cache, so that it can cooperate with the non-private cache instruction to play the optimization function.

[0027] like Figure 2 As shown, the embodiment of the present invention specifically provides an example of performance optimization. Figure 2In (a), assume that the copy of the data block is initially stored in the private cache of the computing core of CPU4. When CPU3 wants to write, it must find the owner of the current data block through the LLC0 directory based on write-invalidate, and then invalidate the copy of the data block in the owner's private cache. After retrieving the data, it will be forwarded to CPU3. The whole process involves four steps of information communication, which is called four-hop communication. Similarly, when CPU2 also wants to write to the data block, it must go through the four-hop communication described above. If a non-private cache strategy is introduced, such as Figure 2 As shown in (b), after CPU3 completes the write operation, the computing core automatically invalidates the data block and writes back the entire dirty block to directory LLC0. Then, when CPU2 issues a write request, it can directly find the data block in LLC0, thereby shortening the memory access latency of CPU2.

[0028] like Figure 3 As shown in Figure 1, the method of starting the non-private cache policy can be distinguished by specific instructions. However, on the LLC side, based on the data sharing status at the time, there are several scenarios that must be considered, and their changes are as follows: Figure 3 shown. Figure 3 This is a state change diagram based on the second-level cache consistency protocol (uncolored background); the gray background is the state that the LLC needs to configure for the data block when it is not a private cache. In the second-level cache consistency protocol, the processing of shared transactions between computing cores and core particles is completely decoupled. Each first-level directory issues a sharing permission request to the second-level directory on behalf of all computing cores in the core particle to which it belongs; for example: if there are multiple cores in the core particle reading and sharing, the first-level directory only needs to issue a read transaction request to the second-level directory when the first read miss occurs. Subsequent read misses can find the accessed data block from the LLC to which it belongs, greatly reducing the pressure on the communication and memory access bandwidth of the second-level directory. In other words, the second-level directory will configure the so-called sharing permission for the first-level directory in the LLC based on the current sharing situation between core particles; there are basically only two types of this permission: write permission (M) and read permission (S).

[0029] When the first-level directory is granted write permission (M), the shared state of the shared block in the first-level directory can be recorded and executed according to the standard MESI protocol in the industry; when the first-level directory is granted read permission (S), the shared state of the shared block in the first-level directory can only be recorded as S or I. This method is an innovative approach different from the industry standard to save the maintenance cost of the overall protocol when the data is only read and shared. It is worth noting that the I state under the write permission, i.e. 301, and the I state under the read permission, i.e. 302, do not mean that the data is invalid, but that there is no computing core in the chip to which the LLC belongs to share the data block temporarily. If it is replaced and the sharing permission is M at this time, the data block is still regarded as a dirty block and must be written back to the memory and the second-level directory is updated to I; the practices of each company are different. In order to save storage costs, the record of the I state in the second-level directory can also be omitted, because a data block has never been requisitioned by any computing core, and there will be no relevant column in the directory at all. When a request arrives and the record cannot be found in the directory, a column can be opened to track it. If the sharing permission is S when it is replaced, the data block is consistent with the memory, and you can choose not to notify the second-level directory. The state I of 300 means that the data has not been requisitioned by any computing core, so it cannot exist in any LLC.

[0030] When the second-level directory receives a read or write request from other cores, if the permission granted to the first-level directory of the next core is (M), then its permission will be downgraded to (S) or completely invalid (that is, the private cache of the current sharer will be invalid, and the sharing permission and sharing status will be changed to (I) at the same time), and the data it holds will be a dirty block and must be returned to the second-level directory or even written back to the memory. If LLC replacement occurs in the core, the cache block must be completely invalidated, and the dirty block must be written back to the memory after the second-level directory protocol operation.

[0031] If the permission granted to the first-level directory of a certain core is (S), and the second-level directory receives a write request from other cores, then its permission will be completely invalidated, because the data is not a dirty block, and only the second-level directory needs to be notified that the invalidation request is completed without data. If the second-level directory receives a read request from other cores, the secondary directory can immediately reply to the requester's data block, without invalidating other LLCs currently in read sharing. In addition, if LLC replacement occurs in a shared core, the cache block must be completely invalidated, and only the second-level directory needs to be notified that the invalidation request is completed without data.

[0032] The protocol operation of the second-level directory is carried out using the industry-standard MSI; it treats each chip as a node with only one LLC cache, regardless of how many computing cores are in the chip, thus achieving the goal of high scalability.

[0033] As for the part of the non-private cache strategy, it can be regarded as an extension of the sharing permission M. The second-level directory cannot see it, but the data block is in a dirty state at this time, so the gray background part is represented by UM. The first scenario is that the relevant data block is in a private cacheable state at the beginning. Then, when the non-private cache instruction is accessed, the first-level directory will convert the current sharing permission to UM and does not need to record the number of the sharer because the computing core will definitely write back after execution; however, if the current sharing permission is S, the first-level directory needs to obtain the sharing permission M from the second-level directory, because the non-private cache instruction is a write instruction, and its operation will eventually change the value of a part of the data block, but the invalidation transaction caused by it between core particles will be handled by the second-level directory; if the current sharing permission is M, there is no need to request the second-level directory. Since the data block will be rewritten by a computing core at this time, the sharing state is also set to M. When the non-private cache instruction is executed and the data block must be written back to LLC, the shared state becomes the I state, i.e., the non-requisitioned state 303 under the non-private cache strategy, indicating that it is temporarily not requisitioned, so that LLC can directly output it when the next non-private cache instruction accesses it.

[0034] Since data blocks are not in the non-private cache policy for a long time, but are constantly switching between private cache and non-private cache policies, there are conditions attached when switching policies in some scenarios. When the first non-private cache instruction accesses the memory architecture, it may hit in its own private cache. At this time, when the computing core writes the data block back to the LLC, it must first merge with the data block in the private cache, and invalidate all copies in the private cache architecture along the way; if it has never hit in the private cache, the data is directly transferred from the LLC, then when the computing core writes back, it does not need to perform the operation of invalidating the private cache copy along the way. Another scenario is that when the first non-private cache instruction arrives at the LLC, it finds that the cache block is shared by other cores. At this time, the LLC needs to invalidate the shared data block through write-invalidate and call it back to the LLC, and then it can enter the UM's authority and respond to the request of the non-private cache instruction.

[0035] If a cache block under the non-private cache policy encounters the next instruction that is privately cacheable, the policy must switch back to the write-invalidate method; if it is a read request from the same core, the sharing permission can only be changed to M, not S, because the data has been changed; regardless of whether the current instruction is a read or write request, the sharing status can only be M, because it is already exclusive and has been changed. If the private cache request comes from another core, the processing is different; if it is a write request from another core, the cache block will be removed from the LLC, and the data will be written back to the second-level directory. If it is a read request from another core, the data will also be written back to the second-level directory, but the sharing permission will be downgraded to S and the sharing status will be determined based on the sharing status value of the UM sharing permission at that time. If it is not I, it will become S, otherwise it will remain unchanged.

[0036] In addition, parallel programming often requires various synchronization operations, such as locks and barriers, to ensure normal operation. In order to ensure the mutual exclusivity of these synchronization operations, different algorithms can be used to avoid bottlenecks, but the use of atomic instructions is still unavoidable. Atomic instructions generally read first and then write, but these two memory accesses need to occur together and cannot be separated. It is common that the data block is often requisitioned by other computing cores after writing, resulting in a one-time write behavior. The present invention can improve the performance of such synchronization operations. In many parallel algorithms, the time when atomic instructions compete for the same variable or block address one after another is usually very close. Based on the non-private cache strategy, when these atomic instructions are queued in front of the LLC arbiter, they will benefit from the UM permissions in this patent. The LLC can immediately forward the cache block written back by the current computing core to the next waiting atomic instruction without changing the state through the LLC, thereby shortening the delay of shared data transfer; the main reason is that under the UM sharing permission, as mentioned above, the shared cache block does not need to record the number of the sharer, and the computing core will ensure that the non-private cache data is immediately written back to the LLC after executing the atomic instruction, to ensure that starvation or deadlock will not occur.

[0037] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A two-level cache consistency protocol based on many-core computing architecture, capable of supporting two different cache strategies at the same time; characterized in that: The second-level cache coherence protocol includes: Many-core computing architecture, including multiple computing clusters, each computing cluster is composed of multiple cores interconnected with an I / O and memory controller, and an I / O and memory controller is interconnected with at least one memory DDR; each core is composed of multiple computing cores interconnected with a shared cache LLC; each computing core has its own private cache, when the data is not in the private cache, the memory access request will continue to the next level of shared cache LLC or memory DDR until the latest data is obtained; the shared cache LLC can access the memory DDR only after it is connected to the I / O and memory controller through high-speed interconnection; The second-level cache consistency protocol directory consists of the first-level directory and the second-level directory; the first-level directory is built into each core particle and is used to manage the sharing status of data between computing cores stored in the same core particle; the second-level directory is built into the memory and I / O controller and is used to manage the sharing permissions of data between multiple core particles of the computing cluster to which it belongs.

2. A two-level cache consistency protocol based on a many-core computing architecture as described in claim 1, characterized in that: The structure of the first-level directory includes sharing permissions, sharing status, and sharer vector bits.

3. A two-level cache consistency protocol based on a many-core computing architecture as described in claim 1, characterized in that: The two different cache strategies include a private cache strategy and a non-private cache strategy; wherein the private cache strategy is used to cache data blocks in a private cache and a shared cache LLC, and the non-private cache strategy can skip the private cache and only cache the data blocks at the serialization point of the shared cache LLC.

4. A two-level cache consistency protocol based on a many-core computing architecture as described in claim 1, characterized in that: The strategy for whether data needs private caching or not depends on the type of instruction used by the computing core to access memory. The instruction that starts the non-private cache strategy is called a non-private cache instruction. The non-private cache instructions include the following two types: using the atomic instructions that already exist in the instruction set as the key to start the non-private cache strategy, and all related cache blocks accessed by the atomic instructions will immediately write the latest data back to the shared cache LLC after the data is written; by extending the instruction set, adding some new memory write instructions specifically for one-time data writing scenarios, and the programmer or compiler will start the non-private cache strategy according to the needs of the scenario to achieve performance optimization.

5. A two-level cache consistency protocol based on a many-core computing architecture as claimed in claim 1, characterized in that: When there are multiple computing cores in the core particle reading and sharing, the first-level directory only needs to send a read transaction request to the second-level directory when the first read miss occurs. Subsequent read misses find the accessed data block from the shared cache LLC to which it belongs. That is, the second-level directory will configure sharing permissions for the first-level directory in the shared cache LLC according to the current sharing situation between core particles. The sharing permissions include write permission M and read permission S.

6. A two-level cache consistency protocol based on a many-core computing architecture as claimed in claim 3, characterized in that: The non-private cache strategy is supported by extending the standard MESI finite state machine.

7. A two-level cache consistency protocol based on a many-core computing architecture as claimed in claim 1, characterized in that: The shared cache blocks in the last level cache of the many-core computing architecture can freely switch between private cache and non-private cache strategies according to the needs of performance optimization.

8. A two-level cache consistency protocol based on a many-core computing architecture as claimed in claim 1, characterized in that: When the first non-private cache instruction accesses the memory architecture, if it hits the private cache, when the computing core writes back the data block to the shared cache LLC, it must first merge it with the data block in the private cache, and invalidate all copies in the private cache architecture along the way; if it does not hit the private cache, the data is directly transferred from the shared cache LLC, so the computing core does not need to perform the operation of invalidating the private cache copy when writing back; when the first non-private cache instruction arrives at the shared cache LLC, it finds that the cache block is shared by other cores. At this time, the shared cache LLC needs to invalidate the shared data block through write-invalidate and call it back to the shared cache LLC, and then it can enter the UM's authority and respond to the request of the non-private cache instruction.

Images