Multi-core GPU interconnection architecture and its adaptive cache allocation method

Abstract

The present application belongs to the field of semiconductor technology and discloses a multi-core GPU interconnection architecture and its adaptive cache allocation method. The multi-core GPU interconnection architecture includes several GPU clusters and a global cluster. The global cluster includes a global router, a global scheduler, a directory, a memory and several GPU clusters. The GPU cluster is composed of several GPU cores. Each GPU core includes several independent first-level data caches, second-level data caches, a local router and a network interface. The network interface determines whether the current request is processed by the GPU core in the GPU cluster based on whether the target address in the target node belongs to the GPU cluster, thereby forming a high-level multi-core interconnection architecture that supports cross-cluster communication. The method optimizes performance by adapting to different access modes, switches between private and shared cache modes in real time, reduces single data access latency and improves overall system efficiency, and is used in multi-core heterogeneous systems with multiple GPUs.

Description

Technical Field

[0001] The present invention belongs to the field of semiconductor technology, and in particular discloses a multi-core GPU interconnection architecture and an adaptive cache allocation method thereof. Background Art

[0002] As semiconductor manufacturing advances to more advanced nodes, the ability to boost performance solely through transistor size reduction is becoming increasingly limited. The chiplet architecture breaks down complex processor modules into multiple small chips, integrating them through advanced packaging techniques to create a single-chip-like operating mechanism. Compared to large, monolithic system-on-chips, smaller chiplets improve wafer yield and reduce overall manufacturing costs. Furthermore, different chiplets can be selected as needed to form different computing modules, increasing system scalability. Finally, different computing tasks can be executed on the most suitable chiplet, reducing energy consumption and improving overall energy efficiency.

[0003] In a multi-chip architecture, the cache allocation mechanism is crucial to system performance and consistency maintenance. The choice of cache sharing or privacy directly affects data access latency, bandwidth utilization, and consistency maintenance overhead. Shared cache reduces data redundancy and improves cache hit rate by reducing repeated storage of the same data. However, shared cache needs to be accessed across the scope of the chip, increasing communication latency and consistency maintenance costs. In contrast, private cache can reduce the complexity of consistency management, increase access speed, and allow each computing unit to independently manage its cache data, so that the same address can be read in parallel in multiple private caches, thereby improving parallelism and access efficiency. However, it can easily lead to increased data redundancy and cache miss rate, and remote access in the event of a miss requires a lot of overhead. Summary of the Invention

[0004] In order to solve the problems of large cache consistency overhead and high remote access latency in multi-GPU core architecture, the present application provides a multi-core GPU interconnection architecture and its adaptive cache allocation method. The method optimizes performance by adapting to different access modes. According to the statistical proportion of remote request behavior during cache access, the present application will switch between private cache mode and shared cache mode in real time, thereby reducing the single data access latency and improving the overall system efficiency. It is suitable for various multi-core heterogeneous systems with multiple GPUs.

[0005] In order to achieve the above objectives, this application is implemented through the following technical solutions:

[0006] The present application is a multi-core GPU interconnection architecture, which is a high-level multi-core interconnection architecture that supports cross-cluster communication. The multi-core GPU interconnection architecture includes several GPU clusters and a global cluster, reducing the load of a single-layer router. The global cluster includes a global router, a global scheduler, a directory, a memory and several GPU clusters. The GPU cluster is composed of several GPU cores, each of which includes several independent first-level data caches, several independent second-level data caches, a local router for data forwarding, a network interface connecting the local router and the first-level data cache and the local router and the second-level data cache. The network interface has a request path determination function, which determines whether the current request is processed by the GPU core in the GPU cluster based on whether the target address in the target node belongs to the GPU cluster. If the target address does not fall within the processing range of the GPU cluster, the request is forwarded to the global router of the global cluster through the local router, thereby forming a high-level multi-core interconnection architecture that supports cross-cluster communication.

[0007] A further improvement of the present application is that: each of the first-level data cache and each of the second-level data cache includes a plurality of physical shard storage units, each physical shard storage unit is controlled by a cache controller, and the cache controller of the first-level data cache and the cache controller of the second-level data cache are used to receive and output message packets, namely, address mapping information, control information and cache data;

[0008] The network interface is used to transfer the address mapping information, control information, and cached data between the cache controller of the secondary data cache and the local router, and between the cache controller of the primary data cache and the local router, and the network interface is integrated with a cache allocation control bit. According to the current state of the cache allocation control bit, the address mapping information input by the cache controller of the primary data cache and the cache controller of the secondary data cache is mapped differently to generate a final target node, and the address mapping information, control information, and cached data are sent to the output port of the corresponding local router;

[0009] The local router receives the address mapping information, control information and cache data output by the network interface and completes the forwarding operation according to the search result of the target address.

[0010] A further improvement of the present application is that: the cache allocation control bit of the network interface is dynamically updated by the global scheduler or execution logic according to the kernel function execution status or remote access characteristics, and is used to adaptively switch between private cache mode and shared cache mode, specifically: when the cache allocation control bit is 0, the target address is calculated according to the private cache allocation mechanism. Under the private cache allocation mechanism, the cache controller of each first-level data cache only accesses the cache controller of the second-level data cache corresponding to the GPU core through the local router, thereby ensuring data isolation. When the cache allocation control bit is 1, the target address is calculated according to the shared cache allocation mechanism. Under the shared cache allocation mechanism, the cache controller of any first-level data cache accesses the cache controller of any second-level cache data within the mapping range of the cache controller of the first-level data cache through the local router and the global router, thereby realizing data sharing across clusters.

[0011] A further improvement of the present application is that the address mapping mechanism for mapping the address mapping information input by the cache controller of the first-level data cache and the cache controller of the second-level data cache is based on the address mapping information plus the GPU cluster number. When the cache allocation control bit is 0, that is, under the private cache allocation mechanism, in order to ensure that cache access does not conflict and to ensure that each first-level data cache accesses its own second-level cache data, the GPU cluster number is usually introduced as an offset in the address mapping information, that is:

[0012]

[0013] in, Physical shard storage unit number for the secondary data cache The corresponding GPU cluster number, is the number of L2 data caches per GPU cluster, The decimal value corresponding to the physical shard storage unit number in the address mapping information. Each L1 data cache accesses the L2 cache data to which the L1 data cache belongs, thus achieving physical isolation of the L2 data cache and allowing multiple copies of a data block to exist.

[0014] When the cache allocation control bit is switched to 1, that is, in the shared cache allocation mechanism, the mapping mode of the address mapping information is adjusted so that different first-level data caches can access the same data block and reduce address conflicts caused by cache mode switching. Specifically:

[0015]

[0016] Where T is the decimal number corresponding to the lower two binary digits. In the shared cache allocation mechanism, a method is proposed to determine the GPU cluster number of the target L2 data cache by borrowing the decimal number corresponding to the lower two binary digits, T, to achieve address mapping. By introducing address tags as a key factor in the calculation, this method ensures a certain degree of randomness in the distribution of different addresses when mapping to shared cache units, preventing hotspot addresses from being concentrated in specific cache units, thereby effectively mitigating cache access conflicts.

[0017] The present application also provides an adaptive cache allocation method for a multi-core GPU interconnect architecture, the method comprising the following steps:

[0018] Step 1: Each GPU core detects the received request and determines its status. Based on the status determination, it selects an allocation mechanism to adapt to different tasks by continuously adjusting the dynamic threshold of delay. The allocation mechanism includes a shared cache allocation mechanism and a private cache allocation mechanism.

[0019] Step 2: After the global scheduler receives a cumulative number of requests, it selects a cache allocation mapping mechanism based on a dynamic threshold of all received remote access requests;

[0020] Step 3: The global scheduler formally issues a request to switch the state. After the switch is completed, it issues an instruction to switch the data cache allocation mechanism to the first-level data cache of the corresponding target address. When the global scheduler formally issues the request to switch the state, it first sends a global manual refresh request and a global invalidation cache synchronization request to the first-level data cache of the corresponding GPU cluster, allowing the first-level data cache to synchronize the stored data and stipulate that the first-level data cache forwards the synchronization request to all second-level data caches for synchronization events, so as to avoid cache consistency problems of some addresses caused by the switching of the allocation mechanism. Then, an instruction to switch the cache allocation mechanism is issued to the corresponding first-level data cache;

[0021] Step 4: The first-level data cache completes its own cache allocation mechanism update and sends a switching instruction to the cache allocation control bit in the network interface. Then, the first-level data cache sends the switching request to the corresponding second-level data cache.

[0022] Step 5: Upon receiving the switch request, the L2 data cache module first checks whether the manual refresh request and global invalidation cache synchronization request forwarded by the first L1 data cache to all L2 data caches in Step 3 are complete. If not, the switch operation will be temporarily paused and blocked. After the data synchronization process is completed, the state switch will be continued. After the switch is completed, the L2 data cache will return a confirmation message of the state switch completion to the L1 data cache control unit.

[0023] Step 6. After the first-level data cache receives the switching event response sent by the second-level data cache, it sends a response to the global scheduler indicating that the entire event is completed. Until the global scheduler receives a response corresponding to the number of requests, notifying the global scheduler that the switching has been completed, the global scheduler can confirm that the switching task of this round of cache allocation mechanism has been successfully completed and enter the next scheduling cycle.

[0024] A further improvement of the present application is that in step 1, when the secondary data cache in each GPU core receives a read request that misses the primary data cache in the GPU core, the secondary data cache determines whether to select a shared cache allocation mechanism or a private cache allocation mechanism:

[0025] If the current secondary data cache is allocated according to the private cache allocation mechanism, the secondary data cache collects access data within the global cluster and between GPU clusters. The secondary data cache compares its own physical shard storage unit number with the master node number corresponding to the target address mapping information of the address mapper to see if it matches. If the physical shard storage unit number matches the master node number, the access count of the local read request within the GPU cluster is increased in the counter of the global scheduler. If the physical shard storage unit number does not match the master node number, the remote access count is increased in the counter of the global scheduler.

[0026] If the current secondary data cache is allocated according to the shared cache allocation mechanism, the secondary data cache compares its own physical shard storage unit and the requesting primary data cache to see if they belong to the same GPU cluster. If the physical shard storage unit and the primary data cache belong to the same GPU cluster, this indicates that this is a local access, and the number of local read request accesses within the GPU cluster needs to be increased in the global scheduler's counter. If the physical shard storage unit and the primary data cache do not belong to the same GPU cluster, this indicates that this is a cross-chip access, and the number of cross-chip request accesses needs to be increased in the global scheduler's counter.

[0027] A further improvement of the present application is that in step 1, under the shared cache allocation mechanism, read requests for the first-level data cache are divided into local read requests and cross-chip read requests mapped to the second-level data caches of different GPU clusters. The latency calculation under the shared cache allocation mechanism is:

[0028] ;

[0029] Under the private cache allocation mechanism, the read requests for the first-level data cache are divided into local read requests and remote read requests that need to be made to the master node when the local second-level data cache misses. The delay calculation under the private cache allocation mechanism is:

[0030]

[0031] in, is the average access delay, is the read request latency under the shared cache allocation mechanism, is the cross-chip read request latency under the shared cache allocation mechanism, is the proportion of cross-chip read requests, Delay for local read requests, The read request delay from the secondary data cache to the directory when the secondary data cache is missed, is the L2 data cache miss rate, is the read request latency under the private cache allocation mechanism, The latency of remote access under the private cache allocation mechanism, The percentage of remote access.

[0032] A further improvement of the present application is that, in step 3, sending a global manual refresh request and a global invalidation cache synchronization request to the first level 1 data cache of the corresponding cluster is the key path of the switch, which specifically includes the following steps:

[0033] Step 3.1: Pause write operations on the first-level data cache: After the switch is triggered, first suspend write requests from the first-level data cache to the second-level data cache to ensure that no new dirty data is generated during the switch;

[0034] Step 3.2, mark dirty data and perform refresh operation: perform dirty data scan on the cache control unit to be switched, and actively write valid data back to the memory to avoid data loss; among them, valid data has data;

[0035] Step 3.3: Invalidate the current cache copy: After writing back to memory, clear all cache line metadata in the cache control unit under the private cache allocation mechanism to invalidate it, preventing subsequent accesses from hitting outdated data.

[0036] Step 3.4: Switch cache mapping and access control policy: Officially enable the shared cache allocation mechanism or the access path with the cache allocation mechanism, and allow the first-level data cache to resume normal access.

[0037] The beneficial effects of this application are:

[0038] This application designs a mechanism that supports adaptive dynamic switching of private and shared caches, which is applicable to various multi-core heterogeneous systems with multiple GPUs.

[0039] In order to improve the access efficiency under the multi-core architecture, this application introduces shared cache support. By building a logical shared topology at the cache layer and combining it with a dynamic switching mechanism based on remote access statistics, it can flexibly adjust between private and shared caches, thereby taking into account both consistency maintenance costs and access latency optimization.

[0040] This application expands upon the existing proprietary direct-connect multi-GPU architecture, implementing a multi-GPU core architecture with hierarchical interconnect support and a dynamically switching adaptive cache allocation mechanism. This architecture is achieved by introducing cache allocation control bits and their addressing logic into the network interface. To support efficient switching between different cache modes, a unified address mapping strategy is designed to ensure that each computing unit can correctly access target data in different modes, while reducing access conflicts and resource waste caused by mode switching.

[0041] During system operation, the secondary data cache controller continuously counts access requests, focusing on recording remote access events caused by local secondary data cache misses and calculating the remote access ratio. Set an adjustable remote access threshold. The system sets an adjustable remote access threshold and updates and judges the statistical results in combination with a periodic scheduling mechanism. When the remote access ratio continues to be higher than the set threshold for multiple consecutive monitoring cycles, the system will actively initiate a cache mode switching request, switching from private cache mode to shared cache mode. This method also supports dynamic adjustment of threshold policies based on historical hit rates to improve the adaptability and robustness of judgments. During the mode switching execution phase, the system will initiate necessary cache consistency maintenance operations, including cache refresh (Flush) and cache invalidation (Invalidate), according to the definition of the consistency protocol, to ensure the consistency and correct synchronization of the cache state during the switching process.

[0042] This application can dynamically judge the utilization efficiency of cache resources based on the access behavior characteristics during actual runtime, and adjust the cache mode accordingly, achieving flexible switching between private and shared states, thereby improving cache resource utilization, reducing remote access bandwidth pressure, and enhancing the operating performance and adaptability of multi-core systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Schematic diagram of the multi-core GPU interconnection architecture proposed in this application.

[0044] Figure 2 It is a schematic diagram of the network interface of this application.

[0045] Figure 3 This is a schematic diagram of the address bit allocation of this application.

[0046] Figure 4 This is a flow chart of the adaptive cache allocation method for multi-core GPU interconnection architecture of this application.

[0047] Figure 5 This is a schematic diagram of access types for applying for a private cache allocation mechanism and a shared cache allocation mechanism.

[0048] Figure 6 This is a flowchart of the application's selection of an allocation mechanism based on status determination.

[0049] Figure 7 This is a schematic diagram comparing the bandwidth performance of different cache allocation mechanisms. DETAILED DESCRIPTION

[0050] The following diagrams illustrate embodiments of the present invention. For clarity, many practical details are included in the following description. However, it should be understood that these practical details are not intended to limit the present invention. In other words, in some embodiments of the present invention, these practical details are not essential.

[0051] like Figure 1 As shown, the present application proposes a multi-core GPU interconnection architecture, which is a high-level multi-core interconnection architecture that supports cross-cluster communication. It is characterized in that: the multi-core GPU interconnection architecture includes several GPU clusters and a global cluster, reducing the load of a single-layer router. The global cluster includes a global router, a global scheduler, a directory, a memory and several GPU clusters. The GPU cluster is composed of several GPU cores, each of which includes several independent first-level data caches, several independent second-level data caches, a local router for data forwarding, a network interface connecting the local router and the first-level data cache and connecting the local router and the second-level data cache. The network interface has a request path determination function, and determines whether the current request is processed by the GPU core in the GPU cluster based on whether the physical slice storage unit number to which the target address in the target node belongs belongs to the GPU cluster. If the target address does not fall within the processing range of the GPU cluster, the request is forwarded to the global router of the global cluster through the local router, thereby forming a high-level multi-core interconnection architecture that supports cross-cluster communication.

[0052] like Figure 3 The address bit allocation diagram shows that each of the first-level data cache and each of the second-level data cache in the present application includes several physical shard storage units, each physical shard storage unit is controlled by a cache controller, and the second-level data cache address mapping mechanism is processed based on the address mapping information plus the GPU cluster number.

[0053] Address bits are assigned as follows Figure 3 As shown, taking the configuration used in this application as an example: using 64-bit address mapping information, 64-byte cache block capacity, 8 L2 cache physical shard storage units, and 8MB L2 cache with 16-way set associativity, the address bit division can be roughly as follows:

[0054] Address bits 0-5: offset within the block

[0055] 6-8 bits of address: storage unit index

[0056] Address digits 9-21: Group number

[0057] Remaining high address: label

[0058] When the cache allocation control bit is 0, that is, under the private cache allocation mechanism, to ensure that cache access does not conflict and that each L1 data cache accesses its own L2 cache data, the GPU cluster number is usually introduced as an offset in the address mapping information, that is:

[0059]

[0060] in, Physical shard storage unit number for the secondary data cache The corresponding GPU cluster number, is the number of L2 data caches per GPU cluster, It is the decimal value corresponding to the physical shard storage unit number in the corresponding address mapping information. Each first-level data cache accesses the second-level cache data to which the first-level data cache belongs, realizing the physical isolation of the second-level data cache and allowing multiple copies of a data block to exist; the data exchange process is carried out in units of data blocks.

[0061] When the cache allocation control bit is switched to 1, that is, in the shared cache allocation mechanism, the mapping mode of the address mapping information is adjusted so that different first-level data caches can access the same data block and reduce address conflicts caused by cache mode switching. Specifically:

[0062]

[0063] Where T is the decimal number corresponding to the lower two binary digits. In the shared cache allocation mechanism, a method is proposed to determine the GPU cluster number of the target L2 data cache by borrowing the decimal number corresponding to the lower two binary digits, T, to achieve address mapping. By introducing address tags as a key factor in the calculation, this method ensures a certain degree of randomness in the distribution of different addresses when mapping to shared cache units, preventing hotspot addresses from being concentrated in specific cache units, thereby effectively mitigating cache access conflicts.

[0064] The cache controller of the first-level data cache and the cache controller of the second-level data cache are used to receive and output message packets, namely, address mapping information, control information and cache data.

[0065] like Figure 2As shown, the network interface is used to transfer the address mapping information, control information, and cached data between the cache controller of the secondary data cache and the local router, and between the cache controller of the primary data cache and the local router, and the network interface is integrated with a cache allocation control bit. According to the current state of the cache allocation control bit, different mapping processes are performed on the address mapping information input by the cache controller of the primary data cache and the cache controller of the secondary data cache to generate a final target node, and the address mapping information, control information, and cached data are sent to the output port of the corresponding local router;

[0066] The local router receives the address mapping information, control information and cache data output by the network interface and completes the forwarding operation according to the search result of the target address.

[0067] The network interface's cache allocation control bits are dynamically updated by the global scheduler or execution logic based on kernel function execution status or remote access characteristics, enabling adaptive switching between private and shared cache modes. This architectural design offers excellent scalability and adaptability, effectively supporting the adaptive caching mechanism proposed in this paper.

[0068] Specifically: when the cache allocation control bit is 0, the target address is calculated according to the private cache allocation mechanism. Under the private cache allocation mechanism, the cache controller of each first-level data cache only accesses the cache controller of the second-level data cache corresponding to the GPU core through the local router, thereby ensuring data isolation. When the cache allocation control bit is 1, the target address is calculated according to the shared cache allocation mechanism. Under the shared cache allocation mechanism, the cache controller of any first-level data cache accesses the cache controller of any second-level cache data within the mapping range of the cache controller of the first-level data cache through the local router and the global router, thereby realizing cross-cluster data sharing. Through this design, the present application can flexibly support dynamic switching between private and shared cache modes, thereby improving the overall system performance and resource utilization efficiency.

[0069] like Figure 4 The present application also provides an adaptive cache allocation method for a multi-core GPU interconnect architecture, which specifically includes the following steps:

[0070] Step 1: Each GPU core detects the received request and makes a status determination. The global scheduler records whether it is a hit and adaptively adjusts the threshold to suit different tasks. An allocation mechanism is selected based on the status determination, and the dynamic threshold of the delay is continuously adjusted to suit different tasks. The allocation mechanism includes a shared cache allocation mechanism and a private cache allocation mechanism.

[0071] like Figure 6As shown, in step 1, when the secondary data cache in each GPU core receives a read request that misses the primary data cache in the GPU core, the secondary data cache determines whether to select a shared cache allocation mechanism or a private cache allocation mechanism:

[0072] If the current secondary data cache is allocated according to the private cache allocation mechanism, the secondary data cache collects access data within the global cluster and between GPU clusters. The secondary data cache compares its own physical shard storage unit number with the master node number corresponding to the target address mapping information of the address mapper to see if it matches. If the physical shard storage unit number matches the master node number, the access count of the local read request within the GPU cluster is increased in the counter of the global scheduler. If the physical shard storage unit number does not match the master node number, the remote access count is increased in the counter of the global scheduler.

[0073] If the current secondary data cache is allocated according to the shared cache allocation mechanism, the secondary data cache compares its own physical shard storage unit and the requesting primary data cache to see if they belong to the same GPU cluster. If the physical shard storage unit and the primary data cache belong to the same GPU cluster, this indicates that this is a local access, and the number of local read request accesses within the GPU cluster needs to be increased in the global scheduler's counter. If the physical shard storage unit and the primary data cache do not belong to the same GPU cluster, this indicates that this is a cross-chip access, and the number of cross-chip request accesses needs to be increased in the global scheduler's counter.

[0074] Under the shared cache allocation mechanism, read requests to the L1 data cache are divided into local read requests and cross-chip read requests mapped to the L2 data cache of different GPU clusters. The latency calculation under the shared cache allocation mechanism is:

[0075] ;

[0076] Under the private cache allocation mechanism, the read requests for the first-level data cache are divided into local read requests and remote read requests that need to be made to the master node when the local second-level data cache misses. The delay calculation under the private cache allocation mechanism is:

[0077]

[0078] in, is the average access delay, is the read request latency under the shared cache allocation mechanism, is the cross-chip read request latency under the shared cache allocation mechanism, is the proportion of cross-chip read requests, Delay for local read requests, The read request delay from the secondary data cache to the directory when the secondary data cache is missed, is the L2 data cache miss rate, is the read request latency under the private cache allocation mechanism, The latency of remote access under the private cache allocation mechanism, The percentage of remote access.

[0079] The first half of the calculation is mainly about the message transmission delay within and between GPU cores, where we should distinguish between local transmission delay and long-distance transmission delay, which is the main bottleneck. The second half of the calculation is about the message delay required to pass the message from the GPU cluster to the directory and memory controller when a miss occurs.

[0080] This application believes that to find a good remote access ratio switching threshold, we need to find a situation where the latency is the same in the private and shared cache situations. Starting with the private cache allocation mechanism, the ideal remote access ratio in the private state is 20%, and the ideal cross-chip access ratio in the shared state is 80%. The specific execution process is as follows:

[0081] Initially, the task is run in private cache mode. Based on collected data, the ideal switching threshold is calculated for the ideal configuration. Above this threshold, the shared cache is considered superior, and the corresponding status code is incremented. Below this threshold, the corresponding status code is decremented. Similarly, for shared cache mode, if the proportion of cross-chip accesses exceeds the ideal ratio, the private cache is considered superior, and the corresponding status code is decremented; otherwise, it is incremented.

[0082] When switching from private 00, to private 01, to shared 10, to shared 11, the current cache miss rate, local latency, and remote access latency are recorded during the shared cache switching process. When the shared cache is about to switch to the private cache, the switching threshold for the remote access ratio in the private state is adjusted based on the current cache miss rate, local latency, cross-chip latency, and the recorded remote access latency. By continuously adjusting the switching threshold, we can adapt to various scenarios.

[0083] Step 2: After the global scheduler receives a cumulative number of requests, it selects a cache allocation mapping mechanism based on a dynamic threshold of all received remote access requests;

[0084] Step 3: The global scheduler formally issues a request to switch the state. After the switch is completed, it issues an instruction to switch the data cache allocation mechanism to the first-level data cache of the corresponding target address. When the global scheduler formally issues a request to switch the state, it first sends a global manual refresh request and a global invalid cache synchronization request to the first-level data cache of the corresponding GPU cluster, allowing the first-level data cache to synchronize the stored data, and designates the first first-level data cache to forward to the second-level data cache for synchronization events, to avoid cache consistency problems at some addresses caused by the switching of the allocation mechanism, and then issues an instruction to switch the cache allocation mechanism to the corresponding first-level data cache. In this step, sending a global manual refresh request and a global invalid cache synchronization request to the first-level data cache of the corresponding cluster is the key path for switching, which specifically includes the following steps:

[0085] Step 3.1: Pause write operations on the first-level data cache: After the switch is triggered, first suspend write requests from the first-level data cache to the second-level data cache to ensure that no new dirty data is generated during the switch;

[0086] Step 3.2: Mark dirty data and perform a refresh operation: Scan the cache control unit to be switched for dirty data and actively write valid data back to the memory to avoid data loss.

[0087] Step 3.3: Invalidate the current cache copy: After writing back to memory, clear all cache line metadata in the cache control unit under the private cache allocation mechanism to invalidate it, preventing subsequent accesses from hitting outdated data.

[0088] Step 3.4: Switch cache mapping and access control policy: Officially enable the shared cache allocation mechanism or the access path with the cache allocation mechanism, and allow the first-level data cache to resume normal access.

[0089] The introduction of this process is an essential component of the mode switching mechanism. Because private caches may contain a large amount of dirty data that has been locally modified but not yet written back to main memory, switching to shared mode before processing can easily lead to data inconsistencies or dirty reads. The manual refresh and invalidation mechanism described above allows the system to clear potential conflicts and establish a clean shared cache view before switching, thereby ensuring data consistency, repeatability, and system reliability.

[0090] In this step, the present application sets four switching states, including private 00, about to switch to private 01, about to switch to shared 10 and shared 11, corresponding to the cache allocation mechanism control unit of the cluster local router. For the private cache allocation mechanism, when the remote access ratio exceeds the threshold set by the dynamic adjustment policy, a judgment is triggered to switch to shared, and the current switching state code is reduced by one. When the switching state is about to switch to shared 10, after receiving the judgment of switching to shared, the event of switching allocation mechanism will be entered. For the shared cache allocation mechanism, it is to judge whether the proportion of collected cross-chip read request results is greater than the ideal proportion of cross-chip proportions, and make corresponding switching decisions through the switching status code. Similarly, the shared cache allocation mechanism will trigger the event of switching private cache allocation state only when it receives the judgment of switching to private allocation mechanism when it is in the state of about to switch to private.

[0091] Step 4: After receiving the state switching instruction, the controller of the first-level data cache will parse the instruction to switch the cache allocation mechanism issued in step 3 and convert it into an internally consistent state change request, which will trigger the corresponding state switching event. The first-level data cache will complete its own cache allocation mechanism update and send a switching instruction to the cache allocation control bit in the network interface. Then, the first-level data cache will send the switching request to the corresponding second-level data cache.

[0092] Step 5: Upon receiving the switch request, the L2 data cache module first checks whether the manual refresh request and global invalidation cache synchronization request forwarded by the first L1 data cache to all L2 data caches in Step 3 have been completed. If not, the switch operation will be temporarily paused and blocked. After the data synchronization process is completed, the state switch will be continued. After the switch is completed, the L2 data cache will return a confirmation message of the state switch completion to the L1 data cache control unit.

[0093] Step 6. After the first-level data cache receives the switching event response sent by the second-level data cache, it sends a response to the global scheduler indicating that the entire event is completed. Until the global scheduler receives a response corresponding to the number of requests, notifying the global scheduler that the switching has been completed, the global scheduler can confirm that the switching task of this round of cache allocation mechanism has been successfully completed and enter the next scheduling cycle.

[0094] This application uses the gem5 simulator for simulation experiments. The GPU model needs to be driven by the roc-1.6.x branch of ROCm. Since roc-1.6.x will have many incompatibility issues in the current Ubuntu 22.04 environment, this application runs in the Ubuntu 16.04 environment by building a docker environment.

[0095] In order to evaluate the results of the cache allocation mechanism used in this application, this application selected BFS, Backpropgation, pathfinder, srad_v2, hotspot, square, dwt2d, and nw as the final results of the benchmark evaluation. Since the original implementation of the above benchmarks is mainly based on the CUDA version, and the gem5 simulator used in this application is mainly based on AMD's GPU operating environment, the benchmark program is ported from the CUDA version to adapt to the HIP operating environment.

[0096] Figure 7 The overall bandwidth using the adaptive allocation mechanism is shown. Figure 7 The data in this table is normalized by the number of vector operations executed per cycle for each benchmark to reflect throughput performance. For benchmarks with poor locality and a high proportion of remote accesses, such as backprop and dwt2d, the reduced remote access ratio achieved by using a shared cache allocation mechanism can lead to better performance than the high parallelism of private caches. Therefore, timely switching to a shared cache can yield certain performance improvements. For scenarios with a low overall proportion of remote accesses, such as Srad_v2, hotspot, and nw, the high parallelism of private caches and the reduced cross-chip memory access overhead compared to shared cache allocation mechanisms enable even higher performance.

[0097] The adaptive cache allocation mechanism proposed in this application can ensure performance and stability improvements compared to a single cache allocation mechanism, enabling it to adapt to different scenarios and make the choice of a better cache allocation mechanism at the moment.

[0098] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A multi-chip GPU interconnect architecture, wherein the multi-chip GPU interconnect architecture is a high-level multi-chip interconnect architecture that supports cross-cluster communication, characterized by: The multi-core GPU interconnection architecture includes several GPU clusters and a global cluster. The global cluster includes a global router, a global scheduler, a directory, a memory and several GPU clusters. The GPU cluster is composed of several GPU cores. Each of the GPU cores includes several independent first-level data caches, several independent second-level data caches, a local router for data forwarding, a network interface connecting the local router and the first-level data cache and the local router and the second-level data cache. The network interface determines whether the current request sent by the first-level data cache to the second-level data cache is from the GPU cluster based on whether the target address in the target node belongs to the GPU cluster. If the target address does not belong to the GPU cluster processing range, the request is forwarded to the global router of the global cluster through the local router, thereby forming a high-level multi-chip interconnection architecture that supports cross-cluster communication. The address mapping mechanism for mapping the address mapping information input by the cache controller of the first-level data cache and the cache controller of the second-level data cache is based on the address mapping information plus the GPU cluster number. Specifically, when the cache allocation control bit is 0, that is, under the private cache allocation mechanism, the first-level data cache accesses the second-level cache data belonging to the first-level data cache, and the GPU cluster number is introduced as an offset in the address mapping information, that is: ， in, Physical shard storage unit number for the secondary data cache The corresponding GPU cluster number, is the number of L2 data caches per GPU cluster, The decimal value corresponding to the physical slice storage unit number in the corresponding address mapping information; When the cache allocation control bit is switched to 1, that is, in the shared cache allocation mechanism, the mapping mode of the address mapping information is adjusted, specifically: ， Wherein, T is the decimal number corresponding to the lower two binary digits, and the address mapping is achieved by borrowing the decimal number corresponding to the decimal number T corresponding to the lower two binary digits to determine the GPU cluster number of the target secondary data cache.

2. The multi-core GPU interconnect architecture according to claim 1, wherein: Each of the first-level data cache and each of the second-level data cache includes a plurality of physical slice storage units, each physical slice storage unit is controlled by a cache controller, and the cache controller of the first-level data cache and the cache controller of the second-level data cache are used to receive and output message packets, namely, address mapping information, control information and cache data; The network interface is used to transfer the address mapping information, control information, and cached data between the cache controller of the secondary data cache and the local router, and between the cache controller of the primary data cache and the local router. The network interface is integrated with a cache allocation control bit. According to the current state of the cache allocation control bit, the address mapping information input by the cache controller of the primary data cache and the cache controller of the secondary data cache is mapped differently to generate a final target node, and the address mapping information, control information, and cached data are sent to the output port of the corresponding local router; The local router receives the address mapping information, control information and cache data output by the network interface and completes the forwarding operation according to the search result of the target address.

3. The multi-core GPU interconnect architecture according to claim 2, wherein: The cache allocation control bit of the network interface is dynamically updated by the global scheduler or execution logic according to the kernel function execution status or remote access characteristics, and is used to adaptively switch between private cache mode and shared cache mode. Specifically, when the cache allocation control bit is 0, the target address is calculated according to the private cache allocation mechanism. Under the private cache allocation mechanism, the cache controller of each first-level data cache only accesses the cache controller of the second-level data cache corresponding to the GPU core through the local router to ensure data isolation. When the cache allocation control bit is 1, the target address is calculated according to the shared cache allocation mechanism. Under the shared cache allocation mechanism, the cache controller of any first-level data cache accesses the cache controller of any second-level cache data within the mapping range of the cache controller of the first-level data cache through the local router and the global router to achieve data sharing across clusters.

4. An adaptive cache allocation method for the multi-core GPU interconnect architecture of claim 1, characterized in that: The adaptive cache allocation method specifically comprises the following steps: Step 1: Each GPU core detects the received request and makes a status determination. Based on the status determination, an allocation mechanism is selected to adapt to different tasks by adjusting the dynamic threshold of delay. The allocation mechanism includes a shared cache allocation mechanism and a private cache allocation mechanism. Step 2: After the global scheduler receives a cumulative number of requests, it selects a cache allocation mapping mechanism based on a dynamic threshold of all received remote access requests; Step 3: The global scheduler formally issues a request to switch the state. After the switch is completed, it issues an instruction to switch the data cache allocation mechanism to the first-level data cache of the corresponding target address. When the global scheduler formally issues the request to switch the state, it first sends a global manual refresh request and a global invalidation cache synchronization request to the first-level data cache of the corresponding GPU cluster, allowing the first-level data cache to synchronize the stored data, and designates the first first-level data cache to forward the cache synchronization message to all second-level data caches for synchronization events, and issues an instruction to switch the cache allocation mechanism. Step 4: The first-level data cache completes its own cache allocation mechanism update and sends a switching instruction to the cache allocation control bit in the network interface. Then, the first-level data cache sends the switching request to the corresponding second-level data cache. Step 5: When the secondary data cache receives the switch request, it first checks whether the manual refresh request and global invalidation cache synchronization request forwarded by the first primary data cache to all secondary data caches in step 3 are completed. If not, it will temporarily pause and block the switch operation. After the data synchronization process is completed, it will continue to execute the state switch. After the switch is completed, the secondary data cache will return a confirmation message of the state switch completion to the primary data cache control unit; Step 6. After the first-level data cache receives the switching event response sent by the second-level data cache, it sends a response to the global scheduler indicating that the entire switching event is completed. Until the global scheduler receives a response corresponding to the number of requests, notifying the global scheduler that the switching has been completed, the global scheduler can confirm that the switching task of this round of cache allocation mechanism has been successfully completed and enter the next scheduling cycle.

5. The adaptive cache allocation method according to claim 4, wherein: In step 1, when the secondary data cache in each GPU core receives a read request that misses the primary data cache in the GPU core, the secondary data cache determines whether to select a shared cache allocation mechanism or a private cache allocation mechanism: If the current secondary data cache is allocated according to the private cache allocation mechanism, the secondary data cache collects access data within the global cluster and between GPU clusters. The secondary data cache compares its own physical shard storage unit number with the master node number corresponding to the target address mapping information of the address mapper to see if it matches. If the physical shard storage unit number matches the master node number, the access count of the local read request within the GPU cluster is increased in the counter of the global scheduler. If the physical shard storage unit number does not match the master node number, the remote access count is increased in the counter of the global scheduler. If the current secondary data cache is allocated according to the shared cache allocation mechanism, the secondary data cache compares its own physical shard storage unit and the requesting primary data cache to see if they belong to the same GPU cluster. If the physical shard storage unit and the primary data cache belong to the same GPU cluster, this indicates that this is a local access, and the number of local read request accesses within the GPU cluster needs to be increased in the global scheduler's counter. If the physical shard storage unit and the primary data cache do not belong to the same GPU cluster, this indicates that this is a cross-chip access, and the number of cross-chip request accesses needs to be increased in the global scheduler's counter.

6. The adaptive cache allocation method according to claim 4, wherein: In step 1, under the shared cache allocation mechanism, read requests to the L1 data cache are divided into local read requests and cross-chip read requests mapped to the L2 data caches of different GPU clusters. The latency calculation under the shared cache allocation mechanism is: ； Under the private cache allocation mechanism, the read requests for the first-level data cache are divided into local read requests and remote read requests that need to be made to the master node when the local second-level data cache misses. The delay calculation under the private cache allocation mechanism is: ， in, is the average access delay, is the read request latency under the shared cache allocation mechanism, is the latency of cross-chip read requests under the shared cache allocation mechanism, is the proportion of cross-chip read requests, Delay for local read requests, The read request delay from the secondary data cache to the directory when the secondary data cache is missed, is the L2 data cache miss rate, is the read request latency under the private cache allocation mechanism, The latency of remote access under the private cache allocation mechanism, The percentage of remote access.

7. The adaptive cache allocation method according to claim 4, wherein: In step 3, a global manual refresh request and a global invalidation cache synchronization request are sent to the first-level data cache of the corresponding cluster. The specific steps include the following: Step 3.1: Pause write operations on the first-level data cache: After the switch is triggered, first suspend write requests from the first-level data cache to the second-level data cache to ensure that no new dirty data is generated during the switch; Step 3.2: Mark dirty data and perform a refresh operation: Scan the cache control unit to be switched for dirty data and actively write valid data back to the memory to avoid data loss. Step 3.3: Invalidate the current cache copy: After writing back to memory, clear all cache line metadata in the cache control unit under the private cache allocation mechanism; Step 3.4: Switch cache mapping and access control policy: Officially enable the shared cache allocation mechanism or the access path with the cache allocation mechanism, and allow the first-level data cache to resume normal access.

Images