A Partitioning Method, Architecture, and Data Access Method for a Multi-Die GPU Cache

Abstract

The present application provides a method, architecture, and data access method for partitioning a multi-core GPU cache, relating to the field of graphics processor design technology. By introducing a topology-aware cache allocation mechanism, system-level optimization of cross-core access performance is achieved, thereby improving the performance of processing large-scale image data. The method comprises: obtaining a multi-core GPU topology structure including a target computing core and at least one associated computing core, wherein the target computing core's to-be-partitioned remote data cache is used to cache image data from a remote video memory, and is connected to the associated computing core via an inter-core interconnection network; if the number of computing cores included in the multi-core GPU does not exceed a first preset threshold, determining the number of inter-core hops between each associated computing core and the target computing core based on the topology structure, and then determining the corresponding capacity of the associated computing core in the to-be-partitioned cache of the target computing core; and partitioning the to-be-partitioned cache according to the capacity corresponding to each associated computing core.

Description

Technical Field

[0001] The present application relates to the technical field of graphics processor design, and in particular to a multi-core GPU cache partitioning method, architecture, and data access method. Background Art

[0002] Graphics Processing Units (GPUs), with their exceptional parallel computing capabilities and high throughput, have become the core computing platform for image data processing. However, as transistor physical limits approach, developing GPUs for large-scale image data processing based on process scaling faces challenges in yield, cost, and development cycle. Therefore, multi-chiplet GPU architectures, integrating multiple compute and storage cores, have become key to advancing large-scale image data processing capabilities.

[0003] In this architecture, multiple graphics processing compute cores (i.e., compute cores) are connected via an inter-core interconnect network, enabling collaborative operation to meet the performance requirements of large-scale image data processing. Each compute core is equipped with local video memory (i.e., storage core), and all video memories are mapped to the entire address space. However, as the number of compute cores increases, the maximum physical distance (also known as the number of hops) between a compute core and the video memory on other cores increases significantly. This results in significantly increased inter-core access latency during image data processing tasks, which has become a bottleneck restricting the performance of multi-core GPUs.

[0004] To address the above issues, existing research has adopted a caching technology based on remote data cache. This technology sets up an additional cache (i.e., remote data cache) in each computing core of a multi-core GPU to cache image data from remote video memory, and is shared by all computing units in the computing core. When a request from each computing unit misses the L1 cache inside the computing unit, if the requested address is in the address space of local video memory, the remote data cache is skipped and the local last-level cache and video memory are directly accessed. If the request address is in the address space of remote video memory, the remote data cache is accessed first. If the request hits the remote data cache, the data access can be completed within the computing core. At this time, there is no need to access the remote last-level cache and video memory through other computing cores, which greatly reduces the frequency of remote data access and improves the access performance of the multi-core GPU.

[0005] However, the current architecture based on remote data caching is relatively simple and suffers from a low cache hit rate when processing large-scale image data, which affects the further improvement of the overall performance of the multi-core GPU system and urgently needs to be optimized and improved through innovative solutions. Summary of the Invention

[0006] This application provides a multi-core GPU cache partitioning method, architecture, and data access method. By introducing a topology-aware cache allocation mechanism, it achieves system-level optimization of cross-core access performance and improves the performance of processing large-scale image data.

[0007] In a first aspect, a method for partitioning a multi-core GPU cache is provided, comprising:

[0008] Obtaining a topological structure of a multi-core GPU, the multi-core GPU including a target computing coreparticle for processing image data and at least one associated computing coreparticle, the target computing coreparticle being a computing coreparticle to be subjected to a cache partitioning operation, the associated computing coreparticle being connected to the target computing coreparticle via an inter-coreparticle interconnection network, the remote data cache to be partitioned of the target computing coreparticle being used to cache image data from a video memory of the at least one associated computing coreparticle;

[0009] If the number of computing cores included in the multi-core GPU does not exceed a first preset threshold, determining the number of inter-core hops between each associated computing core and the target computing core based on the topological structure;

[0010] Determine the cache capacity ratio corresponding to each associated computing core according to the number of inter-core hops corresponding to each associated computing core, wherein the number of inter-core hops corresponding to each associated computing core is positively correlated with the corresponding cache capacity ratio, and the cache capacity ratio is the ratio of the capacity required by the associated computing core to the capacity of the remote data cache to be divided;

[0011] Determine the capacity of each associated computing core in the remote data cache to be divided according to the cache capacity ratio corresponding to each associated computing core and the capacity of the remote data cache to be divided;

[0012] The remote data cache to be divided is divided according to the capacity corresponding to each associated computing core particle.

[0013] In one possible design, the method includes:

[0014] If the number of computing cores included in the multi-core GPU exceeds a first preset threshold, the multi-core GPU is divided into regions according to the topological structure to obtain a target region and at least one associated region, where the target region is the region to which the target computing core belongs, and each associated region is a region of the multi-core GPU other than the target region, each region including the same number of computing cores, and a maximum distance between computing cores in each region being less than a second preset threshold;

[0015] Based on the topology, determine the number of inter-region hops between each region and the target region;

[0016] Adjusting the number of inter-region hops corresponding to each region based on the first linear scaling factor and the first global offset constant;

[0017] Determine the cache capacity ratio corresponding to each region according to the adjusted inter-region hop count corresponding to each region, wherein the inter-region hop count corresponding to each region is positively correlated with the corresponding cache capacity ratio;

[0018] Determine the capacity of each region in the remote data cache to be divided according to the cache capacity ratio corresponding to each region and the capacity of the remote data cache to be divided;

[0019] The remote data cache to be divided is divided according to the capacity corresponding to each region, wherein the associated computing cores in each region share the storage space allocated to the region in the remote data cache to be divided.

[0020] In a feasible design, after the remote data cache to be divided is divided according to the capacity corresponding to each associated computing core, the capacity of the remote data cache to be divided is periodically re-divided. The step of re-dividing the capacity of the remote data cache to be divided after each division cycle includes:

[0021] Detecting the number of first requests by the target computing core particle to access the video memory of each associated computing core particle in a previous partitioning cycle;

[0022] Determining a first capacity that needs to be re-divided in the remote data cache to be divided;

[0023] Adjusting the first request quantity corresponding to each associated computing core particle based on a second linear scaling factor and a second global offset constant;

[0024] Determine, based on the adjusted first request quantity corresponding to each associated computing core, a first capacity ratio corresponding to each associated computing core, wherein the first capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core to the first capacity, and the adjusted first request quantity corresponding to each associated computing core is positively correlated with the corresponding first capacity ratio;

[0025] Determine the new capacity corresponding to each associated computing core particle according to the first capacity ratio, the original capacity, and the first capacity corresponding to each associated computing core particle, wherein the original capacity is the capacity corresponding to the associated computing core particle in the previous division cycle;

[0026] The part of the remote data cache to be divided is repartitioned according to the new capacity and the original capacity corresponding to each associated calculation core particle.

[0027] In a feasible design, based on the new capacity and original capacity corresponding to each associated calculation core, the portion of the remote data cache to be divided is repartitioned, including:

[0028] Sending a memory access delay awareness request to each associated computing core particle, where the memory access delay awareness request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle;

[0029] Determine the access delay value corresponding to each associated computing core particle according to the sending time and receiving time of each memory access delay awareness request;

[0030] Determining a second capacity that needs to be re-divided in the remote data cache to be divided;

[0031] Adjusting the access delay value corresponding to each associated computing core particle based on a third linear scaling factor and a third global offset constant;

[0032] Determining a second capacity ratio corresponding to each associated computing core according to the adjusted access delay value corresponding to each associated computing core, wherein the second capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core to the second capacity, and the adjusted access delay value corresponding to each associated computing core is positively correlated with the corresponding second capacity ratio;

[0033] Determine the final capacity corresponding to each associated calculation core particle according to the new capacity, the second capacity ratio, and the second capacity corresponding to each associated calculation core particle;

[0034] The final capacity and original capacity corresponding to each associated calculation core are used to repartition the part of the remote data cache to be partitioned.

[0035] In a feasible design, after the remote data cache to be divided is divided according to the capacity corresponding to each area, the capacity of the remote data cache to be divided is periodically re-divided. The steps of re-dividing the capacity of the remote data cache to be divided after each division cycle include:

[0036] Detecting the number of second requests by the target computing core to access the video memory of each region in the last partition cycle, where the video memory of each region includes the video memory of each associated computing core in the region;

[0037] Determining a third capacity that needs to be re-divided in the remote data cache to be divided;

[0038] Adjusting the second request quantity corresponding to each region based on a fourth linear scaling factor and a fourth global offset constant;

[0039] Determine a third capacity ratio corresponding to each region based on the adjusted second request quantity corresponding to each region, where the third capacity ratio is a ratio of the capacity required to be reallocated to the region to the third capacity, and the adjusted second request quantity corresponding to each region is positively correlated with the corresponding third capacity ratio;

[0040] Determine the new capacity corresponding to each region based on the third capacity ratio, the original capacity, and the third capacity corresponding to each region, wherein the original capacity is the capacity corresponding to the region in the previous division cycle;

[0041] The portion of the remote data cache to be divided is re-divided according to the new capacity and original capacity corresponding to each area.

[0042] In a feasible design, the portion of the remote data cache to be divided is repartitioned based on the new capacity and original capacity of each region, including:

[0043] Sending a memory access delay awareness request to each associated computing core particle, where the memory access delay awareness request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle;

[0044] Determine the access delay value corresponding to each associated computing core particle according to the sending time and receiving time of each memory access delay awareness request;

[0045] The sum of the access delay values corresponding to the associated calculation core particles in each region is determined as the access delay value corresponding to each region;

[0046] determining a fourth capacity to be re-divided in the remote data cache to be divided;

[0047] Adjusting the access delay value corresponding to each region based on a fifth linear scaling factor and a fifth global offset constant;

[0048] Determining a fourth capacity ratio corresponding to each region based on the adjusted access delay value corresponding to each region, wherein the fourth capacity ratio is a ratio of the capacity required to be reallocated to the region to the fourth capacity, and the adjusted access delay value corresponding to each region is positively correlated with the corresponding fourth capacity ratio;

[0049] Determine the final capacity corresponding to each area according to the new capacity corresponding to each area, the fourth capacity ratio, and the fourth capacity;

[0050] The remote data cache to be divided is re-divided according to the final capacity and original capacity corresponding to each area.

[0051] In a feasible design, the cache capacity ratio corresponding to each associated computing core particle is determined based on the number of inter-core particle hops corresponding to each associated computing core particle, including:

[0052] Adjusting the number of inter-core hops corresponding to each associated computation core based on a sixth linear scaling factor and a sixth global offset constant;

[0053] According to the adjusted inter-core hop count corresponding to each association calculation core, the cache capacity ratio corresponding to each association calculation core is determined.

[0054] In a second aspect, a multi-core GPU cache architecture is provided, including:

[0055] a topology perception unit, configured to obtain a topology structure of a multi-core GPU, the multi-core GPU comprising a target computing coreparticle for processing image data and at least one associated computing coreparticle, the target computing coreparticle being a computing coreparticle to be subjected to a cache partitioning operation, the associated computing coreparticle being connected to the target computing coreparticle via an inter-coreparticle interconnection network, the target computing coreparticle's remote data cache to be partitioned being configured to cache image data from a video memory of the at least one associated computing coreparticle;

[0056] a topology sensing unit, configured to determine, based on the topology structure, the number of inter-core hops between each associated computing core and the target computing core, if the number of computing cores included in the multi-core GPU does not exceed a first preset threshold;

[0057] a capacity calculation unit, configured to determine a cache capacity ratio corresponding to each associated computing core according to the number of inter-core hops corresponding to each associated computing core, wherein the number of inter-core hops corresponding to each associated computing core is positively correlated with the corresponding cache capacity ratio, and the cache capacity ratio is a ratio of the capacity required by the associated computing core to the capacity of the remote data cache to be divided;

[0058] The capacity calculation unit is further configured to determine the capacity corresponding to each associated computing core in the remote data cache to be divided according to the cache capacity ratio corresponding to each associated computing core and the capacity of the remote data cache to be divided;

[0059] The cache control unit is used to divide the remote data cache to be divided according to the capacity corresponding to each associated computing core particle.

[0060] In a feasible design, the remote data cache to be divided adopts a group-associative mapping structure, including multiple cache groups, each cache group containing multiple cache lines, and each cache line including a partition identification bit, which is used to store a video memory identification, and the video memory identification is used to identify the video memory to which the image data to be cached by the cache line belongs;

[0061] The cache control unit is used to divide the remote data cache to be divided according to the capacity corresponding to each associated computing core particle, including:

[0062] The cache control unit is used to determine the video memory identifier stored in the partition identifier of each cache line according to the capacity corresponding to each associated calculation core particle;

[0063] The cache control unit is used to divide the remote data cache to be divided by writing a corresponding video memory identifier into the division identifier bit of each cache line.

[0064] In a third aspect, a data access method for a multi-core GPU cache is provided, which is applied to a multi-core GPU cache architecture such as the aforementioned example. The multi-core GPU cache architecture includes a cache control unit and a remote data cache. The remote data cache adopts a group-associative mapping structure, including multiple cache groups, each cache group including multiple cache lines, and each cache line including a partition identification bit, the partition identification bit being configured to store a video memory identification or a region identification. The video memory identification is used to identify the video memory to which image data to be cached by the cache line belongs, and the region identification is used to identify the region to which the image data to be cached by the cache line belongs. The method includes:

[0065] The computing unit of the target computing core generates a request including a data source identifier, an access address, and an identifier of the computing unit, where the data source identifier is a video memory identifier or a region identifier;

[0066] If the request misses the L1 cache inside the computing unit and the access address includes the address of the remote video memory, a request is sent to the cache control unit;

[0067] The cache control unit parses the request and obtains the requested data source identifier, access address, and computing unit identifier;

[0068] The cache control unit locates the target cache group according to the access address;

[0069] The cache control unit performs request hit judgment on each cache line in the target cache group according to the storage order based on the access address and data source identifier;

[0070] If a hit occurs, the cache control unit extracts the target image data from the target cache line according to the access address and sends the target image data to the computing unit according to the identifier of the computing unit;

[0071] If there is no hit, the cache control unit sends an indication message to the computing unit, which is used to indicate that the target image data is not stored in the target cache group. After receiving the indication message, the computing unit sends a request to the target associated computing chiplet corresponding to the video memory mapped by the access address through the chiplet interconnection network.

[0072] The multi-core GPU cache partitioning method described in the embodiments of this application overcomes the limitations of traditional remote data cache-based architectures that do not partition the remote data cache. By introducing a topology-aware cache allocation mechanism, it achieves system-level optimization of the performance of cross-core access to image data. Based on a positive correlation design based on the ratio of the number of hops between cores and cache capacity, this application enables associated computing cores with longer access distances (i.e., higher hop counts) used to process image data to obtain a larger proportion of cache space, thereby reducing the probability of cache misses for these associated computing cores when processing image data, reducing access to remote video memory, and improving the performance of processing large-scale image data.

[0073] In addition, by limiting the number of computing cores included in the multi-core GPU to not exceed a first preset threshold, it is avoided that the cache is divided too finely due to an excessive number of computing cores, resulting in fragmentation of the remote data cache for storing image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0075] Figure 1 This is a schematic diagram of a multi-core GPU architecture provided by an exemplary embodiment of the present application;

[0076] Figure 2 This is a schematic flow chart of a multi-core GPU cache partitioning method provided by an exemplary embodiment of the present application;

[0077] Figure 3 This is another schematic diagram of a multi-core GPU architecture provided by an exemplary embodiment of the present application;

[0078] Figure 4 This is another schematic diagram of a multi-core GPU architecture provided by an exemplary embodiment of the present application;

[0079] Figure 5 This is another schematic diagram of a multi-core GPU architecture provided by an exemplary embodiment of the present application;

[0080] Figure 6 This is another schematic diagram of a multi-core GPU architecture provided by an exemplary embodiment of the present application;

[0081] Figure 7 This is a schematic diagram of a multi-core GPU cache architecture provided by an exemplary embodiment of the present application;

[0082] Figure 8 This is a schematic diagram of a cache line structure of a multi-core GPU provided by an exemplary embodiment of the present application;

[0083] Figure 9 This is a schematic diagram of an access address composition provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0085] Remote data cache-based caching technology is an important method for reducing the number of image data accesses to remote video memory and improving memory access performance in multi-chiplet GPUs. Traditional remote data cache-based architectures are relatively simple: image data from the video memory corresponding to each associated compute chip (referred to as remote video memory) uses the corresponding storage space in the remote data cache in the order in which the target compute chip accesses it. However, because the target compute chip has different access distances to different associated compute chips, the cost of requesting a missing address in the target compute chip's remote data cache for an associated compute chip with a smaller distance is different from the cost of requesting a missing address in the video memory space of an associated compute chip with a larger distance.

[0086] Below Figure 1 The multi-core GPU shown below is used as an example to illustrate the above problems:

[0087] Figure 1 The multi-core GPU shown includes four computing cores, namely computing core 0, computing core 1, computing core 2, and computing core 3. Each computing core corresponds to a video memory. The video memory corresponding to computing core 0 is video memory 0, the video memory corresponding to computing core 1 is video memory 1, the video memory corresponding to computing core 2 is video memory 2, and the video memory corresponding to computing core 3 is video memory 3. If there is a solid line connecting two computing cores, it means that the two computing cores can communicate directly without going through other computing cores. In other words, the access distance (i.e., the number of hops) between the two computing cores is 1. Each computing unit inside the computing core accesses the last-level cache through the on-chip interconnection network. Figure 1 For example, in the example of Compute Core 3, its remote data cache stores image data in the address space of Video Memory 0-2. For target Compute Core 3, the distances to access Video Memory 0 and Video Memory 1 are different. Therefore, in the remote data cache, the cost of a request that misses an address in Video Memory 0 is different from the cost of a request that misses an address in Video Memory 1.

[0088] In order to improve the overall performance of the graphics processor in processing image data, this application creatively proposes to allocate the capacity of the remote data cache according to the cost of request misses. Based on this, this application provides a multi-core GPU cache partitioning method, such as Figure 2 As shown, the method includes:

[0089] S110, obtaining a topology structure of a multi-core GPU.

[0090] Among them, the multi-core GPU includes a target computing core particle for processing image data and at least one associated computing core particle. The target computing core particle is the computing core particle to be subjected to cache partitioning operation. The associated computing core particle is connected to the target computing core particle through an inter-core particle interconnection network. The remote data cache to be partitioned of the target computing core particle is used to cache image data from the video memory of at least one associated computing core particle.

[0091] S120 : If the number of computing corelets included in the multi-core GPU does not exceed a first preset threshold, determine the number of inter-core hops between each associated computing corelet and the target computing corelet based on the topological structure.

[0092] To prevent the problem of excessive cache fragmentation due to an excessive number of computing cores, which in turn leads to remote data cache fragmentation, this application adopts different methods to partition the cache of multi-core GPUs whose number of computing cores exceeds a first preset threshold and those whose number does not exceed the first preset threshold, based on the number of computing cores contained in the multi-core GPU. If the number of computing cores contained in the multi-core GPU does not exceed the first preset threshold, the remote data cache to be divided of the target computing core is partitioned using methods S120-S150. The first preset threshold is set according to actual needs and is not limited by this application.

[0093] Exemplarily, the following method is used to determine the number of inter-core-grain hops between the target associated computing core-grain and the target computing core-grain based on the topological structure:

[0094] Get the preset topology of multi-core GPU;

[0095] Determine the target associated computing core particle to be routed based on the topological structure;

[0096] Send a route discovery request to the target associated computing core particle, which is used to record the number of forwarding times on the path between the target computing core particle and the target associated computing core particle;

[0097] The forwarding times are determined as the number of inter-core-grain hops between the target associated computation core-grain and the target computation core-grain.

[0098] like Figure 3As shown, taking the target computing core as computing core 3 and computing cores 0-2 as associated computing cores as an example, the inter-core hop numbers corresponding to computing cores 0-2 are 2, 1, and 1 respectively. Computing core 3 includes multiple computing units, each of which can access the remote data cache and the last-level cache through the on-chip interconnect network. The capacity of the remote data cache partition corresponding to video memory 0 in the remote data cache of computing core 3 is greater than the capacity of the remote data cache partition corresponding to video memory 1 and video memory 2 in the remote data cache of computing core 3, respectively. The capacity of the remote data cache partition corresponding to video memory 1 and video memory 2 in the remote data cache of computing core 3 is the same.

[0099] S130 , determining a cache capacity ratio corresponding to each association calculation core particle according to the number of inter-core particle hops corresponding to each association calculation core particle.

[0100] The cache capacity ratio is the ratio of the capacity required by the associated computing core to the capacity of the remote data cache to be divided. The number of inter-core hops corresponding to each associated computing core is positively correlated with the corresponding cache capacity ratio.

[0101] Exemplarily, this is achieved by the following method: determining the cache capacity ratio corresponding to each association calculation core particle according to the number of inter-core particle hops corresponding to each association calculation core particle:

[0102] Determine the sum of the number of hops between core particles corresponding to each associated calculation core particle;

[0103] The ratio of the number of inter-core hops corresponding to each associated computing core to the sum value is determined as the cache capacity ratio corresponding to the associated computing core.

[0104] In order to further accurately determine the cache capacity ratio, a feasible design is implemented as follows: the cache capacity ratio corresponding to each associated computing core particle is determined based on the number of inter-core hops corresponding to each associated computing core particle:

[0105] Adjusting the number of inter-core hops corresponding to each associated computation core based on a sixth linear scaling factor and a sixth global offset constant;

[0106] According to the adjusted inter-core hop count corresponding to each association calculation core, the cache capacity ratio corresponding to each association calculation core is determined.

[0107] It should be noted that the first linear scaling factor, the second linear scaling factor, the third linear scaling factor, the fourth linear scaling factor, the fifth linear scaling factor, and the sixth linear scaling factor of the present application are all used to linearly scale the number of inter-core hops. The first global offset constant, the second global offset constant, the third global offset constant, the fourth global offset constant, the fifth global offset constant, and the sixth global offset constant of the present application are all used to perform an overall translation operation on the number of inter-core hops after linear scaling.

[0108] Exemplarily, the method is implemented as follows: determining the cache capacity ratio corresponding to each association calculation core particle according to the adjusted inter-core particle hop count corresponding to each association calculation core particle:

[0109] Determine the sum of the adjusted inter-core hop counts corresponding to each associated computation core;

[0110] The ratio of the adjusted inter-core hop count corresponding to each associated computing core to the sum value is determined as the cache capacity ratio corresponding to the associated computing core.

[0111] For example, suppose the entire GPU system has The current requirement is for the computing core particle The remote data cache in the system is divided into capacity, and the associated computing core is set ( ) and computing core particles The distance is Jump, then the video memory The corresponding cache capacity ratio is ,in, and All represent the numbers of the associated computing core particles. Represents the associated computing core particle Computing core particles The distance is Jump. is the sixth linear scaling factor, which may be the image data access delay between chiplets without resource contention, is the sixth global offset constant, which may be the delay of the computing unit in the chiplet accessing the local video memory.

[0112] For example, suppose =30, =300. Figure 3 For example, the number of inter-chiplet hops corresponding to the calculation corelets 0-2 is 2, 1, and 1 respectively. The unadjusted cache capacity ratios are 1 / 2, 1 / 4, and 1 / 4 respectively, and the adjusted cache capacity ratios are 6 / 17, 11 / 34, and 11 / 34 respectively. It is worth mentioning that since inter-chiplet communication usually involves resource contention, Taking 30 is just a special case. and The value of is not limited.

[0113] As can be seen, the above embodiment, by introducing a linear scaling factor and a global offset constant to collaboratively adjust the number of hops between cores, can suppress extreme allocations and make cache capacity division more reasonable. For example, before the adjustment, the cache capacity of computing core 0 accounted for 50% (1 / 2), posing a risk of single-core monopoly. After the adjustment, the cache capacity of computing core 0 decreased to 35.3% (6 / 17 ≈ 0.353), and computing cores 1 and 2 each accounted for 32.35% (11 / 34 ≈ 0.3235). The maximum / minimum capacity ratio was reduced from 2:1 (before the adjustment) to 1.09:1 (after the adjustment), avoiding imbalanced resource allocation. This achieves a smooth attenuation of hop count differences, avoiding polarized allocation differences.

[0114] In addition, the linear scaling factor and global offset constant can be dynamically configured, supporting runtime adjustment of cache allocation strategies based on factors such as load characteristics, thus improving the solution's flexible adjustment capabilities.

[0115] S140 , determining the capacity corresponding to each associated computing core in the remote data cache to be divided according to the cache capacity ratio corresponding to each associated computing core and the capacity of the remote data cache to be divided.

[0116] For example Figure 3 As shown in the figure, for the remote data cache in computing core 3, the total capacity is set to R. The corresponding cache capacity of video memory 0 is 6 / 17R, and the cache capacity of video memory 1 and video memory 2 is 11 / 34R. In the specific implementation process, assuming that the remote data cache is a 256-way cache, after rounding, 90 ways are allocated to cache image data of video memory 0, 83 ways are allocated to cache image data of video memory 1, and 83 ways are allocated to cache image data of video memory 2.

[0117] S150 , dividing the remote data cache to be divided according to the capacity corresponding to each associated computing core.

[0118] Exemplarily, according to the serial numbering sequence of the graphics memories of the associated computing core particles, the graphics memories of the associated computing core particles are mapped to the remote data cache to be divided according to the corresponding capacity.

[0119] The multi-core GPU cache partitioning method described in the embodiments of this application overcomes the limitations of traditional remote data cache-based architectures that do not partition the remote data cache. By introducing a topology-aware cache allocation mechanism, it achieves system-level optimization of the performance of cross-core access to image data. Based on a positive correlation design based on the ratio of the number of hops between cores and cache capacity, this application enables associated computing cores with longer access distances (i.e., higher hop counts) used to process image data to obtain a larger proportion of cache space, thereby reducing the probability of cache misses for these associated computing cores when processing image data, reducing access to remote video memory, and improving the performance of processing large-scale image data.

[0120] In addition, by limiting the number of computing cores included in the multi-core GPU to not exceed a first preset threshold, it is avoided that the cache is divided too finely due to an excessive number of computing cores, resulting in fragmentation of the remote data cache for storing image data.

[0121] The above embodiment method can be used in a multi-core GPU system of any topological shape. According to the remote data cache partitioning method proposed in this application, after determining the distance between different computing cores, the cache partition ratio within each computing core can be calculated.

[0122] For example, Figure 4 The topology shown includes four computing cores connected in series, namely computing core 0, computing core 1, computing core 2, and computing core 3. Each computing core corresponds to a video memory. The video memory corresponding to computing core 0 is video memory 0, the video memory corresponding to computing core 1 is video memory 1, the video memory corresponding to computing core 2 is video memory 2, and the video memory corresponding to computing core 3 is video memory 3. Among them, computing core 3 includes multiple computing units, each of which can access remote data cache and access the last-level cache through the on-chip interconnection network. For computing core 3, the distances between computing cores 0, 1, and 2 are 3 hops, 2 hops, and 1 hop respectively. If =30, =300, and the total cache capacity is R. The cache capacity is divided into 13 / 36R, 1 / 3R, and 11 / 36R, respectively, to cache image data for video memory 0, video memory 1, and video memory 2. It can be seen that the capacity of the remote data cache partition corresponding to video memory 0 in the remote data cache of computing core 3 is larger than the capacity of the remote data cache partition corresponding to video memory 1 in the remote data cache of computing core 3. The capacity of the remote data cache partition corresponding to video memory 1 in the remote data cache of computing core 3 is larger than the capacity of the remote data cache partition corresponding to video memory 2 in the remote data cache of computing core 3.

[0123] For example, Figure 5The topology shown includes four computing cores capable of direct communication between each other: computing core 0, computing core 1, computing core 2, and computing core 3. Each computing core corresponds to a video memory. The video memory corresponding to computing core 0 is video memory 0, the video memory corresponding to computing core 1 is video memory 1, the video memory corresponding to computing core 2 is video memory 2, and the video memory corresponding to computing core 3 is video memory 3. Computing core 3 includes multiple computing units, each of which can access the remote data cache and the last-level cache through the on-chip interconnect network. The number of hops between each computing core is 1. For each computing core's remote data cache, the capacity of the image data cached in the other three video memories is the same, all 1 / 3R. For example, in computing core 3, it can be seen that the capacity of the corresponding remote data cache partitions of video memory 0, video memory 1, and video memory 2 in the remote data cache of computing core 3 is the same.

[0124] In a feasible design, after the remote data cache to be divided is divided according to the capacity corresponding to each associated computing core, the capacity of the remote data cache to be divided is periodically re-divided. The step of re-dividing the capacity of the remote data cache to be divided after each division cycle includes:

[0125] Detecting the number of first requests by the target computing core particle to access the video memory of each associated computing core particle in a previous partitioning cycle;

[0126] Determining a first capacity that needs to be re-divided in the remote data cache to be divided;

[0127] Adjusting the first request quantity corresponding to each associated computing core particle based on a second linear scaling factor and a second global offset constant;

[0128] Determine, based on the adjusted first request quantity corresponding to each associated computing core, a first capacity ratio corresponding to each associated computing core, wherein the first capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core to the first capacity, and the adjusted first request quantity corresponding to each associated computing core is positively correlated with the corresponding first capacity ratio;

[0129] Determine the new capacity corresponding to each associated computing core particle according to the first capacity ratio, the original capacity, and the first capacity corresponding to each associated computing core particle, wherein the original capacity is the capacity corresponding to the associated computing core particle in the previous division cycle;

[0130] The part of the remote data cache to be divided is repartitioned according to the new capacity and the original capacity corresponding to each associated calculation core particle.

[0131] The second linear scaling factor and the second global offset constant are set according to actual needs, and this application does not limit this.

[0132] For example, to calculate the core particle As an example, after static partitioning, the calculation core particle Associative computing core in remote data cache The number of cache lines corresponding to the video memory on (i.e., original capacity), where , is the number of computing cores included in the multi-core GPU system. A clock cycle is a division cycle for re-dividing the remote data cache (for example After statically partitioning the remote data cache to be partitioned based on the capacity corresponding to each associated computing core, periodically repartitioning the first capacity portion of the remote data cache to be partitioned by calculating the new capacity corresponding to each associated computing core. Specifically, each repartition of the remote data cache to be partitioned includes the following steps:

[0133] (1) Detection of computing core particles The number of times (i.e., the first request quantity) that the target computing core (i.e., the target computing core) accesses each associated computing core in the last partition cycle. The first requested amount of video memory is ,in Indicates the number of the associated computing core particle.

[0134] (2) Determine that the first capacity of the remote data cache to be re-divided is 1 / m1 of the total number of cache lines, where m1 is a preset constant.

[0135] (3) Based on the second linear scaling factor and a second global offset constant , adjust the first request quantity corresponding to each associated computing core particle to obtain the adjusted first request quantity, wherein the associated computing core particle The corresponding adjusted first request quantity is ( ).

[0136] (4) Determine the capacity ratio based on the adjusted first request quantity: Sum the first request quantities corresponding to each associated computing core particle to obtain .in, Represents the associated computing core particle The corresponding first request quantity. Therefore, the associated computing core particle The corresponding first capacity ratio is .

[0137] (5) Determine the new capacity corresponding to each associated computing core particle based on the first capacity ratio, original capacity, and first capacity corresponding to each associated computing core particle. Specifically, it can be considered that 1 / m1 of the capacity corresponding to the original capacity of each associated computing core particle is first taken out to participate in the redivision process. Then, the capacity taken out from each associated computing core particle constitutes the first capacity, and the remaining capacity corresponding to each associated computing core particle is (m1-1) / m1 of the original capacity. At this time, the capacity required to be redistributed by the associated computing core particle refers to the capacity that needs to be allocated to the associated computing core particle in the first capacity during this redivision process. Therefore, for the associated computing core particle , new capacity = original capacity * (m1-1) / m1 + first capacity ratio * first capacity.

[0138] (6) Considering that only part of the cache in the remote data cache is repartitioned, the image data in the cache that has not been repartitioned can be retained to improve the partitioning efficiency. Therefore, by comparing the new capacity and the original capacity corresponding to each associated computing core, the cache of the remote data cache to be partitioned is repartitioned. Among them, if the new capacity corresponding to the associated computing core is the same as the original capacity, there is no need to allocate a new cache line to the associated computing core. If the new capacity corresponding to the associated computing core is less than the original capacity, the image data of the cache line of the corresponding capacity of the associated computing core is cleared to facilitate allocation to other associated computing cores. If the new capacity corresponding to the associated computing core is greater than the original capacity, the cache line that originally belonged to other associated computing cores but had its image data cleared is allocated to the associated computing core.

[0139] The dynamic cache partitioning mechanism of the present application combines periodic detection with progressive adjustment to achieve efficient utilization of cache resources and dynamic adaptation of loads when a multi-core GPU system processes large-scale image data. Specifically, the number of requests for the target computing core to access the video memory of each associated computing core is periodically detected, and the number of historical access requests is converted into a weight ratio in combination with the second linear scaling factor and the second global offset constant, so that the frequently accessed associated computing cores obtain more cache capacity to cache more image data. In addition, the introduction of the linear scaling factor and the global offset enhances the robustness of the algorithm. When the request amount of a certain associated computing core is zero, the offset can prevent its cache capacity from returning to zero, so as to ensure the baseline resources of the zero-request associated computing core.

[0140] Furthermore, by limiting the dynamic reallocation of only part of the cache capacity at a time, the stability of most of the cache can be maintained, avoiding performance fluctuations caused by global adjustments, and effectively balancing the flexible adjustment performance and stable performance of the system cache.

[0141] In a feasible design, before dividing the cache based on the number of requests corresponding to each associated computing core in the previous division cycle, the cache can be further divided based on the access latency factor corresponding to the graphics memory of each associated computing core during operation. The specific steps include:

[0142] Sending a memory access delay awareness request to each associated computing core particle, where the memory access delay awareness request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle;

[0143] Determine the access delay value corresponding to each associated computing core particle according to the sending time and receiving time of each memory access delay awareness request;

[0144] Determining a second capacity that needs to be re-divided in the remote data cache to be divided;

[0145] Adjusting the access delay value corresponding to each associated computing core particle based on a third linear scaling factor and a third global offset constant;

[0146] Determining a second capacity ratio corresponding to each associated computing core according to the adjusted access delay value corresponding to each associated computing core, wherein the second capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core to the second capacity, and the adjusted access delay value corresponding to each associated computing core is positively correlated with the corresponding second capacity ratio;

[0147] Determine the final capacity corresponding to each associated calculation core particle according to the new capacity, the second capacity ratio, and the second capacity corresponding to each associated calculation core particle;

[0148] The final capacity and original capacity corresponding to each associated calculation core are used to repartition the part of the remote data cache to be partitioned.

[0149] Among them, the third linear scaling factor and the third global offset constant are set according to actual needs, and this application does not limit this.

[0150] For example, target computing core As an example, the remote data cache partition in the example is used to determine the associated computing core particles based on the number of requests. The corresponding new capacity is Further combining the access delay factors corresponding to the graphics memory of each associated computing core in the running process, the remote data cache is divided into the following steps:

[0151] (1) Send a memory access delay awareness request to each associated computing core particle. When the associated computing core particle receives the memory access delay awareness request, it immediately returns the request.

[0152] (2) By recording the time when the request is issued and the time when the request is received, the target computing core particle in the current state is calculated. The access delay value of each associated chip memory is calculated (i.e. the time difference between the time when the request is received and the time when the request is issued). The corresponding access delay value is recorded as .

[0153] (3) Determine that the second capacity to be re-divided in the remote data cache to be divided is 1 / h1 of the total number of cache lines in the cache group, where h1 is a preset constant.

[0154] (4) Based on the third linear scaling factor and the third global offset constant , adjust the access delay value corresponding to each associated computing core particle to obtain the adjusted access delay value, where the associated computing core particle The corresponding adjusted access delay value is ( ).

[0155] (5) Determine the second capacity ratio based on the adjusted access delay value: Sum the access delay values corresponding to each associated calculation core particle to obtain .in, Represents the associated computing core particle The corresponding access delay value. Therefore, the associated computing core The corresponding second capacity ratio is .

[0156] (6) Determine the final capacity corresponding to each associated computing core particle based on the new capacity, the second capacity ratio, and the second capacity corresponding to each associated computing core particle. Specifically, it can be regarded as first taking out 1 / h1 of the capacity corresponding to the new capacity of each associated computing core particle to participate in the redivision process. Then, the capacity taken out corresponding to each associated computing core particle constitutes the second capacity, and the remaining capacity corresponding to each associated computing core particle is (h1-1) / h1 of the new capacity. At this time, the capacity required to be redistributed by the associated computing core particle refers to the capacity that needs to be allocated to the associated computing core particle in the second capacity during this redivision process. Therefore, for the associated computing core particle , final capacity = new capacity * (h1-1) / h1 + second capacity ratio * second capacity.

[0157] (7) Considering that only part of the cache in the remote data cache is repartitioned, the image data in the cache that is not repartitioned can be retained to improve the partitioning efficiency. Therefore, by comparing the final capacity corresponding to each associated computing core particle with the original capacity, the part of the cache to be partitioned of the remote data cache is repartitioned. The specific method is referred to the scheme of repartitioning the part of the cache to be partitioned of the remote data cache by comparing the new capacity corresponding to each associated computing core particle with the original capacity in the aforementioned embodiment, which will not be repeated here.

[0158] In order to further accurately and dynamically divide the remote data cache and further improve the performance of the multi-core GPU system in processing image data, this application not only dynamically divides the remote data cache based on the number of requests corresponding to each associated computing core, but also takes into account the additional delay factors caused by bandwidth contention on the path when the target computing core goes to different associated computing cores. Specifically, the time delay of the target computing core in the current state accessing each associated computing core is periodically detected, and the access delay value is converted into a weight ratio in combination with the third linear scaling factor and the third global offset constant, so that the associated computing cores with higher access delays obtain more cache capacity to cache more image data. In addition, the introduction of the linear scaling factor and the global offset achieves a smooth attenuation of the hop count difference and avoids polarization of distribution differences.

[0159] Furthermore, by limiting the dynamic reallocation of only part of the cache capacity at a time, the stability of most of the cache can be maintained, avoiding performance fluctuations caused by global adjustments, and effectively balancing the flexible adjustment performance and stable performance of the system cache.

[0160] It should be understood that the cache may also be periodically dynamically partitioned based solely on the access delay values corresponding to the memory of each associated computing core particle, the steps comprising: after each partitioning period, sending a memory access delay awareness request to each associated computing core particle, the memory access delay awareness request being used to trigger round-trip communication between the target computing core particle and the associated computing core particle;

[0161] Determine the access delay value corresponding to each associated computing core particle according to the sending time and receiving time of each memory access delay awareness request;

[0162] Determining a second capacity that needs to be re-divided in the remote data cache to be divided;

[0163] Adjusting the access delay value corresponding to each associated computing core particle based on a third linear scaling factor and a third global offset constant;

[0164] Determining a second capacity ratio corresponding to each associated computing core particle according to the adjusted access delay value corresponding to each associated computing core particle, wherein the adjusted access delay value corresponding to each associated computing core particle is positively correlated with the corresponding second capacity ratio;

[0165] Determine the final capacity corresponding to each associated calculation core particle according to the original capacity, the second capacity ratio, and the second capacity corresponding to each associated calculation core particle;

[0166] The final capacity and original capacity corresponding to each associated calculation core are used to repartition the part of the remote data cache to be partitioned.

[0167] For specific examples and effects, please refer to the above embodiments and will not be repeated here.

[0168] It should be noted that the above-mentioned solution for dynamically partitioning the remote data cache is applicable to multi-core GPU systems with any topology, such as Figure 4 and Figure 5 The topological structure shown is described in detail in the above examples, and this application will not elaborate on it here.

[0169] This application takes into account the limited number of cache lines in each cache group to be divided into for the remote data cache. When the number of computing cores in a multi-core GPU system is greater than the number of cache lines, the video memory of each computing core must be allocated to at least one cache line in each cache group, making it impossible to divide the data according to the scheme of the aforementioned embodiment. Furthermore, even if the number of cache lines is greater than the number of computing cores in a multi-core GPU system, when the number of computing cores in a multi-core GPU system is too large, the storage space required for the partitioning flag bits in the cache lines used to record the cache video memory identifier is relatively large, wasting on-chip storage resources and reducing the efficiency of accessing image data in remote video memory in large-scale image data processing tasks.

[0170] To address the above issues, this application implements remote data cache partitioning by partitioning large-scale multi-core GPUs. The method includes:

[0171] If the number of computing cores included in the multi-core GPU exceeds a first preset threshold, the multi-core GPU is divided into regions according to the topological structure to obtain a target region and at least one associated region, where the target region is the region to which the target computing core belongs, and each associated region is a region of the multi-core GPU excluding the target region, that is, the associated region is a region of the multi-core GPU that does not include the target computing core, each region includes the same number of computing cores, and the maximum distance between the computing cores in each region is less than a second preset threshold;

[0172] Based on the topology, determine the number of inter-region hops between each region and the target region;

[0173] Adjusting the number of inter-region hops corresponding to each region based on the first linear scaling factor and the first global offset constant;

[0174] Determine the cache capacity ratio corresponding to each region according to the adjusted inter-region hop count corresponding to each region, wherein the inter-region hop count corresponding to each region is positively correlated with the corresponding cache capacity ratio;

[0175] Determine the capacity of each region in the remote data cache to be divided according to the cache capacity ratio corresponding to each region and the capacity of the remote data cache to be divided;

[0176] The remote data cache to be divided is divided according to the capacity corresponding to each region, wherein the associated computing cores in each region share the storage space allocated to the region in the remote data cache to be divided.

[0177] Among them, the first linear scaling factor and the first global offset constant are set according to actual needs, and this application does not limit this.

[0178] The second preset threshold is set according to actual needs and is not limited in this application.

[0179] It should be noted that since the target area includes associated computing core particles in addition to the target computing core particles, it is also necessary to allocate a certain capacity of cache for the target area in the remote data cache to be divided to cache the video memory image data of the associated computing core particles in the target area.

[0180] The following example illustrates the area division method:

[0181] like Figure 6 As shown, the multi-core GPU includes 16 computing cores, namely computing core 0, computing core 1, computing core 2, computing core 3, computing core 4, computing core 5, computing core 6, computing core 7, computing core 8, computing core 9, computing core 10, computing core 11, computing core 12, computing core 13, computing core 14, and computing core 15. Each computing core corresponds to a video memory, and the video memories corresponding to computing cores 0-15 are video memory 0, video memory 1, video memory 2, video memory 3, video memory 4, video memory 5, video memory 6, video memory 7, video memory 8, video memory 9, video memory 10, video memory 11, video memory 12, video memory 13, video memory 14, and video memory 15, respectively. Computing core 15 includes multiple computing units, each of which can access remote data cache and access the last-level cache through the on-chip interconnect network.

[0182] Figure 6 The topological structure shown can be divided into four areas, namely area 0, area 1, area 2, and area 3. The structure inside each area is the same as Figure 3It can be seen that when dividing the regions, the computational core particles that are adjacent in the topological space are divided into the same region.

[0183] In this case, the remote cache capacity of each compute corelet is divided into regions. In other words, a cache line is allocated to each region in the remote data cache corresponding to the target compute corelet. For example, the target compute corelet is compute corelet 15, located in region 3. The video memory address space it needs to cache corresponds to region 0 (video memory 0-5), region 1 (video memory 2-7), region 2 (video memory 8-13), and region 3 (video memory 10-14).

[0184] The following example illustrates how to calculate the corresponding capacity of a region:

[0185] Assume the entire system has Computing core particles, divided into Regions, currently need to be region (i.e., target area) Remote data cache is divided into capacity. Set area (i.e. associated area) ) and region The distance is Jump, then area The corresponding cache capacity ratio is .in, 、 、 These are the area numbers. Indicates area and region The distance is Jump.

[0186] For example Figure 6 As shown in the figure, for the computational core particle 15, it belongs to region 3, its distance from region 0 is 2, its distance from regions 1 and 2 is 1, and its distance from region 3 is 0. is 30, is 300. Using the above formula, we can calculate that 3 / 11 of the capacity of the computing core 15 is used to cache image data in region 0, 1 / 4 of the capacity is used to cache image data in region 1, 1 / 4 of the capacity is used to cache image data in region 2, and 5 / 22 of the capacity is used to cache image data in region 3. Assuming that the cache group includes a 256-way cache, the capacities corresponding to regions 0-3 are 70-way, 64-way, 64-way, and 58-way, respectively. The 70-way cache is shared by requests from video memory 0, 1, 4, and 5 in region 1, and the others are similar. It can be seen that the capacity of the remote data cache partition corresponding to region 0 in the remote data cache of the computing core 15 is the largest, the capacities of the remote data cache partitions corresponding to regions 1 and 2 in the remote data cache of the computing core 15 are the same, and the capacity of the remote data cache partition corresponding to region 3 in the remote data cache of the computing core 15 is the smallest.

[0187] To prevent cache fragmentation and reduced utilization caused by an excessive number of compute cores, the multi-core GPU cache partitioning method described in an embodiment of the present application partitions the multi-core GPU into regions when the number of compute cores in the multi-core GPU exceeds a first preset threshold. The region partitioning condition is "each region contains the same number of compute cores, and the maximum distance between the compute cores in each region is less than a second preset threshold." Multiple adjacent compute cores are merged into the same region, allowing the compute cores within the same region to share remote data cache space. The equal partitioning rule of "each region contains the same number of compute cores" ensures fairness in cache allocation within each region. The constraint of "the maximum distance is less than the second preset threshold" ensures that the maximum distance between cores within a region is controllable, preventing high hop counts between compute cores within a region from offsetting the performance gains of optimizing capacity allocation based on inter-region hop counts. Consequently, the present application allocates cache capacity based on inter-region hop counts, allowing more distant regions to receive a larger proportion of cache space, reducing the frequency of access to remote video memory for image data and increasing the hit rate of remote video memory image data, thereby improving overall performance.

[0188] Additionally, the algorithm's robustness is enhanced by adjusting the inter-region hop count using a first linear scaling factor and a first global offset constant. When the inter-region hop count is zero (i.e., the number of hops between the target region and the target region is zero), the first global offset constant prevents the corresponding cache capacity from being reset to zero, thus preventing the associated compute cores in the target region from being left without capacity in the target compute core's remote data cache.

[0189] In summary, the above method can ensure that the cache capacity of different areas is different, increase the hit rate of image data in farther video memory, and improve overall performance; it can also prevent the excessive number of cores from causing the cache to be over-partitioned and resulting in a decrease in utilization.

[0190] In a feasible design, after the remote data cache to be divided is divided according to the capacity corresponding to each area, the capacity of the remote data cache to be divided is periodically re-divided. The steps of re-dividing the capacity of the remote data cache to be divided after each division cycle include:

[0191] Detecting the number of second requests by the target computing core to access the video memory of each region in the last partition cycle, where the video memory of each region includes the video memory of each associated computing core in the region;

[0192] Determining a third capacity that needs to be re-divided in the remote data cache to be divided;

[0193] Adjusting the second request quantity corresponding to each region based on a fourth linear scaling factor and a fourth global offset constant;

[0194] Determine a third capacity ratio corresponding to each region based on the adjusted second request quantity corresponding to each region, where the third capacity ratio is a ratio of the capacity required to be reallocated to the region to the third capacity, and the adjusted second request quantity corresponding to each region is positively correlated with the corresponding third capacity ratio;

[0195] Determine the new capacity corresponding to each region based on the third capacity ratio, the original capacity, and the third capacity corresponding to each region, wherein the original capacity is the capacity corresponding to the region in the previous division cycle;

[0196] The portion of the remote data cache to be divided is re-divided according to the new capacity and original capacity corresponding to each area.

[0197] Among them, the fourth linear scaling factor and the fourth global offset constant are set according to actual needs, and this application does not limit this.

[0198] It should be understood that the second request quantity corresponding to each region is the sum of the request quantities of the target computing core particles accessing the region from the associated computing core particles.

[0199] For example, in the region Target computing core in As an example, after static partitioning, the calculation core particle In the remote data cache, the area The number of cache lines corresponding to the video memory is (i.e., original capacity), where ), The number of regions included in the multi-core GPU system. A clock cycle is a division cycle for re-dividing the remote data cache (for example After statically partitioning the remote data cache to be partitioned according to the capacity corresponding to each region, periodically repartitioning the portion of the cache with the third capacity in the remote data cache to be partitioned by calculating the new capacity corresponding to each region. Specifically, each repartitioning of the remote data cache to be partitioned includes the following steps:

[0200] (1) Detection of computing core particles The number of times (i.e., the second request quantity) that the target computing core accesses the video memory of each region in the last partition cycle. The corresponding second request quantity is , For the region The sum of the number of requests corresponding to each associated calculation core particle in Indicates the area number.

[0201] (2) Determine that the third capacity to be re-divided in the remote data cache to be divided is 1 / m2 of the total number of cache lines in the cache group, where m2 is a preset constant.

[0202] (3) Based on the fourth linear scaling factor and the fourth global offset constant , adjust the second request quantity corresponding to each region to obtain the adjusted second request quantity, where region The corresponding adjusted second request quantity is ( ).

[0203] (4) Determine the capacity ratio based on the adjusted second request quantity: Sum the second request quantities corresponding to each area to obtain , therefore, the area The corresponding third capacity ratio is .

[0204] (5) Determine the new capacity corresponding to each area based on the third capacity ratio, original capacity and third capacity corresponding to each area. Specifically, it can be seen as first taking out 1 / m2 of the original capacity corresponding to each area to participate in the redivision process. Then the capacity taken out from each area constitutes the third capacity, and the remaining capacity corresponding to each area is (m2-1) / m2 of the original capacity. At this time, the capacity required to be redistributed by the area refers to the capacity that needs to be allocated to the area in the third capacity during this redivision process. Therefore, for the area , new capacity = original capacity * (m2-1) / m2 + third capacity ratio * third capacity.

[0205] (6) Considering that only part of the cache in the remote data cache is repartitioned, the image data in the cache that is not repartitioned can be retained to improve the partitioning efficiency. Therefore, by comparing the size of the new capacity and the original capacity corresponding to each area, the part of the cache to be partitioned in the remote data cache is repartitioned. Among them, if the area If the new capacity is the same as the original capacity, no new cache line needs to be allocated for the region. If the corresponding new capacity is smaller than the original capacity, the image data of the cache line of the corresponding capacity of the area will be cleared to facilitate allocation to other areas. If the corresponding new capacity is larger than the original capacity, cache lines released from other areas will be allocated to this area.

[0206] The dynamic cache partitioning mechanism of the present application realizes efficient utilization of cache resources and dynamic load adaptation in a multi-core GPU system by combining periodic detection with progressive adjustment. Specifically, the target calculation core is detected periodically to calculate the number of requests for accessing the video memory of each area. Combined with the fourth linear scaling factor and the fourth global offset constant, the number of historical access requests is converted into a weight ratio, so that the frequently accessed areas obtain more cache capacity to cache more image data. In addition, the introduction of the linear scaling factor and the global offset enhances the robustness of the algorithm. When the number of requests in a certain area is zero, the offset can prevent its cache capacity from returning to zero, so as to ensure the baseline resources of the computing cores in the zero-request area.

[0207] Furthermore, by limiting the dynamic reallocation of only part of the cache capacity at a time, the stability of most of the cache can be maintained, avoiding performance fluctuations caused by global adjustments, and effectively balancing the flexible adjustment performance and stable performance of the system cache.

[0208] In a feasible design, before dividing the cache based on the number of requests corresponding to each area in the previous division cycle, the cache can be further divided based on the access latency factors corresponding to the video memory in each area during operation. The specific steps include:

[0209] Sending a memory access delay awareness request to each associated computing core particle, where the memory access delay awareness request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle;

[0210] Determine the access delay value corresponding to each associated computing core particle according to the sending time and receiving time of each memory access delay awareness request;

[0211] The sum of the access delay values corresponding to the associated calculation core particles in each region is determined as the access delay value corresponding to each region;

[0212] determining a fourth capacity to be re-divided in the remote data cache to be divided;

[0213] Adjusting the access delay value corresponding to each region based on a fifth linear scaling factor and a fifth global offset constant;

[0214] Determining a fourth capacity ratio corresponding to each region based on the adjusted access delay value corresponding to each region, wherein the fourth capacity ratio is a ratio of the capacity required to be reallocated to the region to the fourth capacity, and the adjusted access delay value corresponding to each region is positively correlated with the corresponding fourth capacity ratio;

[0215] Determine the final capacity corresponding to each area according to the new capacity corresponding to each area, the fourth capacity ratio, and the fourth capacity;

[0216] The remote data cache to be divided is re-divided according to the final capacity and original capacity corresponding to each area.

[0217] For example, the region Target computing core in As an example, the remote data cache partition in the example is set to be determined by the number of requests. The corresponding new capacity is Further combining the access delay factors corresponding to each area of video memory during operation, the remote data cache is divided into the following steps:

[0218] (1) Send a memory access delay awareness request to each associated computing core particle. When the associated computing core particle receives the memory access delay awareness request, it immediately returns the request.

[0219] (2) By recording the time when the request is issued and received, the delay between the target computing core a and the core memory of each associated computing core is calculated in the current state.

[0220] (3) The sum of the access delay values corresponding to each associated calculation core particle in each region is determined as the access delay value corresponding to each region. The corresponding access delay value is recorded as , For the region The sum of the access delay values corresponding to each associated calculation core particle.

[0221] (4) Determine that the fourth capacity to be re-divided in the remote data cache to be divided is 1 / h2 of the total number of cache lines in the cache group, where h2 is a preset constant.

[0222] (5) Based on the fifth linear scaling factor and the fifth global offset constant , adjust the access delay value corresponding to each area to obtain the adjusted access delay value, where the area The corresponding adjusted access delay value is ( ).

[0223] (6) Determine the fourth capacity ratio based on the adjusted access delay value: Sum the access delay values corresponding to each area to obtain .in, Indicates area The corresponding access delay value. Therefore, the area The corresponding fourth capacity ratio is .

[0224] (7) Determine the final capacity corresponding to each region based on the new capacity, the fourth capacity ratio, and the fourth capacity corresponding to each region. Specifically, it can be considered that 1 / h2 of the capacity corresponding to each region is first taken out from the new capacity corresponding to each region to participate in the redivision process. Then the capacity taken out from each region constitutes the fourth capacity, and the remaining capacity corresponding to each region is (h2-1) / h2 of the new capacity. At this time, the capacity required to be redistributed by the region refers to the capacity that needs to be allocated to the region in the fourth capacity during this redivision process. Therefore, for the region , final capacity = new capacity * (h2-1) / h2 + fourth capacity ratio * fourth capacity.

[0225] (8) Considering that only part of the cache in the remote data cache is repartitioned, the image data in the cache that is not repartitioned can be retained to improve the partitioning efficiency. Therefore, by comparing the final capacity corresponding to each area with the original capacity, the part of the cache to be partitioned of the remote data cache is repartitioned. The specific method is referred to the scheme of repartitioning the part of the cache to be partitioned of the remote data cache by comparing the new capacity corresponding to each area with the original capacity in the aforementioned embodiment, which will not be repeated here.

[0226] In order to further accurately and dynamically divide the remote data cache, this application not only dynamically divides the remote data cache based on the number of requests corresponding to each area, but also takes into account the additional delay factors caused by bandwidth contention on the path when the target computing core particle travels to the associated computing core particles in different areas. Specifically, the sum of the time delays of the target computing core particle accessing the associated computing core particles in each area in the current state is periodically detected, and the access delay value corresponding to each area is converted into a weight ratio in combination with the fifth linear scaling factor and the fifth global offset constant, so that areas with higher access delays obtain more cache capacity to cache more image data. In addition, the introduction of linear scaling factors and global offsets achieves a smooth attenuation of hop count differences, avoiding polarization of distribution differences.

[0227] Furthermore, by limiting the dynamic reallocation of only part of the cache capacity at a time, the stability of most of the cache can be maintained, avoiding performance fluctuations caused by global adjustments, and effectively balancing the flexible adjustment performance and stable performance of the system cache.

[0228] It should be understood that the cache may also be divided based only on the access delay value corresponding to the video memory in each area, and the steps include:

[0229] After each partitioning cycle, a memory access delay awareness request is sent to each associated computing core particle. The memory access delay awareness request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle.

[0230] Determine the access delay value corresponding to each associated computing core particle according to the sending time and receiving time of each memory access delay awareness request;

[0231] The sum of the access delay values corresponding to the associated calculation core particles in each region is determined as the access delay value corresponding to each region;

[0232] determining a fourth capacity to be re-divided in the remote data cache to be divided;

[0233] Adjusting the access delay value corresponding to each region based on a fifth linear scaling factor and a fifth global offset constant;

[0234] determining a fourth capacity ratio corresponding to each area according to the adjusted access delay value corresponding to each area, wherein the adjusted access delay value corresponding to each area is positively correlated with the corresponding fourth capacity ratio;

[0235] Determine the final capacity corresponding to each area according to the original capacity, the fourth capacity ratio, and the fourth capacity corresponding to each area;

[0236] The remote data cache to be divided is re-divided according to the final capacity and original capacity corresponding to each area.

[0237] For specific examples and effects, please refer to the above embodiments and will not be repeated here.

[0238] It should be noted that the above-mentioned solution of dynamically dividing the remote data cache to be divided when the multi-core GPU is divided into regions is applicable to a multi-core GPU system with any topology. The implementation method is shown in the above example and will not be described in detail in this application.

[0239] like Figure 7 As shown, the present application also provides a multi-core GPU cache architecture, including:

[0240] a topology perception unit, configured to obtain a topology structure of a multi-core GPU, the multi-core GPU comprising a target computing coreparticle for processing image data and at least one associated computing coreparticle, the target computing coreparticle being a computing coreparticle to be subjected to a cache partitioning operation, the associated computing coreparticle being connected to the target computing coreparticle via an inter-coreparticle interconnection network, the target computing coreparticle's remote data cache to be partitioned being configured to cache image data from a video memory of the at least one associated computing coreparticle;

[0241] a topology sensing unit, configured to determine, based on the topology structure, the number of inter-core hops between each associated computing core and the target computing core, if the number of computing cores included in the multi-core GPU does not exceed a first preset threshold;

[0242] a capacity calculation unit, configured to determine a cache capacity ratio corresponding to each associated computing core according to the number of inter-core hops corresponding to each associated computing core, wherein the number of inter-core hops corresponding to each associated computing core is positively correlated with the corresponding cache capacity ratio, and the cache capacity ratio is a ratio of the capacity required by the associated computing core to the capacity of the remote data cache to be divided;

[0243] The capacity calculation unit is further configured to determine the capacity corresponding to each associated computing core in the remote data cache to be divided according to the cache capacity ratio corresponding to each associated computing core and the capacity of the remote data cache to be divided;

[0244] The cache control unit is used to divide the remote data cache to be divided according to the capacity corresponding to each associated computing core particle.

[0245] In a feasible design, the remote data cache to be divided adopts a group-associative mapping structure, including multiple cache groups, each cache group containing multiple cache lines, and each cache line including a partition identification bit, which is used to store a video memory identification, and the video memory identification is used to identify the video memory to which the image data to be cached by the cache line belongs;

[0246] The cache control unit is used to divide the remote data cache to be divided according to the capacity corresponding to each associated computing core particle, including:

[0247] The cache control unit is used to determine the video memory identifier stored in the partition identifier of each cache line according to the capacity corresponding to each associated calculation core particle;

[0248] The cache control unit is used to divide the remote data cache to be divided by writing a corresponding video memory identifier into the division identifier bit of each cache line.

[0249] For example, the remote data cache to be divided includes S sets using a set-associative mapping structure, each set has E cache lines (E ways), and the cache consists of S*E cache lines. The cache block size in each cache line is B bytes, and the capacity of the cache is S*E*B bytes. It should be noted that in this application, the capacity of the cache can also be expressed in terms of the number of cache lines. Figure 8 For example, each cache line includes a valid bit, a tag bit, a partition flag bit, and a cache block. The valid bit is used to store a value indicating whether the image data in the cache line is valid; the tag bit is used to store the cache tag (Tag), which is used for tag comparison when performing cache hit judgment; the cache block is used to store image data in the remote video memory; and the partition flag bit is used to store the video memory identifier or region identifier to which the cache line is allocated.

[0250] In a feasible design, the topology perception unit is further configured to, if the number of computing cores included in the multi-core GPU exceeds a first preset threshold, divide the multi-core GPU into regions according to the topological structure to obtain a target region and at least one associated region, wherein the target region is a region to which the target computing core belongs, and each associated region is a region of the multi-core GPU other than the target region, each region includes the same number of computing cores, and the maximum distance between the computing cores in each region is less than a second preset threshold;

[0251] The topology awareness unit is also used to determine the number of inter-area hops between each area and the target area based on the topology structure;

[0252] The capacity calculation unit is further configured to adjust the number of inter-area hops corresponding to each area based on the first linear scaling factor and the first global offset constant;

[0253] The capacity calculation unit is further configured to determine a cache capacity ratio corresponding to each region based on the adjusted inter-region hop count corresponding to each region, wherein the inter-region hop count corresponding to each region is positively correlated with the corresponding cache capacity ratio;

[0254] The capacity calculation unit is further configured to determine the capacity corresponding to each region in the remote data cache to be divided according to the cache capacity ratio corresponding to each region and the capacity of the remote data cache to be divided;

[0255] The cache control unit is further configured to partition the remote data cache to be partitioned according to the capacity corresponding to each region, wherein the associated computing cores in each region share the storage space allocated to the region in the remote data cache to be partitioned.

[0256] In a feasible design, the remote data cache to be divided adopts a group-associative mapping structure, including multiple cache groups, each cache group containing multiple cache lines, and each cache line including a partition identification bit, which is used to store a region identification, and the region identification is used to identify the region to which the image data to be cached by the cache line belongs;

[0257] The cache control unit is configured to divide the remote data cache to be divided according to the capacity corresponding to each area, including:

[0258] The cache control unit is used to determine the region identifier stored in the partition identifier bit of each cache line according to the capacity corresponding to each region;

[0259] The cache control unit is used to divide the remote data cache to be divided by writing a corresponding region identifier into the division identifier bit of each cache line.

[0260] In a feasible design, the architecture further includes a request quantity detection unit. After dividing the remote data cache to be divided according to the capacity corresponding to each associated computing core, the cache control unit is further used to periodically re-divide the capacity of the remote data cache to be divided. After each division cycle,

[0261] The request quantity detection unit is used to detect the first request quantity of the target computing core particle accessing the video memory of each associated computing core particle in the last partition cycle;

[0262] The cache control unit is further configured to determine a first capacity that needs to be re-divided in the remote data cache to be divided;

[0263] The capacity calculation unit is further configured to adjust the first request quantity corresponding to each associated calculation core particle based on a second linear scaling factor and a second global offset constant;

[0264] The capacity calculation unit is further configured to determine a first capacity ratio corresponding to each associated computing core particle based on the adjusted first request quantity corresponding to each associated computing core particle, wherein the first capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core particle to the first capacity, and the adjusted first request quantity corresponding to each associated computing core particle is positively correlated with the corresponding first capacity ratio;

[0265] The capacity calculation unit is further configured to determine a new capacity corresponding to each associated calculation core particle according to the first capacity ratio, the original capacity, and the first capacity corresponding to each associated calculation core particle, wherein the original capacity is the capacity corresponding to the associated calculation core particle in the previous division cycle;

[0266] The cache control unit is further configured to re-divide the portion of the remote data cache to be divided according to the new capacity and the original capacity corresponding to each associated calculation core particle.

[0267] In a feasible design, the architecture also includes a memory access latency awareness unit, wherein,

[0268] The memory access delay perception unit is used to send a memory access delay perception request to each associated computing core particle, and the memory access delay perception request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle;

[0269] The memory access delay sensing unit is further configured to determine an access delay value corresponding to each associated computing core particle according to a sending time and a receiving time of each memory access delay sensing request;

[0270] The cache control unit is further configured to determine a second capacity that needs to be re-divided in the remote data cache to be divided;

[0271] The capacity calculation unit is further configured to adjust the access delay value corresponding to each associated calculation core particle based on a third linear scaling factor and a third global offset constant;

[0272] The capacity calculation unit is further configured to determine a second capacity ratio corresponding to each associated computing core according to the adjusted access delay value corresponding to each associated computing core, wherein the second capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core to the second capacity, and the adjusted access delay value corresponding to each associated computing core is positively correlated with the corresponding second capacity ratio;

[0273] The capacity calculation unit is further configured to determine a final capacity corresponding to each associated calculation core particle according to the new capacity corresponding to each associated calculation core particle, the second capacity ratio, and the second capacity;

[0274] The cache control unit is further configured to calculate the final capacity and the original capacity corresponding to each associated core particle, and to re-divide the portion of the cache to be divided into the remote data cache.

[0275] In a feasible design, after the remote data cache to be divided is divided according to the capacity corresponding to each area, the cache control unit is further used to periodically re-divide the capacity of the remote data cache to be divided. After each division cycle,

[0276] The request quantity detection unit is further configured to detect a second request quantity of the target computing core particle accessing the video memory of each region in a previous partition cycle, where the video memory of each region includes the video memory of each associated computing core particle in the region;

[0277] The cache control unit is further configured to determine a third capacity to be re-divided in the remote data cache to be divided;

[0278] The capacity calculation unit is further configured to adjust the second request quantity corresponding to each region based on a fourth linear scaling factor and a fourth global offset constant;

[0279] The capacity calculation unit is further configured to determine a third capacity ratio corresponding to each region based on the adjusted second request quantity corresponding to each region, wherein the adjusted second request quantity corresponding to each region is positively correlated with the corresponding third capacity ratio;

[0280] The capacity calculation unit is further configured to determine a new capacity corresponding to each region based on the third capacity ratio corresponding to each region, the original capacity, and the third capacity, wherein the third capacity ratio is a ratio of the capacity required to be reallocated to the region to the third capacity, and the original capacity is the capacity corresponding to the region in the previous division cycle;

[0281] The cache control unit is further configured to re-divide the portion of the cache to be divided into the remote data cache according to the new capacity and the original capacity corresponding to each area.

[0282] In a feasible design, the memory access delay perception unit is further configured to send a memory access delay perception request to each associated computing core particle, the memory access delay perception request being used to trigger round-trip communication between the target computing core particle and the associated computing core particle; after determining the access delay value corresponding to each associated computing core particle based on the sending time and the receiving time of each memory access delay perception request, the sum of the access delay values corresponding to each associated computing core particle in each region is determined as the access delay value corresponding to each region;

[0283] The cache control unit is further configured to determine a fourth capacity to be re-divided in the remote data cache to be divided;

[0284] The capacity calculation unit is further configured to adjust the access delay value corresponding to each region based on a fifth linear scaling factor and a fifth global offset constant;

[0285] The capacity calculation unit is further configured to determine a fourth capacity ratio corresponding to each region based on the adjusted access delay value corresponding to each region, wherein the fourth capacity ratio is a ratio of the capacity required to be reallocated to the region to the fourth capacity, and the adjusted access delay value corresponding to each region is positively correlated with the corresponding fourth capacity ratio;

[0286] The capacity calculation unit is further configured to determine a final capacity corresponding to each area based on the new capacity corresponding to each area, the fourth capacity ratio, and the fourth capacity;

[0287] The cache control unit is further configured to re-divide the remote data cache to be divided according to the final capacity and original capacity corresponding to each area.

[0288] In a feasible design, the capacity calculation unit is implemented in the following manner, determining the cache capacity ratio corresponding to each associated calculation core particle according to the number of inter-core particle hops corresponding to each associated calculation core particle:

[0289] Adjusting the number of inter-core hops corresponding to each associated computation core based on a sixth linear scaling factor and a sixth global offset constant;

[0290] According to the adjusted inter-core hop count corresponding to each association calculation core, the cache capacity ratio corresponding to each association calculation core is determined.

[0291] For other implementations and effects of the multi-core GPU cache architecture, please refer to the description in the cache partitioning method embodiment, which will not be repeated here.

[0292] For example, based on the above cache architecture, the cache control unit implements the initialization and dynamic management of the partition flags in the following manner:

[0293] During initialization, the cache control unit writes the partition flag of the corresponding cache line into the corresponding video memory number or region number according to the calculation result of the partition algorithm.

[0294] During dynamic adjustment, the cache control unit updates the partition identification bits of part of the cache according to the periodic partition results to ensure that the cache line only stores image data from a specified source.

[0295] Based on the above multi-core GPU cache architecture, the present application provides a data access method for a multi-core GPU cache, the method comprising:

[0296] The computing unit of the target computing core generates a request including a data source identifier, an access address, and an identifier of the computing unit, where the data source identifier is a video memory identifier or a region identifier;

[0297] If the request misses the L1 cache inside the computing unit and the access address includes the address of the remote video memory, a request is sent to the cache control unit;

[0298] The cache control unit parses the request and obtains the requested data source identifier, access address, and computing unit identifier;

[0299] The cache control unit locates the target cache group according to the access address;

[0300] The cache control unit performs request hit judgment on each cache line in the target cache group according to the storage order based on the access address and data source identifier;

[0301] If a hit occurs, the cache control unit extracts the target image data from the target cache line according to the access address and sends the target image data to the computing unit according to the identifier of the computing unit;

[0302] If there is no hit, the cache control unit sends an indication message to the computing unit, which is used to indicate that the target image data is not stored in the target cache group. After receiving the indication message, the computing unit sends a request to the target associated computing chiplet corresponding to the video memory mapped by the access address through the chiplet interconnection network.

[0303] The multi-chip GPU cache data access method provided in the above embodiment adds data source identification data to the access request when generating the access request. This allows for request hit determination to be performed by adding data source identification verification to the traditional comparison of the check bits in the access address. This can be combined with a multi-chip GPU cache partitioning structure based on partition identification bits to achieve efficient cross-chip memory access.

[0304] The following combination Figure 7 and Figure 8 The cache architecture shown below illustrates the above data access method:

[0305] (1) The access address consists of the following three parts (such as Figure 9 As shown in the following example, a request is generated:

[0306] Block offset (b bit): indicates the byte-level offset position of the target image data within the cache block;

[0307] Cache set index (s bit): used to locate the target cache set (set) of the address mapping;

[0308] Cache tag (t bit): matches the tag stored in the cache line to verify data validity.

[0309] Accordingly, when the computing unit of the target computing core initiates an access request, it generates a request packet containing the following information:

[0310] Data source identifier: identifies the video memory (video memory identifier) or area (area identifier) to which the target image data belongs;

[0311] Access address: contains the offset within the block, cache group index, and cache tag;

[0312] Computing unit ID: Identifies the computing unit that initiates the request.

[0313] (2) Cache access process:

[0314] (2.1) If the request hits the L1 cache inside the computing unit, the target image data is obtained from the L1 cache.

[0315] If the request misses the L1 cache inside the computing unit and the access address points to the remote video memory (i.e., non-local video memory address), the request is forwarded to the cache control unit.

[0316] (2.2) The cache control unit parses the request and extracts the following key information:

[0317] Data source identification (memory identification / region identification);

[0318] Access address (block offset, cache group index, cache tag);

[0319] Compute unit identifier.

[0320] (2.3) The cache control unit locates the target cache group according to the cache group index (s-bit set index) in the access address using the preset mapping rules.

[0321] (4.%2) Cache line hit judgment

[0322] In the target cache group, the cache control unit performs hit detection on each cache line in the order of storage. The detection conditions include:

[0323] Valid bit is 1: cache line data is valid;

[0324] Data source identifier match: The requested data source identifier is consistent with the identifier stored in the partition identifier bit of the cache line. The identifier stored in the partition identifier bit is written by the cache control unit to complete cache initialization when the remote data cache to be partitioned is partitioned. Its value is the video memory number or region number calculated by the partition algorithm, which is used to limit the source of image data that can be stored in the cache line;

[0325] Tag match: The cache tag of the access address matches the tag stored in the cache line.

[0326] (2.5) If all of the above conditions are met, it is considered a cache hit and the following is executed:

[0327] (2.5.1) extracting the target image data from the cache block of the hit cache line according to the offset within the block;

[0328] (2.5.2) Return the target image data to the computing unit that initiated the request according to the computing unit identifier.

[0329] (2.6): If any of the conditions are not met, it is determined to be a cache miss and the following is executed:

[0330] (2.6.1) The cache control unit sends an indication to the computing unit, stating that the target image data is not cached;

[0331] (2.6.2) The computing unit initiates a data request to the corresponding target associated computing chiplet through the inter-chiplet interconnection network based on the remote memory location mapped by the access address.

[0332] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0333] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0334] The block diagrams of the devices, devices, equipment, and systems involved in this application are intended only as illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0335] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0336] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0337] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for partitioning a multi-core GPU cache, characterized in that: include: Acquire a topology structure of a multi-core GPU, the multi-core GPU including a target computing coreparticle for processing image data and at least one associated computing coreparticle, the target computing coreparticle being a computing coreparticle to be subjected to a cache partitioning operation, the associated computing coreparticle being connected to the target computing coreparticle via an inter-coreparticle interconnection network, and the to-be-partitioned remote data cache of the target computing coreparticle being used to cache image data from a video memory of the at least one associated computing coreparticle; If the number of computing corelets included in the multi-core GPU does not exceed a first preset threshold, determining the number of inter-core hops between each of the associated computing corelets and the target computing corelet based on the topological structure; Determining a cache capacity ratio corresponding to each associated computing core according to the number of inter-core hops corresponding to each associated computing core, wherein the number of inter-core hops corresponding to each associated computing core is positively correlated with the corresponding cache capacity ratio, and the cache capacity ratio is a ratio of the capacity required by the associated computing core to the capacity of the remote data cache to be divided; Determining the capacity corresponding to each of the associated computing core particles in the remote data cache to be divided according to the cache capacity ratio corresponding to each of the associated computing core particles and the capacity of the remote data cache to be divided; The remote data cache to be divided is divided according to the capacity corresponding to each of the associated calculation core particles.

2. The method according to claim 1, characterized in that The method comprises: If the number of computing cores included in the multi-core GPU exceeds a first preset threshold, the multi-core GPU is divided into regions according to the topological structure to obtain a target region and at least one associated region, wherein the target region is a region to which the target computing core belongs, and each associated region is a region of the multi-core GPU other than the target region, each region includes the same number of computing cores, and a maximum distance between computing cores in each region is less than a second preset threshold; Based on the topological structure, determining the number of inter-area hops between each area and the target area; Adjusting the number of inter-region hops corresponding to each region based on the first linear scaling factor and the first global offset constant; Determine the cache capacity ratio corresponding to each region according to the adjusted inter-region hop count corresponding to each region, wherein the inter-region hop count corresponding to each region is positively correlated with the corresponding cache capacity ratio; Determining the capacity of each region in the remote data cache to be divided according to the cache capacity ratio corresponding to each region and the capacity of the remote data cache to be divided; The remote data cache to be divided is divided according to the capacity corresponding to each region, wherein the associated computing cores in each region share the storage space allocated to the region in the remote data cache to be divided.

3. The method according to claim 1, characterized in that After the remote data cache to be divided is divided according to the capacity corresponding to each of the associated computing core particles, the capacity of the remote data cache to be divided is periodically re-divided, and the step of re-dividing the capacity of the remote data cache to be divided after each division cycle includes: detecting a first number of requests by the target computing core particle to access the video memory of each associated computing core particle in a previous partitioning cycle; Determining a first capacity that needs to be re-divided in the remote data cache to be divided; Adjusting the first request quantity corresponding to each of the associated calculation core particles based on a second linear scaling factor and a second global offset constant; Determining a first capacity ratio corresponding to each of the associated computing core particles based on the adjusted first request quantity corresponding to each of the associated computing core particles, wherein the first capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core particle to the first capacity, and the adjusted first request quantity corresponding to each of the associated computing core particles is positively correlated with the corresponding first capacity ratio; Determine the new capacity corresponding to each of the associated computing core particles according to the first capacity ratio, the original capacity, and the first capacity corresponding to each of the associated computing core particles, wherein the original capacity is the capacity corresponding to the associated computing core particle in the previous partitioning cycle; The part of the remote data cache to be divided is re-divided according to the new capacity corresponding to each associated calculation core particle and the original capacity.

4. The method according to claim 3, characterized in that The re-dividing the portion of the remote data cache to be divided according to the new capacity corresponding to each associated calculation core particle and the original capacity includes: Sending a memory access delay awareness request to each of the associated computing core particles, wherein the memory access delay awareness request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle; Determining an access delay value corresponding to each of the associated computing core particles according to a sending time and a receiving time of each of the memory access delay awareness requests; Determining a second capacity that needs to be re-divided in the remote data cache to be divided; Adjusting the access delay value corresponding to each of the associated calculation core particles based on a third linear scaling factor and a third global offset constant; Determining a second capacity ratio corresponding to each of the associated computing core particles according to the adjusted access delay value corresponding to each of the associated computing core particles, wherein the second capacity ratio is a ratio of the capacity required to be reallocated by the associated computing core particle to the second capacity, and the adjusted access delay value corresponding to each of the associated computing core particles is positively correlated with the corresponding second capacity ratio; determining a final capacity corresponding to each of the associated calculation core particles according to the new capacity corresponding to each of the associated calculation core particles, the second capacity ratio, and the second capacity; The part of the remote data cache to be divided is re-divided according to the final capacity corresponding to each associated calculation core particle and the original capacity.

5. The method according to claim 2, characterized in that After the remote data cache to be divided is divided according to the capacity corresponding to each area, the capacity of the remote data cache to be divided is periodically re-divided, and the step of re-dividing the capacity of the remote data cache to be divided after each division cycle includes: detecting a second number of requests by the target computing core to access the video memory of each region in a previous partitioning cycle, where the video memory of each region includes the video memory of each associated computing core in the region; Determining a third capacity that needs to be re-divided in the remote data cache to be divided; Adjusting the second request quantity corresponding to each region based on a fourth linear scaling factor and a fourth global offset constant; Determining a third capacity ratio corresponding to each region based on the adjusted second request quantity corresponding to each region, wherein the third capacity ratio is a ratio of the capacity required to be reallocated to the region to the third capacity, and the adjusted second request quantity corresponding to each region is positively correlated with the corresponding third capacity ratio; Determine a new capacity corresponding to each area according to the third capacity ratio, the original capacity, and the third capacity corresponding to each area, wherein the original capacity is the capacity corresponding to the area in the previous division cycle; The part of the remote data cache to be divided is re-divided according to the new capacity and the original capacity corresponding to each area.

6. The method according to claim 5, characterized in that The re-dividing the portion of the remote data cache to be divided according to the new capacity and the original capacity corresponding to each area includes: Sending a memory access delay awareness request to each of the associated computing core particles, wherein the memory access delay awareness request is used to trigger round-trip communication between the target computing core particle and the associated computing core particle; Determining an access delay value corresponding to each associated computing core particle according to a sending time and a receiving time of each memory access delay awareness request; Determine the sum of the access delay values corresponding to the associated calculation core particles in each region as the access delay value corresponding to each region; Determining a fourth capacity that needs to be re-divided in the remote data cache to be divided; Adjusting the access delay value corresponding to each region based on a fifth linear scaling factor and a fifth global offset constant; Determining a fourth capacity ratio corresponding to each region based on the adjusted access delay value corresponding to each region, wherein the fourth capacity ratio is a ratio of the capacity required to be reallocated to the region to the fourth capacity, and the adjusted access delay value corresponding to each region is positively correlated with the corresponding fourth capacity ratio; determining a final capacity corresponding to each area according to the new capacity corresponding to each area, the fourth capacity ratio, and the fourth capacity; The remote data cache to be divided is re-divided according to the final capacity corresponding to each area and the original capacity.

7. The method according to any one of claims 1 to 6, characterized in that Determining the cache capacity ratio corresponding to each of the associated computing core particles according to the number of inter-core particle hops corresponding to each of the associated computing core particles includes: Adjusting the number of inter-core-grain hops corresponding to each of the associated calculation core-grains based on a sixth linear scaling factor and a sixth global offset constant; The cache capacity ratio corresponding to each of the association calculation core particles is determined according to the adjusted inter-core particle hop count corresponding to each of the association calculation core particles.

8. A multi-core GPU cache architecture, characterized in that: include: a topology perception unit, configured to obtain a topology structure of a multi-core GPU, the multi-core GPU comprising a target computing coreparticle for processing image data and at least one associated computing coreparticle, the target computing coreparticle being a computing coreparticle to be subjected to a cache partitioning operation, the associated computing coreparticle being connected to the target computing coreparticle via an inter-coreparticle interconnection network, the target computing coreparticle's remote data cache to be partitioned being configured to cache image data from a video memory of the at least one associated computing coreparticle; The topology sensing unit is configured to determine, based on the topology structure, the number of inter-core hops between each associated coreparticle and the target coreparticle if the number of coreparticles included in the multi-core GPU does not exceed a first preset threshold; a capacity calculation unit, configured to determine a cache capacity ratio corresponding to each associated computing core according to the number of inter-core hops corresponding to each associated computing core, wherein the number of inter-core hops corresponding to each associated computing core is positively correlated with the corresponding cache capacity ratio, and the cache capacity ratio is a ratio of the capacity required by the associated computing core to the capacity of the remote data cache to be divided; The capacity calculation unit is further configured to determine the capacity corresponding to each of the associated computing core particles in the remote data cache to be divided according to the cache capacity ratio corresponding to each of the associated computing core particles and the capacity of the remote data cache to be divided; The cache control unit is used to divide the remote data cache to be divided according to the capacity corresponding to each of the associated calculation core particles.

9. The multi-core GPU cache architecture according to claim 8, wherein: The remote data cache to be divided adopts a group-associative mapping structure, including multiple cache groups, each cache group includes multiple cache lines, and each cache line includes a division identification bit, the division identification bit is used to store a video memory identification, and the video memory identification is used to identify the video memory to which the image data of the cache line is allocated for caching; The cache control unit is configured to divide the remote data cache to be divided according to the capacity corresponding to each of the associated calculation core particles, including: The cache control unit is configured to determine the video memory identifier stored in the partition identifier of each cache line according to the capacity corresponding to each associated calculation core; The cache control unit is used to divide the remote data cache to be divided by writing a corresponding video memory flag into the division flag bit of each cache line.

10. A data access method for a multi-core GPU cache, characterized in that: The method is applied to the multi-core GPU cache architecture according to claim 8 or 9, wherein the multi-core GPU cache architecture includes a cache control unit and a remote data cache, wherein the remote data cache adopts a set-associative mapping structure and includes a plurality of cache groups, each cache group includes multiple cache lines, and each cache line includes a partition identification bit, wherein the partition identification bit is configured to store a video memory identification or a region identification, wherein the video memory identification is used to identify the video memory to which image data to be cached by the cache line belongs, and the region identification is used to identify the region to which the image data to be cached by the cache line belongs, and wherein the method comprises: The computing unit of the target computing core generates a request including a data source identifier, an access address, and an identifier of the computing unit, wherein the data source identifier is a video memory identifier or a region identifier; If the request does not hit the L1 cache inside the computing unit, and the access address includes the address of the remote video memory, sending the request to the cache control unit; The cache control unit parses the request to obtain the data source identifier, the access address, and the identifier of the computing unit of the request; The cache control unit locates the target cache group according to the access address; The cache control unit performs request hit determination on each cache line in the target cache group in storage order according to the access address and the data source identifier; If a hit occurs, the cache control unit extracts the target image data from the target cache line according to the access address, and sends the target image data to the computing unit according to the identifier of the computing unit; If there is no hit, the cache control unit sends an indication message to the computing unit, where the indication message is used to indicate that the target image data is not stored in the target cache group. After receiving the indication message, the computing unit sends the request to the target associated computing chiplet corresponding to the video memory mapped by the access address through the chiplet interconnection network.

Images