GPGPU thread block scheduling method and system based on data space locality

Abstract

The invention belongs to the field of GPGPU chip design, and particularly relates to a GPGPU thread block scheduling method and system based on data space locality, and the method comprises the steps: carrying out the statistics of Bank feature information of access data of all thread blocks, and classifying the thread blocks accessing the same Bank into the same Bank group according to the Bank feature information; preferentially distributing the thread blocks of the same Bank group to the same programmable multiprocessor until the resources of the programmable multiprocessor are saturated; setting a private row cache space for each Bank in each programmable multiprocessor; when the data in the private line cache space is updated, synchronously updating the corresponding data in the L1 cache, the L2 cache and the DRAM; counting the hit rate of the private line cache in real time, and closing the private line cache function when the hit rate is lower than a preset threshold value. High-delay external storage access is reduced, the data reuse rate is improved, and the execution efficiency is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of GPGPU chip design, specifically relating to a GPGPU thread block scheduling method and system based on data spatial locality. Background Technology

[0002] The thread block scheduling strategy is closely related to the overall performance of a GPGPU. Traditional GPGPUs use a pure round-robin thread block allocation strategy: the scheduler allocates one thread block to each Streaming Multiprocessor (SM, the computational core that executes the thread block in the GPGPU) in turn; if there are still SM resources (registers, shared memory, etc.) remaining after the first round of allocation, the second and third rounds of allocation begin, until all SM resources are saturated. Unallocated thread blocks must wait for running thread blocks to release resources before they can be scheduled.

[0003] Modern GPGPUs typically divide their global memory (DRAM) into multiple banks, each with its own row buffer. When a warp accesses a global address that maps to a different physical row within the same bank, the bank needs to perform precharge-activation-read / write operations sequentially, introducing significant latency known as bank collisions.

[0004] In practical applications, the thread ID of a thread block and the global address it accesses often exhibit a linear relationship: thread blocks with consecutive thread block IDs tend to access contiguous regions in the global memory address space. If a pure polling strategy is adopted, these contiguous thread blocks will be distributed across different memory management spaces (SMs). Since different SMs may concurrently issue access requests to different rows of the same bank, bank conflicts increase significantly, thereby amplifying memory access latency and reducing the parallel efficiency of the GPGPU. Summary of the Invention

[0005] To address the problem of spatial locality being violated between thread blocks due to traditional polling strategies, this invention provides a GPGPU thread block scheduling method and system based on data spatial locality to solve the aforementioned technical problem.

[0006] In a first aspect, the present invention provides a GPGPU thread block scheduling method based on data spatial locality, comprising the following steps: S1. Statistically analyze the Bank feature information of all thread block access data. The Bank feature information is determined based on the BA1~BA0 fields in the actual physical address of the thread block access data. Based on the Bank feature information, thread blocks accessing the same Bank are grouped into the same Bank group. S2. Prioritize allocating thread blocks from the same Bank group to the same programmable multiprocessor until the programmable multiprocessor's resources are saturated; S3. Within each programmable multiprocessor, a private row cache space is set up for each Bank. When a thread block within the programmable multiprocessor initiates a DRAM memory access request, the DRAM cache row data is stored in the private row cache of the corresponding Bank. When subsequent thread blocks access data in the same Bank, data is read from the private row cache space first. S4. When the data in the private line cache space is updated, the corresponding data in the L1, L2 cache and DRAM are updated synchronously. S5. After the thread block finishes execution and releases the programmable multiprocessor resources, the released resources are preferentially allocated to the Bank group with the most remaining thread blocks; at the same time, the private line cache hit rate is calculated in real time, and the private line cache function is turned off when the hit rate is lower than the preset threshold.

[0007] By statistically analyzing the distribution characteristics of thread block accesses to DRAM banks, thread blocks accessing the same bank are preferentially grouped together and run on the same SM (Streaming Service), significantly reducing bank conflicts caused by concurrent accesses to different rows of the same bank by different SMs. Within each SM, a private row cache is added for each bank, allowing subsequent thread blocks to directly hit the cache without re-accessing DRAM, reducing access latency. Private row caches with low hit rates are disabled in real-time to avoid wasted power consumption, achieving a dynamic balance between performance, power consumption, and area.

[0008] As a further limitation of the technical solution of the present invention, S1 specifically includes: During GPGPU runtime, for each thread block to be scheduled, the physical addresses of all its memory access requests are parsed, the Bank identifier field in the DDR address is extracted, the access frequency of the thread block to each Bank is counted, and the thread block is assigned to the group corresponding to the Bank with the highest access frequency. If multiple Banks have the same frequency, one of them is selected.

[0009] During GPGPU runtime, the physical address of thread block memory access requests is precisely parsed, the Bank identifier field is extracted, and the access frequency is counted. Thread blocks are then grouped into the group corresponding to the Bank with the highest access frequency. This precise statistical and grouping method can more accurately grasp the access tendency of thread blocks to different Banks, making the grouping more closely reflect the actual memory access situation. Compared with the traditional pure polling strategy, it can better group thread blocks accessing the same Bank together, reducing Bank conflicts caused by the scattered allocation of thread blocks from the source, effectively utilizing data spatial locality, improving memory access efficiency, and thus improving the overall performance of GPGPU.

[0010] As a further limitation of the technical solution of the present invention, S1 also includes: When the difference in access frequency of the same thread block to multiple Banks is less than a preset threshold, the thread block is assigned to the group with the smallest Bank identifier field value.

[0011] When the frequency difference of access to multiple banks by the same thread block is less than a preset threshold, it is grouped into the group with the smallest value in the Bank identifier field. This provides a clear and stable rule for thread block grouping. It avoids the arbitrariness of grouping when access frequencies are similar, making the grouping results more predictable and consistent. This helps subsequent group-based scheduling strategies execute more effectively, ensuring that thread blocks in the same Bank group can be more rationally and centrally allocated, reducing Bank conflicts, guaranteeing system performance stability, and overcoming the uncertainty of traditional pure polling strategies in this regard.

[0012] As a further limitation of the technical solution of the present invention, S2 includes: First round of allocation: Thread blocks are allocated to the corresponding programmable multiprocessors according to the Bank grouping order, until the programmable multiprocessor resources are saturated; Second round of allocation: If the number of programmable multiprocessors is greater than the number of banks, the remaining thread blocks are allocated to unsaturated programmable multiprocessors using a round-robin method.

[0013] A two-round allocation method is adopted. In the first round, thread blocks are allocated to corresponding programmable multiprocessors according to the Bank grouping order. This ensures that thread blocks within the same Bank group are allocated as centrally as possible, fully utilizing data spatial locality and minimizing concurrent access to different rows of the same Bank by different SMs, thus reducing Bank conflicts. In the second round, when the number of programmable multiprocessors exceeds the number of Banks, the remaining thread blocks are allocated using a round-robin method. This avoids some programmable multiprocessors being idle and improves resource utilization. Compared to the traditional pure round-robin strategy, this allocation method considers both data locality to reduce conflicts and ensures full utilization of resources, effectively improving the overall performance of the GPGPU.

[0014] As a further limitation of the technical solution of the present invention, the first round of allocation specifically includes: Based on the Bank group, thread blocks belonging to the same Bank group are allocated sequentially to the same programmable multiprocessor until the registers and shared memory resources of the programmable multiprocessor are exhausted. When the number of thread blocks in a certain Bank group is insufficient to fill all programmable multiprocessors, thread blocks are selected from other Bank groups to fill the remaining programmable multiprocessors in a round-robin manner.

[0015] In the first round of allocation, thread blocks within the same bank group are prioritized and assigned to the same programmable multiprocessor until resources are exhausted. This maximizes the aggregation of thread blocks that frequently access the same bank, reducing data access conflicts and cross-bank access, and lowering the probability of bank conflicts. When the number of thread blocks in a particular bank group is insufficient, a round-robin approach is used to select thread blocks from other groups to fill the remaining programmable multiprocessors, ensuring full resource utilization and avoiding resource waste due to insufficient thread blocks in some groups. Compared to the traditional pure round-robin strategy, this allocation method, while making full use of resources, places greater emphasis on data locality, effectively reducing bank conflicts and improving system resource utilization efficiency and memory access performance.

[0016] As a further limitation of the technical solution of the present invention, the second round of allocation specifically includes: After the first round of allocation, the remaining programmable multiprocessors replenish thread blocks from the unsaturated Bank groups in a round-robin order until all programmable multiprocessors are fully loaded or all thread blocks have been allocated.

[0017] The second round of allocation replenishes thread blocks from the unsaturated Bank groups in a round-robin order after the first round, until all programmable multiprocessors are fully loaded or all thread blocks have been allocated. This allocation method further optimizes the thread block allocation process, ensuring that all programmable multiprocessors are fully utilized and avoiding situations where some processors are underloaded while others are overloaded, resulting in a more balanced allocation of system resources. Compared to the traditional pure round-robin strategy, this approach, while considering balanced resource allocation, incorporates the Bank group-based allocation from the first round, better utilizing data spatial locality, reducing Bank conflicts, and improving overall performance.

[0018] As a further limitation of the technical solution of the present invention, S3 also includes: The private row cache adopts a write-through strategy, where any write request to the same DRAM row is immediately written back to DRAM, and the corresponding row in the L1 and L2 caches is updated before the write-back. When the private line cache space is full and a new DRAM line needs to be loaded, the Least Recently Used (LRU) replacement strategy is used to select the line to be replaced. Before replacement, it is determined whether dirty data has occurred. If so, a write-back operation is performed first.

[0019] The private line cache employs a write-through strategy, immediately writing write requests back to DRAM and updating the corresponding lines in the L1 and L2 caches. This ensures timely data updates, avoids data inconsistency issues, and guarantees the accuracy of data reads by subsequent thread blocks. When the private line cache space is full and a new DRAM line needs to be loaded, an LRU replacement strategy is used to select the line to be replaced. Before replacement, dirty data is checked and a write-back operation is performed, effectively managing cache space and improving cache utilization. Compared to traditional no-cache or poorly managed cache scenarios, these strategies better utilize data spatial locality, reduce the number of DRAM accesses, lower memory access latency, and avoid errors caused by cache data inconsistency, thereby improving system memory access performance.

[0020] It should be noted that in the GPGPU architecture, L1 and L2 caches are two levels of on-chip storage subsystems between compute units (SMs) and off-chip DRAM. The L1 cache (L1 Data Cache / Texture Cache) is located inside each SM and is private to that SM. Its capacity is usually 16 KB to 48 KB, and it uses 128 B or 256 B cache lines. It supports read / write coalescing and broadcast mechanisms.

[0021] The L2 cache is located between the GPU Global Interconnect Network (GITN) and the DRAM controller, and is shared by all SMs. It has a capacity of 512 KB to 6 MB, and its cache line size is the same as that of the DRAM line buffer (64 B or 128 B). L2 serves as both a global data cache and a cache for textures, constants, and some instruction data.

[0022] L1 reduces repeated DRAM accesses by thread bundles within the same SM, capturing thread-level data reuse; it supports write merging, atomic operations, and consistent broadcasting. L2 captures data reuse across SMs, filters DRAM access traffic, and reduces off-chip bandwidth pressure; it also serves as a unified backup after L1 failure, ensuring data consistency seen by all SMs (the GPU's weak consistency model is implemented through L2's write merging / failure queue).

[0023] In this application, the private line cache is located within the SM (Search Engine Controller) and is directly mapped to a 64-byte line buffer in a single DRAM bank, with higher priority than L1. When the private line cache misses, the request first enters L1; if L1 misses, it enters L2; ​​only if L2 misses does it access DRAM. The private line cache adopts a write-through strategy, updating L1, L2, and DRAM simultaneously when writing data, ensuring consistency of lines at each cache level; L2 then notifies other SMs of L1 line misses via a global broadcast / failure queue, maintaining cross-SM consistency.

[0024] As a further limitation of the technical solution of the present invention, the size of the private row cache space is consistent with the size of the DRAM row buffer.

[0025] The private line cache size is kept consistent with the DRAM line buffer size, ensuring that the cache space can completely store a row of DRAM data, avoiding the situation where data is stored or retrieved multiple times due to insufficient cache space. This reduces the number of data accesses and access latency, improves cache hit rate, and allows thread blocks to retrieve the required data from the private line cache more quickly, reducing DRAM accesses and further mitigating the impact of bank conflicts. Compared to cases where cache space sizes are mismatched, this effectively improves memory access efficiency, better utilizes data spatial locality, and enhances the overall performance of the GPGPU.

[0026] Secondly, the present invention provides a GPGPU thread block scheduling system based on data spatial locality, comprising: The thread block memory access feature statistics unit is used to count the Bank feature information of all thread block access data. The Bank feature information is determined based on the BA1~BA0 fields in the actual physical address of the thread block access data, and the thread blocks accessing the same Bank are grouped into the same Bank group according to the Bank feature information. The thread block scheduling unit is used for two-stage scheduling and allocation based on the Bank grouping results; A line cache unit, located within each programmable multiprocessor, is used to store the DRAM line cache data for the corresponding Bank; The cache consistency maintenance unit is used to synchronously update the cache and DRAM at all levels when the private row cache data is updated; The dynamic cache control unit is used to dynamically enable or disable the line cache function based on the line cache hit rate.

[0027] This system accurately counts and groups the Bank characteristics of thread block access data through a thread block memory access feature statistics unit, providing a foundation for rational scheduling and overcoming the shortcomings of traditional pure polling strategies that do not consider data locality. The thread block scheduling unit performs two-stage scheduling allocation based on the grouping results, fully utilizing data spatial locality, reducing Bank conflicts, and improving resource utilization. The row cache unit stores DRAM row buffer data for the corresponding Bank, reducing data access latency. The cache consistency maintenance unit ensures consistency between caches at all levels and DRAM during data updates, avoiding data errors. The dynamic cache control unit dynamically enables or disables row cache functionality based on row cache hit rate, optimizing system performance. All units work collaboratively to comprehensively improve the efficiency and performance of GPGPU thread block scheduling, effectively solving the problems of traditional strategies.

[0028] As a further limitation of the technical solution of the present invention, the thread block scheduling unit is specifically used to allocate thread blocks belonging to the same Bank group to the same programmable multiprocessor in sequence based on Bank grouping, until the registers and shared memory resources of the programmable multiprocessor are exhausted; when the number of thread blocks in a certain Bank group is insufficient to fill all programmable multiprocessors, thread blocks are selected from other Bank groups to fill the remaining programmable multiprocessors in a round-robin manner; after the first round of allocation, the remaining programmable multiprocessors supplement thread blocks from the Bank groups that are not yet saturated in a round-robin order, until all programmable multiprocessors are fully loaded or all thread blocks are allocated.

[0029] The thread block scheduling unit performs a two-stage scheduling allocation according to specific rules. In the first stage, thread blocks from the same bank group are prioritized for allocation to the same programmable multiprocessor, fully utilizing data spatial locality and reducing concurrent access to different rows of the same bank by different service managers (SMs), thus minimizing bank conflicts. When a group of thread blocks is insufficient, a round-robin approach is used to fill the remaining processors, ensuring resource utilization. In the second stage, thread blocks are replenished according to the round-robin order after the first round, resulting in a more balanced resource allocation. Compared to the traditional pure round-robin strategy, this detailed scheduling method can allocate thread blocks more accurately and efficiently, reducing bank conflicts from multiple aspects, improving system resource utilization and memory access performance, ensuring stable and efficient system operation, and enhancing the overall performance of the GPGPU.

[0030] The beneficial effects of this invention are that it addresses the problems of traditional GPGPU pure polling thread block allocation strategies, which easily lead to Bank conflicts, amplify memory access latency, and reduce parallel efficiency. It proposes a GPGPU thread block scheduling method based on spatial locality of data. By statistically analyzing and grouping the Bank characteristics of the data accessed by thread blocks, thread blocks in the same Bank group are preferentially allocated to the same programmable multiprocessor (MMP). This concentrates thread blocks accessing the same Bank, reducing the probability of concurrent accesses from different SMs to different rows of the same Bank, effectively reducing Bank conflicts. A private row cache space is set up within the MMP. Utilizing spatial locality of data, subsequent thread blocks preferentially read data from the private row cache, reducing the number of DRAM accesses and further reducing memory access latency. Simultaneously, when the private row cache data is updated, all levels of cache and DRAM are updated synchronously to ensure data consistency. After a thread block completes execution, resources are rationally allocated, and cache functionality is dynamically adjusted based on the private row cache hit rate, comprehensively improving the parallel efficiency and overall performance of the GPGPU. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention.

[0033] Figure 2 This is a system architecture diagram of one embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the specific embodiments. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] like Figure 1 As shown, the present invention provides a GPGPU thread block scheduling method based on data spatial locality, comprising the following steps: S1. Statistically analyze the Bank feature information of all thread block access data. The Bank feature information is determined based on the BA1~BA0 fields in the actual physical address of the thread block access data. Based on the Bank feature information, thread blocks accessing the same Bank are grouped into the same Bank group. BA1~BA0 are the 10th~9th bits of the physical address bus (assuming the definition of DDR4 address bus), used to identify 4 Banks (00~11 correspond to Bank1~4). S2. Prioritize allocating thread blocks from the same Bank group to the same programmable multiprocessor until the programmable multiprocessor's resources are saturated; S3. Within each programmable multiprocessor, a private row cache space is set up for each Bank. When a thread block within the programmable multiprocessor initiates a DRAM memory access request, the DRAM cache row data is stored in the private row cache of the corresponding Bank. When subsequent thread blocks access data in the same Bank, data is read from the private row cache space first. S4. When the data in the private line cache space is updated, the corresponding data in the L1, L2 cache and DRAM are updated synchronously. S5. After the thread block finishes execution and releases the programmable multiprocessor resources, the released resources are preferentially allocated to the Bank group with the most remaining thread blocks; at the same time, the private line cache hit rate is calculated in real time, and the private line cache function is turned off when the hit rate is lower than the preset threshold.

[0036] By statistically analyzing the distribution characteristics of thread blocks accessing DRAM banks, thread blocks accessing the same bank are preferentially grouped together and run on the same SM (Multi-Processor), significantly reducing bank conflicts caused by concurrent access to different rows of the same bank by different SMs. A private row cache is added to each bank within each SM, allowing subsequent thread blocks to directly hit the cache without re-accessing DRAM, reducing memory access latency. Private row caches with low hit rates are disabled in real-time to avoid wasted power consumption, achieving a dynamic balance between performance, power consumption, and area. This method statistically analyzes the bank characteristics of data accessed by each thread block. Based on these characteristics, thread blocks with the same access bank are allocated to the same programmable multiprocessor as much as possible. Simultaneously, a private row cache space is created within the SM to store the data already loaded into the DRAM row buffer for the corresponding bank. When subsequent consecutive thread blocks read data, there is a high probability that they can directly retrieve the relevant data from this cache space without high-latency external storage access, increasing data reuse and improving execution efficiency. The private row cache size is consistent with the DRAM (DDR4) row buffer size at 64 bytes. When a new bank row is activated, the private row cache, L1, and L2 caches are updated. Compared to the L1 and L2 caches shared by multiple SMs, this increases the probability of data being hit in the private cache, thus increasing data reuse. Simultaneously, when data in the private cache is updated, the data in L1, L2, and DRAM should be updated promptly to maintain data consistency.

[0037] In some embodiments, S1 specifically includes: During GPGPU runtime, for each thread block to be scheduled, the physical addresses of all its memory access requests are parsed, the Bank identifier field in the DDR address is extracted, the access frequency of the thread block to each Bank is counted, and the thread block is assigned to the group corresponding to the Bank with the highest access frequency. If multiple Banks have the same frequency, one of them is selected.

[0038] Using the highest access frequency as the sole criterion for grouping thread blocks, the hardware implementation only requires addition comparisons, which is simple in logic and has low area overhead; at the same time, it ensures that each thread block belongs to only one Bank group, eliminating group ambiguity and simplifying subsequent scheduling logic.

[0039] In some embodiments, S1 further includes: When the difference in access frequency of the same thread block to multiple Banks is less than a preset threshold, the thread block is assigned to the group with the smallest Bank identifier field value.

[0040] By introducing a frequency difference threshold, weakly related accesses of thread blocks to multiple banks can be filtered out, avoiding frequent cross-group migrations. Furthermore, the practice of grouping into the smallest group of bank identifiers allows the hardware to directly use the lowest two address lines as selection signals, further reducing control complexity.

[0041] In some embodiments, S2 includes: First round of allocation: Thread blocks are allocated to the corresponding programmable multiprocessors according to the Bank grouping order, until the programmable multiprocessor resources are saturated; Second round of allocation: If the number of programmable multiprocessors is greater than the number of banks, the remaining thread blocks are allocated to unsaturated programmable multiprocessors using a round-robin method.

[0042] In some embodiments, the first round of allocation specifically includes: Based on the Bank group, thread blocks belonging to the same Bank group are allocated sequentially to the same programmable multiprocessor until the registers and shared memory resources of the programmable multiprocessor are exhausted. When the number of thread blocks in a certain Bank group is insufficient to fill all programmable multiprocessors, thread blocks are selected from other Bank groups to fill the remaining programmable multiprocessors in a round-robin manner.

[0043] In the first round, the remaining amount of registers and shared memory is used as a hard limit to ensure that there is no runtime overflow caused by excessive allocation of resources; unfilled SMs are automatically rolled back to polling filling, so that the strategy smoothly transitions between locality priority and resource saturation, and no additional state machine is required in the hardware.

[0044] In some embodiments, the second round of allocation specifically includes: After the first round of allocation, the remaining programmable multiprocessors replenish thread blocks from the unsaturated Bank groups in a round-robin order until all programmable multiprocessors are fully loaded or all thread blocks have been allocated.

[0045] In some embodiments, S3 further includes: The private row cache adopts a write-through strategy, where any write request to the same DRAM row is immediately written back to DRAM, and the corresponding row in the L1 and L2 caches is updated before the write-back. When the private line cache space is full and a new DRAM line needs to be loaded, the Least Recently Used (LRU) replacement strategy is used to select the line to be replaced. Before replacement, it is determined whether dirty data has occurred. If so, a write-back operation is performed first.

[0046] The write-through strategy is implemented by simultaneously driving the write enable signals of the private row cache, L1, and L2 through hardware circuitry; the LRU replacement strategy maintains an 8-bit counter to record the access timestamp of each row, and the row with the smallest counter value is replaced first.

[0047] By modularizing the five functional units of memory access feature statistics, scheduling, caching, consistency maintenance and dynamic control, the system can reuse the front-end pipeline of the existing GPGPU, requiring only the insertion of a small number of registers and comparators inside the SM; the overall modification is limited to the back-end design of the chip, does not affect the instruction set and programming model, and is completely transparent to the software.

[0048] It should be noted that in the GPGPU architecture, L1 and L2 caches are two levels of on-chip storage subsystems between compute units (SMs) and off-chip DRAM. The L1 cache (L1 Data Cache / Texture Cache) is located inside each SM and is private to that SM. Its capacity is usually 16 KB to 48 KB, and it uses 128 B or 256 B cache lines. It supports read / write coalescing and broadcast mechanisms.

[0049] The L2 cache is located between the GPU Global Interconnect Network (GITN) and the DRAM controller, and is shared by all SMs. It has a capacity of 512 KB to 6 MB, and its cache line size is the same as that of the DRAM line buffer (64 B or 128 B). L2 serves as both a global data cache and a cache for textures, constants, and some instruction data.

[0050] L1 reduces repeated DRAM accesses by thread bundles within the same SM, capturing thread-level data reuse; it supports write merging, atomic operations, and consistent broadcasting. L2 captures data reuse across SMs, filters DRAM access traffic, and reduces off-chip bandwidth pressure; it also serves as a unified backup after L1 failure, ensuring data consistency seen by all SMs (the GPU's weak consistency model is implemented through L2's write merging / failure queue).

[0051] In this application, the private line cache is located within the SM (Search Engine Controller) and is directly mapped to a 64-byte line buffer in a single DRAM bank, with higher priority than L1. When the private line cache misses, the request first enters L1; if L1 misses, it enters L2; ​​only if L2 misses does it access DRAM. The private line cache adopts a write-through strategy, updating L1, L2, and DRAM simultaneously when writing data, ensuring consistency of lines at each cache level; L2 then notifies other SMs of L1 line misses via a global broadcast / failure queue, maintaining cross-SM consistency.

[0052] In some embodiments, the size of the private line cache space is consistent with the size of the DRAM line buffer.

[0053] The scheduling unit reuses the same set of polling pointers and resource counters in the two-stage allocation, resulting in high hardware resource reuse. Its logic depth is only two levels of comparison and one level of addition, with critical path latency of less than 2 clock cycles, so it will not become a bottleneck for SM scheduling. At the same time, it ensures that the local benefits of Bank group priority and global load balancing are achieved simultaneously.

[0054] like Figure 2 As shown, this embodiment of the invention provides a GPGPU thread block scheduling system based on data spatial locality, including: The thread block memory access feature statistics unit is used to count the Bank feature information of all thread block access data. The Bank feature information is determined based on the BA1~BA0 fields in the actual physical address of the thread block access data, and the thread blocks accessing the same Bank are grouped into the same Bank group according to the Bank feature information. The thread block scheduling unit is used for two-stage scheduling and allocation based on the Bank grouping results; A line cache unit, located within each programmable multiprocessor, is used to store the DRAM line cache data for the corresponding Bank; The cache consistency maintenance unit is used to synchronously update the cache and DRAM at all levels when the private row cache data is updated; The dynamic cache control unit is used to dynamically enable or disable the line cache function based on the line cache hit rate.

[0055] The thread block memory access feature statistics unit includes a thread block memory access feature value collection table to perform statistics on the memory access features of all thread blocks. Specifically, the memory access feature statistics unit analyzes the BA1~BA0 fields in the actual physical address (DDR address) of the thread block accessed data to determine the corresponding Bank. For example, when BA1:BA0 = 2`b00 (2 bits of binary 00), it indicates that the accessed data is located in the first Bank; when BA1:BA0 = 2`b11, it indicates that the accessed data is located in the fourth Bank. The thread blocks are then grouped according to the Bank corresponding to the accessed data. Specifically, when the data accessed by a thread block is located in multiple Banks, the access data features are statistically analyzed a second time, and the data is grouped into the Bank group with the most accessed data.

[0056] The thread block scheduling unit includes an SM resource management unit and a Bank task queue management unit, which realize the allocation and scheduling of thread block tasks.

[0057] The row cache unit, i.e. the data storage unit, is responsible for storing the row cache data of the Bank corresponding to the SM.

[0058] This system accurately counts and groups the Bank characteristics of thread block access data through a thread block memory access feature statistics unit, providing a foundation for rational scheduling and overcoming the shortcomings of traditional pure polling strategies that do not consider data locality. The thread block scheduling unit performs two-stage scheduling allocation based on the grouping results, fully utilizing data spatial locality, reducing Bank conflicts, and improving resource utilization. The row cache unit stores DRAM row buffer data for the corresponding Bank, reducing data access latency. The cache consistency maintenance unit ensures consistency between caches at all levels and DRAM during data updates, avoiding data errors. The dynamic cache control unit dynamically enables or disables row cache functionality based on row cache hit rate, optimizing system performance. All units work collaboratively to comprehensively improve the efficiency and performance of GPGPU thread block scheduling, effectively solving the problems of traditional strategies.

[0059] In some embodiments, the thread block scheduling unit is specifically used to prioritize the allocation of thread blocks belonging to the same Bank group to the same programmable multiprocessor in sequence, based on Bank grouping, until the registers and shared memory resources of the programmable multiprocessor are exhausted; when the number of thread blocks in a certain Bank group is insufficient to fill all programmable multiprocessors, thread blocks are selected from other Bank groups to fill the remaining programmable multiprocessors in a round-robin manner; after the first round of allocation, the remaining programmable multiprocessors supplement thread blocks from the Bank groups that are not yet saturated in a round-robin order, until all programmable multiprocessors are fully loaded or all thread blocks have been allocated.

[0060] The thread block scheduling unit performs a two-stage scheduling allocation according to specific rules. In the first stage, thread blocks from the same bank group are prioritized for allocation to the same programmable multiprocessor, fully utilizing data spatial locality and reducing concurrent access to different rows of the same bank by different service managers (SMs), thus minimizing bank conflicts. When a group of thread blocks is insufficient, a round-robin approach is used to fill the remaining processors, ensuring resource utilization. In the second stage, thread blocks are replenished according to the round-robin order after the first round, resulting in a more balanced resource allocation. Compared to the traditional pure round-robin strategy, this detailed scheduling method can allocate thread blocks more accurately and efficiently, reducing bank conflicts from multiple aspects, improving system resource utilization and memory access performance, ensuring stable and efficient system operation, and enhancing the overall performance of the GPGPU.

[0061] The thread block memory access characteristic statistics unit analyzes the memory access information of thread blocks issued by the host. This module identifies the banks containing the data accessed by each thread block based on the addresses of the data the thread block needs to access, and then regroups thread blocks in the same bank into a thread block task queue organized by bank. This is illustrated using an external DRAM with four banks as an example. When bank conflicts occur, access can be performed according to the order of access; however, bank conflicts cannot be completely avoided.

[0062] The thread block scheduling unit and the SM resource management unit statistically analyze the registers and other resources within each SM. Based on whether the resources within an SM can meet the resource requirements of a thread block, they decide whether to assign the thread block to that SM for execution. Specifically, each thread block contains information about its required registers and other computing resources. The SM resource management unit maintains a table of available register resources within an SM and the register resources required by the thread block to be allocated. When the available register resources within an SM exceed the register resources required by the thread block to be allocated, the thread block can be allocated to that SM. The Bank task queue management unit schedules the thread block tasks. The scheduling priority has little impact on task execution efficiency. This patent primarily aims to protect data spatial locality; therefore, the scheduling priority is not studied in detail. The scheduling method is as follows: First, the task allocation is performed based on the Bank task queue: 1. Assign Bank1 related task queues to SM1 until SM1 resources are saturated and thread block TB tasks cannot be assigned. If there are no Bank1 related task queues, assign Bank2 related task queues first. 2. In this manner, the relevant task queues of Bank2 / 3 / 4 are allocated to SM2 / 3 / 4 respectively, until the resources of SM2 / 3 / 4 are saturated and cannot allocate thread block TB tasks; Typically, the number of SM cores in a GPGPU exceeds the number of DRAM banks; this example uses 8 SMs for illustration. To achieve task balance across SM cores and fully utilize the computing resources of each SM core, a second round of allocation based on round-robin is performed, such as... Figure 2 As shown, thread blocks TB(1,5) and TB(1,6) are assigned to SM5, TB(2,3), TB(2,4), and TB(2,5) to SM6, TB(3,5), TB(3,6), and TB(3,7) to SM7, and TB(4,4), TB(4,5), and TB(4,6) to SM8. To address the issue of uneven SM load caused by a bank having significantly more thread blocks than other banks, the SM resources released after a bank's task queue is completed are preferentially allocated to the queue of the bank with the most remaining tasks. The number of SMs in a GPGPU can typically reach hundreds or thousands, while the number of DRAM banks is typically 4, 8, or 16.

[0063] In the SM computing resources, a corresponding row cache unit is allocated for each SM. When a thread block TB within the SM initiates a DRAM memory access request, the data in the DRAM cache line is stored in the row cache unit. Due to the spatial locality of thread block data, the data accessed by other thread blocks is also likely to exist in the cache line. Therefore, when other TBs within the SM initiate a memory access request, they first access the row cache unit. If the required data exists, the relevant data can be obtained directly without performing DRAM access with a large access latency, thus improving data reuse and execution efficiency.

[0064] Meanwhile, to address the issue of frequent line cache invalidation, the line cache hit rate is monitored in real time. When the line cache hit rate falls below 30%, the line cache is disabled to avoid additional overhead. Furthermore, for cases where a thread block suddenly changes its memory access bank during execution, the scheduling unit does not perform further processing due to the low probability of this occurring. After a smaller number of bank task queues have completed execution and released SM resources, the released SM resources are preferentially allocated to banks with larger task queues. Since the total number of tasks is fixed, this allocation strategy will not affect execution efficiency due to a larger number of tasks in a particular bank.

[0065] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A GPGPU thread block scheduling method based on data spatial locality, characterized in that, Includes the following steps: S1. Statistically analyze the Bank feature information of all thread block access data. The Bank feature information is determined based on the BA1~BA0 fields in the actual physical address of the thread block access data. Based on the Bank feature information, thread blocks accessing the same Bank are grouped into the same Bank group. S2. Prioritize allocating thread blocks from the same Bank group to the same programmable multiprocessor until the programmable multiprocessor's resources are saturated; S3. Within each programmable multiprocessor, a private row cache space is set up for each Bank. When a thread block within the programmable multiprocessor initiates a DRAM memory access request, the DRAM cache row data is stored in the private row cache of the corresponding Bank. When subsequent thread blocks access data in the same Bank, data is read from the private row cache space first. S4. When the data in the private line cache space is updated, the corresponding data in the L1, L2 cache and DRAM are updated synchronously. S5. After the thread block finishes execution and releases the programmable multiprocessor resources, the released resources are preferentially allocated to the Bank group with the most remaining thread blocks; at the same time, the private line cache hit rate is calculated in real time, and the private line cache function is turned off when the hit rate is lower than the preset threshold.

2. The GPGPU thread block scheduling method based on data spatial locality according to claim 1, characterized in that, S1 specifically includes: During GPGPU runtime, for each thread block to be scheduled, the physical addresses of all its memory access requests are parsed, the Bank identifier field in the DDR address is extracted, the access frequency of the thread block to each Bank is counted, and the thread block is assigned to the group corresponding to the Bank with the highest access frequency. If multiple Banks have the same frequency, one of them is selected.

3. The GPGPU thread block scheduling method based on data spatial locality according to claim 2, characterized in that, S1 also includes: When the difference in access frequency of the same thread block to multiple Banks is less than a preset threshold, the thread block is assigned to the group with the smallest Bank identifier field value.

4. The GPGPU thread block scheduling method based on data spatial locality according to claim 3, characterized in that, S2 include: First round of allocation: Thread blocks are allocated to the corresponding programmable multiprocessors according to the Bank grouping order, until the programmable multiprocessor resources are saturated; Second round of allocation: If the number of programmable multiprocessors is greater than the number of banks, the remaining thread blocks are allocated to unsaturated programmable multiprocessors using a round-robin method.

5. The GPGPU thread block scheduling method based on data spatial locality according to claim 4, characterized in that, The first round of allocation specifically includes: Based on the Bank group, thread blocks belonging to the same Bank group are allocated sequentially to the same programmable multiprocessor until the registers and shared memory resources of the programmable multiprocessor are exhausted. When the number of thread blocks in a certain Bank group is insufficient to fill all programmable multiprocessors, thread blocks are selected from other Bank groups to fill the remaining programmable multiprocessors in a round-robin manner.

6. The GPGPU thread block scheduling method based on data spatial locality according to claim 5, characterized in that, The second round of allocation specifically includes: After the first round of allocation, the remaining programmable multiprocessors replenish thread blocks from the unsaturated Bank groups in a round-robin order until all programmable multiprocessors are fully loaded or all thread blocks have been allocated.

7. The GPGPU thread block scheduling method based on data spatial locality according to claim 6, characterized in that, S3 also includes: The private row cache adopts a write-through strategy, where any write request to the same DRAM row is immediately written back to DRAM, and the corresponding row in the L1 and L2 caches is updated before the write-back. When the private line cache space is full and a new DRAM line needs to be loaded, the Least Recently Used (LRU) replacement strategy is used to select the line to be replaced. Before replacement, it is determined whether dirty data has occurred. If so, a write-back operation is performed first.

8. The GPGPU thread block scheduling method based on data spatial locality according to claim 7, characterized in that, The size of the private line cache space is consistent with the size of the DRAM line buffer.

9. A GPGPU thread block scheduling system based on data spatial locality, characterized in that, include: The thread block memory access feature statistics unit is used to count the Bank feature information of all thread block access data. The Bank feature information is determined based on the BA1~BA0 fields in the actual physical address of the thread block access data, and the thread blocks accessing the same Bank are grouped into the same Bank group according to the Bank feature information. The thread block scheduling unit is used for two-stage scheduling allocation based on the Bank grouping results; the two-stage scheduling allocation includes a first round of allocation according to the Bank grouping order and a second round of round-robin allocation. A line cache unit, located within each programmable multiprocessor, is used to store the DRAM line cache data for the corresponding Bank; The cache consistency maintenance unit is used to synchronously update the cache and DRAM at all levels when the private row cache data is updated; The dynamic cache control unit is used to dynamically enable or disable the line cache function based on the line cache hit rate.

10. The GPGPU thread block scheduling system based on data spatial locality according to claim 9, characterized in that, The thread block scheduling unit is specifically used to prioritize the allocation of thread blocks belonging to the same Bank group to the same programmable multiprocessor in sequence, based on Bank grouping, until the registers and shared memory resources of the programmable multiprocessor are exhausted; when the number of thread blocks in a certain Bank group is insufficient to fill all programmable multiprocessors, thread blocks are selected from other Bank groups to fill the remaining programmable multiprocessors in a round-robin manner; after the first round of allocation, the remaining programmable multiprocessors supplement thread blocks from the Bank groups that are not yet saturated in a round-robin order, until all programmable multiprocessors are fully loaded or all thread blocks have been allocated.

Images