Heterogeneous computing thread block optimal scheduling method and system based on dynamic topology mapping

Abstract

The invention belongs to the field of parallel computing architecture optimization, and relates to a matrix multiplication acceleration method and system based on dynamic computing resource mapping, and the method comprises the steps: constructing a dynamic topology model driven by tensor dimension features, and generating a thread block distribution mode according to matrix parameters and GPU hardware information; constructing a multi-dimensional resource scheduling strategy library, dynamically selecting an optimal thread block distribution strategy from the multi-dimensional resource scheduling strategy library, and generating a binding relationship between the thread blocks and the data blocks; calculating collaborative access logic of thread blocks and storage hierarchies based on block parameters and dynamic mapping function optimization; distributed calculation is carried out, calculation and data transmission are parallelized through pipelining and a double-buffering mechanism, and result aggregation across calculation units is completed synchronously through atomic operation and a barrier. According to the method, discontinuous memory access conflicts can be effectively reduced, the execution efficiency of the calculation instruction and the utilization rate of the cache space are improved, the parallel calculation process of accelerating and optimizing the general matrix multiplication is realized, and the data processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of parallel computing architecture optimization, and in particular to a method and system for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping. Background Art

[0002] In today's digital age, the scale and complexity of data processing continue to rise, posing a severe challenge to computing efficiency. To address this challenge, parallel computing technology has emerged. By breaking down complex tasks into multiple subtasks and executing them simultaneously on multiple processing units, it significantly shortens computing time and greatly improves data processing efficiency. However, in the practice of parallel computing, the organization and indexing of data have a crucial impact on computing efficiency. Traditional index calculation methods, such as simple linear indexing or two-dimensional indexing, have gradually exposed many limitations when processing large-scale multi-dimensional data. Linear indexing often leads to poor data locality when processing multi-dimensional data, thereby reducing cache utilization. On the other hand, two-dimensional indexing is not flexible enough when dealing with higher-dimensional data and cannot meet the needs of complex data structures.

[0003] In complex computing tasks, such as general matrix multiplication (GEMM) and other computationally intensive tasks, performance bottlenecks often manifest as limited memory bandwidth, low cache utilization, and high idle rates of compute units. These issues severely impact overall computing efficiency. Traditional static scheduling methods employ fixed thread block partitioning strategies, which struggle to adapt to dynamically changing computational loads and resource availability. Furthermore, the lack of flexibility in the binding relationship between thread blocks and data blocks leads to frequent non-contiguous memory accesses, further reducing the efficiency of core instruction issuance. While some existing approaches attempt to improve data locality through block-based computing, these approaches lack an effective dynamic adaptation mechanism between block granularity and resource characteristics, making global optimization difficult in complex computing scenarios. Another area of ​​research seeks to reduce resource contention through pipeline scheduling. However, these studies lack systematic modeling of the spatiotemporal distribution characteristics of thread blocks, resulting in significant deficiencies in balancing the synergistic relationship between computation and memory access.

[0004] As the demand for data processing continues to grow, the limitations of existing technologies are becoming increasingly apparent. In large-scale data processing, traditional indexing methods and scheduling strategies can no longer meet increasingly complex needs. Dynamic changes in load and resource status require scheduling methods that can perceive and adaptively adjust the distribution of tasks in real time. However, existing block computing and pipeline scheduling methods have obvious shortcomings in dynamic adaptation and global optimization. Therefore, there is a need for a scheduling method that can dynamically perceive resource status and adaptively adjust the distribution of tasks. It can flexibly respond to dynamically changing computing loads, optimize data locality, and improve cache utilization, thereby comprehensively improving the efficiency and performance of data processing. Summary of the Invention

[0005] In order to solve the bottleneck problems such as unbalanced resource allocation and inefficient storage access in matrix multiplication and similar computationally intensive tasks under the existing heterogeneous computing architecture, the present invention proposes a collaborative optimization method and system based on dynamic topology mapping, which can flexibly respond to dynamically changing computing loads, optimize data locality, improve cache utilization and enhance data processing efficiency.

[0006] The first object of the present invention is to provide a heterogeneous computing thread block optimization scheduling method based on dynamic topology mapping.

[0007] The second object of the present invention is to provide a heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping.

[0008] The purpose of the present invention can be achieved by taking the following technical solutions:

[0009] The method for optimizing the scheduling of heterogeneous computing thread blocks based on dynamic topology mapping includes the following steps:

[0010] S1. Build a dynamic topology model driven by tensor dimension features and generate thread block distribution patterns based on matrix parameters and GPU hardware information.

[0011] S2. Build a multi-dimensional resource scheduling policy library. Based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, dynamically select the optimal thread block distribution policy from the multi-dimensional resource scheduling policy library and generate a binding relationship between the thread block and the data block.

[0012] S3. Define block parameters and establish dynamic mapping functions, and optimize the collaborative access logic between computing thread blocks and storage layers based on the block parameters and dynamic mapping functions.

[0013] S4. Distributed computing is performed based on the binding relationship between thread blocks and data blocks. The computing and data transmission are parallelized through pipelining and double buffering mechanisms. The results across computing units are aggregated through atomic operations and barrier synchronization, and the final computing results are written back to the global memory.

[0014] Specifically, the step S1 includes:

[0015] S11, obtaining input matrix parameters and GPU hardware information, extracting array dimension feature parameters of tensor operations based on the matrix parameters, performing validity check on the matrix dimension feature parameters, and obtaining valid parameters;

[0016] S12, performing multi-dimensional constraint analysis on the valid parameters, and selecting a thread block distribution pattern from a topology mapping function library according to the dimensional characteristic parameters of the matrix operation and their associated constraints;

[0017] S13, verifying the feasibility of the selected thread block distribution mode, and generating a dynamic binding relationship between the thread block and the data block when the thread block distribution mode is feasible;

[0018] S14. Dynamically adjust the block granularity parameters according to the utilization of computing resources and the latency of data access to optimize the computing efficiency, and obtain the final dynamic topology model. The dynamic topology model dynamically generates the optimal thread block distribution pattern according to the matrix parameters and GPU hardware information.

[0019] Specifically, the step S2 includes:

[0020] S21. Build a multi-dimensional resource scheduling policy library, where the multi-dimensional resource scheduling policy library includes at least one of the following thread block distribution policies:

[0021] The vertical blocking strategy divides the computing blocks according to the row-priority principle and establishes a binding relationship between thread blocks and data rows. Each thread block is responsible for processing multiple consecutive rows of data.

[0022] The horizontal block strategy generates a serpentine topology based on the column-first principle. Each thread block is responsible for processing multiple consecutive columns of data.

[0023] Composite blocking strategy: constructing a chessboard-like thread block distribution structure with bidirectional row and column blocking, where each thread block is responsible for processing a sub-matrix;

[0024] Adaptive hybrid strategy dynamically adjusts the block direction and granularity based on the real-time cache hit rate, and dynamically selects the optimal block direction and granularity according to the cache usage and hit rate.

[0025] S22. Select an optimal thread block distribution strategy from a multi-dimensional resource scheduling strategy library based on runtime parameters, generate a mapping relationship between thread blocks and data blocks, and output a mapping matrix between thread blocks and data blocks;

[0026] S23. According to the mapping matrix between thread blocks and data blocks, the blocks of matrix A and matrix B are mapped to the thread blocks of the GPU and loaded into the shared memory.

[0027] Specifically, the step S22 includes:

[0028] S221. Monitor the operating parameters of the heterogeneous computing system and collect computing load and cache status data in real time;

[0029] S222, based on the operating parameters and thread block distribution mode of the heterogeneous computing system, select a thread block distribution strategy that best suits the current runtime conditions from a strategy library using a Monte Carlo tree search algorithm;

[0030] S223 . Generate a binding relationship between thread blocks and data blocks according to the selected thread block distribution strategy, and output a mapping matrix between thread blocks and data blocks.

[0031] Specifically, step S3 includes:

[0032] S31. Define block parameters and control the alignment rules of data blocks by setting the size and arrangement direction of data blocks;

[0033] S32. Establish a dynamic mapping function between thread block index and cache line, and map the thread block index to the cache line through the dynamic mapping function. The dynamic mapping function is expressed as:

[0034] cache_line=(threadIdx·sc)⊕(blockIdx·sb);

[0035] Where cache_line represents the index of the cache line, threadIdx represents the index of the thread in the thread block, blockIdx represents the index of the thread block in the global grid, sc is the cache alignment coefficient, sb is the block span parameter, and ⊕ represents the exclusive-or operation.

[0036] Specifically, the step S4 includes:

[0037] A heterogeneous pipeline architecture is used to divide computing tasks into four stages: data prefetching, matrix partitioning, core computing, and result aggregation. Each stage is responsible for the specified computing tasks.

[0038] In the data prefetching phase, while calculating the current block, the data of the next block is prefetched and loaded into the shared memory (buffer). Buffer A and buffer B are introduced. When buffer A is used for calculation, buffer B is used to prefetch the data of the next block; when buffer A is used to prefetch the data of the next block, buffer B is used for calculation.

[0039] In the matrix block stage, the pre-fetched block data is split into small blocks that fit into cache lines according to the thread block distribution strategy, and the thread block reading order is rearranged.

[0040] In the core calculation phase, the rearranged matrix blocks are assigned to GPU thread blocks for parallel calculation. Each thread block is responsible for calculating the product of a sub-matrix to obtain the calculation results of each thread block.

[0041] In the result aggregation stage, the calculation results of each thread block are saved in the shared memory, and the result aggregation across the computing units is completed through atomic operations and barrier synchronization to obtain the final matrix multiplication result, and the final matrix multiplication result is written back to the global memory.

[0042] A heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping, comprising:

[0043] Dynamic modeling module, used to build a dynamic topology model driven by tensor dimension features and generate thread block distribution patterns based on matrix parameters and GPU hardware information;

[0044] The policy scheduling module is used to build a multi-dimensional resource scheduling policy library. Based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, it dynamically selects the optimal thread block distribution policy from the multi-dimensional resource scheduling policy library and generates the binding relationship between thread blocks and data blocks.

[0045] The access optimization module is used to define the block parameters and establish the dynamic mapping function, and optimize the collaborative access logic between the computing thread block and the storage layer based on the block parameters and the dynamic mapping function;

[0046] The distributed computing module performs distributed computing based on the binding relationship between thread blocks and data blocks. It parallelizes computing and data transmission through pipelining and double buffering mechanisms, aggregates results across computing units through atomic operations and barrier synchronization, and writes the final computing results back to the global memory.

[0047] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0048] The present invention provides a collaborative optimization method and system based on dynamic topological mapping. By constructing a dynamic topological model driven by tensor dimensional features, it can dynamically perceive resource status and adaptively adjust the task distribution form, optimize data locality and cache utilization, and thus comprehensively improve the efficiency and performance of data processing. By constructing a multi-dimensional resource scheduling policy library, based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, the optimal thread block distribution strategy is dynamically selected from the multi-dimensional resource scheduling policy library. Through adaptive block granularity control and memory access optimization, non-continuous memory access conflicts can be effectively reduced, the execution efficiency of computing instructions and cache space utilization can be improved, and the parallel computing process of general matrix multiplication can be accelerated and optimized. The parallelism and efficiency of the calculation are improved through pipelining, the parallelization of calculation and data transmission is achieved through a double buffering mechanism, and the result aggregation across computing units is completed synchronously through atomic operations and barriers, the accuracy and reliability of the calculation are improved, and the accuracy and integrity of the results can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0050] Figure 1 This is a flow chart of a heterogeneous computing thread block optimization scheduling method based on dynamic topology mapping in an embodiment of the present invention;

[0051] Figure 2 This is a flow chart of pipeline optimization and dynamic result aggregation in an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of thread block remapping in an example of the present invention;

[0053] Figure 4 This is an architecture diagram of a heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is obvious that the embodiments described are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited to these. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] Example 1:

[0056] This embodiment proposes a method for optimizing the scheduling of heterogeneous computing thread blocks based on dynamic topology mapping. This method accelerates matrix multiplication based on dynamic computing resource mapping. Through adaptive block granularity control and memory access optimization, it effectively reduces non-contiguous memory access conflicts, improves instruction execution efficiency and cache space utilization, and accelerates and optimizes the parallel computing process of general matrix multiplication. By introducing dynamic topology mapping and an adaptive scheduling mechanism, the limitations of existing technologies can be effectively addressed.

[0057] like Figure 1 As shown, a heterogeneous computing thread block optimization scheduling method based on dynamic topology mapping includes the following steps:

[0058] S1. Build a dynamic topology model driven by tensor dimension features and generate thread block distribution patterns based on matrix parameters and GPU hardware information.

[0059] S11. Obtain input matrix parameters and GPU hardware information, extract array dimension feature parameters of tensor operations based on the matrix parameters, perform validity check on the matrix dimension feature parameters, and obtain valid parameters.

[0060] Specifically, matrix parameters include matrix size, block granularity, and data sparsity. GPU hardware information includes the GPU's maximum shared memory capacity, number of banks, and maximum number of threads per thread block. These hardware parameters directly affect data storage and access methods, as well as the allocation of computing resources.

[0061] Before performing matrix multiplication operations, it is first necessary to fully obtain the detailed parameters of matrix A and matrix B as well as relevant information about the GPU hardware. These parameters are the basis for formulating subsequent optimization strategies. Specifically, the matrix size: the size of matrix A is M×K, and the size of matrix B is K×N. These size parameters determine the scale and complexity of matrix multiplication. Block granularity: The block granularity is T_M×T_N, usually 64×64. This size is the result of a trade-off between computational granularity and data locality, and can effectively balance computational efficiency and data access efficiency. Data sparsity: The sparsity characteristic of the matrix, that is, the proportion of zero elements in the matrix, is important for optimizing storage and reducing unnecessary calculations.

[0062] For example, in a typical test scenario, the dimensions of matrices A and B are both 2048×2048, the block size is 32×32, the shared memory capacity is 48KB, the number of banks is 32, and the data type is float (4 bytes / element). These parameters will be used in the subsequent dynamic topology model construction and resource scheduling strategy selection.

[0063] The matrix parameters are used to extract the matrix-dimensional characteristic parameters of the tensor operation. These are expressed as (M, N, K), where M, N, and K represent the number of rows, columns, and depth of the tensor, respectively. The input parameters are first validated. If the parameters are invalid, the system handles the exception, logs the error, and terminates the process. If the parameters are valid, the process continues with multidimensional constraint resolution.

[0064] S12. Perform multi-dimensional constraint analysis on valid parameters, and select a thread block distribution pattern from a topology mapping function library based on dimensional characteristic parameters and associated constraints of matrix operations.

[0065] Analyze the dimensional characteristic parameters and associated constraints of matrix operations. The associated constraints include but are not limited to the sparsity, data locality, and computational dependency of tensors. Sparsity refers to the property that most elements in a tensor are zero or close to zero. Sparse tensors reduce memory usage and computational redundancy by storing only non-zero elements and their indices (such as COO and CSR formats). Data locality refers to the physical storage proximity of frequently accessed data during the calculation process, and the cache mechanism is used to reduce memory access latency. Computational dependency refers to the order constraints between operations, which may limit parallelism and cause pipeline stalls. Based on these analysis results, select a thread block distribution pattern that suits the current matrix characteristics from the topology mapping function library, such as linear distribution, two-dimensional grid distribution, or three-dimensional cube distribution.

[0066] The topology mapping function library is used to store thread block distribution patterns under different dimensional combinations, including but not limited to linear distribution, two-dimensional grid distribution and three-dimensional cube distribution. Each distribution pattern corresponds to a specific thread block shape and arrangement.

[0067] S13. Verify the feasibility of the thread block distribution pattern selected from the topology mapping function library. When the thread block distribution pattern is feasible, generate a dynamic binding relationship between the thread block and the data block to ensure that each thread block can correctly access and process the corresponding data block.

[0068] Specifically, the selected thread block distribution pattern is subject to feasibility verification. If the thread block distribution pattern is feasible, the system will generate dynamic binding relationships to ensure that each thread block can correctly access and process the corresponding data block. If the pattern is not feasible, the system will trigger the mode switching logic and select another thread block distribution pattern.

[0069] S14. Dynamically adjust the block granularity parameters according to the utilization of computing resources and the latency of data access to optimize the computing efficiency, and obtain the final dynamic topology model. The dynamic topology model is used to dynamically generate the optimal thread block distribution pattern according to the matrix parameters and GPU hardware information.

[0070] Specifically, the block granularity parameter represents the number of threads per thread block across different dimensions. By monitoring computing resource utilization and data access latency, the block granularity parameter is dynamically adjusted to optimize computing efficiency. The convergence criteria are then checked to determine whether the parameter has reached or is near-optimal. If so, the final optimized topology model is obtained, providing an efficient and flexible solution for large-scale data processing. If not, the system continues to optimize the parameters based on feedback from the performance monitoring feedback loop. Through the above process, the dynamic topology model dynamically generates the optimal thread block distribution pattern based on the input matrix characteristics and hardware status, ensuring efficient execution of computing tasks. This model not only optimizes the distribution of computing tasks and improves data locality, but also adapts to dynamic load changes, achieving global optimization. The adaptive mapping of multidimensional computing tasks to heterogeneous computing resources refers to the dynamic mapping of multidimensional computing tasks to heterogeneous computing resources through the dynamic topology model. This mapping is dynamic and can be adjusted based on the input matrix characteristics and hardware status to achieve optimal computing efficiency.

[0071] For example, if matrices A and B are both 2048×2048 and the block size is 32×32, the dynamic topology model might choose a two-dimensional grid distribution with thread blocks of 32×32 arranged in a 64×64 grid. This distribution fully utilizes the parallel computing capabilities of the GPU while reducing data access conflicts and latency.

[0072] S2. Build a multi-dimensional resource scheduling strategy library. Based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, dynamically select the optimal thread block distribution strategy from the multi-dimensional resource scheduling strategy library and generate a mapping relationship between thread blocks and data blocks.

[0073] S21. Build a multi-dimensional resource scheduling policy library, where the multi-dimensional resource scheduling policy library includes at least one of the following thread block distribution policies:

[0074] The vertical blocking strategy divides the computation blocks according to the row-priority principle and establishes a binding relationship between thread blocks and data rows. Each thread block is responsible for processing multiple consecutive rows of data, improving the locality of data access.

[0075] The horizontal block strategy generates a serpentine topology based on the column-first principle. Each thread block is responsible for processing multiple consecutive columns of data. The serpentine topology reduces data dependencies between thread blocks.

[0076] A composite blocking strategy builds a chessboard-like thread block distribution structure with bidirectional row and column blocking. Each thread block is responsible for processing a sub-matrix. This bidirectional row and column blocking improves computational parallelism and data reuse.

[0077] The adaptive hybrid strategy dynamically adjusts the block direction and granularity based on the real-time cache hit rate, and dynamically selects the optimal block direction and granularity according to the cache usage and hit rate to maximize cache utilization and computing efficiency.

[0078] S22. Based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, a mapping relationship between thread blocks and data blocks is generated, and a thread block-data block mapping matrix is ​​output. During this process, the system needs to comprehensively consider the current computing load, hardware resource status, and data characteristics.

[0079] S221. Monitor the operating parameters of the heterogeneous computing system and collect computing load and cache status data in real time.

[0080] By monitoring parameters such as computing resource utilization, cache hit rate, and data access latency, real-time runtime information is provided for policy scheduling.

[0081] S222. Based on the operating parameters and thread block distribution mode of the heterogeneous computing system, select a thread block distribution strategy that best suits the current runtime conditions from a strategy library based on a Monte Carlo tree search algorithm.

[0082] The optimal thread block distribution strategy is selected from the strategy library based on the Monte Carlo Tree Search (MCTS) algorithm. The specific process includes:

[0083] In the selection phase, starting from the root node, the Monte Carlo tree search algorithm is used. Based on the child node's win rate and exploration count, the upper confidence bound (UCT) algorithm is used to select the most promising child nodes for expansion. In this embodiment, this means selecting the scheduling strategy most likely to lead to performance improvement for further simulation. This process involves simulating and evaluating different scheduling strategies to determine the most effective strategy under the current computing environment.

[0084] In the expansion phase, the tree is expanded at the selected node, adding one or more new child nodes, representing scheduling strategies that have not been tried. These new strategies may change the size of thread blocks, adjust data access patterns, or change the distribution of thread blocks.

[0085] During the simulation phase, starting with the newly added nodes, a simulation is performed to evaluate the performance of the new strategy. During the simulation, the system simulates the actual execution of a computing task, including steps such as data loading, computation execution, and result aggregation. The goal of the simulation is to evaluate the performance of the new strategy in handling the current computing task, including computation speed, resource utilization, and data access efficiency.

[0086] In the update phase, simulation results (such as execution time and resource utilization) are fed back into the search tree. All nodes along the path from the root node to the simulated node have their statistics, such as win rate and number of visits, updated based on the simulation results. These updates help the algorithm make more accurate decisions in future selection steps.

[0087] Through multiple simulations, the MCTS algorithm identifies the strategy most likely to improve computational efficiency under current conditions. Based on this strategy, the system then generates a mapping between thread blocks and data blocks, ensuring that each thread block can efficiently access and process the corresponding data block. This dynamic, adaptive scheduling strategy selection method automatically selects the optimal resource scheduling strategy under varying computing environments and task requirements, thereby improving the execution efficiency and resource utilization of computing tasks.

[0088] S223. Generate a mapping relationship between thread blocks and data blocks based on the selected optimal thread block distribution strategy, and output a mapping matrix between thread blocks and data blocks to ensure that each thread block can correctly access and process the corresponding data block.

[0089] Thread block distribution strategies are crucial for improving data locality and L2 cache hit rates. Dynamically selecting a multi-dimensional resource scheduling strategy based on runtime parameters can significantly optimize data access patterns and improve computational efficiency. Careful selection of thread block distribution directions, such as row-first, column-first, or mixed, is based on matrix characteristics and hardware status. Different distribution directions are suitable for different data access patterns and computational requirements.

[0090] Distribution density setting: Reasonably set the distribution density of thread blocks. The distribution density of thread blocks is used to finely control the distribution pattern of thread blocks, which affects the locality of data and cache utilization.

[0091] S23. According to the mapping matrix between thread blocks and data blocks, the blocks of matrices A and B are mapped to the thread blocks of the GPU and loaded into the shared memory.

[0092] Rationally distribute matrix blocks to different thread blocks, ensuring that each thread block can efficiently process a block while taking into account data locality and computational balance. Thread blocks are precisely distributed across the GPU's computational grid, ensuring collaboration and efficient data access between thread blocks. Thread blocks efficiently load corresponding block data from global memory into shared memory, fully preparing for subsequent computations.

[0093] For example, a block of matrix A may be mapped to thread block (2,3), which will be responsible for loading the block data from global memory into shared memory to provide data support for subsequent matrix multiplication calculations.

[0094] S3. Define block parameters and establish dynamic mapping functions. Based on the block parameters and dynamic mapping functions, optimize the collaborative access logic between computing thread blocks and storage layers to improve cache utilization and data access efficiency.

[0095] S31. Define block parameters and control the alignment rules of data blocks by setting the size and arrangement direction of data blocks.

[0096] Specifically, the block parameters (count, dir) are defined to control the data block alignment rules, where count represents the size of each data block and dir represents the arrangement direction of the data blocks.

[0097] S32: Establish a dynamic mapping function between the thread block index and the cache line, and map the thread block index to the cache line through the dynamic mapping function.

[0098] The dynamic mapping function is expressed as:

[0099] cache_line=(threadIdx·sc)⊕(blockIdx·sb);

[0100] Where cache_line represents the cache line index, threadIdx represents the thread index within the thread block, blockIdx represents the thread block index within the global grid, sc represents the cache alignment factor, sb represents the block span parameter, which controls the alignment of data blocks, and ⊕ represents the exclusive-or operation. A dynamic mapping function enables efficient mapping between thread block indices and cache lines, reducing cache conflicts and data access latency.

[0101] A mapping function transforms thread block indices into new distribution locations, optimizing data access patterns and reducing data transmission latency and contention. This mapping function ensures that thread blocks can access data blocks in a manner that reduces cache contention and data access latency. This mapping strategy is crucial for improving data locality and cache hit rates, significantly boosting computational efficiency. By dynamically selecting multi-dimensional resource scheduling strategies, the system can optimize data access patterns based on runtime parameters, adapting to varying computational requirements and hardware characteristics.

[0102] For example, in an actual application scenario, if the row-first distribution direction is selected and the distribution density is set to 9, the thread block index will be converted to a new position through the mapping function to achieve more efficient data access and calculation.

[0103] By properly setting the size and orientation of data blocks to control data block alignment, and using dynamic mapping functions to map thread block indices to cache lines or other storage resources, cache conflicts and data access latency are reduced. By reconstructing the collaborative access logic between compute thread blocks and the storage hierarchy, data is intelligently distributed across different levels of cache to improve overall cache efficiency.

[0104] S4. Distributed computing is performed based on the binding relationship between thread blocks and data blocks. The computing and data transmission are parallelized through pipelining and double buffering mechanisms. The results across computing units are aggregated through atomic operations and barrier synchronization, and the final computing results are written back to the global memory.

[0105] like Figure 2 The figure shows a flow chart of pipeline optimization and dynamic result aggregation. In this embodiment, a heterogeneous pipeline architecture is used to divide the computing task into four stages: data prefetching, matrix partitioning, core computing, and result aggregation. The four stages are responsible for the set computing tasks respectively. The parallelism and efficiency of the computing are improved through pipelining. The parallelization of computing and data transmission is achieved through a double buffering mechanism. The result aggregation across computing units is completed synchronously through atomic operations and barriers. The results are temporarily stored in shared memory. The final results are written back to the global memory in the result write-back stage to complete the computing task. Among them:

[0106] In the data prefetching stage, while calculating the current block, the data of the next block is prefetched and loaded into the shared memory (buffer). Buffer A and buffer B are introduced. When buffer A is used for calculation, buffer B is used to prefetch the data of the next block; when buffer A is used to prefetch the data of the next block, buffer B is used for calculation.

[0107] Specifically, while computing the current block, prefetching the next block's data and loading it into shared memory (buffer) reduces data transmission latency. A double-buffering mechanism enables parallelization of computation and data transmission. By introducing buffers A and B, buffers A and B alternately load block data and perform computation, enabling parallelization of data transmission and computation. This mechanism significantly reduces the impact of data transmission on computation, improving overall performance.

[0108] In the matrix blocking stage, the pre-fetched block data is split into small blocks that adapt to the cache line according to the thread block distribution strategy, and the thread block reading order is rearranged to ensure data locality and computational efficiency.

[0109] like Figure 3 Figure 2 shows a schematic diagram of thread block remapping in an example of the present invention, demonstrating a comparative example of thread block mapping optimization. In traditional methods, each thread block accesses data sequentially in fixed blocks, such as 81 blocks for matrix A and 27 blocks for matrix B. This results in poor data locality and a low cache hit rate (measured L1 hit rate <50%). Furthermore, address concentration during writes to matrix C causes cache contention. This linear distribution also results in uneven load on SM computing units, with some SM utilization rates below 30%. A dynamic remapping strategy changes the thread block read order, breaking large blocks into smaller blocks that fit into cache lines. For example, matrix A is changed to a 27-block method, and a zigzag read order is used to disperse data access. Furthermore, the thread block-to-SM mapping relationship is adjusted based on hardware feedback, distributing logically adjacent thread blocks to different physical units. After optimization, the cache hit rate increased to 89%, SM load balance increased by 2.1 times, and the peak computational efficiency of matrix multiplication reached 92% of the theoretical value. Furthermore, the strategy automatically adapts to task scale and hardware characteristics.

[0110] In the core calculation stage, the rearranged matrix blocks are assigned to the GPU thread blocks for parallel calculation. Each thread block is responsible for calculating the product of a sub-matrix, and the calculation results of each thread block are obtained, which can fully utilize the parallel computing capabilities of the GPU.

[0111] Specifically, within each thread block, threads are assigned to different computational tasks, with each thread responsible for calculating one element in a submatrix. Threads perform efficient matrix multiplications using data stored in shared memory, and the fast access capabilities of shared memory significantly improve computational efficiency.

[0112] In the result aggregation stage, the calculation results of each thread block are saved in the shared memory, and the result aggregation across the computing units is completed through atomic operations and barrier synchronization to obtain the final matrix multiplication result, and the final matrix multiplication result is written back to the global memory.

[0113] During the result aggregation phase, computation results are first stored in shared memory and then aggregated through atomic operations. This ensures that the computation results from different thread blocks are correctly merged, ultimately resulting in an accurate matrix multiplication result. This improves computational accuracy and reliability, ensuring the accuracy and integrity of the results. For example, when computing the product of submatrices A and B, each thread in the thread block calculates an element in C and accumulates the results to shared memory through atomic operations. Finally, the main thread of the thread block writes the computation results back to global memory.

[0114] In summary, the present invention provides a collaborative optimization method based on dynamic topological mapping, which constructs a dynamic topological model driven by tensor dimensional features, dynamically perceives resource status, adaptively adjusts the task distribution form, optimizes data locality and cache utilization, and thus comprehensively improves the efficiency and performance of data processing. By constructing a multi-dimensional resource scheduling policy library, based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, the optimal thread block distribution strategy is dynamically selected from the multi-dimensional resource scheduling policy library. Through adaptive block granularity control and memory access optimization, non-continuous memory access conflicts can be effectively reduced, the execution efficiency of computing instructions and cache space utilization can be improved, and the parallel computing process of general matrix multiplication can be accelerated and optimized. The parallelism and efficiency of the calculation are improved through pipelining, the parallelization of calculation and data transmission is achieved through a double buffering mechanism, and the result aggregation across computing units is completed synchronously through atomic operations and barriers, the accuracy and reliability of the calculation are improved, and the accuracy and integrity of the results can be ensured.

[0115] This invention can efficiently schedule and optimize large-scale data processing tasks across heterogeneous computing platforms. In particular, when heterogeneous computing systems have multi-level cache structures, further parallelizing data reading and computation can significantly improve the speed of computing task completion. It can be applied to scenarios requiring large-scale parallel processing, such as scientific computing and artificial intelligence training. It can flexibly respond to dynamically changing computing loads, optimize data locality, and improve cache utilization, effectively completing large-scale data processing tasks across various heterogeneous computing platforms.

[0116] Example 2:

[0117] like Figure 4 As shown, this is an architecture diagram of a heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping. Based on an embodiment, a heterogeneous computing thread block optimization scheduling method based on dynamic topology mapping is provided. This embodiment provides a heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping. The system realizes efficient processing of large-scale matrix operations through dynamic topology mapping and resource scheduling optimization, and significantly improves computing performance.

[0118] Specifically, a heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping includes:

[0119] The dynamic modeling module is used to build a dynamic topology model driven by tensor dimensional features and generate a thread block distribution pattern based on matrix parameters and GPU hardware information. The dynamic modeling module includes a dimensionality parsing unit, a topology mapping function library, and an adaptive adjustment algorithm. The dimensionality parsing unit deeply analyzes the dimensional characteristics of the matrix, extracts key parameters such as the number of rows, columns, and depth, and takes into account complex characteristics such as data sparsity, locality, and computational dependencies. The topology mapping function library provides a variety of thread block distribution modes, including linear distribution, two-dimensional grid distribution, and three-dimensional cube distribution. Each mode is optimized for specific matrix characteristics and computing scenarios. The adaptive adjustment algorithm dynamically adjusts the block granularity parameters according to the real-time load and optimizes computing efficiency in real time by monitoring computing resource utilization and data access latency.

[0120] The construction of a dynamic topology model driven by tensor dimension features generates a thread block distribution pattern based on matrix parameters and GPU hardware information, including:

[0121] Obtain input matrix parameters and GPU hardware information, extract array dimension feature parameters of tensor operations based on matrix parameters, perform validity check on matrix dimension feature parameters, and obtain valid parameters;

[0122] Perform multi-dimensional constraint analysis on valid parameters, and select thread block distribution patterns from the topology mapping function library based on the dimensional characteristic parameters of matrix operations and their associated constraints;

[0123] Verify the feasibility of the selected thread block distribution mode. If the thread block distribution mode is feasible, generate a dynamic binding relationship between the thread block and the data block.

[0124] According to the utilization of computing resources and the latency of data access, the block granularity parameters are dynamically adjusted to achieve the optimal computing efficiency, and the final dynamic topology model is obtained. The dynamic topology model dynamically generates the optimal thread block distribution pattern based on matrix parameters and GPU hardware information.

[0125] The policy scheduling module is used to build a multi-dimensional resource scheduling policy library. Based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, it dynamically selects the optimal thread block distribution strategy from the multi-dimensional resource scheduling policy library and generates the binding relationship between thread blocks and data blocks. The policy scheduling module includes a parameter monitoring unit, a policy decision engine, and a binding relationship generator. The runtime parameter monitoring unit collects parameters such as computing load and cache status in real time to provide a basis for policy selection. The policy decision engine selects the scheduling strategy that best suits the current runtime conditions from the policy library based on the Monte Carlo tree search algorithm. The binding relationship generator generates a mapping relationship between thread blocks and data blocks based on the selected strategy, ensuring that each thread block can efficiently access and process the corresponding data.

[0126] The access optimization module is used to define block parameters and establish dynamic mapping functions. Based on these parameters, it optimizes the collaborative access logic between compute thread blocks and the storage hierarchy. The access optimization module aims to optimize storage access logic. The block parameter definition section defines block parameters, such as block granularity and alignment, to control data block alignment rules and access patterns. The dynamic mapping function establishment section dynamically maps thread block indices to cache lines, reducing cache conflicts and data access latency.

[0127] The distributed computing module performs distributed computing based on the binding relationship between thread blocks and data blocks. Pipelining and double-buffering enable parallelization of computation and data transfer. Atomic operations and barriers synchronize results across computing units, and the final results are written back to global memory. The distributed computing module divides computing tasks into four stages: data prefetching, matrix partitioning, core computation, and result aggregation. Pipelining improves computational parallelism and efficiency. Double-buffering parallelizes computation and data transfer, reducing data transmission blockage. Atomic operations ensure the accuracy and completeness of computation results by aggregating results across computing units.

[0128] The workflow of the heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping includes:

[0129] Starting with parameter acquisition, the input module obtains matrix and hardware parameters, including matrix size, block granularity, data sparsity, and GPU hardware information. Subsequently, the dynamic modeling module constructs a dynamic topology model based on matrix characteristics and hardware status, selecting the optimal thread block distribution pattern. The policy scheduling module selects the optimal scheduling policy from the policy library and generates a mapping between thread blocks and data blocks. The data mapping phase maps matrix blocks to thread blocks and loads them into shared memory in preparation for computation. The distributed computing module performs parallel computations, temporarily storing the results in shared memory. Finally, the result writeback phase writes the final results back to global memory, completing the computation.

[0130] A heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping ensures that the thread block distribution matches the computing requirements by dynamically generating the optimal topology structure based on matrix characteristics and hardware status in a dynamic topology model. Thread block distribution optimization improves data locality and cache hit rate by changing the mapping relationship between thread blocks and data blocks. The heterogeneous pipeline architecture decouples computing tasks, achieves spatiotemporal overlap between computing and data transmission, and maximizes resource utilization. The present invention can efficiently schedule and optimize large-scale data processing tasks on different heterogeneous computing platforms. In particular, when the heterogeneous computing system has a multi-level cache structure, the completion speed of computing tasks can be significantly improved by further parallelizing data reading and computing. It can be applied to scenarios that require large-scale parallel processing, such as scientific computing and artificial intelligence training. It can flexibly respond to dynamically changing computing loads, optimize data locality, improve cache utilization, and efficiently complete large-scale data processing tasks on various heterogeneous computing platforms.

[0131] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A heterogeneous computing thread block optimization scheduling method based on dynamic topology mapping, characterized in that: The following steps are involved: S1. Build a dynamic topology model driven by tensor dimension features and generate thread block distribution patterns based on matrix parameters and GPU hardware information. S2. Build a multi-dimensional resource scheduling policy library. Based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, dynamically select the optimal thread block distribution policy from the multi-dimensional resource scheduling policy library and generate a binding relationship between the thread block and the data block. S3. Define block parameters and establish dynamic mapping functions, and optimize the collaborative access logic between computing thread blocks and storage layers based on the block parameters and dynamic mapping functions. S4. Distributed computing is performed based on the binding relationship between thread blocks and data blocks. The computing and data transmission are parallelized through pipelining and double buffering mechanisms. The results across computing units are aggregated through atomic operations and barrier synchronization, and the final computing results are written back to the global memory.

2. The method for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping according to claim 1, characterized in that: The step S1 comprises: S11, obtaining input matrix parameters and GPU hardware information, extracting array dimension feature parameters of tensor operations based on the matrix parameters, performing validity check on the matrix dimension feature parameters, and obtaining valid parameters; S12, performing multi-dimensional constraint analysis on the valid parameters, and selecting a thread block distribution pattern from a topology mapping function library according to the dimensional characteristic parameters of the matrix operation and their associated constraints; S13, verifying the feasibility of the selected thread block distribution mode, and generating a dynamic binding relationship between the thread block and the data block when the thread block distribution mode is feasible; S14. Dynamically adjust the block granularity parameters according to the utilization of computing resources and the latency of data access to optimize the computing efficiency, and obtain the final dynamic topology model. The dynamic topology model dynamically generates the optimal thread block distribution pattern according to the matrix parameters and GPU hardware information.

3. The method for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping according to claim 2, characterized in that: The matrix parameters include matrix size, block granularity, and data sparsity; the GPU hardware information includes the maximum shared memory capacity of the GPU, the number of banks, and the maximum number of threads in the thread block.

4. The method for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping according to claim 2, characterized in that: The associated constraints include tensor sparsity, data locality and computational dependency; the distribution modes include linear distribution, two-dimensional grid distribution and three-dimensional cube distribution.

5. The method for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping according to claim 2, characterized in that: The step S2 includes: S21. Build a multi-dimensional resource scheduling policy library, where the multi-dimensional resource scheduling policy library includes at least one of the following thread block distribution policies: The vertical blocking strategy divides the computing blocks according to the row-priority principle and establishes a binding relationship between thread blocks and data rows. Each thread block is responsible for processing multiple consecutive rows of data. The horizontal block strategy generates a serpentine topology based on the column-first principle. Each thread block is responsible for processing multiple consecutive columns of data. Composite blocking strategy: constructing a chessboard-like thread block distribution structure with bidirectional row and column blocking, where each thread block is responsible for processing a sub-matrix; Adaptive hybrid strategy dynamically adjusts the block direction and granularity based on the real-time cache hit rate, and dynamically selects the optimal block direction and granularity according to the cache usage and hit rate. S22. Select an optimal thread block distribution strategy from a multi-dimensional resource scheduling strategy library based on runtime parameters, generate a mapping relationship between thread blocks and data blocks, and output a mapping matrix between thread blocks and data blocks; S23. According to the mapping matrix between the thread block and the data block, the blocks of the matrix A and the matrix B are mapped to the thread block of the GPU and loaded into the shared memory.

6. The method for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping according to claim 5, characterized in that: The step S22 includes: S221. Monitor the operating parameters of the heterogeneous computing system and collect computing load and cache status data in real time; S222, based on the operating parameters and thread block distribution mode of the heterogeneous computing system, select a thread block distribution strategy that best suits the current runtime conditions from a strategy library using a Monte Carlo tree search algorithm; S223 . Generate a binding relationship between thread blocks and data blocks according to the selected thread block distribution strategy, and output a mapping matrix between thread blocks and data blocks.

7. The method for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping according to claim 1, characterized in that: The step S3 comprises: S31. Define block parameters and control the alignment rules of data blocks by setting the size and arrangement direction of data blocks; S32. Establish a dynamic mapping function between thread block index and cache line, and map the thread block index to the cache line through the dynamic mapping function. The dynamic mapping function is expressed as: cache_line=(threadIdx·sc)⊕(blockIdx·sb); Where cache_line represents the index of the cache line, threadIdx represents the index of the thread in the thread block, blockIdx represents the index of the thread block in the global grid, sc is the cache alignment coefficient, sb is the block span parameter, and ⊕ represents the exclusive-or operation.

8. The method for optimizing and scheduling heterogeneous computing thread blocks based on dynamic topology mapping according to claim 1, characterized in that: The step S4 comprises: A heterogeneous pipeline architecture is used to divide computing tasks into four stages: data prefetching, matrix partitioning, core computing, and result aggregation. Each stage is responsible for the specified computing tasks. In the data prefetching phase, while calculating the current block, the data of the next block is prefetched and loaded into the shared memory (buffer). Buffer A and buffer B are introduced. When buffer A is used for calculation, buffer B is used to prefetch the data of the next block; when buffer A is used to prefetch the data of the next block, buffer B is used for calculation. In the matrix block stage, the pre-fetched block data is split into small blocks that fit into cache lines according to the thread block distribution strategy, and the thread block reading order is rearranged; In the core calculation phase, the rearranged matrix blocks are assigned to GPU thread blocks for parallel calculation. Each thread block is responsible for calculating the product of a sub-matrix to obtain the calculation results of each thread block. In the result aggregation stage, the calculation results of each thread block are saved in the shared memory, and the result aggregation across the computing units is completed through atomic operations and barrier synchronization to obtain the final matrix multiplication result, and the final matrix multiplication result is written back to the global memory.

9. A heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping, characterized in that: include: Dynamic modeling module, used to build a dynamic topology model driven by tensor dimension features and generate thread block distribution patterns based on matrix parameters and GPU hardware information; The policy scheduling module is used to build a multi-dimensional resource scheduling policy library. Based on the operating parameters of the heterogeneous computing system and the generated thread block distribution pattern, it dynamically selects the optimal thread block distribution policy from the multi-dimensional resource scheduling policy library and generates the binding relationship between thread blocks and data blocks. The access optimization module is used to define the block parameters and establish the dynamic mapping function, and optimize the collaborative access logic between the computing thread block and the storage layer based on the block parameters and the dynamic mapping function; The distributed computing module performs distributed computing based on the binding relationship between thread blocks and data blocks. It parallelizes computing and data transmission through pipelining and double buffering mechanisms, aggregates results across computing units through atomic operations and barrier synchronization, and writes the final computing results back to the global memory.

10. The heterogeneous computing thread block optimization scheduling system based on dynamic topology mapping according to claim 9, characterized in that: The construction of a dynamic topology model driven by tensor dimension features generates a thread block distribution pattern based on matrix parameters and GPU hardware information, including: Obtain input matrix parameters and GPU hardware information, extract array dimension feature parameters of tensor operations based on matrix parameters, perform validity check on matrix dimension feature parameters, and obtain valid parameters; Perform multi-dimensional constraint analysis on valid parameters, and select thread block distribution patterns from the topology mapping function library based on the dimensional characteristic parameters of matrix operations and their associated constraints; Verify the feasibility of the selected thread block distribution mode. If the thread block distribution mode is feasible, generate a dynamic binding relationship between the thread block and the data block. According to the utilization of computing resources and the latency of data access, the block granularity parameters are dynamically adjusted to achieve the optimal computing efficiency, and the final dynamic topology model is obtained. The dynamic topology model dynamically generates the optimal thread block distribution pattern based on matrix parameters and GPU hardware information.

Images