Method for determining memory access partitioning strategy, computer equipment, readable storage medium and program product

Abstract

The invention relates to a memory access partitioning strategy determination method, computer equipment, a readable storage medium and a program product. The method comprises the following steps: aiming at all memory access operations on a global memory in a kernel function, respectively carrying out memory access continuity analysis to obtain a candidate block size corresponding to each memory access operation; based on the block features of the candidate block sizes corresponding to the memory access operations, determining block scores of the candidate block sizes; and determining a target block size from the candidate block sizes based on the block fraction of each candidate block size, and determining a memory access block strategy for the kernel function based on the target block size. By adopting the method, the development efficiency of compiler optimization can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a memory access block strategy determination method, computer equipment, a readable storage medium and a program product. BACKGROUND

[0002] With the expansion of deep learning model size and the diversification of hardware ecology, the demand for efficient deployment of models across platforms is urgent, and deep learning compilers have emerged. The core is to convert the model defined by the high-level framework into low-level code that adapts to specific hardware, to improve inference speed, reduce latency and resource consumption.

[0003] Memory optimization is the core of compiler performance improvement. In view of the problem that the memory access speed of artificial intelligence chips is much lower than the calculation speed, the block technology (Tiling) has become the mainstream solution, which cuts large tensors into small blocks, stores them in shared memory and other high-speed storage, reduces global memory access, and improves the calculation memory ratio.

[0004] However, the existing block technology needs to manually design memory addresses to ensure that the data logical address is continuous to realize the merging of hardware memory access, and the block size needs to match the hardware characteristics. The realization of these requirements is highly dependent on the developer's understanding of hardware information, and the automation degree is very low, which greatly limits the improvement of development efficiency. SUMMARY

[0005] Therefore, it is necessary to provide a memory block strategy determination method, computer equipment, computer readable storage medium and computer program product capable of improving development efficiency in view of the above technical problems.

[0006] In a first aspect, the present application provides a memory block strategy determination method, comprising:

[0007] For all memory access operations in the kernel function, memory continuity analysis is performed respectively, and the candidate block size corresponding to each memory access operation is obtained;

[0008] Based on the block characteristics of each candidate block size corresponding to the memory access operation, the block score of each candidate block size is determined;

[0009] Based on the block score of each candidate block size, the target block size is determined from each candidate block size, and the memory block strategy for the kernel function is determined based on the target block size.

[0010] In one embodiment, the memory continuity analysis is performed respectively for all memory access operations in the kernel function, and the candidate block size corresponding to each memory access operation is obtained, comprising:

[0011] For any memory access operation in the kernel function, from each dimension of the memory access tensor data to be accessed corresponding to the memory access operation, search for the dimension with a step size of 1 as the target dimension in order from the high dimension to the low dimension.

[0012] Based on the highest dimension among the dimensions and the target dimension, the candidate block size corresponding to the memory access operation is determined.

[0013] In one embodiment, determining the target block size from the candidate block sizes based on the block fractions of each candidate block size includes:

[0014] Based on the block score of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, the target block size of the kernel function is determined from the candidate block sizes.

[0015] In one embodiment, the block characteristics of the candidate block size include the amount of data corresponding to the candidate block size, the operation type of the memory access operation, and the thread bundle size.

[0016] In one embodiment, determining the block score for each candidate block size based on the block characteristics of the candidate block size corresponding to each memory access operation includes:

[0017] For any candidate block size corresponding to any memory access operation, an initial score for the candidate block size is determined based on the amount of data corresponding to the candidate block size.

[0018] The first memory access factor for the candidate block size is determined based on the operation type of the memory access operation.

[0019] Determine whether the candidate block size satisfies the constraint of the thread bundle size, and determine the second memory access factor of the candidate block size based on the determination result;

[0020] The initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size.

[0021] In one embodiment, adjusting the initial score of the candidate block size based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size includes:

[0022] For any candidate block size corresponding to any memory access operation, the initial score of the candidate block size corresponding to the memory access operation is adjusted based on the first memory access factor and the second memory access factor to determine the basic block score of the candidate block size corresponding to the memory access operation.

[0023] From each of the memory access operations, determine the associated memory access operations corresponding to the same candidate block size, and accumulate the basic block scores of the candidate block size corresponding to each of the associated memory access operations to obtain the block score of the candidate block size.

[0024] In one embodiment, determining the target block size of the kernel function from the candidate block sizes based on the block scores of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation includes:

[0025] The candidate block sizes are sorted in descending order of block score to obtain the candidate queue;

[0026] Traverse the candidate queue. For the currently traversed candidate block size, if the candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation satisfy the access constraint conditions, then the candidate block size is used as the target block size of the kernel function.

[0027] Secondly, this application also provides a device for determining a memory access block strategy, comprising:

[0028] The analysis module is used to perform memory access continuity analysis on all memory access operations to global memory in the kernel function, and obtain the candidate block size corresponding to each memory access operation.

[0029] The first determining module is used to determine the block score of each candidate block size based on the block characteristics of the candidate block size corresponding to each memory access operation;

[0030] The second determining module is used to determine a target block size from the candidate block sizes based on the block score of each candidate block size, and to determine a memory access block strategy for the kernel function based on the target block size.

[0031] In one embodiment, the process of performing memory access continuity analysis on all memory access operations in the kernel function to obtain the candidate block size corresponding to each memory access operation includes:

[0032] For any memory access operation in the kernel function, from each dimension of the memory access tensor data to be accessed corresponding to the memory access operation, search for the dimension with a step size of 1 as the target dimension in order from the high dimension to the low dimension.

[0033] Based on the highest dimension among the dimensions and the target dimension, the candidate block size corresponding to the memory access operation is determined.

[0034] In one embodiment, determining the target block size from the candidate block sizes based on the block fractions of each candidate block size includes:

[0035] Based on the block score of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, the target block size of the kernel function is determined from the candidate block sizes.

[0036] In one embodiment, determining the block score for each candidate block size based on the block characteristics of the candidate block size corresponding to each memory access operation includes:

[0037] For any candidate block size corresponding to any memory access operation, an initial score for the candidate block size is determined based on the amount of data corresponding to the candidate block size.

[0038] The first memory access factor for the candidate block size is determined based on the operation type of the memory access operation.

[0039] Determine whether the candidate block size satisfies the constraint of the thread bundle size, and determine the second memory access factor of the candidate block size based on the determination result;

[0040] The initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size.

[0041] In one embodiment, adjusting the initial score of the candidate block size based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size includes:

[0042] For any candidate block size corresponding to any memory access operation, the initial score of the candidate block size corresponding to the memory access operation is adjusted based on the first memory access factor and the second memory access factor to determine the basic block score of the candidate block size corresponding to the memory access operation.

[0043] From each of the memory access operations, determine the associated memory access operations corresponding to the same candidate block size, and accumulate the basic block scores of the candidate block size corresponding to each of the associated memory access operations to obtain the block score of the candidate block size.

[0044] In one embodiment, determining the target block size of the kernel function from the candidate block sizes based on the block scores of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation includes:

[0045] The candidate block sizes are sorted in descending order of block score to obtain the candidate queue;

[0046] Traverse the candidate queue. For the currently traversed candidate block size, if the candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation satisfy the access constraint conditions, then the candidate block size is used as the target block size of the kernel function.

[0047] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the above-mentioned embodiments.

[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0049] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above-mentioned embodiments.

[0050] The above-mentioned method for determining memory access block strategies, computer devices, computer-readable storage media, and computer program products. This method performs memory access continuity analysis on all memory access operations involving global memory in the kernel function, automatically extracts and generates candidate block sizes matching each memory access operation, determines the block score of each candidate block size based on the block characteristics of the candidate block sizes corresponding to each memory access operation, and determines the target block size from the candidate block sizes based on the block scores of each candidate block size. Finally, it determines the memory access block strategy for the kernel function based on the target block size. The memory access block strategy determination method, computer device, computer-readable storage medium, and computer program product provided in this application automatically analyze memory access continuity, automatically generate candidate block sizes, and select the target block size from them. This breaks through the limitation of traditional block strategies relying on manual design, and realizes the automated generation of block strategies that meet the hardware requirements. It ensures that the generated block strategy maximizes the utilization of hardware bandwidth resources while improving the computation-to-memory access ratio. In this way, it not only lowers the technical threshold of compiler optimization, but also greatly improves development efficiency, and can effectively improve the deployment efficiency of artificial intelligence models on diverse hardware platforms. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating a method for determining a memory access block strategy in one embodiment;

[0053] Figure 2 This is a structural block diagram of an artificial intelligence chip in one embodiment;

[0054] Figure 3 This is a flowchart illustrating step 102 in one embodiment;

[0055] Figure 4 This is a flowchart illustrating step 104 in one embodiment;

[0056] Figure 5 This is a flowchart illustrating step 408 in one embodiment;

[0057] Figure 6 This is a flowchart illustrating the steps for determining the target block in one embodiment;

[0058] Figure 7 This is a schematic diagram illustrating a method for determining a memory access block strategy in one embodiment;

[0059] Figure 8 This is a structural block diagram of a device for determining a memory access block strategy in one embodiment;

[0060] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0063] To enable those skilled in the art to better understand the embodiments of this application, the terms involved in the embodiments of this application are explained below.

[0064] Tiling is a key technology used in deep learning compilers and high-performance computing to optimize memory access efficiency, especially suitable for parallel computing hardware such as GPUs (Graphics Processing Units) and other artificial intelligence chips. Its core idea is to break down large computations or data into smaller, parallelizable sub-blocks (called tiles) to optimize memory access and parallel execution efficiency.

[0065] Bank Conflict: In GPU shared memory, an access conflict occurs when multiple threads within the same warp simultaneously access different addresses of the same memory bank, causing parallel access to degenerate into serial execution and significantly reducing memory bandwidth utilization.

[0066] Cache line: The smallest continuous unit of data transmission and management in the cache of an artificial intelligence chip, used to efficiently transfer data in batches between memory levels.

[0067] like Figure 1 As shown, a method for determining a memory access block strategy is provided, which is applied to an artificial intelligence chip. In this embodiment, the artificial intelligence chip is any one of GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose Graphics Processing Unit). This embodiment does not make specific limitations on this one, and the following description uses GPGPU as an example.

[0068] Reference Figure 2 The diagram shows a schematic of a GPGPU. A GPGPU is actually an array of Streaming Processor Clusters (SPCs), including, for example,... Figure 2 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a graphics processing unit (GPU), one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0069] likeFigure 2 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 2 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations other than matrix calculations such as matrix multiplication and convolution, including operations like accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit contains multiple cores (also called computational kernels), each including an Arithmetic Logic Unit (ALU), a floating-point unit, etc., which are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 2 The register file and shared cache in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared cache in a computing unit is used to share data between the cores of that computing unit.

[0070] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 2 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0071] In each computing unit, the thread beam scheduling / distribution module ( Figure 2 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can be executed concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory execution instructions are issued to the shared cache within the computing unit or further issued to intermediate-level caches, global caches, or global memory for read and write operations, etc.

[0072] like Figure 2 As shown, the streaming processor cluster 1 also includes a tensor operation unit, which is used to perform tensor calculations, such as matrix multiplication, convolution operations, etc.

[0073] Reference Figure 1 As shown in the embodiment of this application, a method for determining a memory access block strategy includes the following steps 102 to 106. Wherein:

[0074] Step 102: For all memory access operations to global memory in the kernel function, perform memory access continuity analysis to obtain the candidate block size corresponding to each memory access operation.

[0075] In this embodiment, the kernel function refers to a user-defined function executed on AI chips such as GPUs, and is the core execution unit for parallel computing. Its execution frequently involves memory access operations, i.e., read and write operations, to global memory. The efficiency of these memory access operations directly affects the overall performance of the kernel function; therefore, it is necessary to perform targeted analysis for each memory access operation.

[0076] The core of memory access continuity analysis is to determine the continuous region of memory access by parsing the dimensional information of tensor data. Dimensional information can include the various dimensions of the tensor data and the stride of each dimension. For example, we can examine the tensor data sequentially from the highest to the lowest dimension to find the first dimension with continuous memory access, and then construct the candidate block size corresponding to the memory access operation based on this dimension. For instance, the region consisting of this dimension and all higher dimensions can be considered as the continuous memory access region, and the size (or shape) of this continuous memory access region is the candidate block size corresponding to the memory access operation.

[0077] Assume the operands in the memory access operation (hereinafter referred to as the memory tensor data) have a shape of [M, N, K] and a stride of [N*K, 1, K]. That is, the memory tensor data includes dimension 0 (corresponding to M), dimension 1 (corresponding to N), and dimension 2 (corresponding to K). Then, the first dimension with a stride of 1 is dimension 1, and the continuous memory access region is the region constructed by dimension 0 and dimension 1. The candidate block size corresponding to this memory access operation is M×N.

[0078] By following this logic, the candidate block size corresponding to all memory access operations can be obtained.

[0079] Step 104: Determine the block score of each candidate block size based on the block characteristics of the candidate block size corresponding to each memory access operation.

[0080] In this embodiment of the application, block features refer to attribute features that can reflect the hardware characteristics that the candidate block size adapts to, and may include the amount of data corresponding to the candidate block size, memory access operation type, thread bundle size, etc.

[0081] The candidate block size corresponds to the total number of elements contained in the candidate block, such as 32, 64, 48, etc. The memory access operation type refers to the kernel function's read or write operation on the global cache; that is, the memory access operation type can include both read and write operation types. The thread bundle size is the number of threads in the basic parallel execution unit of an AI chip architecture, referring to the total number of threads contained in a thread bundle (Warp).

[0082] For example, each block feature can correspond to a different score or score calculation method. Based on the above score or score calculation method, the block score of the candidate block size corresponding to each memory access operation can be determined. The block score is used to characterize the degree of fit between the candidate block size and the kernel function and the hardware. The higher the block score, the higher the degree of fit between the candidate block size and the kernel function and the hardware.

[0083] Step 106: Based on the block score of each candidate block size, determine the target block size from each candidate block size, and determine the memory access block strategy for the kernel function based on the target block size.

[0084] In this embodiment, the target block size is the core parameter of the kernel function memory access block strategy. A memory access block strategy for the kernel function can be constructed based on the determined target block size. This memory access block strategy can be used to guide the compiler to generate specific block code, such as cutting a large tensor into sub-blocks of the target block size, ensuring that the sub-blocks can efficiently utilize the hardware's shared memory, registers and other high-speed storage, realize memory access merging and data reuse, thereby significantly improving the execution efficiency of the kernel function.

[0085] After determining the candidate block sizes for all memory access operations in the kernel function, the optimal block size can be selected as the target block size. For example, the target block size can be determined based on the block scores of the candidate block sizes. This can be achieved by analyzing the fit between each candidate block size and each memory access operation in the kernel function, in descending order of block scores, and then selecting the first candidate block size that best fits all memory access operations as the target block size.

[0086] The aforementioned method for determining memory access block strategies analyzes the memory access continuity of all memory access operations involving global memory in the kernel function, automatically extracts and generates candidate block sizes matching each memory access operation, determines the block score of each candidate block size based on the block characteristics of the candidate block sizes corresponding to each memory access operation, and determines the target block size from the candidate block sizes based on the block scores of each candidate block size. Finally, the memory access block strategy for the kernel function is determined based on the target block size. The method for determining memory access block strategies provided in this application automatically analyzes memory access continuity, automatically generates candidate block sizes, and selects the target block size from them. This overcomes the limitations of traditional block strategies that rely on manual design, achieving automated generation of block strategies that meet hardware requirements. It ensures that the generated block strategy maximizes the utilization of hardware bandwidth resources while improving the computational memory access ratio. This not only lowers the technical threshold for compiler optimization but also significantly improves development efficiency, effectively enhancing the deployment efficiency of artificial intelligence models on diverse hardware platforms.

[0087] In one exemplary embodiment, such as Figure 3 As shown, in step 102, for all memory access operations to global memory in the kernel function, a memory access continuity analysis is performed to obtain the candidate block size corresponding to each memory access operation. This may include steps 302 to 304. Wherein:

[0088] Step 302: For any memory access operation in the kernel function, search for the dimension with a step size of 1 from each dimension of the memory access tensor data to be accessed, in order from high dimension to low dimension, as the target dimension.

[0089] Step 304: Determine the candidate block size corresponding to the memory access operation based on the highest dimension and the target dimension among all dimensions.

[0090] In this embodiment, the tensor data to be accessed refers to tensor data, such as matrices and feature maps, that need to be read or written during memory access operations. The tensor data to be accessed has a defined dimensional structure, such as an n-dimensional tensor with shape [A0, A1, A2, ..., An-1], where A0 is the highest dimension and An-1 is the lowest dimension. Each dimension corresponds to a step size, representing the memory address interval between adjacent elements in that dimension. The core criterion for memory access continuity analysis is whether the step size is 1. When the step size of a certain dimension is 1, the memory addresses of adjacent elements in that dimension are continuous. Therefore, the dimensions of the tensor data to be accessed can be traversed from high to low dimensions to find the first dimension with a step size of 1 and marked as the target dimension.

[0091] For example, consider a three-dimensional tensor with shape [M, N, K] and stride [stride0, stride1, stride2]. Dimension 0 is the highest dimension, and dimension 2 is the lowest dimension. If stride0 is not 1 and stride1 is 1, then the first dimension with a stride of 1 is dimension 1, i.e., the target dimension is 1. If neither stride0 nor stride1 is 1 and stride2 is 1, then the first dimension with a stride of 1 is dimension 2, i.e., the target dimension is 2. If stride0 is 1, then dimension 0 is directly used as the target dimension, without needing to check lower dimensions.

[0092] The candidate block size is the size of the region constructed from all dimensions from the highest dimension to the target dimension. In other words, elements within this region constitute a contiguous access area in memory. Since the target dimension is the first dimension with a step size of 1, its combination with higher dimensions can form contiguous memory access blocks. Multiplying the size of each dimension from the highest to the target dimension gives the candidate block size.

[0093] For example, consider a 3D tensor with shape [M, N, K] and stride [stride0, stride1, stride2]. If the target dimension is 1, meaning dimension 1 is the first dimension with a stride of 1, then the highest dimension 0 to the target dimension 1 includes both dimensions 0 and 1, and the candidate block size is M×N. If the target dimension is 2, meaning dimension 2 is the first dimension with a stride of 1, then the highest dimension 0 to the target dimension 2 includes all dimensions, and the candidate block size is M×N×K. If the target dimension is 0, meaning dimension 0 is the first dimension with a stride of 1, then only dimension 0 is included, and the candidate block size is M.

[0094] The candidate block size determined in this way can accurately reflect the range of continuous memory access that the memory access operation can achieve. Based on the candidate block size corresponding to each memory access operation, the target block size in the memory access block strategy can be automatically analyzed, which can greatly improve the efficiency of memory access optimization.

[0095] In one exemplary embodiment, such as Figure 4 As shown, in step 104, determining the block score for each candidate block size based on the block characteristics of the candidate block sizes corresponding to each memory access operation may include steps 402 to 408. Wherein:

[0096] Step 402: For any memory access operation, determine the initial score of the candidate block size based on the amount of data corresponding to the candidate block size.

[0097] Step 404: Determine the first memory access factor for the candidate block size based on the operation type of the memory access operation;

[0098] Step 406: Determine whether the candidate block size satisfies the constraint of the thread bundle size, and determine the second memory access factor of the candidate block size based on the determination result;

[0099] Step 408: Adjust the initial score of the candidate block size based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size.

[0100] In this embodiment, the data volume corresponding to the candidate block size, i.e., the total number of elements contained in the block, can be used to measure the basic memory access efficiency of the block. The larger the data volume, the more it can amortize the fixed overhead of a single memory access, such as address decoding and bus scheduling, thus resulting in higher basic memory access efficiency. Therefore, the initial score can be determined based on the data volume. This embodiment does not specifically limit the method of determining the initial score. For example, the initial score can be the data volume of the candidate block size, or it can be mapped to a standard quantized initial score through a preset data volume-score mapping algorithm. For example, taking a candidate block size containing 64 elements as an example, its initial score is 64. And so on, the initial score of the candidate block size corresponding to each memory access operation can be determined.

[0101] After determining the initial score, a first memory access factor and a second memory access factor corresponding to the candidate block size can be further determined. The initial score of the candidate block size is then adjusted based on the first and second memory access factors to determine the block score of the candidate block size. The method for adjusting the initial score of the candidate block size based on the first and second memory access factors can include any of the following three:

[0102] Method 1 involves determining an intermediate score based on the initial score and the first memory access factor after determining the first memory access factor, and then determining a second memory access factor. The block score for the candidate block size is then determined based on the intermediate score and the second memory access factor. Method 2 involves determining an intermediate score based on the initial score and the second memory access factor after determining the second memory access factor, and then determining a first memory access factor. The block score for the candidate block size is then determined based on the intermediate score and the first memory access factor. Method 3 involves determining the first memory access factor and the second memory access factor, and then calculating the block score for the candidate block size based on the initial score, the first memory access factor, and the second memory access factor.

[0103] In other words, the order in which the first memory access factor and the second memory access factor are obtained, and the method of calculating the initial score based on the first memory access factor and the second memory access factor, are not specifically limited in the embodiments of this application. The embodiments of this application will be described in accordance with Method 1 in the following embodiments.

[0104] In this embodiment, memory access operations mainly include two types: read and write. A read operation refers to reading data from global memory to high-speed storage, while a write operation refers to writing data from high-speed storage to global memory. Because hardware optimization mechanisms for read and write operations differ—for example, write operations typically require stricter address contiguity to avoid data conflicts—different types of operations exhibit varying sensitivities to block size gains. Therefore, a first memory access factor is needed to quantify these differences. The first memory access factor is an adjustment factor set for the initial score to quantify the differences between different memory access operation types.

[0105] For example, assuming the first memory access factor corresponding to a read operation is a first value and the first memory access factor corresponding to a write operation is a second value, then when the read operation is more profitable than the write operation, the first value will be greater than the second value, or when the write operation is more profitable than the read operation, the second value will be greater than the first value. This application does not specifically limit the values ​​of the first and second values; those skilled in the art can set them based on their needs.

[0106] In one example, let's assume the first value is 1 and the second value is 2. When the memory access operation type is a read operation, the median score for the candidate block size can be determined as the initial score × 1, meaning the current score is maintained. For example, with an initial score of 32, the median score for the read operation would be 32 × 1 = 32. Alternatively, when the memory access operation type is a write operation, the median score for the candidate block size can be determined as the initial score × 2, meaning the median score for the write operation would be 32 × 2 = 64.

[0107] It should be noted that the above method of multiplying the initial score by the first memory access factor is only an example in the embodiments of this application. In fact, the score can be calculated by adding or subtracting the first memory access factor based on the initial score. This embodiment of the application does not make specific limitations on this.

[0108] In this embodiment, the thread bundle size refers to the need for the block size to adapt to the thread bundle parallel granularity of the AI ​​chip. For example, the thread bundle size can be 32 threads, indicating that a thread bundle contains 32 threads. The constraints on the thread bundle size include that the block size is greater than or equal to the thread bundle size and is divisible by the thread bundle size. The second memory access factor of the candidate block size can be determined based on whether the candidate block size satisfies this constraint.

[0109] For example, assuming the candidate block size satisfies the thread bundle size constraint, the corresponding second memory access factor is the third value; if the candidate block size does not satisfy the thread bundle size constraint, the corresponding second memory access factor is the fourth value, and the third value is greater than the fourth value. In this embodiment, the values ​​of the third and fourth values ​​are not specifically limited; those skilled in the art can set them based on their needs.

[0110] For example, taking a thread bundle size of 32 as an example, if the candidate block size is 8×8 (containing 64 elements, which is greater than 32 and divisible by 32), then the constraint of being greater than or equal to the thread bundle size and divisible by the thread bundle size is satisfied, and the second memory access factor can be set to the third value. If the candidate block size is 8×6 (containing 48 elements, which is greater than 32 but not divisible by 32), then the constraint of being greater than or equal to the thread bundle size and divisible by the thread bundle size is not satisfied, and the second memory access factor can be set to the fourth value.

[0111] In one example, let's take a third value of 2 and a fourth value of 1. If the thread bundle size constraint is met, the block size of the candidate block can be determined as the median score (calculated based on the first memory access factor and the initial score) × 2. For example, with a median score of 32, the block score of the candidate block size is 32 × 2 = 64. Alternatively, if the thread bundle size constraint is not met, the block size of the candidate block can be determined as the median score × 1, meaning the block score of the candidate block size is 32 × 1 = 32.

[0112] It should be noted that the above method of multiplying the intermediate score with the second memory access factor is only an example in the embodiments of this application. In fact, the score can be calculated by adding or subtracting the second memory access factor based on the intermediate score. This embodiment of the application does not make specific limitations on this.

[0113] In this embodiment of the application, the block score of the candidate block size obtained by the calculation can intuitively reflect the comprehensive optimization capability of the candidate block size. In turn, the optimal target block size can be determined from each candidate block size based on the block score of the candidate block size, which can greatly improve the memory access optimization efficiency.

[0114] In one exemplary embodiment, such as Figure 5 As shown, in step 408, the initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size. This may include the following steps 502 to 504, wherein:

[0115] Step 502: For any candidate block size corresponding to a memory access operation, adjust the initial score of the candidate block size corresponding to the memory access operation based on the first memory access factor and the second memory access factor to determine the basic block score of the candidate block size corresponding to the memory access operation.

[0116] Step 504: Determine the associated memory access operations corresponding to the same candidate block size from each memory access operation, and sum the basic block scores of the candidate block size corresponding to each associated memory access operation to obtain the block score of the candidate block size.

[0117] In this embodiment, since different memory access operations may correspond to the same candidate block size, when determining the target candidate block size, it is necessary to accumulate the block scores calculated for the same candidate block size under different memory access operations to obtain the final block score for the candidate block size. For clarity, the block scores calculated for the candidate block size under each memory access operation are referred to as the basic block scores. The calculation process can be referred to the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0118] For example, in the process of determining the block size of candidate blocks, memory access operations corresponding to the same candidate block size can be regarded as associated memory access operations. That is, associated memory access operations refer to multiple different memory access operations in the kernel function. If their corresponding candidate block sizes are the same, such as memory access operation A and memory access operation B both having a candidate block size of 64, then memory access operation A and memory access operation B are associated memory access operations.

[0119] Determining the block score for a candidate block size requires considering its overall adaptability to all associated memory access operations. If a candidate block size can accommodate multiple memory access operations simultaneously, the complexity of the block partitioning strategy can be reduced. Therefore, the base block scores of the candidate block sizes corresponding to associated memory access operations need to be accumulated to obtain the final block score for that candidate block size.

[0120] For example, if the candidate block size for memory access operations A and B is 64, and the base block scores are 128 and 256 respectively, then the block score corresponding to this candidate block size is 128 + 256 = 384. Alternatively, if the candidate block size for memory access operation C is 48, the base block score is 512, and there are no other associated memory access operations, then the block score corresponding to this candidate block size is 512.

[0121] By accumulating the base block scores of the associated operations, the final block score of the candidate block size can reflect the overall adaptability of the candidate block size to all memory access operations in the kernel function. The higher the score, the better the adaptability of the candidate block size to more operations, and the more suitable it is as the globally optimal target block size.

[0122] In an exemplary embodiment, step 104, determining the target block size from the candidate block sizes based on the block scores of each candidate block size, and determining the memory access block strategy for the kernel function based on the target block size, may include:

[0123] Based on the block score of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, the target block size of the kernel function is determined from the candidate block sizes.

[0124] In this embodiment, the size information of the tensor data to be accessed refers to the size of each dimension of the tensor. For example, for a tensor with shape [M, N, K], its size information is M, N, and K. Finally, based on the block scores of each candidate block size and the size information of the tensor data to be accessed corresponding to each memory access operation, the target block size of the kernel function can be determined from the candidate block sizes.

[0125] In one exemplary embodiment, such as Figure 6 As shown, determining the target block size of the kernel function from the candidate block sizes, based on the block fractions of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, may include the following steps 602 to 604. Wherein:

[0126] Step 602: Sort the candidate block sizes in descending order of block scores to obtain the candidate queue;

[0127] Step 604: Traverse the candidate queue. For the candidate block size currently being traversed, if the candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation both satisfy the access constraint conditions, then the candidate block size is used as the target block size of the kernel function.

[0128] In this embodiment, the block score is a quantitative evaluation of the overall adaptability of candidate block sizes. The higher the score, the greater the overall optimization value of the candidate block size for memory access operations in the kernel function. A candidate queue can be obtained by sorting the candidates in descending order of their block scores. This method places the candidate size with the best theoretical adaptability at the front of the queue for priority checking, avoiding invalid checks on low-scoring, poorly adaptable sizes, and thus shortening the time required to determine the target block size.

[0129] For example, the block scores of candidate block size (1), candidate block size (2) and candidate block size (3) are 256, 96 and 48 respectively, and the sizes are 8×8, 4×12 and 4×6 respectively. Then the candidate queue after descending order is [(1), (2), (3)], that is, the candidate block size (1) is traversed first, and then the candidate block size (2) and candidate block size (3) are traversed in turn.

[0130] During the traversal of the candidate queue, for the currently traversed candidate block size, if the candidate block size and the size information of the memory access tensor data corresponding to each memory access operation satisfy the access constraints, then the candidate block size is used as the target block size of the kernel function, and the traversal ends. The access constraints include that the candidate block size is compatible with the relevant dimension dimensions of all memory access tensor data; that is, for each memory access tensor data, the size of each dimension from its high dimension to the target dimension (the dimension range constituting the candidate block size) is divisible by the current candidate block size. In this way, the target block size ensures that no residual elements appear during the block division process, i.e., there are no scattered small sub-blocks after block division, thus simplifying the boundary handling logic and avoiding efficiency losses caused by extra processing of scattered sub-blocks.

[0131] For example, suppose that the memory access tensor data corresponding to each memory access operation in the kernel function includes a first tensor data and a second tensor data, where the shape of the first tensor data is [128, 64, 1] and the shape of the second tensor data is [96, 32, 2]. Suppose that the candidate block sizes are arranged in descending order of block score as [32, 32, 1], [16, 16, 2], and [8, 8, 1].

[0132] First, iterate through the first candidate block size [32, 32, 1]. For the first tensor data [128, 64, 1], 128 / 32 = 4 (integer, dimension 0 fits); 64 / 32 = 2 (integer, dimension 1 fits). Dimension 2 with a step size of 1 is ignored, meaning the first tensor data fits. For the second tensor data [96, 32, 2], 96 / 32 = 3 (integer, dimension 0 fits); 32 / 32 = 1 (integer, dimension 1 fits). Dimension 2 with a step size of 1 is ignored, meaning the second tensor data fits. It can be determined that the first candidate block size fits all tensor data, therefore, the iteration can stop, and the first candidate block size is determined as the target block size (shape).

[0133] By using this high-score-first, fully compatible filtering method, the final target block size can maximize the memory access optimization effect and ensure the adaptability to all memory tensors to be accessed. It can automatically generate a practical and optimal memory access block strategy for the kernel function.

[0134] To enable those skilled in the art to better understand the embodiments of this application, the embodiments of this application are described below through specific examples.

[0135] In one example, refer to Figure 7 As shown, determining the memory access block strategy mainly includes the following process:

[0136] Collect memory access operations within the kernel (kernel function), including operations that read or write data from global memory. By parsing the kernel function's code logic (including variable passing, memory address calculation, branching statements, etc.), it can automatically identify and record the dependencies between all memory access operations, forming a deps set, and then iterate through each memory access operation in the deps set.

[0137] For each memory access operation during the traversal, analyze the continuity of memory access. Specifically, starting from the highest dimension, find the first dimension with a step size of 1, denoted as `split`. Then, using this dimension as the boundary, memory accesses up to the highest dimension are continuous, and the number of memory accesses is used as the block size for this memory access operation. Memory accesses up to the lowest dimension are not continuous. For example, if data is loaded from a tensor of size [M, N, K] with a step size of [N×K, 1, K] and dimensions [0, 1, 2], the first dimension with a step size of 1 is 1, so `split` is 1. Therefore, memory accesses in dimensions 0 and 1 are continuous, and the candidate block size for this memory access operation is M×N.

[0138] Each candidate block size is scored. The initial score is based on the amount of data accessed within the candidate block size. This means that the more consecutive memory accesses, the better, as consecutive memory accesses will be merged, reducing the number of memory access transactions. Based on the benefit of merging reads and writes on the target hardware, the score for merged memory access patterns with higher benefits is doubled; otherwise, the current score remains unchanged. For example, if a write operation is more beneficial, the initial score is doubled if the memory access operation is a write operation, and otherwise, it remains unchanged if the memory access operation is a read operation.

[0139] Furthermore, it is determined whether the data volume of the candidate block size is not less than 32 and is divisible by 32. If the data volume of the candidate block size is not less than 32 and is divisible by 32, the score will be doubled; otherwise, the current score remains unchanged. Finally, the scores of candidates with the same block size are summed together to obtain the final score for that candidate block size.

[0140] Sort candidate block sizes in descending order of score to obtain a candidate list. Iterate through the candidate list, filtering for the valid candidate block size with the highest score. This includes finding the first candidate block size that divides every dimension of the tensor to be accessed (excluding dimensions with a size of 1), i.e., the valid candidate block size with the highest score, and using it as the final target block size for the current kernel. In other words, if a candidate block size is valid, it can be used as the target block size, and the current block strategy can be returned, using the target block size as the basis for the current block strategy. If a candidate block size is invalid, continue iterating until all candidate block sizes in the candidate list have been traversed, then end the iteration, determining that the current kernel function will not perform block processing.

[0141] The memory access block strategy determination method, computer device, readable storage medium, and program product provided in this application embodiment systematically analyze all memory access operations in the kernel function that access global memory, perform fine-grained memory access continuity detection for each memory access operation, and then automatically extract and generate candidate block sizes that are suitable for each operation. Based on this, the target block size is selected from the candidate block sizes, and finally the memory access block strategy for the kernel function is formulated based on the target block size.

[0142] Compared to traditional block partitioning techniques that require manual intervention, this approach overcomes the limitations of traditional block partitioning strategies that rely on manual design by leveraging automated analysis of memory access continuity, automatic generation of candidate block sizes, and intelligent selection of target block sizes. It achieves automated generation of block partitioning strategies adapted to hardware requirements. This not only significantly lowers the technical barrier to deep learning compiler optimization and drastically shortens the development cycle of block partitioning strategies, but also effectively improves the deployment efficiency and runtime performance of deep learning models on diverse hardware platforms.

[0143] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0144] Based on the same inventive concept, this application also provides a memory access block strategy determination apparatus for implementing the memory access block strategy determination method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the memory access block strategy determination apparatus provided below can be found in the limitations of the memory access block strategy determination method described above, and will not be repeated here.

[0145] In one exemplary embodiment, such as Figure 8 As shown, a memory access block strategy determination device 800 is provided, including: an analysis module 802, a first determination module 804, and a second determination module 806, wherein:

[0146] Analysis module 802 is used to perform memory access continuity analysis on all memory access operations to global memory in the kernel function, and obtain the candidate block size corresponding to each memory access operation.

[0147] The first determining module 804 is used to determine the block score of each candidate block size based on the block characteristics of the candidate block size corresponding to each memory access operation.

[0148] The second determining module 806 is used to determine a target block size from the candidate block sizes based on the block fraction of each candidate block size, and to determine a memory access block strategy for the kernel function based on the target block size.

[0149] The aforementioned memory access block strategy determination device automatically extracts and generates candidate block sizes matching each memory access operation by performing memory access continuity analysis on all memory access operations involving global memory in the kernel function. Based on the block characteristics of the candidate block sizes corresponding to each memory access operation, it determines the block score of each candidate block size. Then, based on the block scores of each candidate block size, it determines the target block size from the candidate block sizes. Finally, it determines the memory access block strategy for the kernel function based on the target block size. The memory access block strategy determination device provided in this application, by automatically analyzing memory access continuity, automatically generating candidate block sizes, and selecting the target block size from them, overcomes the limitations of traditional block strategies that rely on manual design. It achieves automated generation of block strategies that meet hardware requirements, ensuring that the generated block strategy maximizes the utilization of hardware bandwidth resources while improving the computational memory access ratio. This not only lowers the technical threshold for compiler optimization but also significantly improves development efficiency, effectively enhancing the deployment efficiency of artificial intelligence models on diverse hardware platforms.

[0150] In an exemplary embodiment, the step of performing memory access continuity analysis on all memory access operations in the kernel function to obtain the candidate block size corresponding to each memory access operation includes:

[0151] For any memory access operation in the kernel function, from each dimension of the memory access tensor data to be accessed corresponding to the memory access operation, search for the dimension with a step size of 1 as the target dimension in order from the high dimension to the low dimension.

[0152] Based on the highest dimension among the dimensions and the target dimension, the candidate block size corresponding to the memory access operation is determined.

[0153] In an exemplary embodiment, determining the target block size from the candidate block sizes based on the block fractions of each candidate block size includes:

[0154] Based on the block score of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, the target block size of the kernel function is determined from the candidate block sizes.

[0155] In an exemplary embodiment, determining the block score for each candidate block size based on the block characteristics of the candidate block size corresponding to each memory access operation includes:

[0156] For any candidate block size corresponding to any memory access operation, an initial score for the candidate block size is determined based on the amount of data corresponding to the candidate block size.

[0157] The first memory access factor for the candidate block size is determined based on the operation type of the memory access operation.

[0158] Determine whether the candidate block size satisfies the constraint of the thread bundle size, and determine the second memory access factor of the candidate block size based on the determination result;

[0159] The initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size.

[0160] In an exemplary embodiment, adjusting the initial score of the candidate block size based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size includes:

[0161] For any candidate block size corresponding to any memory access operation, the initial score of the candidate block size corresponding to the memory access operation is adjusted based on the first memory access factor and the second memory access factor to determine the basic block score of the candidate block size corresponding to the memory access operation.

[0162] From each of the memory access operations, determine the associated memory access operations corresponding to the same candidate block size, and accumulate the basic block scores of the candidate block size corresponding to each of the associated memory access operations to obtain the block score of the candidate block size.

[0163] In an exemplary embodiment, determining the target block size of the kernel function from the candidate block sizes based on the block scores of each candidate block size and the size information of the memory access tensor data corresponding to each memory access operation includes:

[0164] The candidate block sizes are sorted in descending order of block score to obtain the candidate queue;

[0165] Traverse the candidate queue. For the currently traversed candidate block size, if the candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation satisfy the access constraint conditions, then the candidate block size is used as the target block size of the kernel function.

[0166] Each module in the aforementioned memory access block strategy determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0167] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it determines a method for memory access block strategy. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0168] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0169] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0170] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0171] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining a memory access block strategy, characterized in that, The method includes: For all memory access operations to global memory in the kernel function, memory access continuity analysis is performed to obtain the candidate block size corresponding to each memory access operation; Based on the block characteristics of the candidate block sizes corresponding to each memory access operation, the block score of each candidate block size is determined; Based on the block score of each candidate block size, a target block size is determined from each candidate block size, and a memory access block strategy for the kernel function is determined based on the target block size.

2. The method according to claim 1, characterized in that, For each memory access operation in the kernel function, a memory access continuity analysis is performed to obtain the candidate block size corresponding to each memory access operation, including: For any memory access operation in the kernel function, from each dimension of the memory access tensor data to be accessed corresponding to the memory access operation, search for the dimension with a step size of 1 as the target dimension in order from the high dimension to the low dimension. Based on the highest dimension among the dimensions and the target dimension, the candidate block size corresponding to the memory access operation is determined.

3. The method according to claim 1 or 2, characterized in that, The step of determining the target block size from the candidate block sizes based on the block scores of each candidate block size includes: Based on the block score of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, the target block size of the kernel function is determined from the candidate block sizes.

4. The method according to claim 3, characterized in that, The block characteristics of the candidate block size include the amount of data corresponding to the candidate block size, the operation type of the memory access operation, and the thread bundle size.

5. The method according to claim 4, characterized in that, The step of determining the block score for each candidate block size based on the block characteristics corresponding to each memory access operation includes: For any candidate block size corresponding to any memory access operation, an initial score for the candidate block size is determined based on the amount of data corresponding to the candidate block size. The first memory access factor for the candidate block size is determined based on the operation type of the memory access operation. Determine whether the candidate block size satisfies the constraint of the thread bundle size, and determine the second memory access factor of the candidate block size based on the determination result; The initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size.

6. The method according to claim 5, characterized in that, The step of adjusting the initial score of the candidate block size based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size includes: For any candidate block size corresponding to any memory access operation, the initial score of the candidate block size corresponding to the memory access operation is adjusted based on the first memory access factor and the second memory access factor to determine the basic block score of the candidate block size corresponding to the memory access operation. From each of the memory access operations, determine the associated memory access operations corresponding to the same candidate block size, and accumulate the basic block scores of the candidate block size corresponding to each of the associated memory access operations to obtain the block score of the candidate block size.

7. The method according to claim 3, characterized in that, The determination of the target block size of the kernel function from the candidate block sizes, based on the block scores of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, includes: The candidate block sizes are sorted in descending order of block score to obtain the candidate queue; Traverse the candidate queue. For the currently traversed candidate block size, if the candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation satisfy the access constraint conditions, then the candidate block size is used as the target block size of the kernel function.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images