Method for determining access block strategy, computer device, readable storage medium and program product

Abstract

The application relates to a memory access block strategy determination method, computer equipment, a readable storage medium and a program product. The method comprises the following steps: for all memory access operations in a kernel function, respectively performing memory access continuity analysis to obtain a candidate block size corresponding to each memory access operation; determining a block score of each candidate block size based on a block feature of the candidate block size corresponding to each memory access operation; determining a target block size from the candidate block sizes based on the block scores of the candidate block sizes, and determining a memory access block strategy for the kernel function based on the target block size. The method can improve the development efficiency of compiler optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a memory access block strategy determination method, computer equipment, a readable storage medium and a program product. BACKGROUND

[0002] With the expansion of deep learning model size and the diversification of hardware ecology, the demand for efficient deployment of models across platforms is urgent, and deep learning compilers have emerged. The core is to convert the model defined by the high-level framework into low-level code that adapts to specific hardware, to improve inference speed, reduce latency and resource consumption.

[0003] Memory optimization is the core of compiler performance improvement. In view of the problem that the memory access speed of artificial intelligence chips is much lower than the calculation speed, the block technology (Tiling) has become the mainstream solution, which cuts large tensors into small blocks, stores them in shared memory and other high-speed storage, reduces global memory access, and improves the calculation memory ratio.

[0004] However, the existing block technology needs to manually design memory addresses to ensure that the data logical address is continuous to realize the merging of hardware memory access, and the block size needs to match the hardware characteristics. The realization of these requirements is highly dependent on the developer's understanding of hardware information, and the automation degree is very low, which greatly limits the improvement of development efficiency. SUMMARY

[0005] Therefore, it is necessary to provide a memory block strategy determination method, computer equipment, computer readable storage medium and computer program product capable of improving development efficiency in view of the above technical problems.

[0006] In a first aspect, the present application provides a memory block strategy determination method, comprising:

[0007] For all memory access operations in the kernel function, memory continuity analysis is performed respectively, and the candidate block size corresponding to each memory access operation is obtained;

[0008] Based on the block characteristics of each candidate block size corresponding to the memory access operation, the block score of each candidate block size is determined;

[0009] Based on the block score of each candidate block size, the target block size is determined from each candidate block size, and the memory block strategy for the kernel function is determined based on the target block size.

[0010] In one embodiment, the memory continuity analysis is performed respectively for all memory access operations in the kernel function, and the candidate block size corresponding to each memory access operation is obtained, comprising:

[0011] For any memory access operation in the kernel function, a dimension with a step size of 1 is found as a target dimension in an order from a high dimension to a low dimension among dimensions of tensor data to be accessed corresponding to the memory access operation;

[0012] Based on the highest dimension among the dimensions and the target dimension, a candidate block size corresponding to the memory access operation is determined.

[0013] In one of the embodiments, the determining of the target block size from the candidate block sizes based on the block scores of the candidate block sizes comprises:

[0014] The target block size of the kernel function is determined from the candidate block sizes based on the block scores of the candidate block sizes and size information of tensor data to be accessed corresponding to the memory access operations.

[0015] In one of the embodiments, the block features of the candidate block size comprise a data amount corresponding to the candidate block size, an operation type of the memory access operation, and a thread bundle size.

[0016] In one of the embodiments, the determining of the block scores of the candidate block sizes based on the block features of the candidate block sizes corresponding to the memory access operations comprises:

[0017] For the candidate block size corresponding to any memory access operation, an initial score of the candidate block size is determined based on a data amount corresponding to the candidate block size.

[0018] A first memory access factor of the candidate block size is determined based on an operation type of the memory access operation.

[0019] It is judged whether the candidate block size satisfies a constraint of the thread bundle size, and a second memory access factor of the candidate block size is determined based on a judgment result.

[0020] The initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor, and a block score of the candidate block size is determined.

[0021] In one of the embodiments, the adjusting of the initial score of the candidate block size based on the first memory access factor and the second memory access factor, and the determining of the block score of the candidate block size comprises:

[0022] For the candidate block size corresponding to any memory access operation, an initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor, and a basic block score of the candidate block size corresponding to the memory access operation is determined.

[0023] determine associated memory access operations corresponding to a same candidate tile size from the memory access operations, and accumulate tile scores of the candidate tile size corresponding to tile scores of the associated memory access operations to obtain a tile score of the candidate tile size.

[0024] In one of the embodiments, the determining the target tile size from the candidate tile sizes based on the tile scores of the candidate tile sizes and size information of the tensor data corresponding to the memory access operations comprises:

[0025] sorting the candidate tile sizes in a descending order of the tile scores to obtain a candidate queue;

[0026] traversing the candidate queue, and if the candidate tile size and size information of the tensor data corresponding to the memory access operations both satisfy the access constraint condition, taking the candidate tile size as the target tile size of the kernel function.

[0027] In a second aspect, the present application further provides a device for determining a memory access tiling strategy, comprising:

[0028] the analysis module is configured to perform memory continuity analysis on all memory access operations in the kernel function respectively to obtain candidate tile sizes corresponding to each of the memory access operations.

[0029] the first determination module is configured to determine tile scores of the candidate tile sizes based on tile features of the candidate tile sizes corresponding to the memory access operations;

[0030] the second determination module is configured to determine a target tile size from the candidate tile sizes based on the tile scores of the candidate tile sizes, and determine the memory access tiling strategy for the kernel function based on the target tile size.

[0031] In one of the embodiments, the performing memory continuity analysis on all memory access operations in the kernel function respectively to obtain candidate tile sizes corresponding to each of the memory access operations comprises:

[0032] for any memory access operation in the kernel function, searching for a dimension with a step of 1 as a target dimension in order from a high dimension to a low dimension in each dimension of the tensor data corresponding to the memory access operation;

[0033] determining the candidate tile size corresponding to the memory access operation based on a highest dimension in the dimensions and the target dimension.

[0034] In one of the embodiments, the determining the target tile size from the candidate tile sizes based on the tile scores of the candidate tile sizes comprises:

[0035] determine the target tile size of the kernel function from the candidate tile sizes based on the tile scores of the candidate tile sizes and the size information of the tensor data to be accessed corresponding to each of the memory access operations.

[0036] In one of the embodiments, the determining the tile score of each of the candidate tile sizes based on the tile feature of the candidate tile size corresponding to each of the memory access operations comprises:

[0037] For any of the candidate tile sizes corresponding to each of the memory access operations, determine an initial score of the candidate tile size based on the data amount corresponding to the candidate tile size;

[0038] determine a first memory access factor of the candidate tile size based on the operation type of the memory access operation;

[0039] determine a second memory access factor of the candidate tile size based on the judgment result of whether the candidate tile size meets the constraint of the thread bundle size;

[0040] adjust the initial score of the candidate tile size based on the first memory access factor and the second memory access factor to determine the tile score of the candidate tile size.

[0041] In one of the embodiments, the adjusting the initial score of the candidate tile size based on the first memory access factor and the second memory access factor to determine the tile score of the candidate tile size comprises:

[0042] For any of the candidate tile sizes corresponding to each of the memory access operations, adjust the initial score of the candidate tile size corresponding to each of the memory access operations based on the first memory access factor and the second memory access factor to determine a basic tile score of the candidate tile size corresponding to each of the memory access operations;

[0043] determine the tile score of the candidate tile size by adding up the basic tile scores of the candidate tile sizes corresponding to the associated memory access operations corresponding to the same candidate tile size from each of the memory access operations.

[0044] In one of the embodiments, the determining the target tile size of the kernel function from the candidate tile sizes based on the tile scores of the candidate tile sizes and the size information of the tensor data to be accessed corresponding to each of the memory access operations comprises:

[0045] sort each of the candidate tile sizes in descending order of the tile score to obtain a candidate queue;

[0046] Traverse the candidate queue, for the currently traversed candidate block size, if the candidate block size and the size information of the tensor data corresponding to each memory access operation satisfy the access constraint condition, the candidate block size is taken as the target block size of the kernel function.

[0047] In a third aspect, the present application also provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method of any one of the above aspects when executing the computer program.

[0048] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the method of any one of the above aspects when executed by a processor.

[0049] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, and the computer program implements the steps of the method of any one of the above aspects when executed by a processor.

[0050] The above-mentioned memory access block strategy determination method, computer device, computer readable storage medium and computer program product. The method automatically extracts and generates candidate block sizes matched with each memory access operation by performing memory access continuity analysis on all memory access operations involved in the kernel function, determines the block score of each candidate block size based on the block characteristics of the candidate block size corresponding to each memory access operation, and determines the target block size from the candidate block sizes based on the block score of each candidate block size. Finally, the memory access block strategy for the kernel function is determined based on the target block size. By automatically analyzing the memory access continuity, automatically generating candidate block sizes, and selecting the target block size from the candidate block sizes, the memory access block strategy determination method, computer device, computer readable storage medium and computer program product provided by the embodiments of the present application break through the limitation of traditional block strategy relying on manual design, realize the automatic generation of block strategy meeting the hardware requirements, and ensure that the generated block strategy not only improves the computation memory ratio, but also maximizes the utilization of hardware bandwidth resources. In this way, not only the technical threshold of compiler optimization is reduced, but also the development efficiency is greatly improved, and the deployment efficiency of artificial intelligence models on diversified hardware platforms can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0052] Figure 1 A flowchart of a method for determining a memory access blocking strategy in an embodiment;

[0053] Figure 2 A structural block diagram of an artificial intelligence chip in an embodiment;

[0054] Figure 3 A flowchart of step 102 in an embodiment;

[0055] Figure 4 A flowchart of step 104 in an embodiment;

[0056] Figure 5 A flowchart of step 408 in an embodiment;

[0057] Figure 6 A flowchart of a method for determining a target blocking step in an embodiment;

[0058] Figure 7 A flowchart of a method for determining a memory access blocking strategy in an embodiment;

[0059] Figure 8 A structural block diagram of a device for determining a memory access blocking strategy in an embodiment;

[0060] Figure 9 An internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0061] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0062] It should be noted that the terms "first", "second", etc. used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "multiple" used in the present application refers to two and more than two. The term "and / or" used in the present application refers to one of the options or any combination of multiple options.

[0063] In order for those skilled in the art to better understand the embodiments of the present application, the following explains the terms related to the embodiments of the present application.

[0064] Tiling: is a key technology in deep learning compiler and high-performance computing for optimizing memory access efficiency, especially suitable for parallel computing hardware such as GPU (Graphics Processing Unit) and other artificial intelligence chips. Its core idea is to divide large-scale computing or data into smaller, parallel processing sub-blocks (called tiles) to optimize memory access and parallel execution efficiency.

[0065] Bank Conflict: Access conflict occurs in GPU shared memory when multiple threads in the same warp access different addresses in the same bank, causing parallel access to degenerate into serial execution, significantly reducing memory bandwidth utilization.

[0066] Cache line: The smallest continuous unit of data transfer and management in the cache of artificial intelligence chips, used to efficiently transfer data in batches between memory levels.

[0067] As shown in Figure 1 , a memory access block strategy determination method is provided, which is applied to an artificial intelligence chip. In the present application, the artificial intelligence chip is any one of GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose Graphics Processing Unit). In the present application, this is not specifically limited, and the following will be described taking GPGPU as an example.

[0068] Referring to Figure 2 , a structure diagram of GPGPU is shown. GPGPU is actually an array of streaming processor clusters (SPC), such as streaming processor cluster 1,..., streaming processor cluster M shown in Figure 2 , M is a positive integer greater than 1. In a graphics processor, one streaming processor cluster processes a computing task, or multiple streaming processor clusters process a computing task. Multiple streaming processor clusters share data through global cache or global memory.

[0069] AsFigure 2 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 2 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations other than matrix calculations such as matrix multiplication and convolution, including operations like accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit contains multiple cores (also called computational kernels), each including an Arithmetic Logic Unit (ALU), a floating-point unit, etc., which are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 2 The register file and shared cache in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared cache in a computing unit is used to share data between the cores of that computing unit.

[0070] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 2 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0071] In each computing unit, the thread beam scheduling / distribution module ( Figure 2 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can be executed concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory execution instructions are issued to the shared cache within the computing unit or further issued to intermediate-level caches, global caches, or global memory for read and write operations, etc.

[0072] like Figure 2 As shown, the streaming processor cluster 1 also includes a tensor operation unit, which is used to perform tensor calculations, such as matrix multiplication, convolution operations, etc.

[0073] Reference Figure 1 As shown in the embodiment of this application, a method for determining a memory access block strategy includes the following steps 102 to 106. Wherein:

[0074] At step 102, memory access continuity analysis is performed on all memory access operations in the kernel function, and candidate block sizes corresponding to each memory access operation are obtained.

[0075] In the embodiments of the present application, the kernel function refers to a user-defined function executed on an artificial intelligence chip such as a GPU, which is the core execution unit of parallel computing. During its execution, it will frequently involve memory access operations, i.e., read-write operations, and the efficiency of these memory access operations will directly affect the overall performance of the kernel function, so each memory access operation needs to be analyzed.

[0076] The core of memory access continuity analysis is to determine the continuous region of memory access by analyzing the dimension information of tensor data, where the dimension information can include each dimension of the tensor data and the stride of each dimension (which can also be represented as stride). For example, the first memory access continuous dimension can be found by checking from the highest dimension to the lowest dimension of the tensor data, and the candidate block size corresponding to the memory access operation can be constructed based on this dimension. For example, the region formed by this dimension and all higher dimensions can be regarded as a continuous memory access region, and the size (or shape) of this continuous memory access region is the candidate block size corresponding to the memory access operation.

[0077] Suppose the shape of the operand (for clarity, referred to as the tensor data to be accessed below) in the memory access operation is [M, N, K] and the stride is [N*K, 1, K], that is, the tensor data to be accessed includes dimension 0 (corresponding to M), dimension 1 (corresponding to N), and dimension 2 (corresponding to K). The first dimension with a stride of 1 is dimension 1, and the continuous memory access region is the region formed by dimension 0 and dimension 1, and the candidate block size corresponding to the memory access operation is MxN.

[0078] Similarly, the candidate block sizes corresponding to all memory access operations can be obtained.

[0079] At step 104, based on the block characteristics of the candidate block sizes corresponding to each memory access operation, the block scores of each candidate block size are determined.

[0080] In the embodiments of the present application, the block characteristics refer to property characteristics that can reflect the adaptation of the candidate block size to the hardware characteristics, which can include the data amount corresponding to the candidate block size, the memory access operation type, the thread bundle size, etc.

[0081] The data amount corresponding to the candidate tile size, i.e., the total number of elements included in the candidate tile size, is, for example, 32, 64, 48, etc. The memory access operation type refers to the read or write operation of the global cache by the kernel function, i.e., the memory access operation type can include a read operation type and a write operation type. The thread bundle size is the number of threads of a basic parallel execution unit in the artificial intelligence chip architecture, and refers to the total number of threads included in a thread bundle (Warp).

[0082] For example, each tile feature can correspond to a different score or score calculation method. Based on the above score or score calculation method, the tile score of the candidate tile size corresponding to each memory access operation can be determined, and the tile score is used to represent the degree of adaptation between the candidate tile size and the kernel function and the hardware. The higher the tile score, the higher the degree of adaptation between the candidate tile size and the kernel function and the hardware.

[0083] Step 106, based on the tile score of each candidate tile size, determine the target tile size from each candidate tile size, and determine the memory tile strategy for the kernel function based on the target tile size.

[0084] In the embodiments of the present application, the target tile size is the core parameter of the memory tile strategy of the kernel function, and the memory tile strategy for the kernel function can be constructed based on the determined target tile size. The memory tile strategy can be used to guide the compiler to generate specific tile code, such as cutting a large tensor into sub-tiles of the target tile size, ensuring that the sub-tiles can efficiently use the shared memory, registers and other high-speed storage of the hardware, realizing memory merging and data reuse, thereby significantly improving the execution efficiency of the kernel function.

[0085] After determining the candidate tile size of all memory access operations in the kernel function, an optimal tile size can be selected as the target tile size. For example, the target tile size can be determined based on the tile score of the candidate tile size, for example, the adaptation degree of each candidate tile size to each memory access operation in the kernel function is analyzed in descending order of the tile score, and the first candidate tile size found to adapt to the global memory access operation is taken as the target tile size.

[0086] The determination method of the memory access blocking strategy can automatically extract and generate candidate blocking sizes matched with each memory access operation by performing memory continuity analysis on all memory access operations involving global memory in the kernel function, determine blocking scores of each candidate blocking size based on blocking characteristics of the candidate blocking sizes corresponding to each memory access operation, determine a target blocking size from the candidate blocking sizes based on the blocking scores of each candidate blocking size, and finally determine the memory access blocking strategy for the kernel function based on the target blocking size. The determination method of the memory access blocking strategy provided in the embodiments of the present application breaks through the limitation of traditional blocking strategies relying on manual design by automatically analyzing memory continuity, automatically generating candidate blocking sizes, and selecting a target blocking size therefrom, realizes automatic generation of a blocking strategy meeting the hardware requirements, ensures that the generated blocking strategy maximizes the use of hardware bandwidth resources while improving the computation-memory ratio, thereby not only reducing the technical threshold of compiler optimization, but also greatly improving the development efficiency, and can effectively improve the deployment efficiency of artificial intelligence models on diversified hardware platforms.

[0087] In one exemplary embodiment, as shown in FIG. 1, the method for determining a memory access blocking strategy for a kernel function includes the following steps 102 to 106. In step 102, memory continuity analysis is performed on all memory access operations involving global memory in the kernel function to obtain candidate blocking sizes corresponding to each memory access operation. Figure 3

[0088] In step 302, for any memory access operation in the kernel function, a dimension with a step size of 1 is searched as a target dimension in order from a high dimension to a low dimension in each dimension of the to-be-accessed tensor data corresponding to the memory access operation.

[0089] In step 304, a candidate blocking size corresponding to the memory access operation is determined based on the highest dimension and the target dimension in each dimension.

[0090] In the embodiments of the present application, the to-be-accessed tensor data refers to tensor data, such as a matrix or a feature map, which needs to be read or written by the memory access operation. The to-be-accessed tensor data has a clear dimension structure, such as an n-dimensional tensor with a shape of [A0, A1, A2,..., An-1], where A0 is the highest dimension and An-1 is the lowest dimension. Each dimension corresponds to a step size, which represents the address interval of adjacent elements in the memory in the dimension. The core basis for the memory continuity analysis is whether the step size is 1, and when the step size of a certain dimension is 1, the memory addresses of adjacent elements in the dimension are continuous. Therefore, the dimensions of the to-be-accessed tensor data can be traversed in order from the high dimension to the low dimension to find the first dimension with a step size of 1, which is marked as the target dimension.

[0091] ​For example, for a three-dimensional tensor data with shape [M, N, K] and strides [stride0, stride1, stride2], dimension 0 is the highest dimension, and dimension 2 is the lowest dimension. If stride0 is not 1 and stride1 is 1, the first dimension with stride 1 is dimension 1, i.e., the target dimension is 1. If stride0 and stride1 are not 1 and stride2 is 1, the first dimension with stride 1 is dimension 2, i.e., the target dimension is 2. If stride0 is 1, dimension 0 is directly taken as the target dimension, and there is no need to check lower dimensions.

[0092] The candidate block size is the size of the region constructed by all dimensions from the highest dimension to the target dimension, i.e., the elements within the region range constructed by all dimensions from the highest dimension to the target dimension form a continuous access region in the memory. Since the target dimension is the first dimension with stride 1, the combination of the target dimension and higher dimensions can form a continuous memory block. The size of each dimension from the highest dimension to the target dimension is multiplied, and the result is the candidate block size.

[0093] For example, still taking the three-dimensional tensor data with shape [M, N, K] and strides [stride0, stride1, stride2] as an example. If the target dimension is 1, i.e., dimension 1 is the first dimension with stride 1, the highest dimension 0 to the target dimension 1 includes dimension 0 and dimension 1, and the candidate block size is MxN. If the target dimension is 2, i.e., dimension 2 is the first dimension with stride 1, the highest dimension 0 to the target dimension 2 includes all dimensions, and the candidate block size is MxNxK. If the target dimension is 0, i.e., dimension 0 is the first dimension with stride 1, only dimension 0 is included, and the candidate block size is M.

[0094] The candidate block size determined in this way can accurately reflect the continuous memory access range that can be achieved by the memory access operation, and thus the target block size in the memory access blocking strategy can be automatically analyzed based on the candidate block size corresponding to each memory access operation, which can greatly improve the memory optimization efficiency.

[0095] In an example embodiment, as shown in FIG. 1, step 104 can include steps 402 to 408. Figure 4

[0096] Step 402, for the candidate block size corresponding to any memory access operation, an initial score of the candidate block size is determined based on the data amount corresponding to the candidate block size.

[0097] ​Step 404, determining a first memory access factor of the candidate block size based on the operation type of the memory access operation;

[0098] Step 406, judging whether the candidate block size meets the constraint of the thread bundle size, and determining a second memory access factor of the candidate block size based on the judging result;

[0099] Step 408, adjusting the initial score of the candidate block size based on the first memory access factor and the second memory access factor, to determine the block score of the candidate block size.

[0100] In the embodiment of the present application, the data amount corresponding to the candidate block size, i.e. the total number of elements contained in the block, can be used to measure the basic memory access efficiency of the block. The larger the data amount is, the more the fixed overhead of single memory access can be reduced, such as address decoding, bus scheduling, etc., that is, the higher the basic memory access efficiency is. Therefore, the determination of the initial score can be based on the size of the data amount. In the embodiment of the present application, the determination method of the initial score is not specifically limited. For example, the initial score can be the data amount of the candidate block size, or the data amount of the candidate block size can be mapped to a standard quantized initial score through a pre-set mapping algorithm of data amount and score. For example, taking a candidate block size containing 64 elements as an example, the initial score thereof is 64. Similarly, the initial scores of the candidate block sizes corresponding to each memory access operation can be finally determined.

[0101] After the initial score is determined, the first memory access factor and the second memory access factor corresponding to the candidate block size can be further determined, and the initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size. The way of adjusting the initial score of the candidate block size based on the first memory access factor and the second memory access factor can include any one of the following three ways:

[0102] Way 1 is to determine the first memory access factor, determine the intermediate score based on the initial score and the first memory access factor, determine the second memory access factor, and determine the block score of the candidate block size based on the intermediate score and the second memory access factor; Way 2 is to determine the second memory access factor, determine the intermediate score based on the initial score and the second memory access factor, determine the first memory access factor, and determine the block score of the candidate block size based on the intermediate score and the first memory access factor; Way 3 is to determine the first memory access factor and the second memory access factor, and calculate the block score of the candidate block size based on the initial score, the first memory access factor and the second memory access factor.

[0103] That is, the order of obtaining the first memory access factor and the second memory access factor, and the way of calculating the initial score based on the first memory access factor and the second memory access factor are not specifically limited in the embodiment of the present application. In the following embodiments, the embodiment of the present application will be described by way of example 1.

[0104] In the embodiments of the present application, the memory access operation types mainly include reading and writing. The reading operation refers to reading data from the global memory to the high-speed storage, and the writing operation refers to writing data from the high-speed storage to the global memory. Since the optimization mechanisms of the hardware for the reading and writing operations are different, for example, the writing operation usually requires stricter address continuity to avoid data conflicts, different types of operations are sensitive to the benefits of the block size, and therefore the difference needs to be quantified by the first memory access factor. The first memory access factor refers to an adjustment factor for the initial score set to quantify the difference between different memory access operation types.

[0105] For example, assuming that the first memory access factor corresponding to the reading operation is a first value, and the first memory access factor corresponding to the writing operation is a second value, when the reading operation has higher benefits than the writing operation, the first value will be greater than the second value, or when the writing operation has higher benefits than the reading operation, the second value will be greater than the first value. In the embodiments of the present application, the values of the first value and the second value are not specifically limited, and a person skilled in the art can set them based on the requirements.

[0106] In one example, taking the first value as 1 and the second value as 2 as an example. When the memory access operation type is the reading operation, the intermediate score of the candidate block size can be determined as the initial score x 1, that is, the current score is maintained. Taking the initial score 32 as an example, the intermediate score corresponding to the reading operation is 32 x 1 = 32. Or, when the memory access operation type is the writing operation, the intermediate score of the candidate block size can be determined as the initial score x 2, that is, the intermediate score corresponding to the writing operation is 32 x 2 = 64.

[0107] It should be noted that the above multiplication of the initial score and the first memory access factor is only an example in the embodiments of the present application. In fact, the first memory access factor can be added or subtracted on the basis of the initial score to calculate the score, which is not specifically limited in the embodiments of the present application.

[0108] In the embodiments of the present application, the thread bundle size refers to the block size that needs to adapt to the thread bundle parallelism granularity of the artificial intelligence chip. For example, the thread bundle size can be 32 threads, indicating that a thread bundle contains 32 threads. The constraint of the thread bundle size includes that the block size is greater than or equal to the thread bundle size and can be divided by the thread bundle size. The second memory access factor of the candidate block size can be determined based on whether the candidate block size meets the constraint.

[0109] For example, it is assumed that the corresponding second memory access factor is a third value when the candidate block size satisfies the constraint of the thread bundle size, and the corresponding second memory access factor is a fourth value when the candidate block size does not satisfy the constraint of the thread bundle size, and the third value is greater than the fourth value. In the embodiment of the present application, the values of the third value and the fourth value are not specifically limited, and a person skilled in the art can set them based on the needs.

[0110] For example, taking the thread bundle size as 32, if the candidate block size is 8x8 (the number of elements contained is 64, which is greater than 32 and can be divided by 32), the constraint of being greater than or equal to the thread bundle size and being divisible by the thread bundle size is satisfied, and the second memory access factor can be set to the third value. If the candidate block size is 8x6 (the number of elements contained is 48, which is greater than 32 but cannot be divided by 32), the constraint of being greater than or equal to the thread bundle size and being divisible by the thread bundle size is not satisfied, and the second memory access factor can be set to the fourth value.

[0111] In one example, taking the third value as 2 and the fourth value as 1 as an example. When the constraint of the thread bundle size is satisfied, the block value of the candidate block size can be determined as the intermediate value (calculated based on the first memory access factor and the initial value) x 2. Taking the intermediate value 32 as an example, the block fraction of the candidate block size is 32x2=64. Alternatively, when the constraint of the thread bundle size is not satisfied, the block value of the candidate block size can be determined as the intermediate value x 1, that is, the block fraction of the candidate block size is 32x1=32.

[0112] It should be noted that the above multiplication of the intermediate value and the second memory access factor is only an example in the embodiment of the present application. In fact, the second memory access factor can be processed by addition, subtraction, etc. on the basis of the intermediate value to calculate the value, which is not specifically limited in the embodiment of the present application.

[0113] In the embodiment of the present application, the block fraction of the candidate block size obtained by the calculation can intuitively reflect the comprehensive optimization capability of the candidate block size, and then the block fraction of the candidate block size can be used to determine the optimal target block size from the candidate block sizes, which can greatly improve the memory optimization efficiency.

[0114] In an exemplary embodiment, as shown in Figure 5 In step 408, adjusting the initial value of the candidate block size based on the first memory access factor and the second memory access factor to determine the block fraction of the candidate block size can include the following steps 502 to 504, wherein:

[0115] Step 502, for any candidate block size corresponding to a memory access operation, adjusting an initial score of the candidate block size corresponding to the memory access operation based on the first memory access factor and the second memory access factor to determine a basic block score of the candidate block size corresponding to the memory access operation;

[0116] Step 504, determining associated memory access operations corresponding to the same candidate block size from each memory access operation, and accumulating the basic block scores of the candidate block size corresponding to each associated memory access operation to obtain a block score of the candidate block size.

[0117] In the embodiments of the present application, since different memory access operations can correspond to the same candidate block size, when determining the target candidate block size, the block scores of the same candidate block size calculated under different memory access operations need to be accumulated as the final block score of the candidate block size. In the following, in order to make the description clear, the block score of the candidate block size calculated under each memory access operation is referred to as a basic block score, and the calculation process can refer to the related description of the foregoing embodiments, which will not be repeated here in the embodiments of the present application.

[0118] For example, in the process of determining the block score of the candidate block size, the memory access operations corresponding to the same candidate block size can be first determined as associated memory access operations, that is, the associated memory access operations refer to multiple different memory access operations in the kernel function, and if they correspond to the same candidate block size, such as the candidate block sizes of the memory access operation A and the memory access operation B are both 64, then the memory access operation A and the memory access operation B are associated memory access operations.

[0119] The determination of the block score of the candidate block size needs to consider the overall adaptability of the size to all associated memory access operations, and if a candidate block size can adapt to multiple memory access operations at the same time, the complexity of the block strategy can be reduced. Therefore, the basic block scores of the candidate block sizes corresponding to the associated memory access operations need to be accumulated to finally obtain the block score of the candidate block size.

[0120] For example, the candidate block sizes of the memory access operation A and the memory access operation B are both 64, and the basic block scores are 128 and 256 respectively, then the block score corresponding to the candidate block size is 128+256=384. In addition, the candidate block size of the memory access operation C is 48, and the basic block score is 512, and there is no other associated memory access operation, then the block score corresponding to the candidate block size is 512.

[0121] By accumulating the basic block scores of the associated operations, the block score of the candidate block size can finally reflect the overall adaptation value of the candidate block size to all memory access operations in the kernel function, and the higher the score is, the better the adaptation of the candidate block size to more operations is, and the more suitable it is as the global optimal target block size.

[0122] In an example embodiment, based on the chunk scores of the candidate chunk sizes, determining the target chunk size from the candidate chunk sizes, and determining the memory access chunk strategy for the kernel function based on the target chunk size, can include:

[0123] Based on the chunk scores of the candidate chunk sizes and the size information of the tensor data to be accessed corresponding to each memory access operation, the target chunk size of the kernel function is determined from the candidate chunk sizes.

[0124] In the embodiments of the present application, the size information of the tensor data to be accessed refers to the size of each dimension of the tensor, such as a tensor with shape [M, N, K], whose size information is M, N, and K. Finally, based on the chunk scores of the candidate chunk sizes and the size information of the tensor data to be accessed corresponding to each memory access operation, the target chunk size of the kernel function can be determined from the candidate chunk sizes.

[0125] In an example embodiment, as shown in Figure 6 Based on the chunk scores of the candidate chunk sizes and the size information of the tensor data to be accessed corresponding to each memory access operation, the target chunk size of the kernel function is determined from the candidate chunk sizes, which can include the following steps 602 to 604. Among them:

[0126] Step 602, sort each candidate chunk size in descending order of chunk score to obtain a candidate queue;

[0127] Step 604, traverse the candidate queue, and for the currently traversed candidate chunk size, if the candidate chunk size and the size information of the tensor data to be accessed corresponding to each memory access operation all satisfy the access constraint condition, the candidate chunk size is taken as the target chunk size of the kernel function.

[0128] In the embodiments of the present application, the chunk score is a quantitative evaluation of the comprehensive adaptation capability of the candidate chunk size. The higher the score, the higher the overall optimization value of the candidate chunk size for the memory access operation in the kernel function. By sorting in descending order of chunk score, a candidate queue can be obtained. This way can put the candidate size with the best theoretical adaptability at the front of the queue and give priority to checking, avoiding invalid checking of low-score and poor-adaptability sizes, which can shorten the time consumption of determining the target chunk size.

[0129] For example, the chunk scores of the candidate chunk size (1), the candidate chunk size (2), and the candidate chunk size (3) are 256, 96, and 48 respectively, and the sizes are 8x8, 4x12, and 4x6 respectively. The descendingly sorted candidate queue is [(1), (2), (3)], that is, the candidate chunk size (1) is traversed first, and then the candidate chunk size (2) and the candidate chunk size (3) are traversed in turn.

[0130] During the traversal of the candidate queue, for the currently traversed candidate block size, if the candidate block size and the size information of the memory access tensor data corresponding to each memory access operation satisfy the access constraints, then the candidate block size is used as the target block size of the kernel function, and the traversal ends. The access constraints include that the candidate block size is compatible with the relevant dimension dimensions of all memory access tensor data; that is, for each memory access tensor data, the size of each dimension from its high dimension to the target dimension (the dimension range constituting the candidate block size) is divisible by the current candidate block size. In this way, the target block size ensures that no residual elements appear during the block division process, i.e., there are no scattered small sub-blocks after block division, thus simplifying the boundary handling logic and avoiding efficiency losses caused by extra processing of scattered sub-blocks.

[0131] For example, suppose that the memory access tensor data corresponding to each memory access operation in the kernel function includes a first tensor data and a second tensor data, where the shape of the first tensor data is [128, 64, 1] and the shape of the second tensor data is [96, 32, 2]. Suppose that the candidate block sizes are arranged in descending order of block score as [32, 32, 1], [16, 16, 2], and [8, 8, 1].

[0132] First, iterate through the first candidate block size [32, 32, 1]. For the first tensor data [128, 64, 1], 128 / 32 = 4 (integer, dimension 0 fits); 64 / 32 = 2 (integer, dimension 1 fits). Dimension 2 with a step size of 1 is ignored, meaning the first tensor data fits. For the second tensor data [96, 32, 2], 96 / 32 = 3 (integer, dimension 0 fits); 32 / 32 = 1 (integer, dimension 1 fits). Dimension 2 with a step size of 1 is ignored, meaning the second tensor data fits. It can be determined that the first candidate block size fits all tensor data, therefore, the iteration can stop, and the first candidate block size is determined as the target block size (shape).

[0133] By using this high-score-first, fully compatible filtering method, the final target block size can maximize the memory access optimization effect and ensure the adaptability to all memory tensors to be accessed. It can automatically generate a practical and optimal memory access block strategy for the kernel function.

[0134] To enable those skilled in the art to better understand the embodiments of this application, the embodiments of this application are described below through specific examples.

[0135] In one example, refer to Figure 7 As shown, determining the memory access block strategy mainly includes the following process:

[0136] Collect memory access operations in the kernel, including operations of reading or writing data from global memory. By parsing the code logic of the kernel, including variable passing, memory address calculation, branching statements, etc., all dependency relationships between memory access operations can be automatically identified and recorded to form a deps set, and each memory access operation in the deps set is traversed.

[0137] Analyze the memory continuity of each memory access operation in the traversal. Specifically, start from the highest dimension to find the first dimension position with a step of 1, denoted as split, then take the dimension as the boundary line, the memory from split to the highest dimension is continuous, and the number of memory accesses is taken as the block size of the memory access operation, and the memory from split to the lowest dimension is not continuous. For example, loading data from a tensor with a size of [M, N, K] and a step of [N x K, 1, K], the dimensions are [0, 1, 2], the first dimension with a step of 1 is 1, so split is 1, and the memory in the 0 and 1 dimensions is continuous, and the candidate block size of the memory access operation is M x N.

[0138] Score each candidate block size. The data amount of the accessed elements in the candidate block size is taken as the basic score, i.e., the initial score, which means that the more continuous memory accesses, the better, because continuous memory accesses will be combined to reduce the number of memory access transactions. According to the benefits of combined read and write on the target hardware, the score of the combined memory access mode with high benefits is doubled, otherwise the current score is kept. For example, the benefits of write operations are higher, then determine whether the memory access operation is a write operation, if the memory access operation is a write operation, the initial score will be doubled, otherwise, if the memory access operation is a read operation, the initial score will remain unchanged.

[0139] Further, it is determined whether the data amount of the candidate block size is not less than 32 and can be divided by 32, if the data amount of the candidate block size is not less than 32 and can be divided by 32, the score will continue to be doubled, otherwise the current score will remain unchanged. Finally, the scores of the same candidate block size are added together as the final score of the candidate block size.

[0140] The candidate block sizes are sorted in descending order of scores to obtain a candidate list. The candidate list is traversed to screen the valid and highest-score candidate block size. This includes: traversing the candidate block sizes to find the first candidate block size that can divide each dimension of all tensors to be accessed (except the dimension with a size of 1), that is, the valid and highest-score candidate block size, as the final target block size of the current kernel. That is, if the traversed candidate block size is valid, the candidate block size can be used as the target block size, the current block strategy is returned, and the target block size is used as the block basis of the current block strategy. If the traversed candidate block size is invalid, the traversal is continued until all candidate block sizes in the candidate list are traversed, and the traversal is ended to determine that the current kernel does not need to be blocked.

[0141] The method for determining a memory access block strategy provided by the embodiments of the present application, the computer device, the readable storage medium and the program product, through systematic analysis of all memory access operations in a kernel function, fine memory continuity detection is performed for each memory access operation, and then candidate block sizes adapted to each operation are automatically extracted and generated, and on this basis, a target block size is screened from the candidate block sizes, and finally a memory access block strategy for the kernel function is formulated according to the target block size.

[0142] Compared with the traditional block technology requiring manual intervention, the automatic analysis of memory continuity, the automatic generation of candidate block sizes and the intelligent screening of target block sizes break through the technical limitation that the traditional block strategy relies on manual design, and realize the automatic generation of a block strategy adapted to hardware requirements. Not only does this greatly reduce the technical threshold of deep learning compiler optimization, significantly shortens the development cycle of the block strategy, but also effectively improves the deployment efficiency and running performance of the deep learning model on diversified hardware platforms.

[0143] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0144] Based on the same inventive concept, the embodiments of the present application also provide a memory access blocking strategy determination apparatus for implementing the memory access blocking strategy determination method described above. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, so the specific limitations in one or more memory access blocking strategy determination apparatus embodiments provided below can refer to the limitations of the memory access blocking strategy determination method in the above text, which will not be described here again.

[0145] In one exemplary embodiment, as shown in Figure 8 A memory access blocking strategy determination apparatus 800 is provided, comprising an analysis module 802, a first determination module 804 and a second determination module 806, wherein:

[0146] The analysis module 802 is configured to perform memory continuity analysis on all memory access operations in the kernel function respectively, to obtain a candidate blocking size corresponding to each memory access operation.

[0147] The first determination module 804 is configured to determine a blocking score of each candidate blocking size based on the blocking feature of the candidate blocking size corresponding to each memory access operation.

[0148] The second determination module 806 is configured to determine a target blocking size from the candidate blocking sizes based on the blocking scores of the candidate blocking sizes, and determine a memory access blocking strategy for the kernel function based on the target blocking size.

[0149] The memory access blocking strategy determination apparatus described above automatically extracts and generates candidate blocking sizes matched with each memory access operation by performing memory continuity analysis on all memory access operations involving global memory in the kernel function, and determines a blocking score of each candidate blocking size based on the blocking feature of the candidate blocking size corresponding to each memory access operation, and determines a target blocking size from the candidate blocking sizes based on the blocking scores of the candidate blocking sizes, and finally determines a memory access blocking strategy for the kernel function based on the target blocking size. By automatically analyzing memory continuity, generating candidate blocking sizes, and selecting a target blocking size from the candidate blocking sizes, the memory access blocking strategy determination apparatus provided by the embodiments of the present application breaks through the limitation of traditional blocking strategies relying on manual design, realizes the automatic generation of blocking strategies that meet the hardware requirements, ensures that the generated blocking strategies not only improve the computation memory ratio, but also maximize the utilization of hardware bandwidth resources. In this way, not only is the technical threshold of compiler optimization reduced, but also the development efficiency is greatly improved, which can effectively improve the deployment efficiency of artificial intelligence models on diversified hardware platforms.

[0150] In an example embodiment, the memory access continuity analysis is performed on each of the memory access operations in the kernel function, and a candidate block size corresponding to each of the memory access operations is obtained, including:

[0151] For any memory access operation in the kernel function, a target dimension is searched from each dimension of the tensor data to be accessed corresponding to the memory access operation in an order from a high dimension to a low dimension, and the target dimension is a dimension with a step size of 1;

[0152] Based on the highest dimension in each of the dimensions and the target dimension, a candidate block size corresponding to the memory access operation is determined.

[0153] In an example embodiment, the target block size of the kernel function is determined from the candidate block sizes based on the block scores of the candidate block sizes, including:

[0154] Based on the block scores of the candidate block sizes and the size information of the tensor data to be accessed corresponding to each of the memory access operations, the target block size of the kernel function is determined from the candidate block sizes.

[0155] In an example embodiment, the block score of each of the candidate block sizes is determined based on the block features of the candidate block sizes corresponding to each of the memory access operations, including:

[0156] For a candidate block size corresponding to any of the memory access operations, an initial score of the candidate block size is determined based on a data amount corresponding to the candidate block size;

[0157] A first memory access factor of the candidate block size is determined based on an operation type of the memory access operation;

[0158] It is determined whether the candidate block size satisfies a constraint of the thread bundle size, and a second memory access factor of the candidate block size is determined based on a determination result;

[0159] The initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor, and a block score of the candidate block size is determined.

[0160] In an example embodiment, the block score of each of the candidate block sizes is determined based on the first memory access factor and the second memory access factor adjusting the initial score of the candidate block size, including:

[0161] For a candidate block size corresponding to any of the memory access operations, an initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor, and a basic block score of the candidate block size corresponding to the memory access operation is determined;

[0162] determining associated memory access operations corresponding to a same candidate tile size from each of the memory access operations, and accumulating a tile score of the candidate tile size based on tile scores of the associated memory access operations corresponding to the candidate tile size.

[0163] In an example embodiment, the determining the target tile size of the kernel function from the candidate tile sizes based on the tile scores of the candidate tile sizes and the size information of the tensor data corresponding to the memory access operations comprises:

[0164] sorting the candidate tile sizes in a descending order of tile scores to obtain a candidate queue;

[0165] traversing the candidate queue, and for a currently traversed candidate tile size, if the candidate tile size and the size information of the tensor data corresponding to the memory access operations all satisfy the access constraint condition, taking the candidate tile size as the target tile size of the kernel function.

[0166] Each of the modules in the memory access tile strategy determination apparatus can be realized by software, hardware, or a combination thereof, in whole or in part. Each of the modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to each of the modules.

[0167] In an example embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 9The computer device shown in the figure includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, near field communication (NFC) or other technologies. The computer program is executed by the processor to realize the determination method of the memory block strategy. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0168] Those skilled in the art can understand that, Figure 9 The skilled in the art can understand that,

[0169] In one embodiment, a computer device is also provided, including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps in the above method embodiments.

[0170] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to realize the steps in the above method embodiments.

[0171] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to realize the steps in the above method embodiments.

[0172] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0173] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. In the embodiments provided in the present application, any reference to memory, database or other medium can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0174] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, any combination of these technical features is deemed to be within the scope of the present application.

[0175] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for determining a memory access block strategy, characterized in that, The method includes: For all memory access operations to global memory in the kernel function, memory access continuity analysis is performed to obtain the candidate block size corresponding to each memory access operation; Based on the block characteristics of the candidate block sizes corresponding to each memory access operation, the block score of each candidate block size is determined; Based on the block score of each candidate block size, a target block size is determined from each candidate block size, and a memory access block strategy for the kernel function is determined based on the target block size.

2. The method according to claim 1, characterized in that, For each memory access operation in the kernel function, a memory access continuity analysis is performed to obtain the candidate block size corresponding to each memory access operation, including: For any memory access operation in the kernel function, from each dimension of the memory access tensor data to be accessed corresponding to the memory access operation, search for the dimension with a step size of 1 as the target dimension in order from the high dimension to the low dimension. Based on the highest dimension among the dimensions and the target dimension, the candidate block size corresponding to the memory access operation is determined.

3. The method according to claim 1 or 2, characterized in that, The step of determining the target block size from the candidate block sizes based on the block scores of each candidate block size includes: Based on the block score of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, the target block size of the kernel function is determined from the candidate block sizes.

4. The method according to claim 3, characterized in that, The block characteristics of the candidate block size include the amount of data corresponding to the candidate block size, the operation type of the memory access operation, and the thread bundle size.

5. The method according to claim 4, characterized in that, The step of determining the block score for each candidate block size based on the block characteristics corresponding to each memory access operation includes: For any candidate block size corresponding to any memory access operation, an initial score for the candidate block size is determined based on the amount of data corresponding to the candidate block size. The first memory access factor for the candidate block size is determined based on the operation type of the memory access operation. Determine whether the candidate block size satisfies the constraint of the thread bundle size, and determine the second memory access factor of the candidate block size based on the determination result; The initial score of the candidate block size is adjusted based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size.

6. The method according to claim 5, characterized in that, The step of adjusting the initial score of the candidate block size based on the first memory access factor and the second memory access factor to determine the block score of the candidate block size includes: For any candidate block size corresponding to any memory access operation, the initial score of the candidate block size corresponding to the memory access operation is adjusted based on the first memory access factor and the second memory access factor to determine the basic block score of the candidate block size corresponding to the memory access operation. From each of the memory access operations, determine the associated memory access operations corresponding to the same candidate block size, and accumulate the basic block scores of the candidate block size corresponding to each of the associated memory access operations to obtain the block score of the candidate block size.

7. The method according to claim 3, characterized in that, The determination of the target block size of the kernel function from the candidate block sizes, based on the block scores of each candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation, includes: The candidate block sizes are sorted in descending order of block score to obtain the candidate queue; Traverse the candidate queue. For the currently traversed candidate block size, if the candidate block size and the size information of the memory tensor data to be accessed corresponding to each memory access operation satisfy the access constraint conditions, then the candidate block size is used as the target block size of the kernel function.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images