Optimization method and device for protocol calculation, computer equipment, readable storage medium and program product

Abstract

The application relates to an optimization method and device for reduction calculation, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: determining the memory mapping addresses corresponding to each thread in a target register by performing address mapping processing on the target register, determining a target initialization value according to the operation type of the reduction calculation, initializing the data stored in the memory mapping addresses corresponding to each thread into the target initialization value, and performing the reduction calculation to obtain a reduction calculation result. The method can improve the calculation efficiency of the reduction calculation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence chip technology, and in particular to an optimization method, apparatus, computer device, readable storage medium, and program product for reduction computing. Background Technology

[0002] In assembly language, Atomic instructions use address mapping to perform Reduce operations on the values ​​stored in registers by multiple threads and the values ​​stored at the corresponding memory addresses. Reduce operations can include SUM, MAX, MIN, etc. After the Reduce calculation is completed, the calculation result is updated to memory.

[0003] In related technologies, memory needs to be initialized before Reduce is executed. Memory initialization is automatically completed by the system before the Kernel (kernel function) is executed. Often, the memory is initialized to a default initial value. This default initial value usually cannot match the requirements of the Reduce type, which seriously affects the efficiency of Reduce calculation and the performance of the assembly program. Summary of the Invention

[0004] Therefore, it is necessary to provide an optimization method, apparatus, computer device, computer-readable storage medium, and computer program product for reducing computation that can improve the efficiency of reducing computation, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides an optimization method for reduction computation, the method comprising:

[0006] The target register is processed for address mapping to determine the memory mapping address corresponding to each thread in the target register;

[0007] Determine the target initialization value based on the operation type calculated according to the protocol;

[0008] The data stored at the memory-mapped addresses corresponding to each thread is initialized to the target initialization value, and the reduction calculation is performed to obtain the reduction calculation result.

[0009] In one embodiment, determining the target initialization value based on the operation type calculated according to the specification includes at least one of the following:

[0010] When the operation type of the reduction calculation is SUM, the target initialization value is determined to be 0;

[0011] When the operation type of the reduction calculation is MAX, the target initial value is determined to be negative infinity;

[0012] When the operation type of the reduction calculation is MIN, the target initialization value is determined to be positive infinity.

[0013] In one embodiment, the method further includes:

[0014] Store the memory mapping addresses corresponding to each thread in the target register in the specified register;

[0015] The process of performing the reduction calculation to obtain the reduction calculation result includes:

[0016] Read the memory-mapped address of each thread from the specified register;

[0017] The reduction calculation is performed based on the memory mapping address and the data stored in each thread to obtain the reduction calculation result.

[0018] In one embodiment, the step of performing address mapping processing on the target register to determine the memory mapping address corresponding to each thread in the target register includes:

[0019] Determine the reference register, and based on the thread index of each thread in the reference register, the position information of the reference register in the tensor data, the memory offset, the stride, and the number of bytes of stored data, determine the memory mapping address corresponding to each thread in the reference register;

[0020] Determine the coordinate difference between the target register and the reference register in the tensor data;

[0021] The memory mapping address of each thread in the target register is determined based on the memory mapping address of each thread in the base register and the coordinate difference value.

[0022] In one embodiment, determining the memory mapping address of each thread in the target register based on the memory mapping address of each thread in the base register and the coordinate difference value includes:

[0023] The memory offset is determined based on the coordinate difference value, the step size, and the number of bytes of the stored data;

[0024] The memory mapping address of each thread in the target register is determined based on the memory mapping address of each thread in the base register and the memory offset.

[0025] In one embodiment, the address mapping process and memory initialization operation are executed in parallel and overlap with the calculation process of the pre-operator, which is an operator that is executed before the reduction calculation.

[0026] Secondly, this application also provides an optimization apparatus for reduction calculation, the apparatus comprising:

[0027] The first determining module is used to perform address mapping processing on the target register and determine the memory mapping address corresponding to each thread in the target register;

[0028] The second determination module is used to determine the target initialization value based on the operation type calculated in the protocol.

[0029] The calculation module is used to initialize the data stored at the memory-mapped address corresponding to each thread to the target initialization value, and to perform the reduction calculation to obtain the reduction calculation result.

[0030] In one embodiment, determining the target initialization value based on the operation type calculated according to the specification includes at least one of the following:

[0031] When the operation type of the reduction calculation is SUM, the target initialization value is determined to be 0;

[0032] When the operation type of the reduction calculation is MAX, the target initial value is determined to be negative infinity;

[0033] When the operation type of the reduction calculation is MIN, the target initialization value is determined to be positive infinity.

[0034] In one embodiment, the device further includes:

[0035] The mapping module is used to store the memory mapping addresses corresponding to each thread in the target register in a specified register;

[0036] The process of performing the reduction calculation to obtain the reduction calculation result includes:

[0037] Read the memory-mapped address of each thread from the specified register;

[0038] The reduction calculation is performed based on the memory mapping address and the data stored in each thread to obtain the reduction calculation result.

[0039] In one embodiment, the step of performing address mapping processing on the target register to determine the memory mapping address corresponding to each thread in the target register includes:

[0040] Determine the reference register, and based on the thread index of each thread in the reference register, the position information of the reference register in the tensor data, the memory offset, the stride, and the number of bytes of stored data, determine the memory mapping address corresponding to each thread in the reference register;

[0041] Determine the coordinate difference between the target register and the reference register in the tensor data;

[0042] The memory mapping address of each thread in the target register is determined based on the memory mapping address of each thread in the base register and the coordinate difference value.

[0043] In one embodiment, determining the memory mapping address of each thread in the target register based on the memory mapping address of each thread in the base register and the coordinate difference value includes:

[0044] The memory offset is determined based on the coordinate difference value, the step size, and the number of bytes of the stored data;

[0045] The memory mapping address of each thread in the target register is determined based on the memory mapping address of each thread in the base register and the memory offset.

[0046] In one embodiment, the address mapping process and memory initialization operation are executed in parallel and overlap with the calculation process of the pre-operator, which is an operator that is executed before the reduction calculation.

[0047] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the above-mentioned embodiments.

[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0049] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above claims.

[0050] The aforementioned optimization method, apparatus, computer device, computer-readable storage medium, and computer program product for reduction computation determine the memory mapping addresses corresponding to each thread in the target register by performing address mapping processing on the target register, determine the target initialization value according to the operation type of the reduction computation, initialize the data stored at the memory mapping addresses corresponding to each thread to the target initialization value, and then perform reduction computation to obtain the reduction computation result. Using the optimization method, apparatus, computer device, computer-readable storage medium, and computer program product for reduction computation provided in this application embodiment, the initialization operation is executed within the kernel. During the initialization operation, the corresponding initialization value can be determined based on the operation type of the reduction computation; that is, the corresponding initialization value can be dynamically adapted based on the operation type of the reduction computation, and the initialization operation can be performed. This allows the initialization result to adapt to the type requirements of the reduction computation, thereby improving the efficiency of reduction computation and the performance of the assembler. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a structural block diagram of an artificial intelligence chip in one embodiment;

[0053] Figure 2 This is a flowchart illustrating the optimization method for reduction computation in one embodiment;

[0054] Figure 3 This is a flowchart illustrating step 206 in one embodiment;

[0055] Figure 4 This is a flowchart illustrating step 202 in one embodiment;

[0056] Figure 5 This is a schematic diagram of register thread-to-memory mapping in one embodiment;

[0057] Figure 6 This is a flowchart illustrating step 406 in one embodiment;

[0058] Figure 7 This is a schematic diagram of register coordinate increments in one embodiment;

[0059] Figure 8 This is a schematic diagram illustrating the process of integrating memory initialization into the Kernel in one embodiment;

[0060] Figure 9 This is a structural block diagram of an optimization device for reduction calculation in one embodiment.

[0061] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0063] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0064] like Figure 2 As shown, an optimization method for reduction computation is provided and applied to artificial intelligence chips. In this embodiment, the artificial intelligence chip can be any one of GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose Graphics Processing Unit). This embodiment does not specifically limit the specific type of chip, and the following description uses GPGPU as an example.

[0065] Reference Figure 1 The diagram shows a schematic of a GPGPU. A GPGPU is actually an array of Streaming Processor Clusters (SPCs), including, for example,... Figure 1The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a graphics processing unit (GPU), one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0066] like Figure 1 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 1 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations other than matrix calculations such as matrix multiplication and convolution, including operations like accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit contains multiple cores (also called computational kernels), each including an Arithmetic Logic Unit (ALU), a floating-point unit, etc., which are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 1 The register file and shared cache in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared cache in a computing unit is used to share data between the cores of that computing unit.

[0067] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0068] In each computing unit, the thread beam scheduling / distribution module ( Figure 1 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can be executed concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory execution instructions are issued to the shared cache within the computing unit or further issued to intermediate-level caches, global caches, or global memory for read and write operations, etc.

[0069] like Figure 1As shown, the streaming processor cluster 1 also includes a tensor operation unit, which is used to perform tensor calculations, such as matrix multiplication, convolution operations, etc.

[0070] Reference Figure 2 As shown in the embodiment of this application, an optimization method for reduction calculation includes the following steps 202 to 206. Wherein:

[0071] Step 202: Perform address mapping processing on the target register to determine the memory mapping address corresponding to each thread in the target register.

[0072] In this embodiment, the target register refers to any register in the set of registers contained in the tensor. Each register in the set contains multiple threads, which are the basic units for parallel reduction computation. The memory mapping address corresponding to the thread is calculated and determined based on the coordinate information of the target register in the tensor, the memory step size parameter, and the memory starting offset, thereby ensuring that the thread data can accurately interact with the data at the target location in memory.

[0073] Step 204: Determine the target initialization value based on the operation type calculated according to the specification.

[0074] In this embodiment, the reduction calculation operation types include three categories: SUM (summation), MAX (maximum value), and MIN (minimum value). The calculation logic uses the initial memory value, i.e., the target initialization value, as the baseline starting point for the reduction calculation. Different operation types require different target initialization values; therefore, the compatibility between the target initialization value and the operation type is crucial to the accuracy of the calculation results.

[0075] In one exemplary embodiment, the target initialization value is determined based on the operation type calculated according to the specification, including at least one of the following:

[0076] When the operation type of the reduction calculation is SUM, the target initialization value is determined to be 0;

[0077] When the operation type of the reduction calculation is MAX, the target initial value is determined to be negative infinity;

[0078] When the operation type of the reduction calculation is MIN, the target initialization value is determined to be positive infinity.

[0079] In this embodiment, during the SUM operation, the summation needs to start from 0 to ensure the accuracy of the summation result; therefore, the target initialization value can be determined to be 0. During the MAX operation, it is necessary to ensure that the data of the first thread can be used as the initial maximum value, and subsequent operations only update this larger value; therefore, the target initialization value can be determined to be negative infinity, i.e., infinitesimal, so that no matter how large the data of the first thread is, it can be written to memory as the initial maximum value. During the MIN operation, it is necessary to ensure that the data of the first thread can be used as the initial minimum value, and subsequent operations only update this smaller value; therefore, the target initialization value can be determined to be positive infinity, i.e., infinity, so that no matter how large the data of the first thread is, it can be written to memory as the initial minimum value.

[0080] In other words, in this embodiment of the application, the corresponding target initialization value can be determined by matching the operation type of the specification calculation, thereby avoiding errors in the result or additional correction steps caused by the mismatch of the system's default initial value.

[0081] Step 206: Initialize the data stored in the memory-mapped addresses corresponding to each thread to the target initialization value, and perform reduction calculation to obtain the reduction calculation result.

[0082] In this embodiment, after determining the target initialization value corresponding to the operation type of the reduction calculation during the initialization phase, the data stored at the memory-mapped addresses corresponding to each thread can be initialized, that is, the data stored at the memory addresses corresponding to each thread can be updated to the target initialization value. After initialization is completed, when performing the reduction calculation, the aggregation operations between the data in the thread and the memory data, such as SUM, MAX, MIN, etc., can be completed to obtain the corresponding reduction calculation results.

[0083] The aforementioned optimization method for reduction computation determines the memory mapping address corresponding to each thread in the target register by performing address mapping processing on the target register, and determines the target initialization value according to the operation type of the reduction computation. Then, the data stored at the memory mapping address corresponding to each thread is initialized to the target initialization value, and the reduction computation is performed to obtain the reduction computation result. Using the optimization method for reduction computation provided in this application embodiment, the initialization operation is performed within the kernel. During the initialization operation, the corresponding initialization value can be determined based on the operation type of the reduction computation. That is, the corresponding initialization value can be dynamically adapted based on the operation type of the reduction computation, and the initialization operation can be performed. This allows the initialization result to adapt to the type requirements of the reduction computation, thereby improving the efficiency of the reduction computation and the performance of the assembler.

[0084] In an exemplary embodiment, the method may further include: storing the memory-mapped addresses corresponding to each thread in the target register into a designated register; such as Figure 3As shown, in step 206, a reduction calculation is performed to obtain the reduction calculation result, which may include the following steps 302 to 304. Wherein:

[0085] Step 302: Read the memory-mapped addresses of each thread from the specified register;

[0086] Step 304: Perform reduction calculation based on the memory mapping address and the data stored in each thread to obtain the reduction calculation result.

[0087] In this embodiment, during memory initialization, the determined memory mapping addresses for each thread can be stored in a designated register for reuse during reduction calculation. The designated register is an intermediate register specifically used to store the address mapping results, enabling address mapping to be calculated once and reused multiple times. During subsequent reduction calculations, it is unnecessary to re-execute these time-consuming address mapping steps for each thread; a simple read instruction is sufficient to read the memory mapping addresses of each thread from the designated register. Finally, the reduction calculation result is obtained by aggregating the read memory mapping addresses with the data stored in each thread and the memory data.

[0088] For example, for the SUM operation, the data in the thread is added to the current value read from the memory-mapped address, and the result is written back to the memory-mapped address; for the MAX operation, the data in the thread is compared with the current value read from the memory-mapped address, and the maximum value is written back to the memory-mapped address; for the MIN operation, the data in the thread is compared with the current value read from the memory-mapped address, and the minimum value is written back to the memory-mapped address.

[0089] Since the memory mapping address is directly obtained from the specified register, this step only requires executing the Atomic instruction operation related to the reduction logic, avoiding redundant calculations in address mapping. Assuming the Tensor contains N registers R, the calculation process from register Thread to memory address mapping requires M instructions, and the Atomic instruction operation itself generates K instructions, where K is much smaller than M. In the traditional reduction calculation operation, the total number of instructions generated by memory initialization and the Reduce process is (N+N)×(M+K). Using the optimized reduction calculation method provided in this application, since the Reduce process does not need to repeat the address mapping process, the total number of instructions is N×(M+K)+N×K. The total number decreases significantly as N increases, thereby significantly improving the execution efficiency of the reduction calculation.

[0090] In one exemplary embodiment, such as Figure 4As shown, in step 202, address mapping processing is performed on the target register to determine the memory mapping address corresponding to each thread in the target register, which may include steps 402 to 406. Wherein:

[0091] Step 402: Determine the reference register. Based on the thread index of each thread in the reference register, the position information of the reference register in the tensor data, the memory offset, the step size, and the number of bytes of stored data, determine the memory mapping address corresponding to each thread in the reference register.

[0092] Step 404: Determine the coordinate difference between the target register and the reference register in the tensor data;

[0093] Step 406: Determine the memory mapping address of each thread in the target register based on the memory mapping address and coordinate difference value of each thread in the base register.

[0094] In this embodiment, the reference register is the first register selected in the Tensor as the reference for address mapping calculation. Since the address mapping logic of all registers in the Tensor has commonality and only differs due to different positions, the complete calculation of the reference register can provide a reuse basis for other registers.

[0095] The coordinate difference value is the difference between the position coordinates of the target register and the base register in the tensor, including row-direction and column-direction differences. The calculation process of the memory mapping address corresponding to each thread in the target register may include: first, determining the base register, and using a traditional address mapping algorithm, calculating the memory mapping address of each thread in the base register based on the thread index of each thread in the base register, the position information of the base register in the tensor data, the memory offset, the stride, and the number of bytes of stored data.

[0096] For example, such as Figure 5 As shown, assume a Tensor contains several registers R, each register R contains TH×TW threads, and the top-left corner of each R has coordinates (h, w) within the entire Tensor. The memory stride is strideH and strideW. Assuming the base register is R1, the steps for calculating the memory mapping address of each thread (thread index in register R1 is (RH, RW), and its coordinates in the Tensor are (h1, w1)) are as follows:

[0097] (1) Use the thread index constraint in the W direction inside R1 to the range of [0,TW] to obtain RW, and use the thread index constraint in the H direction inside R1 to the range of [0,TH] to obtain RH;

[0098] (2) The external cumulative W direction coordinate offset of R1, RW1=RW+w1, and the external cumulative H direction coordinate offset of R1, RH1=RH+h1;

[0099] (3) If the memory itself has a coordinate offset (hm, wm) relative to the starting point, then the above results need to be accumulated, RW2=RW1+wm, RH2=RH1+hm;

[0100] (4) The coordinate offset of each thread in the W direction in memory is multiplied by strideW, RW3=RW2×strideW, and the coordinate offset of each thread in the H direction in memory is multiplied by strideH, RH3=RH2×strideH.

[0101] (5) The coordinate offsets in the H and W directions in memory are accumulated, RM1=RH3+RW3;

[0102] (6) Multiply the coordinate offset in memory by the number of bytes (ElementByte) of data in the thread to get the final memory mapping address of each thread, RM1=RM1×ElementByte.

[0103] Finally, by integrating the above calculation process of the reference register, the calculation formula can be obtained as follows: RM1=((RW+w1+wm)×strideW+(RH+h1+hm)×strideH)×ElementByte.

[0104] After calculating the memory mapping address of the base register, the coordinate difference between the target register and the base register can be calculated. Based on the memory mapping address of each thread in the base register and the coordinate difference value, the memory mapping address of each thread in the target register can be calculated.

[0105] It should be noted that the selection of the reference register is not specifically limited in the embodiments of this application. The first register, the last register, or any register in the middle can be selected.

[0106] In one exemplary embodiment, such as Figure 6 As shown, in step 406, determining the memory mapping address of each thread in the target register based on the memory mapping address and coordinate difference value corresponding to each thread in the base register may include steps 602 to 604. Wherein:

[0107] Step 602: Determine the memory offset based on the coordinate difference value, step size, and number of bytes of stored data;

[0108] Step 604: Determine the memory mapping address of each thread in the target register based on the memory mapping address and memory offset of each thread in the base register.

[0109] In this embodiment of the application, by analyzing the calculation process of the above-mentioned base register, if each register is calculated in this way, most of the calculation steps are repeated. Except for steps (1) and (2), the remaining steps are the same for each register R. Figure 7 As shown, only the coordinate increments (deltaH, deltaW) of register R2 relative to register R1 truly affect the address mapping process. Therefore, the overall calculation process can be simplified by calculating only the coordinate differences in the H and W dimensions of registers, thus achieving instruction optimization.

[0110] For example, such as Figure 7 As shown, the calculation logic for the coordinate difference (deltaH, deltaW) between register R2 (with coordinates (h2, w2) in the Tensor) and register R1 is as follows: deltaW = w2 – w1; deltaH = h2 – h1.

[0111] Based on the calculation logic for the memory mapping addresses of each thread in the reference register R2, the calculation formula for the memory mapping addresses of each thread in the target register R2 is as follows:

[0112] RM2=((RW+w2+wm)×strideW+(RH+h2+hm)×strideH)×ElementByte.

[0113] By substituting the coordinate difference value into the above formula, we can obtain:

[0114] RM2 = ((RW + w1 + deltaW + wm) × strideW + (RH + h1 + deltaH + hm) × strideH) × ElementByte. That is, RM2 = ((RW + w1 + wm) × strideW + (RH + h1 + hm) × strideH) × ElementByte + (deltaW × strideW + deltaH × strideH) × ElementByte.

[0115] Combining the calculation formula for the reference register R1 above, the final calculation formula for RM2 can be derived as follows:

[0116] RM2=RM1+(deltaW×strideW+deltaH×strideH)×ElementByte.

[0117] In this embodiment, the memory offset refers to the address difference between the target register and the base register due to their different locations. Specifically, the coordinate difference values ​​include the H-direction difference deltaH and the W-direction difference deltaW, the step size includes the H-direction step size strideH and the W-direction step size strideW, and the number of bytes of stored data ElementByte is used to convert the coordinate offset into the actual memory address offset.

[0118] Based on the coordinate difference value, step size, and number of bytes of stored data, the formula for calculating the memory offset is: Memory Offset = (deltaW × strideW + deltaH × strideH) × ElementByte. Here, deltaW × strideW represents the memory offset caused by the coordinate difference in the W direction, and deltaH × strideH represents the memory offset caused by the coordinate difference in the H direction. The sum of these two values, multiplied by ElementByte, yields the actual memory address offset in bytes.

[0119] Since the target register and the base register have identical thread layouts, both containing TH×TW threads and sharing the same thread index range, the address difference between the corresponding threads is determined solely by their memory offsets. Therefore, the memory-mapped address of a thread in the target register can be directly obtained by adding the memory offset to the memory-mapped address of the corresponding thread in the base register.

[0120] In this embodiment, by first calculating the memory offset and then accumulating the memory mapping address of the base register, the address mapping of the target register can be quickly derived based on the result of the base register, maximizing the reuse of common calculation logic, fundamentally reducing duplicate instructions, and achieving efficient optimization of address mapping processing.

[0121] In other words, the address mapping algorithm provided in this application embodiment only needs to perform a complete address mapping calculation on the base register after a Tensor is given, while the other registers can be obtained by performing a small amount of multiplication and addition calculation based on the coordinate difference between the base register and the Tensor register.

[0122] Assuming a tensor contains N registers, and each register R requires M instructions for complete address mapping and K instructions for calculating coordinate difference values ​​(where K is much smaller than M), then the traditional address mapping algorithm requires a total of N × M instructions. However, after optimizing the address mapping algorithm using the method provided in this application, the total number of instructions required is M + (N-1) × K. Therefore, the larger N is, the more significant the optimization effect. In other words, this application's embodiment demonstrates a significant optimization effect for tensors containing a large number of registers, greatly enhancing the overall performance of the assembler.

[0123] In one embodiment, address mapping processing and memory initialization operations are performed concurrently with the computation of pre-operators, which are operators that are executed before reduction computation.

[0124] In this embodiment, the pre-operator is executed before the reduction computation. Address mapping and memory initialization are necessary pre-operations for the reduction computation, but they are independent of the computation logic of the pre-operator. They do not need to wait for the pre-operator to complete before initiating address mapping and initialization, nor do they need to wait for address mapping and initialization to complete before executing the pre-operator. Therefore, the execution processes of these three operations can be overlapped. While the pre-operator processes data, address mapping and memory initialization are performed synchronously. This means that they can be executed in parallel and overlapped through a pipeline, eliminating the time-consuming overlap of address mapping, memory initialization, and pre-operator operations, thereby shortening the reduction computation time and improving its efficiency.

[0125] For example, such as Figure 8 As shown, in this embodiment, the correct target initialization value can be selected according to the operation type of Reduce itself, and the result of mapping each register Thread to memory address can be fully calculated. The result is also stored in another designated register (referred to as RM in this embodiment). The calculation process of register Thread to memory address mapping generated by memory initialization can be pipelined with other operator parts before the Reduce operator.

[0126] If the pre-operator ( Figure 8 The computation time of OP1 is 15ms, and the address mapping and memory initialization take 10ms. The total time for serial execution is 15 + 10 = 25ms. However, in parallel overlapping execution, the two can be executed simultaneously. When the pre-operator is completed after 15ms, the address mapping and initialization are also completed synchronously, and the total time is only 15ms. Without affecting the correctness of the calculation, the pre-preparation time of the reduction calculation is embedded into the existing process, which significantly improves the overall execution efficiency of the assembler.

[0127] By comparing the results before and after optimization, it can be seen that the optimization method for the reduction calculation provided in this application integrates memory initialization with the Reduce execution process. This not only allows the pipeline to mask the time consumption of initialization itself, but also allows the consistency between initialization and Reduce memory addresses to complete the Reduce address mapping calculation in advance, thus avoiding the generation of a large number of repetitive calculation instructions.

[0128] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0129] Based on the same inventive concept, this application also provides an optimization apparatus for reducing computation to implement the optimization method for reducing computation as described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the optimization apparatus for reducing computation provided below can be found in the limitations of the optimization method for reducing computation described above, and will not be repeated here.

[0130] In one exemplary embodiment, such as Figure 9 As shown, an optimization device 900 for reduction calculation is provided, comprising: a first determining module 902, a second determining module 904, and a calculation module 906, wherein:

[0131] The first determining module 902 is used to perform address mapping processing on the target register and determine the memory mapping address corresponding to each thread in the target register.

[0132] The second determining module 904 is used to determine the target initialization value based on the operation type calculated according to the protocol.

[0133] The calculation module 906 is used to initialize the data stored at the memory-mapped addresses corresponding to each thread to the target initialization value, and perform reduction calculation to obtain the reduction calculation result.

[0134] In one embodiment, determining the target initialization value based on the operation type calculated according to the specification includes at least one of the following:

[0135] When the operation type of the reduction calculation is SUM, the target initialization value is determined to be 0;

[0136] When the operation type of the reduction calculation is MAX, the target initial value is determined to be negative infinity;

[0137] When the operation type of the reduction calculation is MIN, the target initialization value is determined to be positive infinity.

[0138] In one embodiment, the device further includes:

[0139] The mapping module is used to store the memory mapping addresses corresponding to each thread in the target register in a specified register;

[0140] The process of performing the reduction calculation to obtain the reduction calculation result includes:

[0141] Read the memory-mapped address of each thread from the specified register;

[0142] The reduction calculation is performed based on the memory mapping address and the data stored in each thread to obtain the reduction calculation result.

[0143] In one embodiment, the step of performing address mapping processing on the target register to determine the memory mapping address corresponding to each thread in the target register includes:

[0144] Determine the reference register, and based on the thread index of each thread in the reference register, the position information of the reference register in the tensor data, the memory offset, the stride, and the number of bytes of stored data, determine the memory mapping address corresponding to each thread in the reference register;

[0145] Determine the coordinate difference between the target register and the reference register in the tensor data;

[0146] The memory mapping address of each thread in the target register is determined based on the memory mapping address of each thread in the base register and the coordinate difference value.

[0147] In one embodiment, determining the memory mapping address of each thread in the target register based on the memory mapping address of each thread in the base register and the coordinate difference value includes:

[0148] The memory offset is determined based on the coordinate difference value, the step size, and the number of bytes of the stored data;

[0149] The memory mapping address of each thread in the target register is determined based on the memory mapping address of each thread in the base register and the memory offset.

[0150] In one embodiment, the address mapping process and memory initialization operation are executed in parallel and overlap with the calculation process of the pre-operator, which is an operator that is executed before the reduction calculation.

[0151] The optimized apparatus for reduction computation provided in this application improves the efficiency of Reduce computation by optimizing the memory address mapping process and memory initialization process, thereby avoiding the generation of a large number of repetitive computation instructions.

[0152] Each module in the aforementioned optimization device for specification calculation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0153] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an optimized method for reduction computation. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0154] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An optimization method for reducing computation, characterized by, The method comprises: address mapping processing is performed on the target register according to coordinate information of the target register in tensor data, a memory step and a memory offset, and memory mapping addresses corresponding to threads in the target register are determined; a target initialization value is determined according to an operation type of the reduction calculation; data stored in the memory mapping addresses corresponding to the threads are initialized as the target initialization value, and the reduction calculation is performed to obtain a reduction calculation result.

2. The method of claim 1, wherein, The target initialization value is determined according to the operation type of the reduction calculation, and at least one of the following is included: when the operation type of the reduction calculation is a SUM type, the target initialization value is determined as 0; when the operation type of the reduction calculation is a MAX type, the target initialization value is determined as negative infinity; when the operation type of the reduction calculation is a MIN type, the target initialization value is determined as positive infinity.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: the memory mapping addresses corresponding to the threads in the target register are stored in a specified register; The reduction calculation is performed to obtain a reduction calculation result, and the method comprises: the memory mapping addresses of the threads are read from the specified register; the reduction calculation is performed based on the memory mapping addresses and data stored in the threads to obtain a reduction calculation result.

4. The method of claim 1, wherein, The address mapping processing is performed on the target register according to coordinate information of the target register in tensor data, a memory step and a memory offset, and memory mapping addresses corresponding to threads in the target register are determined, and the method comprises: a reference register is determined, and memory mapping addresses corresponding to threads in the reference register are determined based on thread indexes of the threads in the reference register, position information of the reference register in tensor data, a memory offset, a step and a byte number of stored data; a coordinate difference value of the target register and the reference register in the tensor data is determined; memory mapping addresses corresponding to the threads in the target register are determined according to the memory mapping addresses corresponding to the threads in the reference register and the coordinate difference value.

5. The method of claim 4, wherein, The memory mapping addresses corresponding to the threads in the target register are determined according to the memory mapping addresses corresponding to the threads in the reference register and the coordinate difference value, and the method comprises: a memory offset is determined according to the coordinate difference value, the step and the byte number of the stored data; the memory mapping addresses corresponding to the threads in the target register are determined according to the memory mapping addresses corresponding to the threads in the reference register and the memory offset.

6. The method of claim 1, wherein, The address mapping processing and the memory initialization operation are executed in parallel with a calculation process of a preceding operator, and the preceding operator is an operator executed before the reduction calculation.

7. An optimization device for reduction computation, characterized in that, The device comprises: a first determination module configured to perform address mapping processing on a target register according to coordinate information of the target register in tensor data, a memory step and a memory offset, and to determine memory mapping addresses corresponding to threads in the target register; a second determination module configured to determine a target initialization value according to an operation type of a reduction calculation; a third determination module configured to initialize data stored in the memory mapping addresses corresponding to the threads as the target initialization value, and to perform the reduction calculation to obtain a reduction calculation result. A computing module is configured to initialize data stored in the memory mapping address corresponding to each thread to the target initialization value, and perform the reduction computation to obtain a reduction computation result.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor, when executing the computer program, implements the steps of the method in any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method in any one of claims 1 to 6. The computer program, when executed by the processor, implements the steps of the method in any one of claims 1 to 6.

Images