An operator execution method, apparatus, device, storage medium, and product

Abstract

The application discloses an operator execution method and device, equipment, a storage medium and a product. The method comprises the following steps: applying for a preset number of buffer zones for executing a thread of the operator; wherein the preset number is greater than or equal to 3; for any buffer zone, target data elements in a thread local register array are sequentially sent to the buffer zone by the thread to be accumulated, and an accumulated result in the buffer zone is obtained; wherein the target data elements are data elements corresponding to thread local register indexes in the thread local register array and spaced by the preset number; and the accumulated results of the corresponding buffer zones are accumulated by the thread to obtain a row accumulated result of each row or a column accumulated result of each column in the thread local register array. The embodiment of the application can reduce the execution time of the operator on a chip and improve the performance of the operator.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an operator execution method, apparatus, device, storage medium and product. Background Technology

[0002] During the execution of the chip's fusion operator, it relies on the Thread Local Register (TLR) in the chip's hardware architecture to complete the reduction calculation of the data in the row or column direction (e.g., finding the maximum value, minimum value, or cumulative sum of a row or column), and finally obtains a reduction value for each row or column of the TLR array.

[0003] In existing fusion operator execution schemes, the reduction computation within a thread is performed using two TLRs, resulting in severe data dependencies: subsequent hardware instructions must wait for the computation result of the previous hardware instruction to return before they can start. This lengthens the waiting time for a single hardware instruction, leading to an excessively long total execution time of the operator on the chip, severely limiting operator performance. Summary of the Invention

[0004] This application provides an operator execution method, apparatus, device, storage medium, and product to solve the problem in the prior art where the reduction calculation within a thread has a serious data dependency, which prolongs the waiting time of a single hardware instruction, resulting in an excessively long total execution time of the operator on the chip, severely restricting the performance of the operator.

[0005] To achieve the above objectives, embodiments of this application provide an operator execution method, including:

[0006] For each thread executing the operator, a preset number of buffers are requested; wherein the preset number is ≥3;

[0007] For any of the buffers, the target data elements in the thread local register array are sequentially sent to the buffer for accumulation by the thread to obtain the accumulation result in the buffer; wherein, the target data element is the data element in the thread local register array that is spaced apart by the preset number of thread local register indices;

[0008] The thread accumulates the corresponding buffer's accumulated results to obtain the row-level accumulated result or the column-level accumulated result of each row in the thread's local register array.

[0009] As an improvement to the above scheme, the preset quantity is determined according to any one of the following:

[0010] The order of the thread local register indices in the thread local register array;

[0011] The number of thread-local registers in the thread-local register array.

[0012] As an improvement to the above scheme, for any of the buffers, the process of sequentially sending the target data elements in the thread-local register array to the buffer for accumulation by the thread to obtain the accumulation result in the buffer includes:

[0013] During the current accumulation instruction cycle of the thread, the preset number of thread local register indices are confirmed, and the data elements corresponding to the confirmed thread local register indices in the thread local register array are sent to the buffer of the target buffer index for accumulation until the last accumulation instruction cycle ends, so as to obtain the accumulation result of accumulating the target data elements in each buffer.

[0014] The target buffer index is the remainder when the thread-local register index is divided by the preset number.

[0015] As an improvement to the above scheme, the step of accumulating the accumulation results of the corresponding buffers by the thread to obtain the row accumulation result of each row or the column accumulation result of each column in the thread local register array includes:

[0016] The thread sends the accumulated results belonging to the same row to the same buffer for accumulation, thereby obtaining the row accumulation result in the thread's local register array; or...

[0017] The thread sends the accumulated results belonging to the same column to the same buffer for accumulation, thereby obtaining the column accumulation result in the thread's local register array.

[0018] As an improvement to the above solution, the method further includes:

[0019] The thread sends the row or column accumulation result in the thread local register array to the same buffer for accumulation, thereby obtaining the row and column reduction result in the thread local register array.

[0020] As an improvement to the above scheme, the method further includes: performing cross-thread reduction calculations using the row accumulation result or column accumulation result in the thread local register array.

[0021] To achieve the above objectives, embodiments of this application also provide an operator execution apparatus, comprising:

[0022] The application module is used to apply for a preset number of buffers for the thread executing the operator; wherein the preset number is ≥3;

[0023] The first accumulation module is used to, for any one of the buffers, sequentially send target data elements from the thread local register array to the buffer for accumulation by the thread, and obtain the accumulation result in the buffer; wherein, the target data element is the data element in the thread local register array that corresponds to the thread local register index at a predetermined interval;

[0024] The second accumulation module is used to accumulate the accumulation results of the corresponding buffer through the thread to obtain the row accumulation result of each row or the column accumulation result of each column in the thread local register array.

[0025] To achieve the above objectives, embodiments of this application also provide an operator execution device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the operator execution method as described above when executing the computer program.

[0026] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the operator execution method as described above.

[0027] To achieve the above objectives, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implements the operator execution method as described above.

[0028] Compared with existing technologies, the present application provides an operator execution method, apparatus, device, storage medium, and product. This involves allocating a preset number of buffers for the thread executing the operator; wherein the preset number is ≥3. For any given buffer, the thread sequentially sends target data elements from the thread local register array to that buffer for accumulation, obtaining an accumulation result within the buffer. The target data elements are data elements in the thread local register array corresponding to thread local register indices spaced apart by the preset number. By accumulating the accumulation results of the corresponding buffers, the thread obtains the row accumulation result for each row or the column accumulation result for each column in the thread local register array. Therefore, the present application provides at least three buffers for the thread to perform the accumulation operation within the thread, reducing data dependencies, shortening the waiting time for a single hardware instruction, thereby reducing the total execution time of the operator on the chip and improving operator performance. Attached Figure Description

[0029] Figure 1This is a flowchart of an operator execution method provided in an embodiment of this application;

[0030] Figure 2 This is a structural block diagram of an artificial intelligence chip provided in an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of the layout of a TLR array provided in an embodiment of this application;

[0032] Figure 4 This is another layout schematic diagram of a TLR array provided in an embodiment of this application;

[0033] Figure 5 This is a structural block diagram of an operator execution device provided in an embodiment of this application;

[0034] Figure 6 This is a structural block diagram of an operator execution device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0036] In the description of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0037] In this application description, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0038] In this application description, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The term "based on" means "at least partially based on." The term "according to" means "at least partially according to." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." The term "and / or" means at least one of the connected objects, such as A and / or B, indicating three cases: including only A, only B, and both A and B. Unless otherwise stated, the term "multiple" means two or more.

[0039] refer to Figure 2 This is a structural diagram of an artificial intelligence chip applicable to the embodiments of this application. The hardware unit of the artificial intelligence chip includes at least: a computing core, a TLR, and a buffer. The computing core is used to execute the operator execution method provided in the embodiments of this application. Each thread scheduled by the computing core has its own TLR and completes the reduction calculation of the operator on the TLR array by accumulating in the buffer.

[0040] In addition to the structures described above, the artificial intelligence chip in this application may also include other structures, and this application does not specifically limit such structures.

[0041] The artificial intelligence chip can be any one of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose computing on Graphics Processing Unit), depending on its application to a specific product or technology in the embodiments of this application.

[0042] It is worth noting that, such as Figure 3 (The diagram only shows the specific layout of 4 TLRs; for distinction, each color represents one TLR. The others are similar and will not be shown again.) A TLR array consists of multiple TLRs, and each TLR can store 8... There are 4 data elements, namely K0, K1, ..., K 31 Each data element corresponds to one thread, so 32 data elements correspond to 32 threads. For example, for the blue TLR, data element K0 corresponds to thread 0, data element K1 corresponds to thread 1, and so on; similarly, for the yellow TLR, data element K0 corresponds to thread 0, data element K1 corresponds to thread 1, and so on. A data element refers to the data that one thread can store in a TLR, which is generally 4 bytes. For single-precision floating-point type (fp32 type) data, one thread can store one fp32 data in a TLR, and for half-precision floating-point type (fp16 type) data, one thread can store two fp16 data in a TLR.

[0043] The layout information of a TLR is deterministic, and the order of its TLR indices is also fixed. Data elements in the TLR array can be read through the TLR indices. For example, for 64... A 16-TLR array, its layout is as follows Figure 4 The TLR indexes are arranged in the order of two rows and two columns within the TLR array. The TLR indexes are numbered 0 to 31 according to this order. To better distinguish them from the buffer indexes, they can be denoted as R0 to R31.

[0044] In related technologies, the intra-thread row reduction calculation steps are as follows: Initialize two TLRs: Rdst0 and Rdst1; for each thread, in the row direction, send the data elements corresponding to that thread in the TLR array sequentially to Rdst0 or Rdst1 for accumulation, obtaining the row accumulation result of each row in the TLR array. For example, in... Figure 4 In a TLR array layout, the in-thread row reduction calculation steps are performed by 32 threads (i.e., thread i calculates):

[0045] Rdst0 = K i R0 +K i R1 +K i R4 +K i R5 + … + K i R28 +K i R29 ;

[0046] Rdst1 = K i R2 +K i R3 +K i R6 +K i R7 + … + K i R30 +K i R31 Specific implementation:

[0047] Rdst00 = 0;

[0048] Rdst10 = 0;

[0049] For Rdst00:

[0050] Rdst01 = Rdst00 + K i R0 ;

[0051] Rdst02 = Rdst01 + K i R1 ;

[0052] …

[0053] Rdst0 15 = Rdst0 14 +K i R29 ;

[0054] For Rdst10:

[0055] Rdst11 = Rdst10 + K i R2 ;

[0056] Rdst12 = Rdst11 + K i R3 ;

[0057] …

[0058] Rdst1 15 = Rdst1 14 + K i R31 ;

[0059] In the formula, the index of Rdst0 indicates the number of times Rdst0 is accumulated, and the index of Rdst1 indicates the number of times Rdst1 is accumulated; K i K represents the data element corresponding to thread i in a TLR within a TLR array. i The superscript indicates the TLR index in the TLR array, such as K. i R0 This represents the data element corresponding to thread i in the TLR with TLR index R0 in the TLR array.

[0060] In the above calculation process, each accumulation requires waiting for the result of the previous accumulation. For example, when calculating Rdst02 = Rdst01 + K... i R1 Previously, we needed to wait for Rdst01 = Rdst00 + K. i R0 Execution can only continue after the result of this hardware instruction returns, and the same applies to other hardware instructions. This results in a significant waiting time between each hardware instruction, indicating a clear and serious data dependency. Consequently, the total execution time of the operator on the chip is long, leading to poor operator performance.

[0061] See Figure 1 , Figure 1 This is a flowchart of an operator execution method provided in an embodiment of this application. The operator execution method includes:

[0062] S1. For the thread executing the operator, allocate a preset number of buffers; wherein, the preset number is ≥3;

[0063] S2. For any of the buffers, the target data elements in the thread local register array are sequentially sent to the buffer for accumulation by the thread to obtain the accumulation result in the buffer; wherein, the target data element is the data element in the thread local register array that is spaced apart by the preset number of thread local register indices;

[0064] S3. The thread accumulates the corresponding buffer's accumulation result to obtain the row accumulation result of each row or the column accumulation result of each column in the thread's local register array.

[0065] In this embodiment, a preset number (≥3) of buffers are allocated for each thread. The thread sequentially sends data elements corresponding to TLR indices at preset intervals in the TLR array to the same buffer for accumulation, obtaining the accumulation result of that buffer. Finally, the accumulation results from each buffer are summed to obtain the row-level accumulation result or column-level accumulation result of each row in the TLR array. Therefore, this embodiment uses a space-for-time tradeoff method, allocating at least 3 buffers, accumulating data elements in the buffers, and storing intermediate results of reduction calculations within the thread. This reduces data dependencies, shortens the waiting time for a single hardware instruction, thereby reducing the total execution time of the operator on the chip and improving operator performance.

[0066] The preset number of thread local register indices refers to the difference between the TLR indices at the preset interval, i.e., the next TLR index = the current TLR index + the preset number.

[0067] In one alternative embodiment, the preset quantity is determined according to any of the following:

[0068] The order of the thread local register indices in the thread local register array;

[0069] The number of thread-local registers in the thread-local register array.

[0070] In this embodiment, the number of buffers to be requested (i.e., the preset number) is determined according to the TLR index arrangement order in the TLR array or the size of the TLR array, to ensure the smooth accumulation of subsequent buffers.

[0071] For row accumulation, for example, the TLR indexes are arranged in the order of 2 rows first and 2 columns first, that is, 4 TLR indexes form a cycle. Therefore, the preset number is set to a multiple of 4 of this cycle to ensure the smooth progress of subsequent buffer accumulation.

[0072] For column accumulation, for example, if the number of TLRs in a certain TLR array is 32, then the preset number is set to 32 to ensure that subsequent buffer accumulation proceeds smoothly.

[0073] In an optional embodiment, for any of the buffers, the step of sequentially sending target data elements from the thread-local register array to the buffer for accumulation by the thread to obtain the accumulation result in the buffer includes:

[0074] During the current accumulation instruction cycle of the thread, the preset number of thread local register indices are confirmed, and the data elements corresponding to the confirmed thread local register indices in the thread local register array are sent to the buffer of the target buffer index for accumulation until the last accumulation instruction cycle ends, so as to obtain the accumulation result of accumulating the target data elements in each buffer.

[0075] The target buffer index is the remainder when the thread-local register index is divided by the preset number.

[0076] In this embodiment, within the current accumulation instruction cycle of the thread, a preset number of thread-local register indices to be accumulated are identified. The data elements corresponding to the identified thread-local register indices in the thread-local register array are sent to the buffer of the target buffer index for accumulation, enabling the thread to perform a preset number of accumulation operations within the accumulation instruction cycle. This embodiment relies on a buffer to store intermediate results of the thread's reduction computation, which reduces data dependencies, significantly reduces the waiting time of a single hardware instruction, thereby reducing the total execution time of the operator on the chip and improving operator performance.

[0077] For example, if the preset quantity is 4, and there are 4 buffers with indices 0, 1, 2, and 3 respectively; then within one accumulation instruction cycle n (n is an integer), the 4 TLR indices are confirmed as 4n, 4n+1, 4n+2, and 4n+3. Within this accumulation instruction cycle n, the data element corresponding to TLR index 4n is sent to the buffer with buffer index 0 for accumulation, the data element corresponding to TLR index 4n+1 is sent to the buffer with buffer index 1 for accumulation, the data element corresponding to TLR index 4n+2 is sent to the buffer with buffer index 2 for accumulation, and the data element corresponding to TLR index 4n+3 is sent to the buffer with buffer index 3 for accumulation, so that 4 accumulation operations are performed within one accumulation instruction cycle.

[0078] In other words, thread i sequentially sends the data elements corresponding to TLR indices 0, 4, 8, ..., 4n at intervals of 4 to the buffer with index 0 for accumulation; thread i sequentially sends the data elements corresponding to TLR indices 1, 5, 9, ..., 4n+1 at intervals of 4 to the buffer with index 1 for accumulation; thread i sequentially sends the data elements corresponding to TLR indices 2, 6, 10, ..., 4n+2 at intervals of 4 to the buffer with index 2 for accumulation; and thread i sequentially sends the data elements corresponding to TLR indices 3, 7, 11, ..., 4n+3 at intervals of 4 to the buffer with index 3 for accumulation.

[0079] This application's embodiments target various types of data (such as word vector matrices, image data matrices, and video data matrices) in scenarios such as text, images, and videos. By using operators to calculate reduction values ​​for each row or column of the data, high-dimensional and large-volume original data is compressed into low-dimensional and small-volume reduction results. This not only preserves the key features of the data and avoids losing core information valuable for subsequent tasks (such as feature fusion and classification detection), but also significantly reduces the amount of data that subsequent chips need to process, improving the hardware's data processing efficiency.

[0080] Taking a text-based scenario as an example, the word vector matrix [sequence length, embedding dimension] generated after the input text is encoded is [128, 768]. The operator described in this application performs a reduction calculation on the sequence length dimension of 128, compressing the word vector matrix into a single 768-dimensional vector. This vector not only retains the core semantics of the entire sentence and can be directly used as input for subsequent tasks such as sentiment analysis and classification detection, but also, due to the significant reduction in data volume, the chip can process more batches of text data with the same hardware resources, improving the hardware's data processing efficiency. Other scenarios are similar and will not be elaborated further here.

[0081] In an optional embodiment, the step of accumulating the accumulation results of the corresponding buffers by the thread to obtain the row accumulation result of each row or the column accumulation result of each column in the thread local register array includes:

[0082] The thread sends the accumulated results belonging to the same row to the same buffer for accumulation, thereby obtaining the row accumulation result in the thread's local register array; or...

[0083] The thread sends the accumulated results belonging to the same column to the same buffer for accumulation, thereby obtaining the column accumulation result in the thread's local register array.

[0084] In this embodiment, the accumulated results belonging to the same row or column are sent to the same buffer for accumulation, and finally the row accumulation result of each row or the column accumulation result of each column in the TLR array is obtained, thus completing the row reduction or column reduction within the thread.

[0085] For example, in Figure 4 The TLR array layout applies the operator execution method provided in this application embodiment. Based on the order of its TLR indices, four buffers are allocated, including the first buffer tmpTLR0, the second buffer tmpTLR1, the third buffer tmpTLR2, and the fourth buffer tmpTLR3. The following is the execution process for each thread:

[0086] first step:

[0087] tmpTLR0 / 1 / 2 / 3 = 0;

[0088] tmpTLR0 / 1 / 2 / 3 = tmpTLR0 / 1 / 2 / 3 + K i R0 / 1 / 2 / 3 ;

[0089] tmpTLR0 / 1 / 2 / 3 = tmpTLR0 / 1 / 2 / 3 + K i R4 / 5 / 6 / 7 ;

[0090] …

[0091] tmpTLR0 / 1 / 2 / 3 = tmpTLR0 / 1 / 2 / 3 + K i R28 / 29 / 30 / 31 ;

[0092] For ease of understanding, this step is represented in Table 1:

[0093] Table 1

[0094]

[0095] Step Two:

[0096] tmpTLR09= tmpTLR08+ tmpTLR18;

[0097] tmpTLR29= tmpTLR28+ tmpTLR38.

[0098] In the first accumulation instruction cycle, the accumulation instruction tmpTLR01 = tmpTLR00 + K is executed first. i R0 Then execute the accumulation instruction tmpTLR11 = tmpTLR10 + K respectively.i R1 Accumulation instruction tmpTLR21 = tmpTLR20 + K i R2 Accumulation instruction tmpTLR31 = tmpTLR30 + K i R3 The execution process of the remaining accumulation instructions is similar and will not be described in detail here.

[0099] In other words, an accumulation instruction cycle consists of 4 accumulation instructions. Therefore, for the same buffer, instructions with data dependencies are 3 instructions apart from the current instruction. Thus, in terms of chip hardware resources, no additional waiting is required for direct computation, achieving operator optimization. This is because during hardware execution, one instruction is executed every 2 cycles (clock cycles). For example, in cycle 0, tmpTLR01 = tmpTLR00 + K is executed. i R0 Then, in the 8th cycle, tmpTLR02 = tmpTLR01 + K is executed. i R4 The latency (the number of clock cycles required for an instruction to complete) of an instruction is 8 cycles. Therefore, when the instruction is executed in the 8th cycle, the tmpTLR01 of the instruction executed in the 0th cycle is equal to tmpTLR00 + K. i R0 The result has been returned, and no additional waiting time is required, which can minimize the total execution time of the operator on the chip and improve operator performance.

[0100] In an optional embodiment, the method further includes: sending the row accumulation result or column accumulation result in the thread local register array to the same buffer for accumulation by the thread, so as to obtain the row and column reduction result in the thread local register array.

[0101] This application embodiment can also utilize row accumulation results or column accumulation results to perform row and column reduction, obtaining a global reduction result, further reducing the data volume and improving the efficiency of subsequent data processing. Specifically, a thread sends all row accumulation results of that thread in the thread-local register array to the same buffer for accumulation, obtaining the row and column reduction result in the thread-local register array; or, a thread sends all column accumulation results of that thread in the thread-local register array to the same buffer for accumulation, obtaining the row and column reduction result in the thread-local register array.

[0102] In an optional embodiment, the method further includes: performing cross-thread reduction calculations using row accumulation results in the thread-local register array.

[0103] This application embodiment can utilize the cross-thread reduction capability provided by the hardware to achieve reduction calculations between threads. For example, when calculating the maximum and minimum values, the redmatrix (load) instruction can be used for related calculations, and when calculating the cumulative sum, the shuff (shuffle) instruction can be used for related calculations.

[0104] See Figure 5 , Figure 5 This is a structural block diagram of an operator execution device 10 provided in an embodiment of this application. The operator execution device 10 includes:

[0105] Application module 11 is used to apply for a preset number of buffers for the thread executing the operator; wherein the preset number is ≥3;

[0106] The first accumulation module 12 is used to, for any of the buffers, sequentially send target data elements in the thread local register array to the buffer for accumulation by the thread, and obtain the accumulation result in the buffer; wherein, the target data element is the data element in the thread local register array that is spaced apart by the preset number of thread local register indices;

[0107] The second accumulation module 13 is used to accumulate the accumulation results of the corresponding buffer through the thread to obtain the row accumulation result of each row or the column accumulation result of each column in the thread local register array.

[0108] Optionally, the preset quantity is determined according to any one of the following:

[0109] The order of the thread local register indices in the thread local register array;

[0110] The number of thread-local registers in the thread-local register array.

[0111] Optionally, for any of the buffers, the step of sequentially sending target data elements from the thread-local register array to the buffer for accumulation by the thread to obtain the accumulation result in the buffer includes:

[0112] During the current accumulation instruction cycle of the thread, the preset number of thread local register indices are confirmed, and the data elements corresponding to the confirmed thread local register indices in the thread local register array are sent to the buffer of the target buffer index for accumulation until the last accumulation instruction cycle ends, so as to obtain the accumulation result of accumulating the target data elements in each buffer.

[0113] The target buffer index is the remainder when the thread-local register index is divided by the preset number.

[0114] Optionally, the step of accumulating the accumulated results of the corresponding buffers through the thread to obtain the row-level accumulated result of each row or the column-level accumulated result of each column in the thread-local register array includes:

[0115] The thread sends the accumulated results belonging to the same row to the same buffer for accumulation, thereby obtaining the row accumulation result in the thread's local register array; or...

[0116] The thread sends the accumulated results belonging to the same column to the same buffer for accumulation, thereby obtaining the column accumulation result in the thread's local register array.

[0117] Optionally, the device further includes:

[0118] The third accumulation module is used to send the row accumulation result or column accumulation result in the thread local register array to the same buffer for accumulation through the thread, so as to obtain the row and column reduction result in the thread local register array.

[0119] Optionally, the device further includes:

[0120] Cross-thread computation is used to perform cross-thread reduction computation using the row or column accumulation results in the thread-local register array.

[0121] It is worth noting that the working process of each module in the operator execution device 10 described in this application embodiment can refer to the working process of the operator execution method described in the above embodiment, and will not be repeated here.

[0122] An operator execution device 10 provided in this application embodiment allocates a preset number of buffers for the thread executing the operator; wherein the preset number is ≥3; for any buffer, the thread sequentially sends target data elements in the thread local register array to the buffer for accumulation, obtaining the accumulation result in the buffer; wherein the target data element is the data element in the thread local register array corresponding to the thread local register index at intervals of the preset number; by the thread accumulating the accumulation results of the corresponding buffers, the row accumulation result of each row or the column accumulation result of each column in the thread local register array is obtained. Therefore, this application embodiment allocates at least 3 buffers for the thread to perform the accumulation operation within the thread, which can reduce data dependencies, shorten the waiting time of a single hardware instruction, thereby reducing the total execution time of the operator on the chip and improving operator performance.

[0123] Furthermore, this application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the operator execution method as described in any of the above embodiments.

[0124] Furthermore, this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the operator execution method as described in any of the above embodiments.

[0125] See Figure 6 , Figure 6 This is a structural block diagram of an operator execution device 20 provided in an embodiment of this application. The operator execution device 20 includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above-described operator execution method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments.

[0126] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the operator execution device 20.

[0127] The operator execution device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the operator execution device 20 and does not constitute a limitation on the operator execution device 20. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the operator execution device 20 may also include input / output devices, network access devices, buses, etc.

[0128] The processor 21 can be any one of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural Network Processing Unit), DPU (Deep Learning Processing Unit), APU (Accelerated Processing Unit), and GPGPU (General-Purpose Computing on Graphics Processing Unit). The processor 21 is the control center of the electronic device, connecting various parts of the electronic device via various interfaces and lines.

[0129] The memory 22 can be used to store the computer program and / or modules. The processor 21 implements various functions of the operator execution device 20 by running or executing the computer program and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store related data, etc. In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0130] If the modules / units integrated in the operator execution device 20 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0131] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0132] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. An operator execution method, characterized in that, include: For each thread executing the operator, a preset number of buffers are requested; wherein the preset number is ≥3; For any of the buffers, the target data elements in the thread local register array are sequentially sent to the buffer by the thread for accumulation, and the accumulation result in the buffer is obtained; wherein, the target data element is the data element in the thread local register array that is spaced apart by the preset number of thread local register indices, and the difference between the thread local register indices corresponding to two target data elements received sequentially in the same buffer is the preset number. The thread accumulates the corresponding buffer's accumulated results to obtain the row-level accumulated result or the column-level accumulated result of each row in the thread's local register array.

2. The operator execution method as described in claim 1, characterized in that, The preset quantity is determined according to any one of the following: The order of the thread local register indices in the thread local register array; The number of thread-local registers in the thread-local register array.

3. The operator execution method as described in claim 1, characterized in that, For any of the buffers, the thread sequentially sends the target data elements from the thread-local register array to the buffer for accumulation, obtaining the accumulation result in the buffer, including: During the current accumulation instruction cycle of the thread, the preset number of thread local register indices are confirmed, and the data elements corresponding to the confirmed thread local register indices in the thread local register array are sent to the buffer of the target buffer index for accumulation until the last accumulation instruction cycle ends, so as to obtain the accumulation result of accumulating the target data elements in each buffer. The target buffer index is the remainder when the thread-local register index is divided by the preset number.

4. The operator execution method as described in claim 1, characterized in that, The step of accumulating the accumulated results of the corresponding buffers through the thread to obtain the row-level accumulated result of each row or the column-level accumulated result of each column in the thread-local register array includes: The thread sends the accumulated results belonging to the same row to the same buffer for accumulation, thereby obtaining the row accumulation result in the thread's local register array; or... The thread sends the accumulated results belonging to the same column to the same buffer for accumulation, thereby obtaining the column accumulation result in the thread's local register array.

5. The operator execution method as described in claim 1, characterized in that, The method further includes: The thread sends the row or column accumulation result in the thread local register array to the same buffer for accumulation, thereby obtaining the row and column reduction result in the thread local register array.

6. The operator execution method as described in claim 1, characterized in that, The method further includes: performing cross-thread reduction calculations using the row accumulation results or column accumulation results in the thread-local register array.

7. An operator execution device, characterized in that, include: The application module is used to apply for a preset number of buffers for the thread executing the operator; wherein the preset number is ≥3; The first accumulation module is used to, for any buffer, sequentially send target data elements in the thread local register array to the buffer for accumulation by the thread, and obtain the accumulation result in the buffer; wherein, the target data element is the data element in the thread local register array that is spaced apart by the preset number of thread local register indices, and the difference between the thread local register indices corresponding to two target data elements received sequentially in the same buffer is the preset number. The second accumulation module is used to accumulate the accumulation results of the corresponding buffer through the thread to obtain the row accumulation result of each row or the column accumulation result of each column in the thread local register array.

8. An operator execution device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the operator execution method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the operator execution method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the operator execution method as described in any one of claims 1 to 6.

Images