Intelligent processing unit and three-dimensional pooling calculation method

Abstract

The present application discloses a smart processing unit and a three-dimensional pooling operation method, which is used to operate an input tensor to generate an output tensor, wherein the input tensor includes a plurality of input data blocks, and the output tensor includes a plurality of output data blocks. The method includes: reading one of the plurality of input data blocks from an external memory as a target input data block, and storing the target input data block in the memory; reading the target input data block from the memory; performing a first two-dimensional pooling operation R times on the target input data block to generate an intermediate tensor, where R is a positive integer; performing a second two-dimensional pooling operation once on the intermediate tensor to generate a target output data block of a plurality of output data blocks; and storing the target output data block in the memory. By performing two two-dimensional pooling operations to reduce the amount of data for processing the output tensor, the efficiency of the pooling operation is improved, and it has the advantages of low complexity and low cost.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to an intelligent processing unit and a three-dimensional pooling operation method. Background Art

[0002] Convolutional neural networks are one of the common technologies in the current field of artificial intelligence. Convolutional neural networks include convolution operations and pooling operations. The main purpose of pooling operations is to reduce the amount of output data (tensor) of convolution operations. For electronic devices that do not include an intelligence processing unit (IPU) (for example, image processing chips or circuits), pooling operations are usually performed by a central processing unit (CPU) or a graphics processing unit (GPU), but the CPU and GPU are not dedicated to pooling operations, so the efficiency is not high. However, implementing an IPU in an electronic device will increase the complexity and cost of the electronic device. Therefore, designing a low-complexity and / or low-cost IPU is an important topic in this field. Summary of the invention

[0003] The embodiments of the present application provide a smart processing unit and a three-dimensional pooling operation method to improve the efficiency of pooling operation.

[0004] In a first aspect, an embodiment of the present application provides a three-dimensional pooling operation method for performing operations on an input tensor to generate an output tensor, wherein the input tensor includes a plurality of input data blocks, and the output tensor includes a plurality of output data blocks. The method includes: reading one of the plurality of input data blocks from an external memory as a target input data block, and storing the target input data block into the memory; reading the target input data block from the memory; performing a first two-dimensional pooling operation R times on the target input data block to generate an intermediate tensor, where R is a positive integer; performing a second two-dimensional pooling operation once on the intermediate tensor to generate a target output data block of a plurality of output data blocks; and storing the target output data block into the memory.

[0005] In a second aspect, an embodiment of the present application also provides an intelligent processing unit for processing an input tensor and generating an output tensor, wherein the input tensor includes a plurality of input data blocks, and the output tensor includes a plurality of output data blocks. The intelligent processing unit includes: a memory, a direct memory access unit, and a computing circuit. The direct memory access unit is used to read one of the plurality of input data blocks from an external memory as a target input data block, and store the target input data block into the memory. The computing circuit is used to perform the following operations to perform a three-dimensional pooling operation on the target input data block, and the three-dimensional pooling operation method is used to generate a target output data block of a plurality of output data blocks: read the target input data block from the memory; perform a first two-dimensional pooling operation R times on the target input data block to generate an intermediate tensor, where R is a positive integer; and, perform a second two-dimensional pooling operation on the intermediate tensor once to generate a target output data block.

[0006] In the embodiment of the present application, two-dimensional pooling operations are performed twice by using a circuit with low complexity and cost to equivalently implement a three-dimensional pooling operation. In this way, the efficiency of the pooling operation can be improved without significantly increasing the complexity and / or cost of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The features, implementation and effects of the present application are described in detail below with reference to the accompanying drawings.

[0008] Figure 1 Functional block diagram of an electronic device provided for implementation of the present application;

[0009] Figure 2 A schematic diagram of input data, intermediate data, and output data of a pooling operation provided for the implementation of this application;

[0010] Figure 3 A schematic diagram of input tensors or input data blocks provided for implementation of this application;

[0011] Figure 4 A flowchart of a three-dimensional pooling operation provided for the implementation of this application;

[0012] Figure 5 for Figure 4 Detailed flowchart of step S440;

[0013] Figure 6 for Figure 4 Another detailed flowchart of step S440 in FIG. 1 ; and

[0014] Figure 7 for Figure 4 Detailed flowchart of step S450. DETAILED DESCRIPTION

[0015] The technical terms used in the following description refer to the customary terms in the technical field. If some terms are explained or defined in this specification, the interpretation of these terms shall be based on the explanation or definition in this specification.

[0016] The disclosure of this application includes an intelligent processing unit and a three-dimensional pooling operation method. Since some of the components included in the intelligent processing unit of this application may be known components individually, the details of the known components will be omitted in the following description without affecting the full disclosure and feasibility of the device invention. In addition, part or all of the process of the three-dimensional pooling operation method of this application may be in the form of software and / or firmware, and can be executed by the intelligent processing unit of this application or its equivalent device. Without affecting the full disclosure and feasibility of the method invention, the following description of the method invention will focus on the step content rather than the hardware.

[0017] Figure 1 Functional block diagram of an electronic device provided for the implementation of the present application. The electronic device 100 includes a processor 110, a memory 120, and an intelligent processing unit 130. The processor 110, the memory 120, and the intelligent processing unit 130 are coupled to each other, and the processor 110 and the intelligent processing unit 130 can access the memory 120. The intelligent processing unit 130 includes a direct memory access unit (DMA) 132, a cache memory 134, and a computing circuit 136. The computing circuit 136 includes a neural network computing core, and the neural network computing core includes a convolution core 137 (or called a convolution computing circuit) for performing convolution operations and a vector core 138 (or called a vector computing circuit) for performing pooling operations. The vector core 138 is designed for two-dimensional pooling operations (2D pooling), that is, the vector core 138 is a two-dimensional vector core that performs two-dimensional pooling operations.

[0018] Please continue reading Figure 1 The memory 120 stores the result of the convolution operation of the intelligent processing unit 130, which is the output tensor of the convolution operation and the input tensor TSR_in of the pooling operation. The intelligent processing unit 130 generates an intermediate tensor TSR_imt during the pooling operation on the input tensor TSR_in. The intermediate tensor TSR_imt is stored in the cache memory 134, and the result of the pooling operation (i.e., the output tensor TSR_out) is first stored in the cache memory 134, and then the direct memory access unit 132 writes the output tensor TSR_out to the memory 120. In some embodiments, the processor 110 can obtain the output tensor TSR_out from the memory 120.

[0019] Figure 2Schematic diagram of input data, intermediate data and output data of the pooling operation provided for the implementation of the present application. Input data 210 can be the aforementioned input tensor TSR_in or a part of the input tensor TSR_in (for example, an input data tile); intermediate data 220 is the result of one or more two-dimensional pooling operations on the input data 210, that is, the aforementioned intermediate tensor TSR_imt or a part of the intermediate tensor TSR_imt, and output data 230 is the result of one two-dimensional pooling operation on the intermediate data 220. Output data 230 can be the aforementioned output tensor TSR_out or a part of the output tensor TSR_out (for example, an output data tile).

[0020] Currently, common three-dimensional pooling operations include max pooling and average pooling, as shown in equations (1) and (2) respectively.

[0021] MaxPool3D_out(n,d,h,w,c)=max l＝0，...，Kd-1 max p＝0，...，Kh-1 max q＝0，...，Kw-1 input(n,Sd×d+l,Sh×h+p,Sw×w+q,c)(1)

[0022]

[0023]

[0024] Among them, the five parameters (n, d, h, w, c) represent a point in the output vector ("n", "d", "h", "w", "c" represent batch number (N), depth (D), height (H), width (W) and channel (C) respectively), Kd, ​​Kh and Kw are the sizes of the sliding window in the depth (D), height (H) and width (W) directions respectively, and Sd, Sh and Sw are the moving steps (stride) of the sliding window in the depth (D), height (H) and width (W) directions respectively. The principles of equations (1) and (2) are well known to those skilled in the art, so they are not repeated here.

[0025] The present application analyzes the other program (1) and equation (2) and adjusts them into equation (3) and equation (4) below respectively. It can be seen from equation (3) or equation (4) that the three-dimensional pooling operation is equivalent to the two-dimensional pooling operation plus the one-dimensional pooling operation, for example, the height (H) dimension and the width (W) dimension are processed first, and then the depth (D) dimension is processed.

[0026] MaxPool3D_out(n,d,h,w,c)=max l＝0，...，Kd-1 (max p＝0，...，Kh-1 max q＝0，...，Kw-1 input(n, Sd×d+l, Sh×h+p, Sw×w+q, c)) (3)

[0027]

[0028] Please note that the following examples omit the batch number (N) dimension, however, those skilled in the art can apply the present application to tensors with the batch number (N) dimension based on the following examples.

[0029] Please continue reading Figure 2 . Figure 2 The four dimensions of the data are shown: depth (D), height (H), width (W), and channel (C). For example, the input data 210 includes three sub-tensors (sub-tensor 212, sub-tensor 214, and sub-tensor 216) in the depth (D) dimension, and each sub-tensor is a three-dimensional data (the values ​​of height (H), width (W), and channel (C) are H1, W1, and C1, respectively). Figure 2 As shown, after the two-dimensional pooling operation (for example, from the input data 210 to the intermediate data 220, and from the intermediate data 220 to the output data 230), the size of the channel (C) dimension remains unchanged (all C1).

[0030] In some embodiments, the bit width of the cache memory 134 is related to the channel (C) dimension and the data format of the tensor (including the input tensor TSR_in, the intermediate tensor TSR_imt, and the output tensor TSR_out). For example, if the bit width of the cache memory 134 is 256 bits and the data format of the tensor is INT16 (i.e., one channel contains 16 bits of data), each row of the cache memory 134 can store up to 16 channels. Figure 2 In the example, one row of cache memory 134 stores C1 channels; that is, if H1xW1=10, sub-tensor 212 is stored in rows n to n+9 (n is an integer) of cache memory 134, sub-tensor 214 is stored in rows n+10 to n+19 of cache memory 134, and sub-tensor 216 is stored in rows n+20 to n+29 of cache memory 134. C1 is less than or equal to the bit width of cache memory 134 divided by the data format of the tensor.

[0031] Figure 3 Schematic diagram of input tensors or input data blocks provided for implementation of this application. Figure 3As shown, the input data 310 may be a three-dimensional data: the height (H) dimension is H1, the width (W) dimension is W1, and the channel (C) dimension is C2 (=3xC1). However, because the number of channels C2 is greater than the bit width of the cache memory 134, the input data 310 is arranged in the cache memory 134 as follows: Figure 2 The structure shown, that is, sub-tensor 312 (data of the first C1 channels), sub-tensor 314 (data of the middle C1 channels), and sub-tensor 316 (data of the last C1 channels) are arranged at the positions of sub-tensor 212, sub-tensor 214, and sub-tensor 216 respectively; in other words, in cache memory 134, sub-tensor 314 is immediately after sub-tensor 312, and sub-tensor 316 is immediately after sub-tensor 314. In this case, the computing circuit 136 can know from the values ​​of the parameters of the instruction (including the height (H) parameter, the width (W) parameter, and the channel (C) parameter) that the size of the input data 310 is H1xW1x3C1, and then according to the bit width of the cache memory 134 and the data format of the tensor, it is known that the input data 310 includes three sub-tensors (the number of channels of each sub-tensor is C1). In this way, the computing circuit 136 can read sub-tensor 312, sub-tensor 314 and sub-tensor 316 once based on one instruction (compared to reading them three times, reading them once can save time), and process sub-tensor 312, sub-tensor 314 and sub-tensor 316 in parallel (i.e., processing H1xW1x3C1 data in total).

[0032] Please note that Figure 3 In the example, the target number of channels (i.e., 3C1) is greater than the sum of the number of channels of sub-tensor 312 and the number of channels of sub-tensor 314 (i.e., 2C1). In other embodiments, if input data 310 only includes sub-tensor 312 and sub-tensor 314, but does not include sub-tensor 316, the target number of channels is equal to the sum of the number of channels of sub-tensor 312 and the number of channels of sub-tensor 314 (both are 2C1).

[0033] According to the characteristics of the above-mentioned computing circuit 136, the computing circuit 136 can process the input data 210 as four-dimensional data (for example, when the instruction indicates that the dimension of the input data 210 is [D, H, W, C] = [3, H1, W1, C1]), or process the input data 210 as three-dimensional data (when the instruction indicates that the dimension of the input data 210 is [H, W, C] = [H1, W1, 3C1]).

[0034] In some embodiments, when the cache memory 134 cannot store all the input tensors TSR_in, the present application divides the output tensor TSR_out into a plurality of output data blocks (tiles) according to the size of the cache memory 134 in advance, and then finds the position and size of the corresponding input data block according to the position and size of each output data block. The following equations (3) to (6) show the corresponding relationship between the output data blocks and the input data blocks in the depth direction.

[0035] DoHighest=DoLowest+min(tileDo,Do-DoLowest)-1 (3)

[0036] DiLowest=clip(DoLowest×Sd-padding_depth, 0, Di-1) (4)

[0037] DiHighest=min(DoHighest×Sd-padding_depth+Kd-1, Di-1) (5)

[0038] tileDi=DiHighest-DiLowest+1 (6)

[0039] Wherein, DoLowest is the starting position of the output data block, tileDo is the length of the output data block, DoHighest is the final position of the output data block, DiLowest is the starting position of the input data block, tileDi is the length of the input data block, and DiHighest is the final position of the input data block. A person with ordinary knowledge in the technical field can infer the equations for the height direction and the width direction based on equations (3) to (6), so they are not repeated here. It should be noted that because the pooling operation does not change the dimension value in the channel direction, the starting position and size of the input tensor in the channel dimension are the same as the output tensor.

[0040] In actual operation, the direct memory access unit 132 reads the input data blocks from the memory 120 into the cache memory 134 in sequence, and the computing circuit 136 processes the input data blocks in sequence. However, in other embodiments, if the cache memory 134 can store an entire input tensor TSR_in, the direct memory access unit 132 reads an entire input tensor TSR_in from the memory 120 into the cache memory 134, and the computing circuit 136 processes the entire input tensor TSR_in at one time. Figure 2 The input data 210 may represent an entire input tensor TSR_in (equivalent to regarding the input tensor TSR_in as containing only one input data block) or one of a plurality of input data blocks of the input tensor TSR_in.

[0041] Please note that the present invention does not limit the partitioning method of the output tensor TSR_out. In some embodiments, the partitioning method may be determined based on the memory bandwidth required for performing a three-dimensional pooling operation on the entire input tensor TSR_in.

[0042] See also Figure 4 , Figure 4 A flowchart of a three-dimensional pooling operation provided for the implementation of this application. Figure 4 The process is executed by the intelligent processing unit 130. Figure 2 illustrate Figure 4 steps.

[0043] Step S410: Select an input tensor or a target input data block. The intelligent processing unit 130 selects an input tensor or a target input data block. For example, the input tensor or the target input data block can be Figure 2 The input data 210 is stored in Figure 1 in the memory 120.

[0044] Step S420: Use the direct memory access unit to read the input tensor or the target input data block from the external memory into the internal memory. The intelligent processing unit 130 uses the direct memory access unit 132 to read the input tensor or the target input data block from the external memory (i.e., the memory 120) into the internal memory (i.e., the cache 134).

[0045] Step S430 : Read the input tensor or the target input data block from the internal memory. The vector core 138 of the computing circuit 136 reads the input tensor or the target input data block from the cache memory 134 .

[0046] Step S440: Execute a first instruction to perform R two-dimensional pooling operations on the input tensor or the target input data block to obtain an intermediate tensor. The vector core 138 of the computing circuit 136 executes a first instruction to perform R two-dimensional pooling operations on the input tensor or the target input data block (R is a positive integer) to obtain an intermediate tensor (e.g., Figure 2 If the first instruction indicates that the dimension of the input tensor or the target input data block includes a depth (D) dimension, a height (H) dimension, and a width (W) dimension, then R = Kd (please refer to equation (3) or (4)); if the first instruction indicates that the dimension of the input tensor or the target input data block includes only two of the depth (D) dimension, the height (H) dimension, and the width (W) dimension (for example, only the height (H) dimension and the width (W) dimension but not the depth (D) dimension), then R = 1. The details of step S440 will be described below in conjunction with Figure 5 or Figure 6 Describe in detail.

[0047] Step S450: Execute a second instruction to perform a two-dimensional pooling operation on the intermediate tensor to obtain an output tensor or an output data block. The vector core 138 of the computing circuit 136 executes a second instruction to perform a two-dimensional pooling operation on the input tensor or the target input data block to obtain an output tensor or an output data block (e.g., Figure 2 The details of step S450 will be described below. Figure 7 Describe in detail.

[0048] Step S460: Use the direct memory access unit to write the output tensor or the output data block to the external memory. The intelligent processing unit 130 uses the direct memory access unit 132 to write the output tensor or the output data block to the external memory.

[0049] Step S470: Determine whether there are still unprocessed input tensors or target input data blocks. The vector core 138 of the computing circuit 136 determines whether there are still unprocessed input tensors or target input data blocks. If yes, return to step S410; if no, end the three-dimensional pooling operation.

[0050] like Figure 4 As shown, the present application does not directly perform a three-dimensional pooling operation on the data (requiring hardware with high complexity and cost); on the contrary, the present application uses a circuit with low complexity and cost (i.e., the vector core 138) to execute two instructions for the two-dimensional pooling operation (i.e., step S440 and step S450, both of which are performed by the vector core 138) to equivalently implement the three-dimensional pooling operation. Therefore, the calculation circuit 136 of the present application has the advantages of low cost, low complexity and easy implementation, making the intelligent processing unit 130 more competitive.

[0051] Figure 5 for Figure 4 The detailed flow chart of step S440 includes steps S510 to S540. Figure 2 Steps S510 to S540 are described.

[0052] Step S510: Read a sub-tensor of the input tensor or a sub-tensor of the target input data block. The vector core 138 of the computing circuit 136 reads a sub-tensor of the input tensor or a sub-tensor of the target input data block. Figure 2For example, input data 210 includes sub-tensor 212, sub-tensor 214, and sub-tensor 216. In this step, computing circuit 136 reads one of sub-tensor 212, sub-tensor 214, and sub-tensor 216. In some embodiments, computing circuit 136 processes sub-tensor 212, sub-tensor 214, and sub-tensor 216 in sequence.

[0053] Step S520: Perform a two-dimensional pooling operation on the sub-tensor to obtain a portion of the intermediate tensor. The vector core 138 of the computing circuit 136 performs a two-dimensional pooling operation on the sub-tensor to obtain a portion of the intermediate tensor TSR_imt (i.e., a sub-tensor of the intermediate tensor TSR_imt). Figure 2 For example, the intermediate data 220 (ie, the intermediate tensor TSR_imt) includes sub-tensors 222, 224, and 226, which correspond to sub-tensors 212, 214, and 216, respectively. That is, sub-tensors 222, 224, and 226 are the results of the two-dimensional pooling operation of sub-tensors 212, 214, and 216, respectively. Figure 2 In the example, the two-dimensional pooling operation transforms the sub-tensor of H1xW1 into the sub-tensor of H2xW2; for example, if the dimension of sub-tensor 212 (214 or 216) is [D, H, W, C] = [3, 5, 2, 16], the size of the sliding window [Kd, Kh, Kw] = [3, 3, 2] and the moving step of the sliding window [Sd, Sh, Sw] = [1, 1, 1], then the dimension of sub-tensor 222 (224 or 226) is [D, H, W, C] = [3, 3, 1, 16] (i.e., H2xW2 = 3x1 = 3).

[0054] Step S530: Store the portion of the intermediate tensor into the internal memory. The vector core 138 of the computing circuit 136 stores the portion of the intermediate tensor into the internal memory. Figure 2 In the example of , the computing circuit 136 stores the sub-tensor 222 ( 224 or 226 ) into the cache memory 134 at this step.

[0055] Step S540: Determine whether there are still unprocessed sub-tensors. The vector core 138 of the computing circuit 136 determines whether there are still unprocessed sub-tensors. If yes, the computing circuit 136 executes step S510 to read the next sub-tensor; if no, step S440 ends.

[0056] Figure 6 show Figure 4 A detailed flow chart of another embodiment of step S440 includes steps S610 to S630. Figure 6In the process, the computing circuit 136 treats the input data 210 as a three-dimensional tensor ([H, W, DxC]) rather than a four-dimensional tensor ([D, H, W, C]) according to the instruction. Figure 2 Steps S610 to S630 are described.

[0057] Step S610: Read an input tensor or a target input data block according to an instruction. The vector core 138 of the computing circuit 136 reads the input tensor or the target input data block from the cache memory 134 according to an instruction, and the instruction includes a target channel number. In this step, the instruction executed by the computing circuit 136 indicates the data to be processed (in Figure 2 The input data 210 of FIG. 1 includes three dimensional parameters: height H=H1, width W=W1 and channel C=3C1, but does not include a depth (D) parameter, where "3C1" is the target channel number in this example. In this case, the way in which the computing circuit 136 reads the input data 210 has been described in Figure 3 This has been discussed in the description, so I will not repeat it here.

[0058] Step S620: Perform a two-dimensional pooling operation on the input tensor or the target input data block to obtain an intermediate tensor. The vector core 138 of the computing circuit 136 performs a two-dimensional pooling operation on the input tensor or the target input data block to obtain an intermediate tensor TSR_imt. Please note that Figure 5 The process of performing three two-dimensional pooling operations on the input data 210 (once for each of the sub-tensor 212, the sub-tensor 214 and the sub-tensor 216) (ie, R = Kd = 3) is used to obtain the complete intermediate data 220, and Figure 6 The process only needs to perform a two-dimensional pooling operation (i.e., R=1) on the input data 210 once to obtain the complete intermediate data 220. In other words, the vector core 138 of the computing circuit 136 only needs to perform a two-dimensional pooling operation on the input data 210 once in step S620 to obtain the complete intermediate data 220, which is equivalent to processing all sub-tensors of the input data 210 in parallel.

[0059] Step S630: Store the intermediate tensor into the internal memory. The vector core 138 of the computing circuit 136 stores the intermediate tensor into the internal memory.

[0060] Figure 7 for Figure 4 The detailed flow chart of step S450 in the embodiment includes steps S710 to S730. Figure 2 Steps S710 to S730 are described.

[0061] Step S710 : Reading the intermediate tensor. The vector core 138 of the computing circuit 136 reads the intermediate tensor TSR_imt from the cache memory 134 .

[0062] Step S720: Perform a two-dimensional pooling operation on the intermediate tensor to obtain an output tensor or an output data block. The vector core 138 of the computing circuit 136 performs a two-dimensional pooling operation on the intermediate tensor to obtain the output tensor or the output data block. It should be noted that this step is to complete the one-dimensional pooling operation in the depth direction by performing a two-dimensional pooling operation, and the details include the following steps S722 to S726.

[0063] Step S722: Combine dimension H and dimension W into a new dimension. The vector core 138 of the computing circuit 136 combines dimension H and dimension W to generate a new dimension, that is, a new dimension represents the combination of dimension H and dimension W. Figure 2 For example, the dimension of the intermediate tensor TSR_imt is [D, H, W, C] = [3, H2, W2, C1] ("3" represents the three sub-tensors 222, 224 and 226).

[0064] After step S722, the new dimension of the intermediate tensor TSR_imt is [D, L, C] = [3, L2, C1], where L2 = H2xW2. In other words, step S722 is an operation to reduce the dimension of the intermediate tensor TSR_imt. In practice, step S722 can be implemented by adjusting the parameters of the instruction, that is, by reducing one parameter (i.e., replacing the parameters of the height (H) dimension and the parameters of the width (W) dimension with the parameters of the L dimension), the vector core 138 treats the intermediate tensor TSR_imt as data with one less dimension.

[0065] Step S724: Set the size of the sliding window corresponding to the new dimension to 1, set the moving step of the sliding window corresponding to the new dimension to 1, and set the padding corresponding to the new dimension to 0. The vector core 138 of the calculation circuit 136 sets the size of the sliding window corresponding to the new dimension to 1, sets the moving step of the sliding window corresponding to the new dimension to 1, and sets the padding corresponding to the new dimension to 0 (i.e., no padding). In this way, the vector core 138 will not actually process the new dimension (i.e., dimension L).

[0066] Step S726: Perform a two-dimensional pooling operation on the intermediate tensor. The vector core 138 of the computing circuit 136 performs a two-dimensional pooling operation on the intermediate tensor TSR_imt to obtain the output tensor or the output data block. Although the vector core 138 performs a two-dimensional pooling operation on the intermediate tensor TSR_imt in this step, because the vector core 138 does not actually process the new dimension (i.e., dimension L), the vector core 138 is equivalent to performing a one-dimensional pooling operation on the intermediate tensor TSR_imt.

[0067] Step S730: Store the output tensor or the output data block to the internal memory. The vector core 138 of the computing circuit 136 stores the output tensor or the output data block to the internal memory. Figure 2 For example, the vector core 138 stores the output data 230 to the cache memory 134 in this step.

[0068] It should be noted that, in subsequent processing, the shape (or dimension) of the output data 230 can be reshaped from [D, L, C] = [1, 3, C1] to [D, H, W, C] = [1, 3, 1, C1], that is, the output data 230 is increased by one dimension.

[0069] Although equation (3) or equation (4) shows that the three-dimensional pooling operation is equivalent to the two-dimensional pooling operation plus the one-dimensional pooling operation (theoretically performed by a two-dimensional vector core and a one-dimensional vector core respectively), the present application uses the same vector core 138 to perform two two-dimensional pooling operations (i.e., step S440 and step S450, wherein step S450 is equivalent to the one-dimensional pooling operation, please refer to the description of step S720 above), so the intelligent processing unit 130 of the present application has the advantages of low cost and low complexity (i.e., eliminating a one-dimensional vector core).

[0070] In other embodiments, if the computing circuit 136 includes a one-dimensional vector core for performing a one-dimensional pooling operation, then in step S450 , the computing circuit 136 may instead use the one-dimensional vector core to perform a one-dimensional pooling operation on the intermediate data 220 .

[0071] In summary, the present application ingeniously uses one 2D pooling operation to be equivalent to a 1D pooling operation, and then ingeniously uses two 2D pooling operations to be equivalent to a 3D pooling operation. Since the 2D pooling operation core (i.e., 2D vector core) has lower circuit cost and complexity than the 3D pooling operation core (i.e., 3D vector core), and the present application does not require an additional 1D pooling operation core (i.e., 1D vector core), the electronic device using the IPU of the present application can improve efficiency without significantly increasing complexity and / or cost.

[0072] Those skilled in the art can design the computing circuit 136 according to the above disclosure. That is, the computing circuit 136 can be an application specific integrated circuit (ASIC), such as the aforementioned neural network computing core.

[0073] Although the embodiments of the present application are described above, these embodiments are not intended to limit the present application. Those skilled in the art may make changes to the technical features of the present application based on the explicit or implicit contents of the present application. All these changes may fall within the scope of patent protection sought by the present application. In other words, the scope of patent protection of the present application shall be subject to the scope of the patent application defined in this specification.

[0074]

Explanation of symbols

[0075] 100: electronic device;

[0076] 110: processor;

[0077] 120: memory;

[0078] 130: intelligent processing unit;

[0079] 132: direct memory access unit;

[0080] 134: cache memory;

[0081] 136: calculation circuit;

[0082] 137: convolution core;

[0083] 138: vector core;

[0084] TSR_in: input tensor;

[0085] TSR_imt: intermediate tensor;

[0086] TSR_out: output tensor;

[0087] 210, 310: input data;

[0088] 212, 214, 216, 312, 314, 316, 222, 224, 226: subtensors;

[0089] 220: intermediate data;

[0090] 230: output data;

[0091] S410, S420, S430, S440, S450, S460, S470, S510, S520, S530, S540, S610, S620, S630, S710, S720, S722, S724, S726, S730: steps.

Claims

1. A three-dimensional pooling operation method for operating an input tensor to generate an output tensor, wherein the input tensor includes a plurality of input data blocks, and the output tensor includes a plurality of output data blocks, the method comprising: (A) reading one of the plurality of input data blocks from an external memory as a target input data block, and storing the target input data block into the memory; (B) reading the target input data block from the memory; (C) in response to an instruction, performing R first two-dimensional pooling operations on the target input data block to generate an intermediate tensor, where R is a positive integer; (D) performing a second two-dimensional pooling operation on the intermediate tensor to generate target output data blocks of the plurality of output data blocks; as well as (E) storing the target output data block in the memory; Among them, if the instruction indicates that the dimension of the target input data block includes depth dimension, height dimension and width dimension, then R=Kd; if the instruction indicates that the dimension of the target input data block only includes two of the depth dimension, height dimension and width dimension, then R=1, wherein Kd is the size of a first sliding window in the depth direction.

2. The three-dimensional pooling operation method according to claim 1, characterized in that: The intermediate tensor has a first dimension parameter and a second dimension parameter, and the step (D) comprises: Using a third dimension parameter to represent a combination of the first dimension parameter and the second dimension parameter; and A size of a second sliding window corresponding to the third dimensional parameter is set to 1, a moving step of the second sliding window corresponding to the third dimensional parameter is set to 1, and a padding amount corresponding to the third dimensional parameter is set to 0.

3. The three-dimensional pooling operation method according to claim 2, characterized in that: The product of the first dimension parameter and the second dimension parameter is equal to the third dimension parameter.

4. The method according to claim 1, characterized in that: The memory stores a first sub-tensor and a second sub-tensor of the target input data block, wherein the second sub-tensor is adjacent to the first sub-tensor, and the step (C) is used to process the first sub-tensor and the second sub-tensor.

5. The three-dimensional pooling operation method according to claim 4, characterized in that: The first sub-tensor has a first channel dimension, the second sub-tensor has a second channel dimension, and the step (B) is used to read the first sub-tensor and the second sub-tensor in response to the instruction, and the target channel number of the instruction is greater than or equal to the sum of the first channel dimension and the second channel dimension.

6. The three-dimensional pooling operation method according to claim 5, characterized in that: The first channel dimension is equal to the second channel dimension.

7. The three-dimensional pooling operation method according to claim 5, characterized in that: The first channel dimension and the second channel dimension are both equal to a bit width of the memory divided by a data format of the input tensor.

8. The three-dimensional pooling operation method according to claim 1, characterized in that: The step (C) is performed by a two-dimensional vector core, and the step (D) is performed by the two-dimensional vector core.

9. An intelligent processing unit, characterized in that: For processing an input tensor and generating an output tensor, the input tensor includes a plurality of input data blocks, the output tensor includes a plurality of output data blocks, and the intelligent processing unit includes: Memory; a direct memory access unit, configured to read one of the plurality of input data blocks from an external memory as a target input data block, and store the target input data block into the memory; and A computing circuit is configured to perform the following operations to perform a three-dimensional pooling operation on the target input data block, wherein the three-dimensional pooling operation is used to generate a target output data block of the plurality of output data blocks: (A) reading the target input data block from the memory; (B) in response to an instruction, performing R first two-dimensional pooling operations on the target input data block to generate an intermediate tensor, where R is a positive integer; and (C) performing a second two-dimensional pooling operation on the intermediate tensor to generate the target output data block; Among them, if the instruction indicates that the dimension of the target input data block includes depth dimension, height dimension and width dimension, then R=Kd; if the instruction indicates that the dimension of the target input data block only includes two of the depth dimension, height dimension and width dimension, then R=1, wherein Kd is the size of a first sliding window in the depth direction.

10. The intelligent processing unit according to claim 9, characterized in that: The intermediate tensor has a first dimension parameter and a second dimension parameter, and the step (C) comprises: Using a third dimension parameter to represent a combination of the first dimension parameter and the second dimension parameter; and A size of a second sliding window corresponding to the third dimensional parameter is set to 1, a moving step of the second sliding window corresponding to the third dimensional parameter is set to 1, and a padding amount corresponding to the third dimensional parameter is set to 0.

11. The intelligent processing unit according to claim 10, characterized in that: The product of the first dimension parameter and the second dimension parameter is equal to the third dimension parameter.

12. The intelligent processing unit according to claim 9, characterized in that: The memory stores a first sub-tensor and a second sub-tensor of the target input data block, wherein the second sub-tensor is adjacent to the first sub-tensor, and the computing circuit processes the first sub-tensor and the second sub-tensor in the step (B).

13. The intelligent processing unit according to claim 12, characterized in that: The first sub-tensor has a first channel dimension, the second sub-tensor has a second channel dimension, and the computing circuit is used in the step (A) to read the first sub-tensor and the second sub-tensor in response to an instruction, and the target channel number of the instruction is greater than or equal to the sum of the first channel dimension and the second channel dimension.

14. The intelligent processing unit according to claim 13, characterized in that: The first channel dimension is equal to the second channel dimension.

15. The intelligent processing unit according to claim 13, characterized in that: The first channel dimension and the second channel dimension are both equal to a bit width of the memory divided by a data format of the input tensor.

16. The intelligent processing unit according to claim 9, characterized in that: The calculation circuit includes a two-dimensional vector core, and the step (B) and the step (C) are executed by the two-dimensional vector core.

Images