Arithmetic circuit, arithmetic method, and program
By grouping output feature map channels and adaptively performing convolution operations, the arithmetic circuit optimizes CNN hardware to efficiently handle zero matrices, enhancing calculation speed and reducing hardware requirements.
Patent Information
- Application Number
- JP2022567947
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-09
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2040-12-09
AI Technical Summary
Conventional CNN hardware implementations face inefficiencies in calculation speed due to the presence of zero matrices in kernel data, leading to wasted MAC arithmetic units and reduced throughput in convolutional layers.
An arithmetic circuit that groups output feature map channels into sets and assigns multiple MAC calculation units to each set, adaptively performing convolution operations only when necessary, skipping zero matrix calculations to optimize processing.
This approach significantly speeds up calculation by reducing the number of required processes, potentially achieving a 40% reduction in processing time while maintaining a manageable hardware footprint.
Smart Images

Figure 0007587168000001 
Figure 0007587168000002 
Figure 0007587168000003
Abstract
Description
[Technical field]
[0001] The present invention relates to techniques for an arithmetic circuit, an arithmetic method, and a program. [Background technology]
[0002] When performing inference using a trained CNN (Convolutional Neural Network) or when training a CNN, convolution processing is performed in the convolution layer, which is equivalent to repeated multiply-and-accumulate operations. In CNN inference, the above-mentioned multiply-and-accumulate operations (hereinafter also referred to as "MAC operations") account for the majority of the overall processing volume. Even when implementing a CNN inference engine as hardware, the computational efficiency and implementation efficiency of the MAC operation circuit have a significant impact on the overall hardware.
[0003] In the convolution layer, the output feature map data oFmap is obtained by convolving the input feature map data iFmap, which is the result of the previous layer, with the weighting coefficient Kernel. The input feature map data iFmap and the output feature map data oFmap each consist of multiple channels. These are iCH_num (the number of input channels) and oCH_num (the number of output channels), respectively. Since the Kernel is convolved between channels, the Kernel has a number of channels equivalent to (iCH_num × oCH_num). Fig. 13 is a conceptual diagram of a convolution layer. The example in Fig. 13 shows a convolution layer that generates output feature map data oFmap of oCH_num=3 from an input feature map iFmap of iCH_num=2.
[0004] When implementing such convolutional layer processing as hardware, a parallel method is often used to improve throughput by parallelization, in which oCH_num parallel MAC operators are prepared, kernel MAC processing for the same input channel number is performed in parallel, and this processing is repeated iCH_num times.
[0005] FIG. 14 is a diagram showing an example of a MAC calculation circuit and an example of a processing flow. In the configuration of FIG. 14, for example, a convolution layer generates output feature map data oFmap of oCH_num=4 from input feature map data iFmap of iCH_num=5. In this case, for example, four MAC calculators 910 are prepared in parallel, and the MAC calculator 910 is operated five times. Also, each MAC calculator 910 needs a memory 920 for temporarily storing the calculation result of the output feature map data oFmap. The memory 920 needs four memories 921-924 for oCHm (m is an integer from 0 to 3). As shown in FIG. 14, in the (n+1)th processing (n is an integer from 0 to 4), the iFmap data of iCHn is supplied to the four MAC calculators 911-914 as the input feature map data iFmap. As the weighting coefficient data Kernel, the kernel data of iCHn&oCH0 is supplied to a MAC calculator 911, the kernel data of iCHn&oCH1 is supplied to a MAC calculator 912, the kernel data of iCHn&oCH2 is supplied to a MAC calculator 913, and the kernel data of iCHn&oCH3 is supplied to a MAC calculator 914. At the start of each layer, the data in each memory is initialized to 0. The kernel data of a certain channel, where the input channel number is n and the output channel number is m, is represented as "kernel data of iCHn&oCHm."
[0006] In the first process in which the convolution calculation of iCH0 is performed, the MAC calculator 911 performs the convolution accumulation of iCH0*oCH0, and adds and stores the calculation result in the memory 921. The MAC calculator 912 performs the convolution accumulation of iCH0*oCH1, and adds and stores the calculation result in the memory 922. The MAC calculator 913 performs the convolution accumulation of iCH0*oCH2, and adds and stores the calculation result in the memory 923. The MAC calculator 914 performs the convolution accumulation of iCH0*oCH3, and adds and stores the calculation result in the memory 924. Note that the convolution calculation of kernel data whose input channel number is n and whose output channel number is m(iCHn&oCHm) for an input channel whose input channel number is n(iCHn) to obtain an output channel whose output channel number is m(oCHm) is represented as "iCHn*oCHm".
[0007] Subsequently, in the second processing, the input feature map data iFmap of iCH1 is supplied to the MAC calculators 911-914, and each MAC calculator performs the product-sum operation of the kernel. The operation results are stored in memories 921-924 by adding the convolution results of iCH0 and iCH1. That is, in the second processing in which the convolution operation of iCH1 is performed, the product-sum operation result of iCH0*oCH0+iCH1*oCH0 is stored in memory 921, the product-sum operation result of iCH0*oCH1+iCH1*oCH1 is stored in memory 922, the product-sum operation result of iCH0*oCH2+iCH1*oCH2 is stored in memory 923, and the product-sum operation result of iCH0*oCH3+iCH1*oCH3 is stored in memory 924.
[0008] In the fifth processing, the input feature map data iFmap of iCH4 is supplied to the MAC calculators 911 to 914, and each MAC calculator performs a product-sum operation process of the kernel. The calculation results are stored in the memories 921 to 924 by adding the convolution results from iCH0 to iCH4. In such processing, the final calculation result becomes the output feature map data oFmap, so the data in the memory 920 is determined as the oFmap result of this convolution layer. If the next layer is again a convolution layer, the output feature map data oFmap is used as the input feature map data iFmap of the next layer and similar processing is carried out. In the configuration as shown in FIG. 14, the common input feature map data iFmap can be simultaneously subjected to product-sum operations, and it is easy to improve the throughput by parallelization. In addition, in the configuration as shown in FIG. 14, the calculator and the memory are paired one-to-one, and the final convolution result can be obtained simply by adding the calculation result in each iCH to the memory data associated with the calculator, so the circuit configuration is simple. [Prior art documents] [Non-patent literature]
[0009] [Non-Patent Document 1] Norman P. Jouppi, Cliff Young et al., “In-Datacenter Performance Analysis of a Tensor Processing Unit TM”, the 44th International Symposium on Computer Architecture (ISCA), 2017 Summary of the Invention [Problem to be solved by the invention]
[0010] On the other hand, there are many cases where the input feature map data iFmap and the input data of the Kernel are partially 0. In such cases, the product-sum operation is unnecessary (as it is a process of multiplying by 0). In particular, since the size of each channel of the kernel data is generally smaller than that of Fmap, such as 3×3 1×1, there are cases where the kernel data of a channel becomes entirely 0 (zero matrix).
[0011] FIG. 15 is a diagram showing kernel data with sparsity. In FIG. 15, hatched squares 951 represent kernel data that is not zero, and non-hatched squares 952 represent sparse kernel data. In FIG. 15, 8 channels out of 20 channels of kernel data are sparse with zero matrices. In the calculation process, the kernel data is used in the order of i, ii, iii, iv, and v. Also, the MAC calculator 911 is assigned to process kernel data 961 of oCH0, the MAC calculator 912 is assigned to process kernel data 962 of oCH1, the MAC calculator 913 is assigned to process kernel data 963 of oCH2, and the MAC calculator 914 is assigned to process kernel data 964 of oCH4.
[0012] FIG. 16 is a diagram illustrating an example of a processing flow when sparse kernel data is supplied. In the first process in which the convolution calculation of iCH0 is performed, the kernel data of iCH0&oCH1 and the kernel data of iCH0&oCH2 are zero matrices, so 0 is simply added to the data stored in memory 922 and memory 923. For this reason, MAC calculator 912 and MAC calculator 913 do not need to perform calculations. However, since the calculations of MAC calculator 911 and MAC calculator 914 cannot be omitted, in the hardware configuration according to the conventional technology shown in Fig. 14 etc., MAC calculator 912 and MAC calculator 913 have to wait for these calculations to finish, so that the MAC calculator 912 and MAC calculator 913 are wasted. When the input data is sparse in this way, there is a problem in that the conventional techniques cannot be expected to achieve sufficient speedup in calculations.
[0013] In view of the above circumstances, the present invention aims to provide a technique that can efficiently speed up calculations while suppressing an increase in hardware scale when some of the weighting coefficients are zero matrices in product-sum calculation processing in the convolution layer of a neural network. [Means for solving the problem]
[0014] One aspect of the present invention is an arithmetic circuit that performs a convolution operation between input feature map information supplied as multiple channels and coefficient information supplied as multiple channels, the arithmetic circuit comprising: a set including at least two channels of the output feature map based on an output channel; and at least three or more sub-arithmetic circuits, wherein at least two of the sub-arithmetic circuits are assigned to each set, and the sub-arithmetic circuits included in the set perform a convolution operation between the coefficient information included in the set and the input feature map information, and when a specific channel of the output feature map becomes a zero matrix, the sub-arithmetic circuit that performs the convolution operation performs a convolution operation between the coefficient information supplied next and the input feature map information from the channels of the output feature map and the channels of the input feature map included in the set, and outputs the results of the convolution operation for each channel of the output feature map.
[0015] One aspect of the present invention is a computation method for causing a computation circuit including a set including at least two output feature map channels based on an output channel and at least three or more sub-computation circuits to execute a convolution operation of input feature map information and coefficient information supplied as a plurality of channels, the computation method allocating at least two of the sub-computation circuits to each of the sets, causing the sub-computation circuits included in the set to execute a convolution operation of the coefficient information included in the set and the input feature map information, and when a specific channel of the output feature map becomes a zero matrix, causing the sub-computation circuit performing the convolution operation to execute a convolution operation of the coefficient information supplied next and the input feature map information from the channel of the output feature map and the channel of the input feature map included in the set, and outputting the results of the convolution operation for each channel of the output feature map.
[0016] One aspect of the present invention is a program for causing a computer to implement any one of the arithmetic circuits described above. Effect of the Invention
[0017] According to the present invention, in the case where some of the weighting coefficients in the product-sum calculation process in the convolution layer of a neural network are zero matrices, it is possible to efficiently increase the calculation speed while suppressing an increase in hardware scale. [Brief description of the drawings]
[0018] [Figure 1] FIG. 2 is a diagram illustrating an arithmetic circuit according to an embodiment. [Diagram 2] FIG. 13 is a diagram showing an example in which 8 channels out of 20 channels of kernel data are sparse matrices. [Diagram 3] FIG. 11 is a diagram illustrating an example of allocation of MAC calculators in the embodiment. [Figure 4] FIG. 11 is a diagram illustrating an example of a processing order used in kernel data according to the embodiment. [Diagram 5] 11A and 11B are diagrams illustrating an example of a first process when sparseness occurs in kernel data according to the embodiment; [Figure 6] FIG. 11 is a diagram illustrating an example of a second process when sparseness occurs in kernel data according to the embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of a third process when sparseness occurs in kernel data according to the embodiment. [Figure 8] 1A and 1B are diagrams illustrating an example of allocation and configuration of a MAC calculator according to an embodiment. [Figure 9] FIG. 13 is a diagram illustrating allocation of MAC operators to a set of kernel data when k=1. [Figure 10] FIG. 13 is a diagram illustrating allocation of MAC operators to a set of kernel data when k=4. [Figure 11] 11 is a flowchart of an example of a processing procedure of an arithmetic circuit according to the embodiment. [Figure 12] 13 is a flowchart of a procedure for optimizing allocation of MAC calculators to a set of kernel data in a modified example. [Figure 13] FIG. 1 is an image of a convolutional layer. [Figure 14] FIG. 13 illustrates an example of a MAC operation circuit and a process flow. [Figure 15] FIG. 13 is a diagram illustrating kernel data with sparsity. [Figure 16] FIG. 13 is a diagram illustrating an example of a processing flow when kernel data having sparseness is supplied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0019] An embodiment of the present invention will be described in detail with reference to the drawings. Note that the method of the present embodiment is applicable to, for example, a case where inference is performed using a trained CNN, or a case where a CNN is trained.
[0020] <Example of arithmetic circuit configuration> Fig. 1 is a diagram showing an arithmetic circuit of this embodiment. As shown in Fig. 1, the arithmetic circuit 1 includes a sub-arithmetic circuit 10 and a memory 20 for temporarily storing the results of the arithmetic operation. The sub-arithmetic circuit 10 includes a MAC calculator macA (sub-arithmetic circuit), a MAC calculator macB (sub-arithmetic circuit), a MAC calculator macC (sub-arithmetic circuit), and a MAC calculator macD (sub-arithmetic circuit). The memory 20 comprises a memory 21 for oCH0, a memory 22 for oCH1, a memory 23 for oCH2, and a memory 24 for oCH3.
[0021] The arithmetic circuit 1 is an arithmetic circuit in the convolutional layer of the CNN. The arithmetic circuit 1 divides kernel data (coefficient information) which is a weight coefficient into a plurality of sets including several output channels. The arithmetic circuit 1 divides the sets so that no channel belongs to two or more sets. The arithmetic circuit 1 assigns MAC calculators to each set, the number of which is equal to the number of channels in the set. The MAC calculators are also supplied with input feature map data iFmap and weight coefficient data (kernel data) kernel.
[0022] 1 shows an example having four MAC arithmetic units and four memories, the arithmetic circuit 1 may have three or more MAC arithmetic units and three or more memories, or may have five or more MAC arithmetic units and five or more memories. The number of MAC arithmetic units and the number of memories are the same.
[0023] The arithmetic circuit 1 is configured using a processor such as a CPU (Central Processing Unit) and a memory, or an arithmetic circuit and a memory. The arithmetic circuit 1 functions as a MAC arithmetic unit, for example, by a processor executing a program. All or part of the functions of the arithmetic circuit 1 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. The computer-readable recording medium may be, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (for example, an SSD: Solid State Drive), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.
[0024] <Example of input data with sparsity> Next, the case where the kernel data has sparseness will be described with reference to FIGS. FIG. 2 is a diagram showing an example in which 8 channels out of 20 channels of kernel data are sparse matrices. In FIG. 2, hatched squares 101 represent kernel data that is not a sparse matrix, and non-hatched squares 102 represent kernel data that is a sparse matrix. In the embodiment, the channels of sparse kernel data may include channels in which the matrix is such that the majority of data is zero and only a small number of data is meaningful, in addition to channels in which the matrix is zero. The sparse kernel data are iCH0&oCH1, iCH0&oCH2, iCH1&oCH1, iCH2&oCH2, iCH3&oCH1, iCH3&oCH2, iCH3&oCH3, and iCH4&oCH1.
[0025] In conventional parallel processing, kernel data is used in the order of i, ii, iii, iv, and v, as shown in Fig. 15. Also, conventionally, each MAC arithmetic unit was assigned to process kernel data of oCHm, as shown in Fig. 15.
[0026] In contrast, in this embodiment, multiple oCHm are grouped into one set, and multiple MAC calculators are assigned to one set. Fig. 3 is a diagram showing an example of allocation of MAC calculators in this embodiment. In the example of Fig. 3, two oCHm are grouped into one set. A first set 201 (set 0) is a set of oCH0 and oCH1. A second set 202 (set 1) is a set of oCH2 and oCH3. Note that the calculation device 1 is a set including at least two channels of output feature maps based on the output channel included in the kernel data. In this manner, the set of this embodiment is configured based on the channels of the input feature map and the channels of the output feature map in the input feature map data.
[0027] Furthermore, in this embodiment, instead of using a fixed processing order such as iCH0, iCH1, ... as in the conventional method, multiply-and-accumulate processing is adaptively performed within the same set according to the sparseness of the kernel data, thereby achieving faster processing.
[0028] <Kernel data processing order> Next, an example of the processing order used in the kernel data will be described. FIG. 4 is a diagram showing an example of a processing order used in the kernel data according to the present embodiment. In the first set 201 (set 0) of kernel data, the arithmetic circuit 1 uses the kernel data iCH0&oCH0, iCH0&oCH1, iCH1&oCH0, iCH1&oCH1, iCH2&oCH0, iCH2&oCH1, iCH3&oCH0, iCH3&oCH1, iCH4&oCH0, and iCH4&oCH1 in this order.
[0029] In the second set 202 (set 1) of kernel data, the arithmetic circuit 1 uses the kernel data iCH0&oCH2, iCH0&oCH3, iCH1&oCH2, iCH1&oCH3, iCH2&oCH2, iCH2&oCH3, iCH3&oCH2, iCH3&oCH3, iCH4&oCH2, and iCH4&oCH3 in that order.
[0030] (First processing) Next, an example of the first processing when sparseness occurs in the kernel data will be described with reference to FIGS. 5 is a diagram showing an example of the first processing when sparseness occurs in the kernel data according to this embodiment. The MAC calculators macA and macB of the first pair 11 are assigned to process the first set 201 of kernel data (FIG. 3). The MAC calculators macC and macD of the first pair 12 are assigned to process the second set 202 of kernel data (FIG. 3). Data (iCH0 and iCH1) are supplied from the input feature map data iFmap to each of the MAC calculators macA to macD.
[0031] When there is a channel of kernel data that becomes a sparse matrix within each set of kernel data, the calculation circuit 1 performs a convolution operation of the next kernel data in that set and the feature map using the MAC calculation unit to which the kernel data that becomes the sparse matrix was supposed to be assigned. In FIG. 5, the dashed arrow from the MAC calculator to oCHm indicates that kernel data is skipped and therefore no addition to memory is performed.
[0032] In the first set 201, since the kernel data iCH0&oCH1 is a zero matrix, no calculation is required. Therefore, in the first processing, the calculation circuit 1 performs a calculation on the kernel data iCH0&oCH0, but skips the kernel data iCH0&oCH1 and performs a calculation on the next kernel data iCH1&oCH0 in the first set 201.
[0033] 5, the MAC calculator macA adds and stores the convolution accumulation result of iCH0*oCH0 in the oCH0 memory 21. The MAC calculator macB adds and stores the convolution accumulation result of iCH1*oCH0 in the oCH0 memory 21.
[0034] As a result, the calculation result of iCH0*oCH0+iCH1*oCH0 is stored in oCH0 memory 21. No calculation result is added to oCH1 memory 22, which remains at the initial value 0.
[0035] In the second set 202, since the kernel data iCH0&oCH2 is a zero matrix, no calculation is required. Therefore, the calculation circuit 1 skips the kernel data iCH0&oCH2 in the second set 202, and performs calculations on the next kernel data iCH0&oCH3 (skipping one channel), and the convolution calculations on the further next kernel data iCH1&oCH2.
[0036] 5, the MAC calculator macC adds and stores the convolution accumulation result of iCH0*oCH3 in the memory 24 for oCH3. The MAC calculator macD adds and stores the convolution accumulation result of iCH1*oCH2 in the memory 23 for oCH2.
[0037] As a result, the calculation result of iCH1*oCH2 is stored in the oCH2 memory 23. The calculation result of iCH0*oCH3 is stored in the oCH3 memory 24.
[0038] (Second processing) Next, an example of the second processing when sparseness occurs in the kernel data will be described with reference to FIG. 4 and FIG. FIG. 6 is a diagram showing an example of the second processing when sparseness occurs in the kernel data according to this embodiment.
[0039] In the second processing, the kernel data iCH1&oCH1 is a zero matrix in the first set 201. Therefore, the arithmetic circuit 1 skips the kernel data iCH1&oCH1 in the first set 201, performs arithmetic on the next kernel data iCH2&oCH0, and then performs arithmetic on the kernel data iCH2&oCH1.
[0040] 6, the MAC calculator macA adds and stores the convolution accumulation result of iCH2*oCH0 in the memory 21 for oCH0. The MAC calculator macB adds and stores the convolution accumulation result of iCH2*oCH1 in the memory 21 for oCH0.
[0041] As a result, the calculation result of iCH0*oCH0+iCH1*oCH0+iCH2*oCH0 is stored in the oCH0 memory 21. The calculation result of iCH2*oCH1 is stored in the oCH1 memory 22.
[0042] As shown in FIG. 6, the MAC calculator macC adds and stores the convolution accumulation result of iCH1*oCH3 in the memory 24 for oCH3. Also, in the second set 202, the kernel data iCH2&oCH2 is a zero matrix. Therefore, the arithmetic circuit 1 performs an arithmetic operation on the kernel data iCH1&oCH3, skips the kernel data iCH2&oCH2 within the second set 202, and performs an arithmetic operation on the preceding kernel data iCH2&oCH3. The MAC arithmetic unit macD adds the convolution accumulation result of iCH2*oCH3 to the memory 24 for oCH3 and stores it.
[0043] As a result, no new addition is performed on the calculation result stored in oCH2 memory 23, and the calculation result of iCH1*oCH2 remains stored. The calculation result of iCH0*oCH3+iCH1*oCH3+iCH2*oCH3 is stored in oCH3 memory 24.
[0044] (Third processing) Next, a third processing example when sparseness occurs in the kernel data will be described with reference to FIG. 4 and FIG. FIG. 7 is a diagram illustrating an example of the third processing when sparseness occurs in the kernel data according to this embodiment.
[0045] In the third processing, the kernel data iCH3&oCH1 is a zero matrix in the first set 201. Therefore, the arithmetic circuit 1 performs an arithmetic operation on the kernel data iCH3&oCH0, skips the kernel data iCH3&oCH1 in the first set 201, and performs an arithmetic operation on the next kernel data iCH4&oCH0.
[0046] 6, the MAC calculator macA adds and stores the convolution accumulation result of iCH3*oCH0 in the oCH0 memory 21. The MAC calculator macB adds and stores the convolution accumulation result of iCH4*oCH0 in the oCH0 memory 21.
[0047] As a result, the calculation result iCH0*oCH0+iCH1*oCH0+iCH2*oCH0+iCH2*oCH0+iCH4*oCH0 is stored in the oCH0 memory 21. No new addition is performed on the calculation result stored in the oCH1 memory 22, and the result of iCH2*oCH1 is stored. Note that, as shown in Fig. 6, in the first set 201, the kernel data iCH4&oCH1 is a zero matrix, so the processing of the first set 201 ends after the above three times.
[0048] As shown in FIG. 6, in the second set 202, the kernel data iCH3&oCH2 and the kernel data iCH3&oCH3 are zero matrices. Therefore, the arithmetic circuit 1 skips the kernel data iCH2&oCH2 in the second set 202, performs arithmetic on the kernel data iCH4&oCH2 two channels ahead (skipping two channels), and performs arithmetic on the kernel data iCH4&oCH3. The MAC calculator macC adds the convolution accumulation result of iCH4*oCH2 to the memory 23 for oCH2 and stores it. The MAC calculator macD adds the convolution accumulation result of iCH4*oCH3 to the memory 24 for oCH3 and stores it.
[0049] As a result, the calculation result of iCH1*oCH2+iCH4*oCH2 is stored in the oCH2 memory 23. The calculation result of iCH0*oCH3+iCH1*oCH3+iCH2*oCH3+iCH4*oCH3 is stored in the oCH3 memory 24. The processing of the second set 202 ends after the above three times.
[0050] In this manner, in this embodiment, the convolution operation results from iCH0 to iCH4 in each oCH are stored in each memory. Since the operation results stored in the memory of the arithmetic circuit 1 become the final operation results, that is, the output feature map data oFmap, the data in the memory is the convolution layer result.
[0051] However, in the conventional method, five processes were required, whereas in the present embodiment, only three processes are required, so that in this example, the processing time can be reduced by 40%, and a significant increase in the speed of calculation can be achieved.
[0052] In this embodiment, since it is necessary to supply input feature map data iFmap data of a plurality of input channels to the MAC calculator, the bus width of the input data becomes larger than before, but if the bus width is made n times larger than before, it is possible to supply input feature map data iFmap across n channels. Also, in this embodiment, by making n sufficiently large, it is possible to prevent a situation in which skipping is not possible due to insufficient supply capacity of input feature map data iFmap data. However, if n is made sufficiently large, an increase in the circuit scale due to the expansion of the bus width becomes a bottleneck, so for example, the following restrictions may be added.
[0053] Limitation 1. Up to two channels of input feature map data iFmap data can be provided, with n=2. Restriction 2: Do not perform skip processing that requires input feature map data iFmap with (n+1) or more channels.
[0054] In the examples of Figures 4 to 7, if n=2 or more, there is no restriction on skip processing, and there is no need to simultaneously supply n+1=3 channels of input feature map data. Also, even if n=2 or 3, there are many cases where skip processing is not restricted much.
[0055] <Allocation of MAC operators to kernel data sets> Next, allocation of MAC calculators to kernel data sets will be described. Fig. 8 is a diagram showing allocation of MAC calculators to kernel data sets in this embodiment when k = 2. The number of oCHs in one set is k.
[0056] For example, as shown in Figure 8, when oCH0 and oCH1 are treated as one set and two circuits, MAC calculator macA and MAC calculator macB, are assigned, the result of the calculation performed by MAC calculator macA will change for each process, whether it is the mac calculation result of oCH0 or the result of the multiply-and-accumulate calculation of oCH1. For this reason, there is not a one-to-one correspondence between memory and MAC calculator, and wiring is required from one MAC calculator to two memories, as shown in Figure 8. From the memory's perspective, for example, a selector circuit and wiring are required to select one of the two MAC calculators.
[0057] (When k is small) When k is small, for example, in the smallest case where k=1, as shown in Fig. 9, one oCHn is assigned to each of sets 13 to 16, so the number of sets is equal to oCH_num. Fig. 9 is a diagram showing an example of the correspondence between the MAC calculator and the memory when k=1. In the example of Fig. 9, the kernel data is as shown in Fig. 5, and there is a zero matrix. And in this example as well, in the case of a zero matrix, the data is skipped and the preceding kernel data is processed.
[0058] For this reason, MAC calculator macA performs a convolution operation on iCH0*oCH0, adds the results and stores them in memory 21 for oCH0, MAC calculator macB performs a convolution operation on 0+iCH2*oCH1, adds the results and stores them in memory 22 for oCH1. MAC calculator macC performs a convolution operation on 0+iCH1*oCH2, adds the results and stores them in memory 23 for oCH2, and MAC calculator macD performs a convolution operation on iCH0*oCH3, adds the results and stores them in memory 24 for oCH3.
[0059] In the case of k=1, for example, oCH1 has four sparse values out of five, but oCH0 has no sparse values at all. Therefore, the MAC calculator macB in charge of oCH1 finishes the calculation in one processing because it has four skips, but the MAC calculator macA in charge of oCH0 cannot skip any and requires five processings. In this way, in the case of k=1, when a multiply-and-accumulate operation is performed on the next input channel in the corresponding output channel, the MAC calculator often advances ahead only for a specific output channel. For this reason, in the case of k=1 in the examples of Figures 5 and 9, the processing of this convolution layer ends up waiting until the calculation of the MAC calculator macA is completed, and as a result, the speed cannot be increased at all in the five processings.
[0060] In addition, the kernel data tends to have a large bias in sparsity depending on the output channel. That is, it is relatively common for the kernel data of one output channel to be very sparse, while the kernel data of another output channel has almost no sparseness. For this reason, if k is too small, such as k=1, it is necessary to wait until the calculation of a set with few sparse elements is completed, and sufficient speed improvement may not be expected. Therefore, it is preferable that k is 2 or more.
[0061] (When k is large) When k is large, for example when k=the maximum of oCH_num and k=4, the number of sets 17 is one, and all oCHs are assigned to one set, as shown in Fig. 10. Fig. 10 is a diagram showing the allocation of MAC operators to kernel data sets when k=4.
[0062] When K=oCH_num, if the kernel data is sparse, it is possible to advance the MAC to any output channel. In this case, the kernel data can be packed as tightly as possible and placed in the MAC calculator, so it is possible to maximize the speed.
[0063] On the other hand, since the MAC calculator may perform calculations for all oCHs, the correspondence between the MAC calculator and the memory requires fully connected wiring. In the example of Figure 9, 4x4 fully connected wiring is required between the MAC calculator side and the memory side. Due to this wiring, a selector circuit for selecting oCH_num is required on the side of the memory 21 for oCH0 to the memory 24 for oCH3 to determine each time which calculation result of the oCH_num MAC calculators should be received. In the convolutional layers of modern CNNs, the number of oCH_num is often several tens to several hundreds, so implementing wiring and selector circuits for oCH_num fully connected states poses hardware bottlenecks in terms of circuit area and power consumption. For this reason, it is desirable that the value of k is not too large.
[0064] For this reason, in this embodiment, the value of k is set to, for example, 2 or more and less than the maximum value.
[0065] <Example of processing procedure> Next, an example of a processing procedure will be described. FIG. 11 is a flowchart of an example of a processing procedure of the arithmetic circuit according to this embodiment.
[0066] The arithmetic circuit 1 assigns the MAC calculators by determining in advance the set of output channels for each set. The arithmetic circuit 1 assigns at least two MAC calculators (sub-arithmetic circuits) to each set (step S1).
[0067] The arithmetic circuit 1 initializes the value of each memory to 0 (step S2).
[0068] The arithmetic circuit 1 selects data to be used in the arithmetic operation from the kernel data (step S3).
[0069] The arithmetic circuit 1 determines whether the selected kernel data is a zero matrix (step S4). If the arithmetic circuit 1 determines that the selected kernel data is a zero matrix (step S4; YES), the arithmetic circuit 1 proceeds to processing in step S5. If the arithmetic circuit 1 determines that the selected kernel data is not a zero matrix (step S4; NO), the arithmetic circuit 1 proceeds to processing in step S6.
[0070] The arithmetic circuit 1 skips the selected kernel data and reselects the next kernel data. Note that the arithmetic circuit 1 also judges whether the reselected kernel data is a zero matrix, and if the reselected kernel data is also a zero matrix, it skips again and reselects the next kernel data (step S5).
[0071] The arithmetic circuit 1 determines a memory for storing the calculation result calculated by the MAC calculator based on the presence or absence of a skip and the number of skips (step S6).
[0072] Each MAC calculator performs convolution calculation using the kernel data (step S7).
[0073] Each MAC calculator adds up the calculation results and stores the results in memory (step S8).
[0074] The arithmetic circuit 1 determines whether or not the calculation of all kernel data has been completed (step S9). If the arithmetic circuit 1 determines that the calculation of all kernel data has been completed (step S9; YES), it ends the process. If the arithmetic circuit 1 determines that the calculation of all kernel data has not been completed (step S9; NO), it returns to the process of step S3.
[0075] The processing procedure described with reference to FIG. 11 is an example, and is not limited thereto. For example, the arithmetic circuit 1 may perform a procedure for determining a memory for storing the calculation result calculated by the MAC calculator based on the presence or absence of a skip and the number of skips when selecting or reselecting kernel data. Furthermore, the kernel data is obtained by learning, and is known in advance when the inference process is executed. For this reason, in the process, the presence or absence of a skip and the memory determination procedure may be determined in advance before the inference process.
[0076] In the above-described embodiment, an example of the MAC calculation process in the convolutional layer of a CNN has been described, but the method of this embodiment can also be applied to other networks.
[0077] As described above, in this embodiment, a plurality of oCHs (weighting coefficients) are treated as one set, and a plurality of MAC calculators are assigned to each set. As a result, according to this embodiment, it is possible to eliminate waiting in the circuit that may occur when implementing the convolution processing of a convolutional neural network such as a CNN in hardware, thereby speeding up the calculation.
[0078] <Modification> As described above, in the allocation of MAC operators to kernel data sets, i.e., in the allocation of channels, if k is too small, the calculation speed cannot be increased efficiently, and if k is too large, the increase in circuit area becomes non-negligible. The value of K is related to the hardware configuration, such as the wiring between the operators and memory, and is therefore determined at the time of hardware design and cannot be changed during inference processing. On the other hand, whether to assign output channels to each set is not related to the hardware configuration, and can be arbitrarily changed during inference processing. For this reason, the arithmetic circuit 1 may predetermine the set of output channels for each set based on each value of the kernel data obtained during inference, thereby optimizing the allocation of MAC arithmetic units so as to maximize the speed of inference processing for the k determined during hardware design.
[0079] FIG. 12 is a flowchart of a procedure for optimizing the allocation of MAC calculators to a set of kernel data in the modified example.
[0080] The arithmetic circuit 1 checks each value of the kernel data obtained during inference (step S101).
[0081] The arithmetic circuit 1 determines the number of sets of kernel data and assigns the kernel data to the MAC calculator. The arithmetic circuit 1 may determine the sets of output channels included in each set based on the number and distribution of zero matrices included in the kernel data, for example, and assign the sets of kernel data to the MAC calculator. Alternatively, when processing is performed while skipping zeros in the kernel data, the arithmetic circuit 1 may determine the sets of output channels included in each set so that there is little bias in the number of calculations of the MAC calculator in each set, and assign the kernel data to the MAC calculator before performing the actual convolution calculation (step S102).
[0082] The arithmetic circuit 1 determines the pair of output channels included in each set, and judges whether the allocation of kernel data and MAC calculators has been optimized. For example, the arithmetic circuit 1 judges that optimization has been achieved if the difference in the number of calculations of the MAC calculators is within a predetermined value (S103). If optimization has been achieved (step S103; YES), the arithmetic circuit 1 ends the process. If optimization has not been achieved (step S103; NO), the arithmetic circuit 1 returns to the process of step S102.
[0083] After the optimization procedure described with reference to Fig. 12, the arithmetic circuit 1 performs the arithmetic processing of Fig. 11. The procedure and method of the optimization process described with reference to Fig. 12 are merely examples, and are not limited thereto.
[0084] As described above, in the modified example, the allocation of kernel data and MAC calculators is optimized, that is, the channels allocated to sets are optimized.
[0085] As a result, according to the modified example, the calculation speed can be further increased.
[0086] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Industrial Applicability]
[0087] The present invention is applicable to various types of inference processing devices. [Explanation of symbols]
[0088] 1...arithmetic circuit, 10...sub-arithmetic circuit, 20...memory, macA, macB, macC, macD...MAC calculator, 21...memory for oCH0, 21, 22...memory for oCH1, 23...memory for oCH2, 24...memory for oCH3
Claims
1. An arithmetic circuit for performing a convolution operation on input feature map information supplied as a plurality of channels and coefficient information supplied as a plurality of channels, a set based on the output channel and including at least two output feature map channels; At least three sub-arithmetic circuits; allocating at least two of the sub-operation circuits to each of the sets; The sub-operation circuit included in the set executes a convolution operation process between the coefficient information included in the set and the input feature map information; When a zero matrix occurs in the coefficient information, a sub-operation circuit that performs the convolution operation executes a convolution operation process between the coefficient information that is next supplied and the input feature map information from the channel of the output feature map and the channel of the input feature map that are included in the set; Outputting the convolution result for each channel of the output feature map. Arithmetic circuit.
2. the sub-arithmetic circuit outputs, for each channel of the output feature map, a sum of the convolution operation results for each channel of the input feature map obtained as a result of performing the operation for each channel of the input feature map information; The arithmetic circuit according to claim 1 .
3. When a zero matrix occurs in the coefficient information, the sub-operation circuit which performs the convolution operation also performs a convolution operation of the coefficient information and the input feature map information which are next supplied from the channel of the output feature map and the channel of the input feature map included in the set when a zero matrix occurs in the coefficient information, and performs a convolution operation of the coefficient information and the input feature map information which are further next supplied from the channel of the output feature map and the channel of the input feature map included in the set.
3. The arithmetic circuit according to claim 1.
4. allocating the sub-operation circuits, the number of which is less than the number of channels, to each of the sets; An arithmetic circuit according to any one of claims 1 to 3.
5. optimizing channels to be assigned to the sets by allocating the sub-arithmetic circuits corresponding to the sets based on each value of kernel data obtained during inference; An arithmetic circuit according to any one of claims 1 to 4.
6. A calculation method for causing a calculation circuit including a set including at least two output feature map channels based on an output channel and at least three or more sub-calculation circuits to execute a convolution calculation of input feature map information and coefficient information supplied as a plurality of channels, the method comprising: Allocating at least two of the sub-operation circuits to each of the sets; causing the sub-arithmetic circuit included in the set to execute a convolution operation between the coefficient information included in the set and the input feature map information; when a zero matrix occurs in the coefficient information, a sub-operation circuit that performs the convolution operation executes a convolution operation process between the coefficient information that is next supplied and the input feature map information from the channel of the output feature map and the channel of the input feature map included in the set; outputting the convolution result for each channel of the output feature map; Calculation method.
7. A program for causing a calculation circuit including a set including at least two output feature map channels based on an output channel and at least three or more sub-calculation circuits to execute a convolution calculation of input feature map information and coefficient information supplied as a plurality of channels, The program causes the arithmetic circuit to Allocating at least two of the sub-operation circuits to each of the sets; causing the sub-arithmetic circuit included in the set to execute a convolution operation between the coefficient information included in the set and the input feature map information; when a zero matrix occurs in the coefficient information, a sub-operation circuit that performs the convolution operation is caused to execute a convolution operation process between the coefficient information that is next supplied and the input feature map information from the channel of the output feature map and the channel of the input feature map included in the set. program.
Citation Information
Patent Citations
Zero coefficient skipping convolution neural network engine
JP2018028908A
System and method for compact and efficient sparse neural networks
US20190108436A1
Arithmetic processing device
WO2019215907A1
