Calculation device, calculation method, and program

Abstract

A calculation device (10) according to the present disclosure calculates the matrix product of a sparse matrix and a weight matrix in a neural network. The sparse matrix has a structure having a predetermined number of non-zero elements in each block of a predetermined size. The sparse matrix is defined by a local index indicating the positions of the non-zero elements in each of the blocks and the values of the non-zero elements. The calculation device (10) acquires elements corresponding to the positions of the non-zero elements of the sparse matrix from the weight matrix by referencing the local index, and executes a process of performing a product-sum operation of the non-zero elements of the sparse matrix and the corresponding elements of the weight matrix.

Description

Arithmetic device, arithmetic method, and program

[0001] The present invention relates to a neural network computing device, computing method, and program.

[0002] Due to the limited power and computing performance of in-vehicle AI processors, lightweight deep neural network (DNN) models are required. A DNN is composed of the multiplication of learned weights and the feature map inside the network. Generally, wide DNNs with a large number of feature map channels have good performance, but the increased amount of calculations makes them difficult to operate on hardware with limited resources, such as in-vehicle devices.

[0003] It is known that by training a wide DNN so that the feature MAP is sparse and omitting operations related to zero, the number of product-sum operations can be reduced while maintaining performance (Non-Patent Document 1).

[0004] Mark Kurtz et al. “Inducing and exploiting activation sparsity for fast inference on deep neural networks.” 119:5533-5543, 2020.

[0005] The calculation method described in Non-Patent Document 1 omits irregular zero patterns, making it difficult to perform efficient calculations using widely used vector-type calculation units such as GPUs. The purpose of this disclosure is to realize a DNN model that has sparse feature representation and is compatible with vector-type calculation units, thereby enabling power-efficient calculations.

[0006] The arithmetic device according to the present disclosure is an arithmetic device that performs a matrix multiplication operation between a sparse matrix and a weight matrix in a neural network, wherein the sparse matrix has a structure having a predetermined number of non-zero elements for each block of a predetermined size, and the sparse matrix is ​​defined by local indices indicating the positions of the non-zero elements in each block and the values ​​of the non-zero elements, and the arithmetic device references the local indices to obtain elements from the weight matrix that correspond to the positions of the non-zero elements in the sparse matrix, and performs a process of performing a multiply-and-accumulate operation between the non-zero elements of the sparse matrix and the corresponding elements of the weight matrix.

[0007] FIG. 1 is a diagram illustrating an overview of arithmetic processing performed by a arithmetic device according to an embodiment. FIG. 2 is a diagram illustrating an example of a sparse matrix handled by the arithmetic device according to an embodiment. FIG. 3 is a diagram illustrating an example of a local index. FIG. 4 is a diagram illustrating a method for mapping an input matrix onto a space of sparse matrices. FIG. 5 is a diagram illustrating processing of a matrix multiplication of a sparse matrix and a weight matrix. FIG. 6 is a diagram illustrating a configuration of a vector arithmetic device according to an embodiment. FIG. 7 is a flowchart illustrating the operation of arithmetic processing by a arithmetic device. FIG. 8 is a diagram illustrating an example of calculating a matrix multiplication with a weight matrix according to a conventional technique. FIG. 9 is a diagram illustrating the results of a comparison of accuracy using CIFAR-10. FIG. 10 is a diagram illustrating the results of a comparison of accuracy using CIFAR-100. FIG. 11 is a diagram illustrating the results of a comparison of accuracy using ImageNet.

[0008] An arithmetic device according to an embodiment is an arithmetic device that performs a matrix multiplication operation between a sparse matrix and a weight matrix in a neural network, wherein the sparse matrix has a structure having N (M>N) non-zero elements for each block having a size of M elements, and the sparse matrix is ​​defined by local indices indicating the positions of the non-zero elements in each block and the values ​​of the non-zero elements, and the arithmetic device references the local indices to obtain elements from the weight matrix that correspond to the positions of the non-zero elements in the sparse matrix, and performs a process of performing a product-sum operation between the non-zero elements of the sparse matrix and the corresponding elements of the weight matrix.

[0009] By using a sparse matrix with a structure in which N out of M elements are non-zero, it is possible to perform calculations on a neural network with a substantially wide feature representation with the same number of product-sum operations as a narrow neural network with only 1 / M channels. In other words, the expressive power of the neural network can be increased while reducing the calculation load. Furthermore, because this sparse matrix always has values ​​for only N elements within a block, it is possible to perform calculations in parallel for each block, enabling parallel calculations on a vector-type computing device such as a GPU.

[0010] The arithmetic device of the present embodiment may be a vector-type arithmetic device. That is, the arithmetic device of the present embodiment may be a vector-type arithmetic device that includes multiple cores and performs a matrix multiplication operation between a sparse matrix and a weight matrix in a neural network, wherein the sparse matrix has a structure having N nonzero elements for each block having M elements, the sparse matrix is ​​defined by local indices indicating the positions of the nonzero elements in each block and values ​​of the nonzero elements, the multiple cores refer to the local indices for different blocks to obtain elements from the weight matrix that correspond to the positions of the nonzero elements in the sparse matrix, and perform multiplication and accumulation operations between the nonzero elements of the sparse matrix and corresponding elements of the weight matrix in parallel, and any one of the cores may perform a process of summing up the results of the multiplication and accumulation operations obtained by the multiple cores.

[0011] The arithmetic device of the embodiment may receive an input matrix of lower rank than the sparse matrix, map the input matrix into the space of the sparse matrix, and perform a mapping process to generate local indices indicating the positions where the elements of the input matrix are mapped.

[0012] In the embodiment, a structured sparse matrix is ​​realized by mapping an input matrix of lower rank than the sparse matrix onto the space of the sparse matrix. For example, if the weight matrix is ​​made sparse, it is possible to set small-valued elements of the weight coefficients obtained by learning to zero (pruning small values), but the same method cannot be used for the input matrix because the values ​​are not known in advance. Mapping a low-rank input matrix onto a high-dimensional space to generate a sparse matrix is ​​a way to make the input matrix sparsity.

[0013] In the mapping process, the arithmetic device according to the embodiment may determine a destination for mapping an element of the input matrix based on the sign of the element and the signs of elements adjacent to the element.

[0014] In the mapping process, the computing device of the embodiment may use the minimum value among a plurality of elements adjacent to the element to be mapped as the value of the element to be mapped to the space, thereby suppressing changes in the values ​​of non-zero elements of the sparse matrix with respect to changes in the values ​​of each element.

[0015] In the arithmetic device of the embodiment, the non-zero elements of the sparse matrix may be quantized data. The arithmetic device of the embodiment can reduce the amount of calculations using a sparse matrix, but the number of parameters increases compared to a dense matrix having only non-zero elements. Quantizing the data reduces memory usage and alleviates the disadvantage of the increased number of parameters. While the use of quantized data results in a corresponding decrease in accuracy, the arithmetic device of the embodiment has a larger feature dimension than a dense matrix, allowing the accuracy to be restored. By using quantized data in this way, a high-performance neural network can be realized while minimizing the disadvantage of an increased number of parameters.

[0016] The computing device according to the embodiment may perform the process of learning the weighting matrix by adjusting a learning rate for each element of the weighting matrix based on the number of times each element has been updated.

[0017] In this embodiment, a sparse matrix is ​​used, so elements of the weight matrix corresponding to zero elements of the weight matrix are not updated during learning. In other words, learning does not progress uniformly for each element of the weight matrix, and the number of updates varies depending on the element. Learning that assumes that learning progresses uniformly for all elements does not work well. In this embodiment, learning can be performed appropriately by adjusting the learning rate for each element based on the number of updates for each element of the weight matrix.

[0018] An arithmetic method according to an embodiment is a arithmetic method for performing a matrix multiplication of a sparse matrix and a weight matrix in a neural network by a arithmetic device, wherein the sparse matrix has a structure having a predetermined number of non-zero elements for each block of a predetermined size, and the sparse matrix is ​​defined by local indices indicating the positions of the non-zero elements in each block and the values ​​of the non-zero elements, and the arithmetic device includes a step of obtaining elements from the weight matrix that correspond to the positions of the non-zero elements in the sparse matrix by referring to the local indices, and a step of performing a multiply-and-accumulate operation between the non-zero elements of the sparse matrix and the corresponding elements of the weight matrix.

[0019] The computing method of the embodiment may include a step in which the computing device receives an input matrix of a lower rank than the sparse matrix, maps the input matrix into the space of the sparse matrix, and generates a local index indicating the position where the element of the input matrix is ​​mapped.

[0020] A program according to an embodiment is a program for performing a matrix multiplication operation between a sparse matrix and a weight matrix in a neural network, wherein the sparse matrix has a structure having a predetermined number of non-zero elements for each block of a predetermined size, and the sparse matrix is ​​defined by local indices indicating the positions of the non-zero elements within each block and the values ​​of the non-zero elements, and causes a computer to execute the steps of: referencing the local indices to obtain elements from the weight matrix that correspond to the positions of the non-zero elements in the sparse matrix; and performing a multiply-and-accumulate operation between the non-zero elements of the sparse matrix and the corresponding elements of the weight matrix.

[0021] The program of the embodiment may cause a computer to execute a step of receiving an input matrix of a lower rank than the sparse matrix, mapping the input matrix into the space of the sparse matrix, and generating local indices indicating the positions where the elements of the input matrix are mapped.

[0022] The arithmetic device of this embodiment will be described below with reference to the drawings. Note that the following description is merely an example of a preferred embodiment and is not intended to limit the invention described in the claims.

[0023] FIG. 1 is a diagram illustrating an overview of the arithmetic processing performed by a calculation device according to an embodiment. The calculation device performs a matrix multiplication of an input matrix and a weight matrix. The input matrix has a lower rank than the sparse matrix. In the example shown in FIG. 1, the input matrix has 4 rows and 4 columns, the sparse matrix has 16 rows and 4 columns, and the weight matrix has 12 rows and 16 columns. Note that the numbers of matrices handled by the calculation device, including the input matrix, sparse matrix, and weight matrix, are not limited to those shown in FIG. 1. The matrix handled by the calculation device may be a matrix consisting of one row or one column.

[0024] An example of an input matrix is ​​a feature map, which is the activation output of a layer constituting a convolutional neural network (CNN). Here, activation refers to the output of a neural network after passing through an activation function. Note that the arithmetic device of the present disclosure is not limited to CNNs, but can also be applied to activation matrices of other DNNs, such as multilayer perceptrons (MLPs) and attention networks.

[0025] The arithmetic unit maps the input matrix into a space of sparse matrices. In the example shown in FIG. 1, the 4-by-4 input matrix is ​​mapped into a 16-by-4 space. Through this mapping, the value of each element of the input matrix is ​​mapped to an element of the sparse matrix, constituting a non-zero element of the sparse matrix. Elements of the input matrix that are not mapped constitute zero elements of the sparse matrix.

[0026] 2 is a diagram showing an example of a sparse matrix handled by the arithmetic unit of this embodiment. As shown in FIG. 2, the sparse matrix has blocks B1, B2, B3, B4, etc., each consisting of four elements. Although not all blocks are labeled in FIG. 2, the sparse matrix has 16 blocks.

[0027] In the sparse matrix shown in Figure 2, one element in a block of four consecutive elements is a non-zero element. In Figure 2, the shaded elements are non-zero elements, and the unshaded elements are zero elements. The numbers 1 to 4 written on the four elements of block B1 do not indicate the value of the element, but rather the ordinal number of the element within the block. In block B1, the first element is a non-zero element. In other words, the (1, 1) element of the input matrix is ​​mapped to the first element of block B1. The second to fourth elements of block B1 are zero elements.

[0028] In this way, the non-zero elements in a sparse matrix do not exist randomly, but have a rule that each block has one non-zero element, and thus has a fixed structure. Note that, although Fig. 2 shows an example in which the size of a block is four elements, one of which is a non-zero element, the sparse matrix according to the present disclosure has a more general structure in which N (M>N) non-zero elements are included in a block consisting of M consecutive elements.

[0029] Returning to Figure 1, the arithmetic unit maps an input matrix to a sparse matrix space to generate a sparse matrix, and manages the data of the sparse matrix using local indices. The local indices are data that indicate which of the four elements in each block of the sparse matrix are non-zero.

[0030] FIG. 3 is a diagram showing an example of a local index. The example shown in FIG. 3 corresponds to the sparse matrix shown in FIG. 2. That is, index In1 is "1," which indicates that the first element in block B1 is a non-zero element. Note that the value of the non-zero element can be determined by referring to the value of the original input matrix. For example, the value of the non-zero element indicated by index In1 is the value of the element at (1, 1) in the input matrix.

[0031] In this way, a sparse matrix is ​​managed by combining the original input matrix (which is a non-zero matrix) and local indices, and does not have the value of each element of 16 rows and 4 columns as data. The 16 rows and 4 columns diagram shown in Figure 1 is a conceptual diagram for explanation. By using local indices, memory capacity can be reduced more than if the sparse matrix itself were stored.

[0032] Here, we will explain the term "non-zero element." A "non-zero element" is an element that is specified as a non-zero element by a local index. Since elements of the input matrix are mapped to non-zero elements, if an element of the input matrix happens to be zero, the value entered into the non-zero element will be zero, but in this case it is still called a non-zero element.

[0033] 4 is a diagram showing a method for mapping an input matrix to a space of sparse matrices. ｊ The position to map is its element X ｊ and the following element X ｊ＋１ The matrix on the left side of FIG. 4 shows a part of the input matrix, and the elements X ０ ,...X ３ The central diagram in Figure 4 shows the element X ｊ and element X ｊ＋１ The horizontal axis shows the relationship between the sign of X and the position to be mapped. ｊ The vertical axis indicates the sign of X, with the right being positive and the left being negative. ｊ＋１ indicates the sign of the value, with up being positive and down being negative.

[0034] X ｊ is positive, X ｊ＋１ If is positive, it corresponds to "1" in the upper right area, and the element X ｊ is mapped to the first position in the block. ｊ is positive, X ｊ＋１ If is negative, it corresponds to "2" in the lower right area, and the element X ｊ is mapped to the second position in the block. ｊ is negative, X ｊ＋１ If is negative, it corresponds to "3" in the lower left area, and the element X ｊ is mapped to the third position in the block.ｊ is negative, X ｊ＋１ If is positive, it corresponds to "4" in the upper left area, and the element X ｊ is mapped to the fourth element in the block. Thus, the mapping from the input matrix to the sparse matrix space is performed as a function of the input matrix, based on the contents of the input matrix (in this example, the input activations). The rightmost matrix in Figure 4 is a sparse matrix, with one element of the input matrix mapped to each block of the sparse matrix.

[0035] FIG. 5 is a diagram illustrating the process of matrix multiplication between a sparse matrix and a weight matrix. FIG. 5 illustrates an example of a multiplication-and-accumulation operation between the first row R1 of the weight matrix and the first column C1 of the sparse matrix shown in FIG. 1 . To read the value of the first column C1 of the sparse matrix, data in the first column In of the local index is used. The arithmetic unit reads the weight matrix elements at positions corresponding to non-zero elements of the sparse matrix. Specifically, the arithmetic unit obtains four elements indicated by the local index In from the 16 elements in the first row R1 of the weight matrix. The arithmetic unit multiplies the four elements of the obtained weight matrix by each of the four elements in the first row of the sparse matrix, and then adds the obtained products. This obtains the multiplication-and-accumulation operation between the first row R1 of the weight matrix and the first column C1 of the sparse matrix.

[0036] 6 is a diagram showing the configuration of a computing device 10 according to an embodiment. The computing device 10 is a vector computing device, and includes a CPU 11, a GPU 12, and a storage unit 13. A data input unit 20 and an output unit 21 are connected to the computing device 10. The computing device 10 has a function of inputting data to the computing device 10, and the output unit 21 has a function of outputting the results of computation by the computing device 10, etc.

[0037] The GPU 12 has multiple cores and can execute processing tasks in parallel using the multiple cores. The storage unit 13 stores programs and neural network models for executing the processing of the arithmetic device 10. The CPU 11 and GPU 12 read and execute the programs stored in the storage unit 13 to realize the functions of the arithmetic device 10 and perform the operations shown in FIG. 7 .

[0038] 7 is a flowchart showing the operation of the calculation process by the calculation device 10. This figure shows the operation of calculating the matrix product of the input matrix and the weight matrix for one layer of the neural network. The calculation device 10 acquires the input matrix (S10). The input matrix is, for example, the activation matrix output from the previous layer.

[0039] The arithmetic unit 10 maps the input matrix to a space of sparse matrices (S11). As described in Fig. 4, the arithmetic unit 10 determines the mapping position within each block based on the sign of each element of the input matrix, thereby generating a structured sparse matrix (conceptually, see Fig. 2, for example) in which each block contains one non-zero element.

[0040] The arithmetic unit 10 calculates the matrix product of the sparse matrix and the weight matrix (S12). Specifically, the multiply-and-accumulate operation is performed in parallel for each block using multiple cores of the GPU 12. For example, the multiply-and-accumulate operations for blocks B1 to B4, blocks B5 to B8, blocks B9 to B12, and blocks B13 to B16 are each performed by a separate core.

[0041] Each core of the computing device 10 performs an indexing operation to obtain elements corresponding to non-zero elements from the weight matrix by referring to the local index (S13), and performs a multiply-and-accumulate operation between the obtained elements of the weight matrix and the elements of the sparse matrix (S14). This allows the computing device 10 to obtain values ​​for each element of the output matrix. The computing device 10 outputs the obtained output matrix to, for example, the next layer (S15).

[0042] The above has described the arithmetic device 10 and the arithmetic method according to the present embodiment. The arithmetic device 10 according to the present embodiment maps an input matrix to a structured sparse matrix and calculates the matrix product of the sparse matrix and a weight matrix, thereby improving the expressive power of the neural network.

[0043] 8 shows an example of computing the matrix product of a 4-by-4 input matrix with a weight matrix without mapping it to a sparse matrix. As shown in FIG. 8, in order to compute the matrix product with a 4-by-4 input matrix, the number of columns in the weight matrix is ​​limited to four. As is clear from a comparison of FIG. 1 and FIG. 8, in this embodiment, by mapping the input matrix to a sparse matrix, a larger weight matrix can be used, thereby enhancing the expressive power of the neural network.

[0044] Furthermore, the sparse matrix used in this embodiment is a structured sparse matrix in which the number of non-zero elements in each block is fixed, and the positions of the non-zero elements are managed as local indices, so that the multiplication and accumulation operations for zero elements can be omitted by index manipulation, and the operation can be performed with substantially the same load as the operation of a matrix multiplication with the input matrix before sparsification. Furthermore, since the operation can be performed for each block, parallel processing can be achieved in the GPU 12, and the processing time can be shortened.

[0045] Next, learning of the weight matrix used in the arithmetic device 10 of this embodiment will be described. Learning of the weight matrix uses the same concept as learning methods for neural network models such as Adam (Diederik P. Kingma and Jimmy Ba. Adam, "A method for stochastic optimization." December 2014.) and AdamW (Ilya Loshchilov and Frank Hutter. "Decoupled weight decay regularization." In International Conference on Learning Representations, 2019.). However, in learning of the weight matrix of this embodiment, a learning rate is adjusted for each element. Specifically, the learning rate is adjusted based on the number of updates to each weight element.

[0046] Adam and AdamW use an adaptive learning rate based on the statistical value of gradient history, but it is known that this adaptive learning rate may become unstable in the early stages of learning when the number of experienced gradients is small. In the neural network of this embodiment, this problem may become more pronounced because each weight element receives gradients less frequently than in a dense network (on average, 1 / M times less frequently). In this embodiment, we propose a method that modifies the formulation of RAdam. In this specification, this proposed method is called "ERAdam (Experience-RAdam)."

[0047] First, we will explain RAdam, which reduces variance in the early stages of learning by multiplying the adaptive learning rate of Adam or AdamW by rt<1. where t is the optimization step and β is a hyperparameter used to calculate the running average of the gradient. As t → ∞ (training progresses), rt → 1, restoring the original Adam or AdamW update.

[0048] In the proposed ERAdam, the step t for calculating rt of each weight element is scaled proportionally to the number of received gradients. Specifically, instead of t, a scaled time vector t∈R is used: C￣i is used.

[0049] In this embodiment, we combine the optimizer with a scheduler based on CosineDecay (Ilya Loshchilov and Frank Hutter, "SGDR: Stochastic gradient descent with warm restarts," In International Conference on Learning Representations, 2017.) with k-decay (Tao Zhang and Wei Li, "kdecay," arXiv preprint arXiv:2004.05909, 2020.).

[0050] Although the arithmetic device and arithmetic method of the present disclosure have been described above by way of embodiments, the present disclosure is not limited to the above-described embodiments.

[0051] In the above embodiment, an example was given in which an input matrix is ​​mapped to a sparse matrix and then the matrix product of the sparse matrix and a weight matrix is ​​calculated, but a structured sparse matrix may be input as the input matrix. By calculating the matrix product of the structured sparse matrix and a weight matrix using a local index, it is possible to reduce the amount of calculation while increasing the expressive power of the neural network.

[0052] The computing device of the present disclosure may use quantized data as the non-zero elements of the sparse matrix. By using quantized data, the amount of computation can be reduced.

[0053] When mapping an input matrix to a sparse matrix space, the arithmetic unit of the present disclosure may use the minimum value among multiple elements adjacent to the element to be mapped as the value of the element to be mapped. Since the arithmetic unit of the present disclosure determines the mapping destination of the input matrix as a function of the input matrix, the continuity between the input matrix and the sparse matrix may be lost. By using a small value as the element to be mapped, it is possible to suppress changes in the values ​​of non-zero elements when the values ​​of each element change.

[0054] In the examples, the expressive power was compared between when a sparse matrix was used and when it was not used, and the method of the present disclosure for making an input matrix sparse was compared with the method for making a weight matrix sparse.

[0055] We evaluate classification accuracy on CIFAR-10 (Alex Krizhevsky et al., "Cifar-10," Canadian Institute for Advanced Research), CIFAR-100 (Alex Krizhevsky et al., "Cifar-100," Canadian Institute for Advanced Research), and ImageNet (Jia Deng et al., "ImageNet: A large-scale hierarchical image database," In 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, June 2009). Because commonly used networks already achieve good saturation accuracy, we employ narrower versions as base networks to better evaluate the effect of using sparse activations and weights. Specifically, we use a narrower width (W∈R) for all layers except the first and final layers. Co / α×C￣i / α ) network, and approximately α ２ × consumes fewer multi-counts. ReLU, SWS (Structured Weight Sparsification), and the network disclosed herein (hereinafter referred to as "SAS (Structured Activation Sparsification)") consume the same number of multi-counts for the same α. SWS is a technique for making the weight matrix sparse instead of activation, as described in Aojun Zhou et al., "Learning N:M fine-grained structured sparse neural networks from scratch," Feb. 2021.

[0056] For fair comparison, we adopted the same training strategy for all variables, i.e., training epochs, batch size, optimizer, scheduler, etc. For all variables, we used the ERAdam optimizer described in this embodiment. Because ∇X in equation (1) above is almost the same for each weight element, this is equivalent to RAdam for the base network. We adopted the method by Aojun Zhou et al. to train the SWS network because the codebase supports arbitrary 1:M sparsity and allows training from scratch, which allows for fair comparison with SAS. In this example, we use the data argumentation adopted in the ConvNeXt paper (Zhuang Liu et al., "A convnet for the 2020s," In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 11976-11986, 2022.) to train the network from scratch in all experiments. The main experimental setup is summarized in the following table.

[0057] For CIFAR-10 and CIFAR-100, we used ResNet18 (Kaiming He et al., "Deep residual learning for image recognition," In IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016), one of the most common architectures. Figure 9 shows the results for CIFAR-10, and Figure 10 shows the results for CIFAR-100. For ImageNet, we used the method from the ConvNeXt paper. ConvNeXt is computationally efficient due to its network architecture design. For example, it makes extensive use of depthwise convolutions to reduce the number of iterations. Figure 11 shows the results for ImageNet.

[0058] 9 to 11, the vertical axis indicates accuracy, and the horizontal axis indicates the experimental conditions. ReLU on the horizontal axis indicates the results of a network without sparsification, while M=2, M=4, M=8, and M=16 indicate the results of M-fold sparsification. The solid line indicates the results of SAS, and the dotted line indicates the results of SWS.

[0059] This comparative experiment shows that accuracy improves as activation sparsity M increases. Across the entire range of base network widths (with different α), the largest gains are observed from the base ReLU network to the sparse network with M = 2. The improvement saturates at M = 8 or M = 16. Similarly, large gains are observed when the base network capacity is small (large α).

[0060] Although the SWS network also shows improvement for weight sparsity M, the accuracy of SAS significantly outperforms SWS for the same sparsity M.

[0061] SAS improved accuracy across a wide range of network structures and datasets without increasing multicounting and actual inference time. The increased flexibility provided by the increased weights appears to be the main factor behind the improved accuracy of SAS over SWS (with the same computational effort).

Claims

1. A calculation device that performs a matrix multiplication operation between a sparse matrix and a weight matrix in a neural network, wherein the sparse matrix has a structure having a predetermined number of non-zero elements for each block of a predetermined size, and the sparse matrix is ​​defined by a local index indicating the position of the non-zero elements in each block and the values ​​of the non-zero elements, and the calculation device executes a process of obtaining elements from the weight matrix that correspond to the positions of the non-zero elements of the sparse matrix by referring to the local index, and performing a process of performing a multiply-and-accumulate operation between the non-zero elements of the sparse matrix and corresponding elements of the weight matrix.

2. The arithmetic device according to claim 1, which executes a mapping process that receives an input matrix of a lower rank than the sparse matrix, maps the input matrix into the space of the sparse matrix, and generates a local index indicating the position where the elements of the input matrix are mapped.

3. The arithmetic unit according to claim 2, wherein in said mapping process, a destination for mapping an element of said input matrix is ​​determined based on the sign of said element and the signs of elements adjacent to said element.

4. The arithmetic device according to claim 2, wherein in said mapping process, the minimum value among a plurality of elements adjacent to the element to be mapped is used as the value of the element to be mapped into said space.

5. The arithmetic unit according to claim 1 or 2, wherein the non-zero elements of the sparse matrix are quantized data.

6. The arithmetic device according to claim 1 or 2, wherein the process of learning the weighting matrix comprises adjusting a learning rate for each element of the weighting matrix based on the number of times each element of the weighting matrix has been updated to perform learning.

7. A computation method for performing a matrix multiplication operation between a sparse matrix and a weight matrix in a neural network by a computation device, wherein the sparse matrix has a structure having a predetermined number of non-zero elements for each block of a predetermined size, and the sparse matrix is ​​defined by local indices indicating the positions of the non-zero elements in each block and values ​​of the non-zero elements, the computation method comprising: a step in which the computation device refers to the local indices to obtain elements from the weight matrix that correspond to the positions of the non-zero elements in the sparse matrix; and a step in which the computation device performs a multiply-and-accumulate operation between the non-zero elements of the sparse matrix and corresponding elements of the weight matrix.

8. The computation method according to claim 7, further comprising the steps of: receiving an input matrix having a lower rank than the sparse matrix; mapping the input matrix into the space of the sparse matrix; and generating a local index indicating the position where the elements of the input matrix are mapped.

9. A program for performing a multiply-and-accumulate operation between a sparse matrix and a weighting matrix in a neural network, wherein the sparse matrix has a structure having a predetermined number of nonzero elements for each block of a predetermined size, and the sparse matrix is ​​defined by local indices indicating the positions of the nonzero elements in each block and values ​​of the nonzero elements, and the program causes a computer to execute the steps of: referring to the local indices to obtain elements from the weighting matrix that correspond to the positions of the nonzero elements in the sparse matrix; and performing a multiply-and-accumulate operation between the nonzero elements of the sparse matrix and corresponding elements of the weighting matrix.

10. The program according to claim 9, which causes a computer to execute the steps of receiving an input matrix having a lower rank than the sparse matrix, mapping the input matrix into the space of the sparse matrix, and generating local indices indicating positions where elements of the input matrix have been mapped.