Data compression method, data compression system and calculation method for deep learning acceleration chip

Abstract

A data compression method, data compression system, and operation method for a deep learning acceleration chip. The data compression method includes the following steps: obtaining a filter coefficient tensor matrix of a deep learning model. Performing a matrix decomposition procedure based on the filter coefficient tensor matrix to obtain at least one sparse tensor matrix and at least one transformation matrix. The product of the transformation matrix and the filter coefficient tensor matrix is ​​a sparse tensor matrix. The transformation matrix is ​​an orthogonal normal matrix. Compressing the sparse tensor matrix. Storing the sparse tensor matrix and the transformation matrix, or storing the sparse tensor matrix and the reduction matrix in a memory. The deep learning acceleration chip performs operations using the sparse tensor matrix to obtain a convolution operation result. The deep learning acceleration chip then uses the reduction matrix to restore the convolution operation result.

Description

Technical Field

[0001] The present invention relates to a data compression method, a data compression system and an operation method for a deep learning acceleration chip. Background Art

[0002] With the development of deep learning technology, a deep learning acceleration chip has been developed. This deep learning acceleration chip can directly calculate complex convolution operations in hardware, thereby speeding up the operation.

[0003] Deep learning accelerator chips paired with large-capacity memory exchange temporary data via the system bus. Ideally, data transfer and computation proceed in parallel. However, in reality, due to shared buses or physical limitations, data transfer occupies the majority of the processing time, resulting in subpar performance. Furthermore, external memory accesses contribute significantly to power consumption. Amdahl's law also suggests that the efficiency gains from parallelism have a ceiling, and recent research indicates that this performance ratio is capped at 10TOPs / W. Breaking this performance limit has become a major technical bottleneck for deep learning accelerator chips. Summary of the Invention

[0004] The present invention relates to a data compression method, a data compression system and an operation method for a deep learning acceleration chip.

[0005] According to one embodiment of the present invention, a data compression method for a deep learning acceleration chip is proposed. The data compression method for a deep learning acceleration chip includes the following steps. Obtain a filter coefficient tensor matrix of a deep learning model. Perform a matrix decomposition procedure based on the filter coefficient tensor matrix to obtain at least one sparse tensor matrix and at least one transformation matrix. The product of the transformation matrix and the filter coefficient tensor matrix is ​​a sparse tensor matrix. The transformation matrix is ​​an orthonormal matrix. Compress the sparse tensor matrix. Store the sparse tensor matrix and the transformation matrix, or store the sparse tensor matrix and the reduction matrix in a memory. The deep learning acceleration chip uses the sparse tensor matrix to perform operations to obtain a convolution operation result. The deep learning acceleration chip also uses the reduction matrix to restore the convolution operation result.

[0006] According to another embodiment of the present invention, a data compression system for a deep learning acceleration chip is proposed. The data compression system is used to reduce the amount of data moved by a deep learning model. The data compression system includes a decomposition unit, a compression unit, and a transfer unit. The decomposition unit is used to perform a matrix decomposition procedure based on a filter coefficient tensor matrix to obtain at least one sparse tensor matrix and at least one transformation matrix. The product of the transformation matrix and the filter coefficient tensor matrix is ​​a sparse tensor matrix. The transformation matrix is ​​an orthonormal matrix. The compression unit is used to compress the sparse tensor matrix. The transfer unit is used to store the sparse tensor matrix and the transformation matrix in a memory, or to store the sparse tensor matrix and the reduction matrix in a memory. The deep learning acceleration chip uses the sparse tensor matrix to perform operations to obtain convolution operation results. The deep learning acceleration chip also uses the reduction matrix to restore the convolution operation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of a deep learning acceleration chip according to one embodiment;

[0008] Figure 2 Examples illustrate the computing process of deep learning acceleration chips;

[0009] Figure 3 Changes in the data required to be stored in memory;

[0010] Figure 4 is a block diagram of a data compression system of a deep learning acceleration chip according to one embodiment;

[0011] Figure 5 This is a flowchart of a data compression method for a deep learning acceleration chip according to one embodiment;

[0012] Figure 6 is a flowchart of a calculation method of a deep learning acceleration chip according to one embodiment;

[0013] Figure 7 A matrix decomposition procedure for a filter coefficient tensor matrix according to an embodiment;

[0014] Figures 8A to 8E : is a comparison result of the overall data volume of the convolution layer according to one embodiment;

[0015] Figures 8F to 8H : is a comparison result of the overall data volume of a fully connected layer according to one embodiment;

[0016] Figure 9 Data reduction ratio curve for convolutional layers and fully connected layers.

[0017] [Description of Reference Numerals]

[0018] 100: Deep Learning Acceleration Chip

[0019] 110: register

[0020] 120: Arithmetic unit

[0021] 200: Memory

[0022] 300:Data Compression System

[0023] 310: Decomposition unit

[0024] 320: Compression unit

[0025] 330: Transfer unit

[0026] A,A1,A2: transformation matrix

[0027] A T :Restore Matrix

[0028] Conv1, Conv2, Conv3, Conv4, Conv5: convolutional layers

[0029] D0: original data volume

[0030] FC6, FC7, FC8: fully connected layers

[0031] H,H1,H2: filter coefficient tensor matrix

[0032] M: number of divisions

[0033] S110, S120, S130, S140, S210, S220, S230: Steps

[0034] W,W1,W2: sparse tensor matrix

[0035] X: Identification data

[0036] Y, Y′: convolution operation result DETAILED DESCRIPTION

[0037] Please refer to Figure 1 , which is a schematic diagram of a deep learning acceleration chip 100 according to one embodiment. During operation, the deep learning acceleration chip 100 requires an external memory 200 (e.g., DRAM) to store the trained filter coefficient tensor matrix H of the deep learning model. After the filter coefficient tensor matrix H is transferred to the register 110, it is processed by the computing unit 120.

[0038] Researchers have discovered that the most time-consuming and power-intensive process in the deep learning accelerator chip 100 is transferring the filter coefficient tensor matrix H from the memory 200. Therefore, the researchers are working to reduce the amount of data transferred between the memory 200 and the deep learning accelerator chip 100, thereby increasing the processing speed of the deep learning accelerator chip 100 and reducing power consumption.

[0039] Please refer to Figure 2 , which illustrates the operation process of the deep learning acceleration chip 100. During the operation process of the deep learning acceleration chip 100, the filter coefficient tensor matrix H and the identification data X are obtained. Figure 2 As shown in the following formula (1), after the filter coefficient tensor matrix H is operated on the identification data X, a convolution operation result Y can be obtained.

[0040] H×X=Y……………………(1)

[0041] The filter coefficient tensor matrix H is a non-sparse matrix and cannot be compressed, or can only be compressed to a very limited degree. In this embodiment, a sparse tensor matrix W is obtained through a matrix decomposition process. As shown in the following equation (2), the sparse tensor matrix W is the product of a transformation matrix A and the filter coefficient tensor matrix H. The transformation matrix is ​​an orthonormal matrix. The sparse tensor matrix W is a sparse matrix and can be significantly compressed.

[0042] W=A×H……………………(2)

[0043] The memory 200 can store the compressible sparse tensor matrix W and the transformation matrix A. Figure 2 As shown in the following formula (3), the deep learning acceleration chip 100 can first use the sparse tensor matrix W to calculate a convolution operation result Y′.

[0044] W×X=Y′……………………(3)

[0045] Then, if Figure 2 As shown in the following formula (4), the deep learning acceleration chip 100 can use a reduction matrix A T The convolution operation result Y′ is restored to the convolution operation result Y. Restore the matrix A T is a transposed matrix of the transformation matrix A.

[0046] A T ×Y′

[0047] =A T ×(W×X)

[0048] =A T ×((A×H)×X)

[0049] =A T ×A×H×X

[0050] =H×X

[0051] =Y……………………(4)

[0052] Therefore, the convolution operation result Y originally calculated using the filter coefficient tensor matrix H can be changed to use the sparse tensor matrix W to calculate the convolution operation result Y′, and the convolution operation result Y′ can be restored to the convolution operation result Y without loss. Figure 3 , which is the change of data that the memory 200 needs to store. Figure 3 As shown, the data required to be stored in memory 200 is originally the filter coefficient tensor matrix H, which is not easily compressible. Instead, the data required to be stored in memory 200 can be converted to the compressible sparse tensor matrix W and the transformation matrix A. The data size of the transformation matrix A is very small and does not increase the data volume significantly. This significantly reduces the amount of data moved between memory 200 and the deep learning acceleration chip 100, significantly increasing the computing speed of the deep learning acceleration chip 100 and reducing power consumption.

[0053] Please refer to Figure 4 , which is a block diagram of a data compression system 300 of a deep learning acceleration chip 100 according to one embodiment. The data compression system 300 is used to reduce the amount of data moved by a deep learning model. The data compression system 300 includes a decomposition unit 310, a compression unit 320, and a transfer unit 330. The functions of each component are summarized as follows. The decomposition unit 310 is used to perform a matrix decomposition procedure. The compression unit 320 is used to perform a data compression procedure. The transfer unit 330 is used to perform a data transfer procedure. The decomposition unit 310, the compression unit 320, and the transfer unit 330 are, for example, a circuit, a chip, a circuit board, a program code, or a storage device for storing program code. The data compression system 300 of this embodiment performs a matrix decomposition procedure through the decomposition unit 310 to obtain a sparse tensor matrix W and a transformation matrix A. The compression unit 320 can compress the sparse tensor matrix W. In this way, the data required to be stored in the memory 200 can be converted into a compressible sparse tensor matrix W and a transformation matrix A, which significantly reduces the amount of data moved, thereby significantly increasing the computing speed of the deep learning acceleration chip 100 and reducing power consumption. The operation of each of the above components is further explained in detail below through a flowchart.

[0054] Please refer to Figure 5 , which is a flow chart of a data compression method for the deep learning acceleration chip 100 according to one embodiment. In step S110, the decomposition unit 310 obtains the filter coefficient tensor matrix H of the deep learning model. The filter coefficient tensor matrix H is a non-sparse matrix and cannot be compressed, or the degree of compression that can be performed is very small.

[0055] Next, in step S120, the decomposition unit 310 performs a matrix decomposition procedure based on the filter coefficient tensor matrix H to obtain a sparse tensor matrix W and a transformation matrix A. The product of the transformation matrix A and the filter coefficient tensor matrix H is the sparse tensor matrix W, i.e., the above equation (2). The transformation matrix A is an orthogonal normal matrix. After the filter coefficient tensor matrix H is multiplied by the transformation matrix A, the matrix contents undergo finite precision quantization, resulting in a large number of repeated values, i.e., very few non-repeated elements. This type of matrix is ​​called a sparse matrix.

[0056] Then, in step S130 , the compression unit 320 compresses the sparse tensor matrix W. In this step, the compression unit 320 may compress the sparse tensor matrix W using run-length encoding, a compressed sparse column format, or a cross-linked list.

[0057] Next, in step S140, the transfer unit 330 transfers the compressed sparse tensor matrix W and the conversion matrix A to the memory 200, or transfers the compressed sparse tensor matrix W and the restoration matrix A to the memory 200. T The deep learning acceleration chip 100 can use the sparse tensor matrix W to perform operations to obtain the convolution operation result Y′ and the above formula (3). The deep learning acceleration chip 100 also uses the restored matrix A T The convolution operation result Y′ is restored to obtain the convolution operation result Y, that is, the above formula (4). The following further describes in detail the sparse tensor matrix W and the conversion matrix A (or the restoration matrix A) of this embodiment through the flowchart. T )How to perform operations on the deep learning acceleration chip 100.

[0058] Please refer to Figure 6 , which is a flow chart of a calculation method of the deep learning acceleration chip 100 according to one embodiment. In step S210, the deep learning acceleration chip 100 obtains the sparse tensor matrix W and the conversion matrix A, or the sparse tensor matrix W and the restoration matrix A from the memory 200. T The product of the conversion matrix A and the filter coefficient tensor matrix H of the deep learning model is the sparse tensor matrix W. The conversion matrix A is an orthogonal normal matrix. Restore matrix A T is the transposed matrix of the conversion matrix A. The memory 200 does not store the filter coefficient tensor matrix H. In this step, if the conversion matrix A is obtained from the memory 200, the deep learning acceleration chip 100 further obtains the restoration matrix A based on the conversion matrix A. TThe matrix transpose operation does not consume much time and power.

[0059] Then, in step S220, the deep learning acceleration chip 100 performs a calculation using the sparse tensor matrix W to obtain a convolution operation result Y′, such as the calculation of equation (3). The convolution operation result Y′ is not the final result, but because the sparse tensor matrix W is a sparse matrix, the calculation in this step is much faster than the calculation using the filter coefficient tensor matrix H.

[0060] Next, in step S230, the deep learning acceleration chip 100 uses the restored matrix A T The convolution operation result Y′ is restored, for example, by the operation of equation (4). Therefore, the convolution operation result Y originally calculated using the filter coefficient tensor matrix H can be calculated using the sparse tensor matrix W to obtain the convolution operation result Y′. The convolution operation result Y′ can be restored to the convolution operation result Y without loss.

[0061] In addition, researchers also used matrix partitioning technology to further reduce the amount of data moved. Figure 7 , which is a matrix partitioning procedure of the filter coefficient tensor matrix H according to one embodiment. Figure 7 In the example of , the identification data X (such as Figure 2 For example, the image (shown in the figure) is a color image with a length and width of 11 pixels by 11 pixels, with each pixel having three pixel values: red, green, and blue. The deep learning model, for example, has 96 filters. Therefore, the deep learning model needs to train a filter coefficient tensor matrix H of size (11*96)*(11*11*3). Following the above steps, a sparse tensor matrix W of size (11*96)*(11*11*3) and a transformation matrix A of size (11*96)*(11*96) can be obtained in the matrix decomposition process.

[0062] Researchers can use matrix partitioning technology to partition the filter coefficient tensor matrix H into M parts. M is 2^N, N is a natural number, Figure 7In the example, N=1. The filter coefficient tensor matrix H is evenly divided into filter coefficient tensor matrices H1 and H2 of size (11*48)*(11*11*3). The sparse tensor matrices H1 and H2 have the same size. According to the above steps, the filter coefficient tensor matrix H1 can obtain a sparse tensor matrix W1 of size (11*48)*(11*11*3) and a conversion matrix A1 of size (11*48)*(11*48) through a matrix decomposition program; the filter coefficient tensor matrix H2 can obtain a sparse tensor matrix W2 of size (11*48)*(11*11*3) and a conversion matrix A2 of size (11*48)*(11*48) through a matrix decomposition program. The sparse tensor matrices W1 and W2 have the same size, and the conversion matrices A1 and A2 have the same size.

[0063] The data required to be stored in memory 200 are compressible sparse tensor matrices W1 and W2 and transformation matrices A1 and A2. The data size of transformation matrices A1 and A2 is significantly lower than that of transformation matrix A. Therefore, it is possible to further reduce the amount of data moved. The sparsity of the sparse tensor matrices W1 and W2 may be lower than that of the sparse tensor matrix W. However, as long as the reduction in the data size of transformation matrices A1 and A2 relative to transformation matrix A is greater than the increase in the data size of sparse tensor matrices W1 and W2 relative to sparse tensor matrix W, this approach can still be used to reduce the overall data size.

[0064] In addition, the above Figure 7 The example is N = 1. N = 2, 3, 4, ..., may also produce different results. The data compression system 300 can determine the appropriate value of N based on the comparison result of the overall data volume. For example, please refer to Figures 8A to 8E , which is a comparison result of the overall data volume of the convolution layers Conv1 to Conv5 according to one embodiment. Figure 8A As shown, in the convolution layer Conv1, when M = 1, 2, 4, 8, 16, 32 (i.e., N = 0, 1, 2, 3, 4, 5), the overall data volume can be lower than the original data volume D0 of the filter coefficient tensor matrix H. The data compression system 300 can decide to use M = 2 (N = 1) in the convolution layer Conv1 to achieve the maximum data reduction. Figure 8B As shown, in the convolution layer Conv2, the data compression system 300 may decide to use M=1 (N=0) in the convolution layer Conv2 to achieve the maximum data reduction. Figure 8C As shown, in the convolution layer Conv3, the data compression system 300 may decide to use M=4 (N=2) in the convolution layer Conv3 to achieve the maximum data reduction. Figure 8DAs shown, in the convolution layer Conv4, the data compression system 300 may decide to use M=1 (N=0) in the convolution layer Conv4 to achieve the maximum data reduction. Figure 8E As shown, in the convolution layer Conv5, the data compression system 300 may decide to use M=1 (N=0) in the convolution layer Conv5 to achieve the maximum data reduction.

[0065] Please refer to Figures 8F to 8H , which is a comparison result of the overall data volume of the fully connected layers FC6 to FC8 according to one embodiment. Figure 8F As shown, in the fully connected layer FC6, when M=2, 4, 8, 16 (i.e., N=1, 2, 3, 4), the overall data volume can be lower than the original data volume D0 of the filter coefficient tensor matrix H. The data compression system 300 can decide to use M=8 (N=3) in the fully connected layer FC6 to achieve the maximum data reduction. Figure 8G As shown, in the fully connected layer FC7, when M=8 (i.e., N=3), the overall data volume can be lower than the original data volume D0 of the filter coefficient tensor matrix H. Therefore, the data compression system 300 can decide to use M=8 (N=3) in the fully connected layer FC6 to achieve the maximum data reduction. Figure 8H As shown, in the fully connected layer FC8, when M=2, 4, 8, 16 (i.e., N=1, 2, 3, 4), the overall data volume can be lower than the original data volume D0 of the filter coefficient tensor matrix H, wherein the data compression system 300 can decide to use M=4 (N=2) in the fully connected layer FC8 to achieve the maximum data reduction.

[0066] Please refer to Figure 9 , which is the data reduction ratio curve of the convolutional layers Conv1~Conv5 and the fully connected layers FC6~FC8. Figure 9 It can be seen that the convolutional layers Conv1 to Conv5 and the fully connected layers FC6 to FC8 can use the most suitable M and N to perform the matrix segmentation procedure, and a good data reduction ratio can be obtained in the convolutional layers Conv1 to Conv5 and the fully connected layers FC6 to FC8.

[0067] According to the above embodiment, after the data required to be stored in the memory 200 is changed to a compressible sparse tensor matrix W and a conversion matrix A, the amount of data is relatively reduced. In this way, the amount of data moved between the memory 200 and the deep learning acceleration chip 100 can be greatly reduced, thereby greatly accelerating the computing speed of the deep learning acceleration chip 100. In addition, the convolution operation result Y originally calculated using the filter coefficient tensor matrix H can be replaced by using the sparse tensor matrix W to calculate the convolution operation result Y′. The convolution operation result Y′ can be restored to the convolution operation result Y without loss of computational accuracy.

[0068] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data compression method for a deep learning acceleration chip, characterized in that: The data compression method of the deep learning acceleration chip includes: Get the filter coefficient tensor matrix of the deep learning model; performing a matrix decomposition process on the filter coefficient tensor matrix to obtain at least one sparse tensor matrix and at least one transformation matrix, wherein the product of the transformation matrix and the filter coefficient tensor matrix is ​​the sparse tensor matrix, and the transformation matrix is ​​an orthogonal normal matrix; compressing the sparse tensor matrix; and The sparse tensor matrix and the transformation matrix are stored in a memory, or the sparse tensor matrix and the restoration matrix are stored in a memory, where the restoration matrix is ​​the transposed matrix of the transformation matrix. The deep learning acceleration chip uses the sparse tensor matrix and the identification data to be identified by the deep learning model to perform operations to obtain a convolution operation result. The deep learning acceleration chip also uses the restoration matrix to restore the convolution operation result to obtain a restored convolution operation result, wherein the restored convolution operation result is a lossless restoration.

2. The data compression method for a deep learning acceleration chip according to claim 1, wherein in the step of performing the matrix decomposition procedure based on the filter coefficient tensor matrix, the filter coefficient tensor matrix is ​​divided into M parts, the number of the at least one sparse tensor matrix is ​​M, the number of the at least one transformation matrix is ​​M, and M is a natural number.

3. The data compression method for a deep learning acceleration chip according to claim 2, wherein M is 2^N and N is a natural number.

4. The data compression method of a deep learning acceleration chip according to claim 2, wherein the filter coefficient tensor matrix is ​​evenly divided.

5. The data compression method for a deep learning acceleration chip according to claim 2, wherein the sparse tensor matrices have the same size.

6. The data compression method for a deep learning acceleration chip according to claim 2, wherein the transformation matrices have the same size.

7. A data compression system for a deep learning acceleration chip, characterized in that: The data compression system is used to reduce the amount of data moved by deep learning models. The data compression system includes: a decomposition unit configured to perform a matrix decomposition process on the filter coefficient tensor matrix to obtain at least one sparse tensor matrix and at least one transformation matrix, wherein the product of the transformation matrix and the filter coefficient tensor matrix is ​​the sparse tensor matrix, and the transformation matrix is ​​an orthogonal normal matrix; A compression unit, configured to compress the sparse tensor matrix; and A transfer unit is used to store the sparse tensor matrix and the transformation matrix in a memory, or to store the sparse tensor matrix and the restoration matrix in the memory, wherein the restoration matrix is ​​the transposed matrix of the transformation matrix. The deep learning acceleration chip uses the sparse tensor matrix and the identification data to be identified by the deep learning model to perform operations to obtain a convolution operation result. The deep learning acceleration chip also uses the restoration matrix to restore the convolution operation result to obtain a restored convolution operation result, wherein the restored convolution operation result is a lossless restoration.

8. The data compression system of the deep learning acceleration chip according to claim 7, wherein the decomposition unit further divides the filter coefficient tensor matrix into M parts, the number of the at least one sparse tensor matrix is ​​M, the number of the at least one transformation matrix is ​​M, and M is a natural number.

9. The data compression system of the deep learning acceleration chip according to claim 8, wherein M is 2^N and N is a natural number.

10. The data compression system of the deep learning acceleration chip according to claim 8, wherein the filter coefficient tensor matrix is ​​evenly divided.

11. The data compression system of the deep learning acceleration chip according to claim 8, wherein the sparse tensor matrices have the same size.

12. The data compression system of the deep learning acceleration chip according to claim 8, wherein the transformation matrices have the same size.

13. A calculation method for a deep learning acceleration chip, characterized in that: The computing method of the deep learning acceleration chip includes: Obtaining at least one sparse tensor matrix and at least one transformation matrix from a memory, or obtaining at least one sparse tensor matrix and at least one restoration matrix from a memory, wherein the sparse tensor matrix is ​​a sparse tensor matrix obtained by compressing the product of the transformation matrix and a filter coefficient tensor matrix of the deep learning model, the transformation matrix is ​​an orthogonal normal matrix, and the restoration matrix is ​​a transposed matrix of the transformation matrix; Performing a convolution operation on the sparse tensor matrix and the identification data to be identified by the deep learning model to obtain a convolution operation result; and The convolution operation result is restored using the restoration matrix to obtain a restored convolution operation result, wherein the restored convolution operation result is a lossless restoration.

14. The computing method of the deep learning acceleration chip according to claim 13, wherein the number of the at least one sparse tensor matrix is ​​2^N, the number of the at least one transformation matrix is ​​2^N, and N is a natural number.

15. The computing method of the deep learning acceleration chip according to claim 13, wherein the sparse tensor matrices have the same size.

16. The calculation method of the deep learning acceleration chip according to claim 13, wherein the transformation matrices have the same size.

Images