Sparse data compression apparatus and method for neural network tensor processors

Abstract

The present application relates to the field of neural network tensor processors, and particularly relates to a sparse data compression device and method for a neural network tensor processor. The method comprises the following steps: reading configuration data of a compression operation and configuring each module of the sparse data compression device; reading sparse data according to the configuration information; judging and marking sparse feature points; deleting sparse feature point data, generating sparse compressed data and a sparse mapping table; writing out the sparse compressed data; and writing out the sparse mapping table. The method can efficiently realize 0 data deletion and data rearrangement of sparse data by traversing the sparse data, judging and marking the sparse feature points, deleting the sparse feature points and constructing the sparse mapping table, thereby efficiently realizing compression of the sparse data and having the advantages of low compression calculation complexity and high efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of neural network tensor processor, and particularly relates to a sparse data compression device and method for a neural network tensor processor. BACKGROUND

[0002] Neural network algorithm is based on dense data for calculation. The dense data refers to data with fixed length, width and height dimensions and occupying fixed memory space. The number of calculation operations of the dense data is fixed, for example, the number of multiplication operations required for multiplication operation of two dense tensor data with fixed dimensions is fixed. The input data, parameter data, intermediate temporary data and output data in the neural network algorithm have fixed dimensions and occupy fixed memory space, therefore, the neural network algorithm is based on dense data for calculation.

[0003] However, in actual neural network calculation, the calculation process of the neural network generates a lot of 0 data, especially when the activation function of the neural network algorithm is of a type that is more likely to generate 0 values (for example, Relu activation sets all negative numbers to 0). Since 0 multiplied by any number is 0, the multiplication operation of 0 data can actually be skipped, thereby saving calculation energy and reducing calculation time.

[0004] When a tensor data has a lot of 0s, for example, the number of 0s is greater than the number of non-0s, the tensor data is usually referred to as sparse data.

[0005] The spiking neural network algorithm naturally has the sparse data characteristic. The spiking neural network processes pulse data, which refers to data composed of time, coordinates and polarity, which can be directly from an event sensor. For example, a data from an event sensor is composed of Δt, x, y and p, where Δt refers to time value, x and y refer to coordinates in the sensor picture, and p refers to the polarity of the change in illumination of the corresponding coordinate pixel point (for example, p equal to 1 represents an increase in illumination and equal to -1 represents a decrease in illumination). At a certain Δt time, the number of pixel points in the sensor picture that have a change in illumination is limited, therefore, the data with the same Δt is limited (even a small number), therefore, the input data of the spiking neural network algorithm has the sparse data characteristic.

[0006] The traditional neural network tensor processor is based on dense data for storage and calculation. Therefore, in the traditional neural network tensor processor, sparse data must be stored in the manner of dense data: sparse data can be regarded as dense data with many 0s. Sparse data must also be calculated in the manner of dense data: sparse data with a value of 0 will participate in the calculation.

[0007] For traditional neural network tensor processors, sparse data is the same as dense data, with the same memory occupation and computing operation. Therefore, although numerous 0 data in sparse data does not save valid information, it still participates in storage and calculation, which leads to waste of storage and computing resources. This waste of storage and computing resources is particularly evident in spiking neural network computing. SUMMARY

[0008] In view of the above defects of the prior art, the present application proposes a sparse data compression device and method for a neural network tensor processor. In the sparse data compression device and method, the "useless 0" data of the sparse data is deleted, and the sparse data is stored in a compressed form of "0-free".

[0009] The specific scheme is as follows:

[0010] A sparse data compression device for a neural network tensor processor, comprising a sparse data RDMA, a configuration unit, a compression unit, a sparse compressed data WDMA, a sparse mapping table WDMA, sparse data, configuration data, sparse compressed data, and a sparse mapping table; the sparse data compression device is used to realize conversion from sparse data to sparse compressed data;

[0011] The configuration unit is used to read the configuration data, and configure the starting address and size information of the sparse data to the sparse data RDMA, configure the starting address and size information of the sparse compressed data to the sparse compressed data WDMA, and configure the starting address and size information of the sparse mapping table to the sparse mapping table WDMA;

[0012] The sparse data RDMA is used to read the sparse data according to the starting address and size information of the sparse data;

[0013] The compression unit is used to perform sparse data compression operation and generate sparse compressed data and a sparse mapping table;

[0014] The sparse compressed data WDMA is used to respond to a sparse compressed data write request, and write out the sparse compressed data according to the starting address and size information of the sparse compressed data;

[0015] The sparse mapping table WDMA is used to respond to a sparse mapping table write request, and write out the sparse mapping table according to the starting address and size information of the sparse mapping table.

[0016] Further, the sparse data refers to tensor data adopting an n-degree parallel storage scheme (C / n, H, W, n), with a data bit width of 8 bits or 16 bits; the original storage scheme (C, H, W) of the tensor data, W represents width, which is the 0th dimension of data; H represents height, which is the 1st dimension of data; C represents channel, which is the 2nd dimension of data;

[0017] The conversion method of the n-parallel storage scheme (C / n, H, W, n) is: for the original storage scheme (C, H, W) of the tensor data, taking the continuous n data in the C direction, storing in a physically continuous manner, and setting the 0th dimension, the length of the 0th dimension is fixed as n; setting W as the 1st dimension, the length is unchanged; setting H as the 2nd dimension, the length is unchanged; setting C / n as the 3rd dimension, C / n is expressed as the length C divided by n and rounded down.

[0018] Further, the n is an integer multiple of 8.

[0019] Further, the sparse data has at least one sparse feature point; the sparse feature point refers to: in the sparse data, if the values of all the n data in the 0th dimension pointed to by the 1st, 2nd and 3rd coordinates (Z, Y, X) are 0, then the (Z, Y, X) coordinates are referred to as a sparse feature point of the sparse data.

[0020] Further, the sparse data has N sparse feature points, and the number N is less than or equal to (C / n)*H*W.

[0021] Further, the sparse compressed data refers to the sparse data in which all the n numbers in the 0th dimension pointed to by all the sparse feature points are deleted; the sparse mapping table refers to the tensor data with a size of (C / n, H, W) and a data bit width of 1 bit; the sparse mapping table corresponds to the sparse data: each 1-bit data of the sparse mapping table represents whether the values of the n data in the 0th dimension of the corresponding sparse data are all 0.

[0022] A sparse data compression method for a neural network tensor processor, applied to the sparse data compression device for a neural network tensor processor as described above, comprising:

[0023] Reading the configuration data of the compression operation and configuring each module of the sparse data compression device; the configuration data includes the start address and size of the sparse data, the start address and size of the sparse compressed data, and the start address and size of the sparse mapping table;

[0024] Reading the sparse data according to the configuration information; for the sparse data with a size of (C / n, H, W, n), n numbers in the 0th dimension are read at a time, and the sparse data is read in turn in the order of the 1st, 2nd and 3rd dimensions from the start address;

[0025] Judging and marking the sparse feature points: for the n numbers in the 0th dimension read each time, judging whether the values of the n numbers are all 0; if the values of all the n data are 0, then the 1st, 2nd and 3rd coordinates (Z, Y, X) corresponding to the n numbers are marked as a sparse feature point;

[0026] Delete sparse feature point data, generate sparse compression data and sparse mapping table;

[0027] Write sparse compression data: respond to sparse compression data write request, and write the n data to the corresponding address;

[0028] Write sparse mapping table: respond to sparse mapping table write request, and write sparse mapping table data to the corresponding address.

[0029] Further, the method of reading sparse data according to the configuration information is: for sparse data with size (C / n, H, W, n), n numbers in the 0th dimension are read at a time, and the sparse data is read in turn in the order of 1, 2, 3 dimensions from the start address; The method of judging and marking sparse feature points is: for the n numbers in the 0th dimension read each time, judge whether the values of the n numbers are all 0; if the values of all n data are 0, the 1, 2, 3 dimensional coordinates (Z, Y, X) corresponding to the n numbers are marked as a sparse feature point.

[0030] Further, the method of deleting sparse feature point data, generating sparse compression data and sparse mapping table is: skipping the writing stage of n data marked as sparse feature points, not sending the sparse compression data write request of the n data, only sending the sparse mapping table write request with value 0; for data not marked as sparse feature points, send the sparse compression data write request of the n data, and send the sparse mapping table write request with value 1.

[0031] Further, the method of writing sparse compression data is: writing n numbers at a time, and writing data in turn from the start address according to the increasing address.

[0032] Further, the way of writing sparse mapping table is: writing 1 one-bit data at a time, and writing sparse mapping table in turn from the start address according to the increasing address.

[0033] The present application realizes the following technical effects:

[0034] The sparse compression data proposed in the present application expresses the case that the values of all 16 data in the 0th dimension are 0 as a sparse feature point that can be compressed and deleted. The sparse feature point exists widely in the neural network calculation process, especially in the pulse neural network calculation process. The sparse information of sparse data can be greatly mined through the sparse feature point.

[0035] Through the way of traversing sparse data, judging and marking sparse feature points, deleting sparse feature points and constructing sparse mapping table, the 0 data deletion and data rearrangement operation of sparse data can be efficiently realized, so that the compression of sparse data is efficiently realized, which has the advantages of low compression calculation complexity and high efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a functional block diagram of a sparse data compression device of the present application;

[0037] Figure 2 is a schematic diagram of a 16-degree parallel storage scheme (C / 16, H, W, 16) of the present application;

[0038] Figure 3 is an example of sparse compressed data and its corresponding sparse mapping table of the present application;

[0039] Figure 4 is a flowchart of a sparse data compression method of the present application. DETAILED DESCRIPTION

[0040] To further illustrate the embodiments, the present application provides drawings. These drawings are part of the disclosure of the present application and are mainly used to illustrate the embodiments, and can be used to explain the operating principles of the embodiments in conjunction with the related descriptions in the specification. Those of ordinary skill in the art should be able to understand other possible implementations and advantages of the present application in conjunction with these. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0041] The present application will be further described in conjunction with the drawings and specific embodiments.

[0042] The present application proposes a sparse data compression device and method for a neural network tensor processor. The sparse data compression device, as shown in Figure 1 , comprises sparse data RDMA, configuration unit, compression unit, sparse compressed data WDMA, sparse mapping table WDMA, sparse data, configuration data, sparse compressed data and sparse mapping table. The sparse data compression device is used to realize the conversion from sparse data to sparse compressed data.

[0043] The sparse data refers to tensor data using a 16-degree parallel storage scheme (C / 16, H, W, 16), and the data bit width is usually 8 bits or 16 bits. The sparse data can be converted from tensor data using an original storage scheme (C, H, W). In the original storage scheme, W represents width, which is the 0th dimension of the data; H represents height, which is the 1st dimension of the data; and C represents channel, which is the 2nd dimension of the data.

[0044] The conversion method of the 16-degree parallel storage scheme (C / 16, H, W, 16) is: for the original storage scheme (C, H, W), take the continuous 16 data in the C direction, store them in a physically continuous manner, and set them as the 0th dimension, the length of the 0th dimension is fixed as 16; set W as the 1st dimension, the length remains unchanged; set H as the 2nd dimension, the length remains unchanged; set C / 16 as the 3rd dimension, C / 16 is expressed as the length C divided by 16 and rounded down. An example of conversion is shown in Figure 2 .

[0045] The sparse feature point refers to that in the sparse data, if the values of all the 0th dimension data pointed to by the 1st, 2nd and 3rd coordinates (Z, Y, X) are 0, then the (Z, Y, X) coordinate is referred to as a sparse feature point of the sparse data. The sparse data can have N sparse feature points, and the number N is less than or equal to (C / 16)*H*W.

[0046] The sparse compressed data refers to the sparse data in which all the 16 numbers in the 0th dimension pointed to by all the sparse feature points are deleted. Further, in order to make up for the information loss (some data are deleted) caused by the compression of the sparse data, a sparse mapping table is used to save the original complete information of the sparse data. The sparse mapping table refers to a tensor data with a size of (C / 16, H, W) and a data bit width of 1 bit. The sparse mapping table corresponds to the sparse data: each 1-bit data in the sparse mapping table represents whether the values of the 16 data in the 0th dimension of the corresponding sparse data are all 0. For example, the value of a 1-bit data in the sparse mapping table with a size of (C / 16, H, W) pointed to by the 0th, 1st and 2nd coordinates (Z, Y, X) represents whether the 16 data in the 0th dimension of the sparse data with a size of (C / 16, H, W, 16) pointed to by the 1st, 2nd and 3rd coordinates (Z, Y, X) are all 0. When a certain 1-bit data in the sparse mapping table is 0, it represents that the 16 data of the corresponding sparse data are all 0. When a certain 1-bit data in the sparse mapping table is 1, it represents that the 16 data of the corresponding sparse data have at least one non-0 data. An example of sparse compressed data and its corresponding sparse mapping table is shown in Figure 3 .

[0047] The sparse compressed data is converted from the sparse data by a sparse data compression device.

[0048] The sparse data of the sparse data compression device can be feature data or parameter data of a neural network, and the sparse compressed data can be feature data or parameter data of a neural network.

[0049] The sparse data compression device, the configuration unit is used for reading configuration data, and configuring the start address and size information of sparse data to sparse data RDMA, configuring the start address and size information of sparse compressed data to sparse compressed data WDMA, and configuring the start address and size information of sparse mapping table to sparse mapping table WDMA. The size of the sparse data is expressed as (C / 16, H, W, 16), the size of the sparse compressed data is expressed as (C / 16, H, W, 16), and the size of the sparse mapping table is expressed as (C / 16, H, W).

[0050] The sparse data compression device, the sparse data RDMA is used for reading sparse data according to the start address and size information of the sparse data. The sparse data RDMA reads 16 numbers of the 0th dimension at a time, and reads the sparse data in a traversal manner of 1, 2, 3 dimensions in order from the start address.

[0051] The sparse data compression device, the compression unit is used for performing sparse data compression operation and generating sparse compressed data and sparse mapping table. The compression unit sequentially obtains sparse data in order. Each time, 16 numbers of the 0th dimension of the sparse data are read, and it is judged whether the values of the 16 numbers are all 0. If the value of at least one number in the 16 numbers is not 0, the sparse compressed data write request of the 16 data is sent, and the sparse mapping table write request with the value of 1 is sent. If the values of the 16 numbers are all 0, the sparse compressed data write request of the 16 data is not sent (that is, the all-0 data is skipped), and only the sparse mapping table write request with the value of 0 is sent.

[0052] The sparse data compression device, the sparse compressed data WDMA is used for responding to the sparse compressed data write request, and writing out the sparse compressed data according to the start address and size information of the sparse compressed data. The sparse compressed data WDMA writes out 16 numbers at a time, and from the start address, for the valid write request from the compression unit, the sparse compressed data is written out in a manner of sequentially increasing the address.

[0053] The sparse data compression device, the sparse mapping table WDMA is used for responding to the sparse mapping table write request, and writing out the sparse mapping table according to the start address and size information of the sparse mapping table. The sparse mapping table WDMA writes out 1 bit data at a time, and from the start address, for the valid write request from the compression unit, the sparse mapping table is written out in a manner of sequentially increasing the address.

[0054] The compression of the sparse data is lossless, that is, the sparse compressed data and the sparse mapping table after compression can completely express the sparse data before compression. Therefore, the sparse compressed data has complete original sparse data information and can be directly used for neural network calculation, thereby reducing the memory bandwidth requirement and the calculation resource requirement.

[0055] The sparse data compression method is as shown in Figure 4

[0056] (1) Configuration. Read the configuration data of the compression operation, which mainly includes the start address and size of the sparse data, the start address and size of the sparse compressed data, the start address and size of the sparse mapping table, and configure each module of the sparse data compression device.

[0057] (2) Read sparse data. According to the configuration information, read the sparse data. For sparse data with a size of (C / 16, H, W, 16), the reading method is: read 16 numbers in the 0th dimension at a time, and read the sparse data in turn in the order of 1, 2, and 3 dimensions from the start address.

[0058] (3) Judge and mark sparse feature points. For each 16 numbers in the 0th dimension read at a time, judge whether the values of the 16 numbers are all 0. If the values of all 16 data are 0, the 1, 2, and 3 dimensional coordinates (Z, Y, X) corresponding to the 16 numbers are marked as a sparse feature point.

[0059] (4) Delete sparse feature point data, generate sparse compressed data and sparse mapping table. The 16 data marked as sparse feature points are deleted, and the deletion method is: skip the writing stage of the 16 data marked as sparse feature points, do not send the sparse compressed data write request of the 16 data, and only send the sparse mapping table write request with a value of 0. For data not marked as sparse feature points, send the sparse compressed data write request of the 16 data, and send the sparse mapping table write request with a value of 1.

[0060] (5) Write out the sparse compressed data. Respond to the sparse compressed data write request and write the 16 data to the corresponding address. The writing method is: write 16 numbers at a time, and write the data in turn from the start address in the order of increasing address.

[0061] (6) Write out the sparse mapping table. Respond to the sparse mapping table write request and write the sparse mapping table data to the corresponding address. The writing method is: write 1 bit data at a time, and write the sparse mapping table in turn from the start address in the order of increasing address.

[0062] ​In the embodiment, the sparse data refers to tensor data in a 16-degree parallel storage scheme (C / 16, H, W, 16). In specific applications, to adapt to the scale and processing capacity of different tensor processors, the sparse data can be more widely defined as tensor data in an n-degree parallel storage scheme (C / n, H, W, n), where n is an integer multiple of 8. Although the application is specifically shown and described in combination with the preferred embodiments, it should be understood by those skilled in the art that various changes in form and details can be made to the application without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. A sparse data compression device for a neural network tensor processor, characterized in that, It includes sparse data RDMA, a configuration unit, a compression unit, sparse compressed data WDMA, sparse mapping table WDMA, sparse data, configuration data, sparse compressed data, and a sparse mapping table; the sparse data compression device is used to realize the conversion from sparse data to sparse compressed data; The configuration unit is used to read configuration data and configure the starting address and size information of sparse data to sparse data RDMA, configure the starting address and size information of sparse compressed data to sparse compressed data WDMA, and configure the starting address and size information of sparse mapping table to sparse mapping table WDMA. The sparse data RDMA is used to read sparse data based on the starting address and size information of the sparse data. The compression unit is used to perform sparse data compression operations and generate sparse compressed data and a sparse mapping table. The sparse compressed data WDMA is used to respond to sparse compressed data write requests and to write sparse compressed data according to the starting address and size information of the sparse compressed data. The sparse mapping table WDMA is used to respond to sparse mapping table write requests and write the sparse mapping table according to the starting address and size information of the sparse mapping table. The sparse data refers to tensor data using an n-degree parallel storage scheme (C / n, H, W, n), with a data bit width of 8 bits or 16 bits; the original storage scheme of the tensor data (C, H, W) is as follows: W represents width, which is the 0th dimension of the data; H represents height, which is the 1st dimension of the data; and C represents channel, which is the 2nd dimension of the data. The conversion method of the n-degree parallel storage scheme (C / n, H, W, n) is as follows: For the original storage scheme (C, H, W) of tensor data, take n consecutive data in the C direction, store them in a way that is physically consecutive, and set them as the 0th dimension, with the length of the 0th dimension fixed at n; set W as the 1st dimension, with the length unchanged; set H as the 2nd dimension, with the length unchanged; set C / n as the 3rd dimension, and C / n is expressed as the length C divided by n and rounded down; The sparse data has at least one sparse feature point; the sparse feature point refers to: in the sparse data, if the values ​​of all n data in the 0 dimension pointed to by the 1, 2, and 3-dimensional coordinates (Z, Y, X) are all 0, then the (Z, Y, X) coordinate is called a sparse feature point of the sparse data. The sparse compressed data refers to sparse data in which all n zero-dimensional numbers pointed to by all sparse feature points are deleted. The sparse mapping table refers to tensor data of size (C / n, H, W) with a data bit width of 1 bit; Sparse mapping tables correspond to sparse data: each 1-bit data in a sparse mapping table represents whether the value of the nth data in the 0th dimension of the corresponding sparse data is all 0.

2. The sparse data compression apparatus for a neural network tensor processor as described in claim 1, characterized in that, The n is an integer multiple of 8.

3. The sparse data compression apparatus for a neural network tensor processor as described in claim 1, characterized in that, The sparse data has N sparse feature points, where the number of N is less than or equal to (C / n)*H*W.

4. A sparse data compression method for neural network tensor processors, characterized in that, An apparatus for sparse data compression of a neural network tensor processor as described in any one of claims 1-3, comprising: Read the configuration data for the compression operation and configure each module of the sparse data compression device; the configuration data includes the starting address and size of the sparse data, the starting address and size of the sparse compressed data, and the starting address and size of the sparse mapping table; Read sparse data according to the configuration information; for sparse data of size (C / n, H, W, n), read n numbers of the 0th dimension at a time, and read sparse data in the order of 1st, 2nd and 3rd dimensions starting from the starting address. Identify and label sparse feature points: For each nth number read in dimension 0, determine whether all n numbers are 0; if all n numbers are 0, then the 1st, 2nd, and 3rd dimension coordinates (Z, Y, X) corresponding to the n numbers are labeled as a sparse feature point. Delete sparse feature point data and generate sparse compressed data and sparse mapping table; Write sparse compressed data: Respond to the sparse compressed data write request and write the n data to the corresponding address; Write the sparse mapping table: Respond to the sparse mapping table write request and write the sparse mapping table data to the corresponding address.

5. The sparse data compression method for neural network tensor processors as described in claim 4, characterized in that, The method for reading sparse data according to the configuration information is as follows: for sparse data of size (C / n, H, W, n), read the n numbers of the 0th dimension at a time, and read the sparse data sequentially in the order of the 1st, 2nd, and 3rd dimensions, starting from the starting address. The method for identifying and marking sparse feature points is as follows: for each nth number read in the 0th dimension, determine whether the values ​​of the nth number are all 0; if the values ​​of all n data are 0, then the 1st, 2nd, and 3rd dimension coordinates (Z, Y, X) corresponding to the nth number are marked as a sparse feature point.

6. The sparse data compression method for neural network tensor processors as described in claim 4, characterized in that, The method for deleting sparse feature point data and generating sparse compressed data and sparse mapping table is as follows: skip the writing stage of n data that are marked as sparse feature points, do not send sparse compressed data write requests for the n data, and only send sparse mapping table write requests with a value of 0; for data that are not marked as sparse feature points, send sparse compressed data write requests for the n data and sparse mapping table write requests with a value of 1.

7. The sparse data compression method for neural network tensor processors as described in claim 4, characterized in that, The method for writing sparse compressed data is as follows: write n numbers at a time, starting from the starting address and writing the data in ascending order of address.

8. The sparse data compression method for neural network tensor processors as described in claim 4, characterized in that, The way to write a sparse mapping table is to write one 1-bit data at a time, and write the sparse mapping table in ascending order of address starting from the starting address.

Images