Sparse data decompression apparatus and method for neural network tensor processors

Abstract

The present application relates to the field of neural network tensor processors, and discloses a sparse data decompression device and method for a neural network tensor processor.The method comprises reading configuration data of a decompression operation, and configuring modules of the sparse data decompression device; reading a sparse mapping table according to the configuration information; reading sparse compressed data according to the configuration information and the sparse mapping table; decompressing to generate sparse data; and writing out the sparse data.The sparse compressed data proposed by the present application expresses the case that the values of all 16 data of 0 dimension are 0 as sparse feature points that can be filled by decompression, and develops the distribution of the sparse feature points through a sparse mapping table.The sparse feature points can be directly identified by traversing the sparse mapping table, and the decompression of the sparse data can be efficiently realized by filling 0 at the sparse feature points and inserting the written data, so that the present application has the advantages of low decompression calculation complexity and high efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of neural network tensor processor, and particularly relates to a sparse data decompression device and method for a neural network tensor processor. BACKGROUND

[0002] Neural network algorithm is based on dense data for calculation. The dense data refers to data with fixed length, width, height dimensions and occupying fixed memory space. The number of calculation operations of the dense data is fixed, for example, the number of multiplication operations required for multiplication operation of two dense tensor data with fixed dimensions is fixed. The input data, parameter data, intermediate temporary data and output data in the neural network algorithm have fixed dimensions and occupy fixed memory space, therefore, the neural network algorithm is based on dense data for calculation.

[0003] However, in actual neural network calculation, the calculation process of the neural network will generate a lot of 0 data, especially when the activation function of the neural network algorithm is of a type that is more likely to generate 0 values (for example, Relu activation sets all negative numbers to 0). Since 0 multiplied by any number is 0, the multiplication operation of 0 data can actually be skipped, thereby saving calculation energy and reducing calculation time.

[0004] When a tensor data has a lot of 0s, for example, the number of 0s is greater than the number of non-0s, the tensor is usually referred to as sparse data.

[0005] The spiking neural network algorithm naturally has the sparse data characteristic. The spiking neural network processes pulse data, which refers to data composed of time, coordinates and polarity, which can be directly from an event sensor. For example, a data from an event sensor is composed of Δt, x, y, p, where Δt refers to time value, x and y refer to coordinates in the sensor picture, and p refers to the polarity of the change in illumination of the corresponding coordinate pixel point (for example, p equal to 1 represents an increase in illumination and equal to -1 represents a decrease in illumination). At a certain Δt time, the number of pixel points in the sensor picture that have a change in illumination is limited, therefore, the data with the same Δt is limited (even a small number), therefore, the input data of the spiking neural network algorithm has the sparse data characteristic.

[0006] The traditional neural network tensor processor is based on dense data for storage and calculation. Therefore, in the traditional neural network tensor processor, sparse data must be stored in the manner of dense data: sparse data can be regarded as dense data with many 0s. Sparse data must also be calculated in the manner of dense data: sparse data with value 0 will participate in the calculation.

[0007] For a traditional neural network tensor processor, sparse data is the same as dense data, with the same memory occupation and calculation operation. Therefore, although numerous 0 data in sparse data does not save valid information, it still participates in storage and calculation, which causes waste of storage and calculation resources. This waste of storage and calculation resources is particularly obvious in spiking neural network calculation.

[0008] The compression operation on sparse data can generate sparse compressed data. If the sparse compressed data is directly used by processor hardware, memory space can be saved, calculation operation numbers can be reduced, and thus calculation efficiency can be improved. However, the sparse compressed data is not a general data type and can be used only for processor hardware supporting the data type, such as a tensor processor supporting the sparse compressed data. For a processor not supporting the sparse compressed data type, such as a traditional tensor processor, the sparse compressed data cannot be directly used. SUMMARY

[0009] In view of the above defects of the prior art, the present application provides a sparse data decompression device and method for a neural network tensor processor. In the sparse data decompression device and method, sparse compressed data stored in a "0-free" compression form is efficiently filled with 0 data, so as to be losslessly restored into sparse data. The sparse data is a general data type and can be used by other arbitrary type processor hardware.

[0010] The specific scheme is as follows:

[0011] The present application provides a sparse data decompression device for a neural network tensor processor, comprising sparse compressed data RDMA, sparse mapping table RDMA, configuration unit, decompression unit, sparse data WDMA, sparse data, configuration data, sparse compressed data, and sparse mapping table.

[0012] The configuration unit is configured to read the configuration data, and configure the start address and size information of the sparse compressed data to the sparse compressed data RDMA, configure the start address and size information of the sparse mapping table to the sparse mapping table RDMA, and configure the start address and size information of the sparse data to the sparse data WDMA.

[0013] The sparse mapping table RDMA is configured to read the sparse mapping table according to the start address and size information of the sparse mapping table.

[0014] The sparse compressed data RDMA is configured to read the sparse data according to the sparse mapping table and the start address and size information of the sparse compressed data.

[0015] The decompression unit is configured to perform a decompression operation on the sparse compressed data and generate sparse data.

[0016] The sparse data WDMA is used to respond to a sparse data write request and write out sparse data according to a start address and size information of the sparse data.

[0017] Further, the sparse data refers to tensor data using an n-degree parallel storage scheme (C / n, H, W, n), and a data bit width is 8 bits or 16 bits; the original storage scheme (C, H, W) of the tensor data, W representing width, is the 0th dimension of data; H representing height, is the 1st dimension of data; and C representing channel, is the 2nd dimension of data.

[0018] The conversion method of the n-degree parallel storage scheme (C / n, H, W, n) is that, for the original storage scheme (C, H, W) of the tensor data, n continuous data in the C direction are stored in a physically continuous manner and set as the 0th dimension, the length of the 0th dimension being fixed as n; W is set as the 1st dimension, the length being unchanged; H is set as the 2nd dimension, the length being unchanged; and C / n is set as the 3rd dimension, C / n being expressed as the length C divided by n and rounded down.

[0019] Further, the n is an integer multiple of 8.

[0020] Further, the sparse data has at least one sparse feature point; the sparse feature point refers to that, in the sparse data, if the values of all the n data in the 0th dimension pointed to by the 1st, 2nd or 3rd coordinate (Z, Y, X) are 0, the (Z, Y, X) coordinate is referred to as a sparse feature point of the sparse data.

[0021] Further, the sparse data has N sparse feature points, the number N being less than or equal to (C / n)*H*W.

[0022] Further, the sparse compressed data refers to the sparse data in which all the n data in the 0th dimension pointed to by all the sparse feature points are deleted; and the sparse mapping table refers to tensor data with a size of (C / n, H, W) and a data bit width of 1 bit; the sparse mapping table corresponds to the sparse data: each 1-bit data of the sparse mapping table represents whether the values of the n data in the 0th dimension of the corresponding sparse data are all 0.

[0023] A sparse data decompression method for a neural network tensor processor, comprising:

[0024] Configuration data of a decompression operation is read, and each module of the sparse data decompression device is configured; the configuration data mainly includes a start address and size of sparse data, a start address and size of sparse compressed data, and a start address and size of a sparse mapping table;

[0025] The sparse mapping table is read according to the configuration information;

[0026] The sparse compressed data is read according to the configuration information and the sparse mapping table.

[0027] decompressing to generate sparse data;

[0028] writing out sparse data.

[0029] Further, the method of reading the sparse mapping table according to the configuration information is that, for a sparse mapping table with a size of (C / n, H, W), 1 bit of data is read at a time, and data is read in turn in a 0, 1, 2-dimensional sequential traversal manner starting from a starting address.

[0030] Further, the method of reading sparse compressed data according to the configuration information and the sparse mapping table is that, starting from the (0, 0, 0) coordinates of the sparse mapping table, it is judged whether the value of the data corresponding to the sparse mapping table is 0; if it is 0, it means that the sparse compressed data corresponding to the coordinates does not exist, and the reading of the coordinate data is skipped; if it is 1, it means that the sparse compressed data corresponding to the coordinates exists, and the coordinate data is read; the address of data reading starts from the starting address and is sequentially incremented with the reading process.

[0031] Further, the method of decompressing to generate sparse data is that, the sparse mapping table is sequentially obtained in turn, starting from the (0, 0, 0) coordinates of the sparse mapping table, it is judged whether the value of the data corresponding to the sparse mapping table is 0; if it is 0, it means that the sparse compressed data corresponding to the coordinates does not exist, and the sparse compressed data is not obtained and a write request of n numbers with a value of 0 is directly sent; if it is 1, it means that the sparse compressed data corresponding to the coordinates exists, and n sparse compressed data is obtained and a write request of the n data is sent.

[0032] Further, the method of writing out sparse data is that, in response to a sparse data write request, n numbers are written out at a time, and data is written out in a sequentially incremented manner starting from the starting address.

[0033] The present application achieves the following technical effects:

[0034] The sparse compressed data proposed in the present application expresses the case that the values of all n data of 0 dimension are 0 as sparse feature points that can be decompressed and filled, and the distribution of the sparse feature points is developed through the sparse mapping table.

[0035] The sparse feature points can be directly identified through the traversal of the sparse mapping table, and the decompression of the sparse data can be efficiently realized through the filling of 0 at the sparse feature points and the insertion of the written data, which has the advantages of low decompression calculation complexity and high efficiency.

[0036] The decompression of the sparse data is lossless, that is, the sparse compressed data and the sparse mapping table corresponding to the sparse data after decompression can be compressed again. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a functional block diagram of a sparse data decompression apparatus of the present application;

[0038] Figure 2 is a schematic diagram of a 16-degree parallel storage scheme (C / 16, H, W, 16) of the present application;

[0039] Figure 3 is an example of sparse compressed data and its corresponding sparse mapping table of the present application;

[0040] Figure 4 is a flowchart of a sparse data decompression method of the present application. DETAILED DESCRIPTION

[0041] To further illustrate the embodiments, the present application provides accompanying drawings. These drawings are part of the disclosure of the present application and mainly serve to illustrate the embodiments, and can be used to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. Those of ordinary skill in the art should be able to understand other possible implementations and advantages of the present application in conjunction with these. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0042] The present application will be further described in conjunction with the accompanying drawings and specific embodiments.

[0043] The present application proposes a sparse data decompression apparatus and method for a neural network tensor processor. The sparse data decompression apparatus, as shown in Figure 1 , comprises sparse compressed data RDMA, sparse mapping table RDMA, configuration unit, decompression unit, sparse data WDMA, sparse data, configuration data, sparse compressed data, and sparse mapping table. The sparse data decompression apparatus is used to realize the conversion from sparse compressed data to sparse data.

[0044] The sparse data refers to tensor data using a 16-degree parallel storage scheme (C / 16, H, W, 16), and the data bit width is usually 8 bits or 16 bits. The sparse data can be converted from tensor data using an original storage scheme (C, H, W). In the original storage scheme, W represents width, which is the 0th dimension of the data; H represents height, which is the 1st dimension of the data; and C represents channel, which is the 2nd dimension of the data.

[0045] The conversion method of the 16-degree parallel storage scheme (C / 16, H, W, 16) is as follows: for the original storage scheme (C, H, W), take the continuous 16 data in the C direction, store them in a physically address-continuous manner, and set them as the 0th dimension, the length of which is fixed as 16; set W as the 1st dimension, the length of which remains unchanged; set H as the 2nd dimension, the length of which remains unchanged; and set C / 16 as the 3rd dimension, C / 16 being expressed as the length C divided by 16 and rounded down. An example of conversion is as follows:Figure 2 as shown.

[0046] The sparse data has at least one sparse feature point. The sparse feature point refers to: in sparse data, if the values of all 0-dimensional 16 data pointed to by 1, 2, 3-dimensional coordinates (Z, Y, X) are 0, the (Z, Y, X) coordinates are called a sparse feature point of the sparse data. The sparse data can have N sparse feature points, and the number N is less than or equal to (C / 16)*H*W.

[0047] The sparse compressed data refers to sparse data in which all 0-dimensional 16 numbers pointed to by all sparse feature points are deleted. Further, in order to make up for the information loss (some data are deleted) caused by sparse data compression, a sparse mapping table is used to save the original complete information of the sparse data. The sparse mapping table refers to a tensor data with a size of (C / 16, H, W) and a data bit width of 1 bit. The sparse mapping table corresponds to the sparse data: each 1-bit data of the sparse mapping table represents whether the values of the 0-dimensional 16 data of the corresponding sparse data are all 0. For example, the value of a 1-bit data pointed to by 0, 1, 2-dimensional coordinates (Z, Y, X) in the sparse mapping table with a size of (C / 16, H, W) represents whether the 16 data of the 0 dimension pointed to by the 1, 2, 3-dimensional coordinates (Z, Y, X) in the sparse data with a size of (C / 16, H, W, 16) are all 0. When a certain 1-bit data in the sparse mapping table is 0, it represents that the 16 data of the corresponding sparse data are all 0. When a certain 1-bit data in the sparse mapping table is 1, it represents that the 16 data of the corresponding sparse data have at least one non-0 data. An example of a sparse compressed data and its corresponding sparse mapping table is shown as Figure 3 as shown.

[0048] The sparse data is converted from the sparse compressed data by a sparse data decompression device.

[0049] The sparse compressed data of the sparse data decompression device can be feature data or parameter data of a neural network, and the sparse data can be feature data or parameter data of a neural network.

[0050] In the sparse data decompression device, the configuration unit is configured to read configuration data, and configure a start address and size information of the sparse compressed data to the sparse compressed data RDMA, configure a start address and size information of the sparse mapping table to the sparse mapping table RDMA, and configure a start address and size information of the sparse data to the sparse data WDMA. The size of the sparse data is expressed as (C / 16, H, W, 16), the size of the sparse compressed data is expressed as (C / 16, H, W, 16), and the size of the sparse mapping table is expressed as (C / 16, H, W).

[0051] The sparse data decompression device, the sparse map table RDMA is used to read the sparse map table according to the start address and size information of the sparse map table. The sparse map table RDMA reads 1 bit of data at a time, and reads the data in the order of 0, 1, 2 dimensions from the start address.

[0052] The sparse data decompression device, the sparse compressed data RDMA is used to read the sparse data according to the sparse map table and the start address and size information of the sparse compressed data. The sparse compressed data RDMA reads 16 numbers of the 0th dimension at a time, and the reading method is: starting from the (0, 0, 0) coordinates of the sparse map table, judging whether the value of the data corresponding to the sparse map table is 0. If it is 0, it means that the sparse compressed data corresponding to the coordinates does not exist, and the reading of the coordinate data is skipped. If it is 1, it means that the sparse compressed data corresponding to the coordinates exists, and the coordinate data is read. The address of the data reading starts from the start address and increases sequentially with the reading process.

[0053] The sparse data decompression device, the decompression unit is used to perform the decompression operation of the sparse compressed data and generate the sparse data. The decompression unit sequentially obtains the sparse map table, starting from the (0, 0, 0) coordinates of the sparse map table, judges whether the value of the data corresponding to the sparse map table is 0. If it is 0, it means that the sparse compressed data corresponding to the coordinates does not exist, then the sparse compressed data is not obtained, and a write request of 16 values of 0 is directly sent. If it is 1, it means that the sparse compressed data corresponding to the coordinates exists, then 16 sparse compressed data are obtained, and a write request of the 16 data is sent.

[0054] The sparse data WDMA is used to respond to the sparse data write request and write out the sparse data according to the start address and size information of the sparse data. The sparse data WDMA writes out 16 numbers at a time, and from the start address, for valid write requests from the decompression unit, the sparse data is written out in the order of sequentially increasing addresses.

[0055] The decompression of the sparse data is lossless, that is, the corresponding sparse compressed data and sparse map table can be generated again by compressing the decompressed sparse data.

[0056] The sparse data decompression method is as shown in Figure 4 The sparse data decompression method is as shown in

[0057] (1) Configuration. Read the configuration data of the decompression operation, which mainly includes the start address and size of the sparse data, the start address and size of the sparse compressed data, the start address and size of the sparse map table, and configure each module of the sparse data decompression device.

[0058] (2) Reading the sparse mapping table. According to the configuration information, the sparse mapping table is read. For a sparse mapping table with a size of (C / 16, H, W), the reading method is: reading 1 bit of data at a time, and sequentially reading the data in a 0, 1, 2-dimensional order from the start address.

[0059] (3) Reading the sparse compressed data. According to the configuration information and the sparse mapping table, the sparse compressed data is read. The reading method is: starting from the (0, 0, 0) coordinate of the sparse mapping table, judging whether the value of the data corresponding to the sparse mapping table is 0. If it is 0, it means that the sparse compressed data corresponding to the coordinate does not exist, and the reading of the coordinate data is skipped. If it is 1, it means that the sparse compressed data corresponding to the coordinate exists, and the coordinate data is read. The address of the data reading starts from the start address and is sequentially incremented with the reading process.

[0060] (4) Decompression to generate sparse data. The decompression method is: sequentially obtaining the sparse mapping table from the (0, 0, 0) coordinate of the sparse mapping table, judging whether the value of the data corresponding to the sparse mapping table is 0. If it is 0, it means that the sparse compressed data corresponding to the coordinate does not exist, and the sparse compressed data is not obtained and a write request of 16 values of 0 is directly sent. If it is 1, it means that the sparse compressed data corresponding to the coordinate exists, and 16 sparse compressed data are obtained and a write request of the 16 data is sent.

[0061] (5) Writing out the sparse data. Responding to the sparse data write request, the 16 data are written to the corresponding address. The writing method is: writing 16 numbers at a time, and writing the data sequentially from the start address in the order of increasing address.

[0062] In the embodiment, the sparse data refers to tensor data using a 16-degree parallel storage scheme (C / 16, H, W, 16). In specific applications, to adapt to the size and processing capacity of different tensor processors, the sparse data can be more widely defined as tensor data using an n-degree parallel storage scheme (C / n, H, W, n), where n is an integer multiple of 8.

[0063] Although the present application is specifically shown and described in connection with preferred embodiments, those skilled in the art will appreciate that various modifications in form and detail can be made without departing from the spirit and scope of the application as defined by the appended claims.

Claims

1. A sparse data decompression device for a neural network tensor processor, characterized in that, It includes Sparse Compressed Data (RDMA), Sparse Mapping Table (RDMA), Configuration Unit, Decompression Unit, Sparse Data (WDMA), Sparse Data, Configuration Data, Sparse Compressed Data, and Sparse Mapping Table; The configuration unit is used to read configuration data and configure the starting address and size information of sparse compressed data to sparse compressed data RDMA, configure the starting address and size information of sparse mapping table to sparse mapping table RDMA, and configure the starting address and size information of sparse data to sparse data WDMA. The sparse mapping table RDMA is used to read the sparse mapping table according to the starting address and size information of the sparse mapping table; The sparse compressed data RDMA is used to read sparse compressed data according to the sparse mapping table and the starting address and size information of the sparse compressed data. The decompression unit is used to perform decompression operations on sparse compressed data and generate sparse data. The sparse data WDMA is used to respond to sparse data write requests and write sparse data according to the starting address and size information of the sparse data. The sparse data refers to data stored using an n-degree parallel storage scheme.<C / n,H,W,n> The tensor data has a data bit width of 8 bits or 16 bits; the original storage scheme of the tensor data.<C,H,W> W represents width, which is the 0th dimension of the data; H represents height, which is the 1st dimension of the data; C represents channel, which is the 2nd dimension of the data. The n-degree parallel storage scheme<C / n,H,W,n> The conversion method is as follows: for the original storage scheme of tensor data<C,H,W> Take n consecutive data points in direction C, store them in a continuous manner according to their physical addresses, and set them as the 0th dimension. The length of the 0th dimension is fixed at n. Set W as the 1st dimension with no change in length. Set H as the 2nd dimension with no change in length. Set C / n as the 3rd dimension, and C / n is expressed as the length C divided by n and rounded down. The sparse data has at least one sparse feature point; the sparse feature point refers to: in the sparse data, if the values ​​of all n data in the 0 dimension pointed to by the 1, 2, and 3-dimensional coordinates (Z, Y, X) are all 0, then the (Z, Y, X) coordinate is called a sparse feature point of the sparse data. The sparse compressed data refers to sparse data in which all zero-dimensional n numbers pointed to by all sparse feature points have been deleted; the sparse mapping table refers to a table with a size of<C / n,H,W> The tensor data has a data bit width of 1 bit; the sparse map table corresponds to the sparse data: each 1-bit data in the sparse map table represents whether the value of the nth data in the 0th dimension of the corresponding sparse data is all 0.

2. The sparse data decompression apparatus for a neural network tensor processor as described in claim 1, characterized in that, The n is an integer multiple of 8.

3. The sparse data decompression apparatus for a neural network tensor processor as described in claim 1, characterized in that, Sparse data has N sparse feature points, where N is less than or equal to (C / n). H W.

4. A sparse data decompression method for neural network tensor processors, characterized in that, An apparatus for sparse data decompression of a neural network tensor processor as described in any one of claims 1-3, comprising: Read the configuration data for the decompression operation and configure each module of the sparse data decompression device; the configuration data includes the starting address and size of the sparse data, the starting address and size of the sparse compressed data, and the starting address and size of the sparse mapping table; Read the sparse mapping table based on the configuration information; Read sparse compressed data based on configuration information and sparse mapping table; Decompression generates sparse data; Write out the sparse data.

5. The sparse data decompression method for neural network tensor processors as described in claim 4, characterized in that, The method for reading the sparse mapping table based on the configuration information is as follows: for a size of<C / n,H,W> The sparse mapping table reads one 1-bit data at a time, starting from the starting address and traversing the data in 0, 1, and 2-dimensional order.

6. The sparse data decompression method for neural network tensor processors as described in claim 4, characterized in that, The method for reading sparse compressed data based on configuration information and a sparse mapping table is as follows: starting from the coordinates (0, 0, 0) of the sparse mapping table, determine whether the value of the corresponding data in the sparse mapping table is 0; If it is 0, it means that the sparse compressed data corresponding to the coordinate does not exist, and the reading of the coordinate data is skipped; if it is 1, it means that the sparse compressed data corresponding to the coordinate exists, and the coordinate data is read; the address for data reading starts from the starting address and increments sequentially as the reading process proceeds.

7. The sparse data decompression method for neural network tensor processors as described in claim 6, characterized in that, The method for decompressing and generating sparse data is as follows: obtain the sparse mapping table in sequence, starting from the coordinate (0, 0, 0) of the sparse mapping table, and determine whether the value of the corresponding data in the sparse mapping table is 0; if it is 0, it means that the sparse compressed data corresponding to the coordinate does not exist, so do not obtain the sparse compressed data, and directly send a write request for n numbers with a value of 0. If it is 1, it means that the sparse compressed data corresponding to the coordinate exists. Then, obtain n sparse compressed data and send a write request for the n data.

8. The sparse data decompression method for neural network tensor processors as described in claim 6, characterized in that, The method for writing sparse data is to respond to sparse data write requests, write n numbers at a time, and write the data in ascending order of address starting from the starting address.

Images