Sparse data computing apparatus and method for neural network tensor processors

Abstract

The present application relates to the field of neural network tensor processors, and discloses a sparse data computing device and method for a neural network tensor processor. The method comprises the following steps: reading configuration data required for computation, and configuring each module of the sparse data computing device; reading an input feature sparse mapping table and a parameter sparse mapping table according to the configuration information; reading input feature sparse compressed data according to the configuration information and the input feature sparse mapping table, and reading parameter sparse compressed data according to the configuration information and the parameter sparse mapping table; calculating an addition sparse mapping table or a multiplication sparse mapping table; selectively performing addition or multiplication operation according to the value of the 16-bit addition sparse mapping table or the 16-bit multiplication sparse mapping table; writing sparse compressed data; and writing the sparse mapping table. The sparse data computing device and method can save data storage space, reduce the number of operations in neural network computation, and thus improve the computation efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of neural network tensor processor, and particularly relates to a sparse data computing device and method for a neural network tensor processor. BACKGROUND

[0002] Neural network algorithm is based on dense data for calculation. The dense data refers to data with fixed length, width, height dimensions and occupying fixed memory space. The number of calculation operations of the dense data is fixed, for example, the number of multiplication operations required for multiplication operation of two dense tensor data with fixed dimensions is fixed. The input data, parameter data, intermediate temporary data and output data in the neural network algorithm have fixed dimensions and occupy fixed memory space, therefore, the neural network algorithm is based on dense data for calculation.

[0003] However, in actual neural network calculation, the calculation process of the neural network will generate a lot of 0 data, especially when the activation function of the neural network algorithm is of a type that is more likely to generate 0 values (for example, Relu activation sets all negative numbers to 0). Since 0 multiplied by any number is 0, the multiplication operation of 0 data can actually be skipped, thereby saving calculation energy and reducing calculation time.

[0004] When a tensor data has a lot of 0s, for example, the number of 0s is greater than the number of non-0s, the tensor is usually referred to as sparse data.

[0005] The spiking neural network algorithm naturally has the sparse data characteristic. The spiking neural network processes pulse data, which refers to data composed of time, coordinates and polarity, which can be directly from an event sensor. For example, a data from an event sensor is composed of Δt, x, y, p, where Δt refers to time value, x and y refer to coordinates in the sensor picture, and p refers to the polarity of the change in illumination of the corresponding coordinate pixel point (for example, p equal to 1 represents an increase in illumination, and equal to -1 represents a decrease in illumination). At a certain Δt time, the number of pixel points in the sensor picture that have a change in illumination is limited, therefore, the data with the same Δt is limited (even a small number), therefore, the input data of the spiking neural network algorithm has the sparse data characteristic.

[0006] The traditional neural network tensor processor is based on dense data for storage and calculation. Therefore, in the traditional neural network tensor processor, sparse data must be stored in the manner of dense data: sparse data can be regarded as dense data with many 0s. Sparse data must also be calculated in the manner of dense data: sparse data with value 0 will participate in the calculation.

[0007] For a traditional neural network tensor processor, sparse data is the same as dense data, with the same memory occupation and calculation operation. Therefore, although numerous 0 data in sparse data does not save valid information, it still participates in storage and calculation, which causes waste of storage and calculation resources. The waste of storage and calculation resources is particularly obvious in spiking neural network calculation. SUMMARY

[0008] In view of the above defects of the prior art, the present application proposes a sparse data calculation device and method for a neural network tensor processor. In the sparse data calculation device and method, "useless 0" data of sparse data is deleted, and sparse data is stored in a compressed form without 0, which can greatly reduce the memory occupation space of sparse data. In the sparse data calculation device and method, the sparse compressed data stored in the compressed form can be directly used for calculation, and "useless 0" data of sparse data is directly skipped, which can greatly reduce the number of calculation operations of sparse data. The specific scheme is as follows:

[0009] A sparse data calculation device for a neural network tensor processor, comprising an input feature sparse mapping table RDMA, a parameter sparse mapping table RDMA, input feature data RDMA, parameter data RDMA, a configuration unit, an operation unit, output feature data WDMA, output feature sparse mapping table WDMA, input feature sparse compressed data, parameter sparse compressed data, input feature sparse mapping table, parameter sparse mapping table, configuration data, output feature sparse compressed data, and output feature sparse mapping table;

[0010] The configuration unit is configured to read configuration data, and configure the start address information and size information of the input feature sparse mapping table RDMA to the input feature sparse mapping table RDMA, configure the start address and size information of the parameter sparse mapping table RDMA to the parameter sparse mapping table RDMA, configure the start address and size information of the input feature sparse compressed data to the input feature data RDMA, configure the start address and size information of the parameter sparse compressed data to the parameter data RDMA, configure the start address and size information of the output feature sparse compressed data to the output feature data WDMA, and configure the start address and size information of the output feature sparse mapping table WDMA to the output feature sparse mapping table WDMA;

[0011] The input feature sparse mapping table RDMA is configured to read the input feature sparse mapping table according to the start address and size information of the input feature sparse mapping table;

[0012] The parameter sparse mapping table RDMA is configured to read the parameter sparse mapping table according to the start address and size information of the parameter sparse mapping table;

[0013] The input feature data RDMA is used to read input feature sparse compression data according to the input feature sparse mapping table and the start address and size information of the input feature data;

[0014] The parameter data RDMA is used to read parameter sparse compression data according to the parameter sparse mapping table and the start address and size information of the parameter data;

[0015] The operation unit is used to perform addition or multiplication operation of input feature sparse compression data and parameter sparse compression data;

[0016] The output feature sparse data WDMA is used to write output feature sparse compression data in response to an output feature sparse compression data write request and according to the start address and size information of the output feature sparse compression data;

[0017] The output feature sparse mapping table WDMA is used to write output feature sparse mapping table in response to an output feature sparse mapping table write request and according to the start address and size information of the output feature sparse mapping table.

[0018] Further, the sparse data refers to tensor data in an n-degree parallel storage scheme (C / n, H, W, n), and the data bit width is 8 bits or 16 bits; the original storage scheme (C, H, W) of the tensor data, W represents width, which is the 0th dimension of the data; H represents height, which is the 1st dimension of the data; C represents channel, which is the 2nd dimension of the data;

[0019] The conversion method of the n-degree parallel storage scheme (C / n, H, W, n) is: for the original storage scheme (C, H, W) of the tensor data, continuously n data in the C direction are taken and stored in a physically continuous manner, and are set as the 0th dimension, the length of the 0th dimension is fixed as n; W is set as the 1st dimension, the length is unchanged; H is set as the 2nd dimension, the length is unchanged; C / n is set as the 3rd dimension, C / n is expressed as the length C divided by n and rounded down.

[0020] Further, the n is an integer multiple of 8.

[0021] Further, the sparse data has at least one sparse feature point; the sparse feature point refers to: in the sparse data, if the values of all the n data in the 0th dimension pointed to by the 1st, 2nd and 3rd coordinates (Z, Y, X) are 0, the (Z, Y, X) coordinate is referred to as a sparse feature point of the sparse data.

[0022] Further, the sparse data has N sparse feature points, and the number of N is less than or equal to (C / n) * H * W.

[0023] Further, the operation unit performs the operation of the n groups of input feature sparse compressed data and the n groups of parameter sparse compressed data in a maximum of n cycles as a round, and performs the operation of 1 group of data in each cycle, and each group of data contains n numbers corresponding to the n numbers of the 0th dimension of the sparse compressed data.

[0024] When the round of operation starts, the operation unit obtains the input feature sparse mapping table of n consecutive bits from the input feature sparse mapping table RDMA and obtains the parameter sparse mapping table of n consecutive bits from the parameter sparse mapping table RDMA; for the addition operation mode, the input feature sparse mapping table of n consecutive bits and the parameter sparse mapping table of n consecutive bits are subjected to a logical OR operation to generate an addition sparse mapping table; for the multiplication operation mode, the input feature sparse mapping table of n consecutive bits and the parameter sparse mapping table of n consecutive bits are subjected to a logical AND operation to generate a multiplication sparse mapping table.

[0025] The n-bit addition or multiplication sparse mapping table corresponds to n groups of input feature sparse compressed data and parameter sparse compressed data: the kth bit of the mapping table corresponds to the kth group of input feature sparse compressed data and the kth group of parameter sparse compressed data, where k is an integer in the interval [0, n).

[0026] The execution mode of the round of operation is to traverse the addition or multiplication sparse mapping table from the 0th bit, judge whether the data value of the addition or multiplication sparse mapping table is 1: if it is 1, then spend one cycle to perform the operation of the corresponding group of data; if it is 0, then skip the operation and do not spend a cycle.

[0027] A sparse data calculation method for a neural network tensor processor, applied to the sparse data calculation device for a neural network tensor processor as described above, comprising:

[0028] Step S1, read the configuration data required for calculation, and configure each module of the sparse data calculation device;

[0029] Step S2, read the input feature sparse mapping table and the parameter sparse mapping table according to the configuration information;

[0030] Step S3, read the input feature sparse compressed data according to the configuration information and the input feature sparse mapping table, and read the parameter sparse compressed data according to the configuration information and the parameter sparse mapping table;

[0031] Step S4, calculating to obtain an addition sparse mapping table or a multiplication sparse mapping table: maximum n cycles as a round, at the beginning of each round operation, the input feature sparse mapping table of continuous n bits and the parameter sparse mapping table of continuous n bits are obtained; for the addition operation mode, the input feature sparse mapping table of continuous n bits and the parameter sparse mapping table of continuous n bits are subjected to a "logical or" operation to generate an addition sparse mapping table; for the multiplication operation mode, the input feature sparse mapping table of continuous n bits and the parameter sparse mapping table of continuous n bits are subjected to a "logical and" operation to generate a multiplication sparse mapping table;

[0032] Step S5, performing addition or multiplication operation: according to the value of the n-bit addition sparse mapping table or the n-bit multiplication sparse mapping table, selectively performing addition or multiplication operation: the value of the sparse mapping table is 1, which represents that the data is valid and the operation is performed, and the value of the sparse mapping table is 0, which represents that the data is invalid and the calculation is skipped;

[0033] Step S6, writing out sparse compressed data: in response to an output feature sparse compressed data write request, the n data are written to the corresponding address;

[0034] Step S7, writing out sparse mapping table: in response to an output feature sparse mapping table write request, the sparse mapping table data are written to the corresponding address.

[0035] Further, in the step S1, the configuration data include the start address and size of the input feature sparse compressed data, the start address and size of the parameter sparse compressed data, the start address and size of the output feature sparse compressed data, the start address and size of the input feature sparse mapping table, the start address and size of the parameter sparse mapping table, and the start address and size of the output feature sparse mapping table.

[0036] The present application realizes the following technical effects:

[0037] The sparse data computing device and method provided by the present application can save data storage space, reduce the number of operations of neural network calculation, and thus improve the calculation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a functional block diagram of the sparse data computing device of the present application;

[0039] Figure 2 is a schematic diagram of a 16-degree parallel storage scheme (C / 16, H, W, 16) of the present application;

[0040] Figure 3 is an example of sparse compressed data and its corresponding sparse mapping of the present application;

[0041] Figure 4 is a flowchart of the sparse data computing method of the present application. DETAILED DESCRIPTION

[0042] For further illustration of the embodiments, the present disclosure provides accompanying drawings. These drawings are part of the disclosure and mainly serve to illustrate the embodiments, and can be used to explain the operation principle of the embodiments in conjunction with the relevant description of the specification. Based on these, those skilled in the art should be able to understand other possible implementations and advantages of the present disclosure.

[0043] The present disclosure will be further described in conjunction with the accompanying drawings and specific embodiments.

[0044] The present disclosure provides a sparse data computing device and method for a neural network tensor processor. The sparse data computing device, as shown in Figure 1 , comprises an input feature sparse mapping table RDMA, a parameter sparse mapping table RDMA, input feature data RDMA, parameter data RDMA, a configuration unit, an operation unit, output feature data WDMA, output feature sparse mapping table WDMA, input feature sparse compressed data, parameter sparse compressed data, input feature sparse mapping table, parameter sparse mapping table, configuration data, output feature sparse compressed data, and output feature sparse mapping table.

[0045] The sparse data refers to tensor data using a 16-degree parallel storage scheme (C / 16, H, W, 16), and the data bit width is usually 8 bits or 16 bits. The sparse data can be converted from tensor data using an original storage scheme (C, H, W). In the original storage scheme, W represents width, which is the 0th dimension of the data; H represents height, which is the 1st dimension of the data; and C represents channel, which is the 2nd dimension of the data.

[0046] The conversion method of the 16-degree parallel storage scheme (C / 16, H, W, 16) is as follows: for the original storage scheme (C, H, W), take the continuous 16 data in the C direction, store them in a physically address-continuous manner, and set them as the 0th dimension, the length of which is fixed as 16; set W as the 1st dimension, the length of which remains unchanged; set H as the 2nd dimension, the length of which remains unchanged; and set C / 16 as the 3rd dimension, C / 16 being expressed as the length C divided by 16 and rounded down. An example of conversion is shown in Figure 2 .

[0047] The sparse data has at least one sparse feature point. The sparse feature point refers to: in the sparse data, if the values of all the 0th dimension data pointed to by the 1st, 2nd, and 3rd coordinates (Z, Y, X) are 0, then the (Z, Y, X) coordinate is referred to as a sparse feature point of the sparse data. The sparse data can have N sparse feature points, and the number N is less than or equal to (C / 16) * H * W.

[0048] The sparse compressed data refers to sparse data in which all 0-dimensional 16 numbers pointed by all sparse feature points are deleted. Further, in order to compensate for information loss (some data is deleted) caused by sparse data compression, a sparse mapping table is used to save the original complete information of the sparse data. The sparse mapping table refers to a tensor data with a size of (C / 16, H, W) and a data bit width of 1 bit. The sparse mapping table corresponds to the sparse data: each 1-bit data in the sparse mapping table represents whether the values of the 0-dimensional 16 data corresponding to the 1-bit data are all 0. For example, the value of a 1-bit data pointed by 0, 1, and 2-dimensional coordinates (Z, Y, X) in the sparse mapping table with a size of (C / 16, H, W) represents whether the 0-dimensional 16 data corresponding to the 1, 2, and 3-dimensional coordinates (Z, Y, X) in the sparse data with a size of (C / 16, H, W, 16) are all 0. When the 1-bit data in the sparse mapping table is 0, it represents that the 16 data in the corresponding sparse data are all 0. When the 1-bit data in the sparse mapping table is 1, it represents that the 16 data in the corresponding sparse data have at least one non-0 data. An example of a sparse compressed data and a corresponding sparse mapping table is shown in FIG. 1. Figure 3

[0049] The sparse data and the sparse compressed data can be converted to each other: the sparse compressed data can be converted from the sparse data by compression, and the sparse data can be converted from the sparse compressed data by decompression.

[0050] In the sparse data calculation device, the configuration unit is configured to read the configuration data, and configure, by RDMA, the start address information and the size information of the input feature sparse mapping table to the input feature sparse mapping table, configure, by RDMA, the start address and the size information of the parameter sparse mapping table to the parameter sparse mapping table, configure, by RDMA, the start address and the size information of the input feature data to the input feature sparse compressed data, configure, by RDMA, the start address and the size information of the parameter data to the parameter sparse compressed data, configure, by WDMA, the start address and the size information of the output feature data to the output feature sparse compressed data, and configure, by WDMA, the start address and the size information of the output feature sparse mapping table to the output feature sparse mapping table.

[0051] The size of the sparse data is expressed as (C / 16, H, W, 16), the size of the sparse compressed data is expressed as (C / 16, H, W, 16), and the size of the sparse mapping table is expressed as (C / 16, H, W).

[0052] ​The sparse data computing device, the input feature sparse mapping table RDMA is used to read the input feature sparse mapping table according to the start address and size information of the input feature sparse mapping table. The input feature sparse mapping table RDMA reads 1 bit of data at a time, and sequentially reads data in the order of 0, 1, and 2 dimensions starting from the start address.

[0053] The sparse data computing device, the parameter sparse mapping table RDMA is used to read the parameter sparse mapping table according to the start address and size information of the parameter sparse mapping table. The parameter sparse mapping table RDMA reads 1 bit of data at a time, and sequentially reads data in the order of 0, 1, and 2 dimensions starting from the start address.

[0054] The sparse data computing device, the input feature data RDMA is used to read the input feature sparse compressed data according to the input feature sparse mapping table and the start address and size information of the input feature data. The input feature data RDMA reads 16 numbers of the 0th dimension of the input feature sparse compressed data at a time, and the reading method is as follows: starting from the (0, 0, 0) coordinate of the input feature sparse mapping table, it is judged whether the value of the corresponding data of the sparse mapping table is 0. If it is 0, it means that the input feature sparse compressed data corresponding to the coordinate does not exist, and the reading of the coordinate data is skipped. If it is 1, it means that the input feature sparse compressed data corresponding to the coordinate exists, and the coordinate data is read. The address of the data reading starts from the start address and is sequentially incremented with the reading process.

[0055] The sparse data computing device, the parameter data RDMA is used to read the parameter sparse compressed data according to the parameter sparse mapping table and the start address and size information of the parameter data. The parameter data RDMA reads 16 numbers of the 0th dimension of the parameter sparse compressed data at a time, and the reading method is as follows: starting from the (0, 0, 0) coordinate of the parameter sparse mapping table, it is judged whether the value of the corresponding data of the sparse mapping table is 0. If it is 0, it means that the parameter sparse compressed data corresponding to the coordinate does not exist, and the reading of the coordinate data is skipped. If it is 1, it means that the parameter sparse compressed data corresponding to the coordinate exists, and the coordinate data is read. The address of the data reading starts from the start address and is sequentially incremented with the reading process.

[0056] The sparse data computing device, the operation unit is used to perform addition or multiplication operation of the input feature sparse compressed data and the parameter sparse compressed data.

[0057] The operation unit performs the operation of the maximum 16 groups of input feature sparse compression data and 16 groups of parameter sparse compression data in each round of 16 cycles. The operation unit performs the operation of 1 group of data in each cycle. Each group of data contains 16 numbers (16 numbers of the 0th dimension of the sparse compression data).

[0058] The operation unit obtains 16 bits of input feature sparse mapping table from the input feature sparse mapping table RDMA and 16 bits of parameter sparse mapping table from the parameter sparse mapping table RDMA at the beginning of each round of operation. For the addition operation mode, the 16 bits of input feature sparse mapping table and the 16 bits of parameter sparse mapping table are subjected to the logical OR operation to generate the addition sparse mapping table. For the multiplication operation mode, the 16 bits of input feature sparse mapping table and the 16 bits of parameter sparse mapping table are subjected to the logical AND operation to generate the multiplication sparse mapping table.

[0059] The 16 bits of addition or multiplication sparse mapping table correspond to 16 groups of input feature sparse compression data and parameter sparse compression data. The kth bit of the mapping table corresponds to the kth group of input feature sparse compression data and the kth group of parameter sparse compression data, where k is an integer in the interval [0, 16). For example, the 0th bit of the mapping table corresponds to the 0th group of input feature sparse compression data and the 0th group of parameter sparse compression data, and the 15th bit of the mapping table corresponds to the 15th group of input feature sparse compression data and the 15th group of parameter sparse compression data.

[0060] The operation of each round of operation is as follows: the operation unit sequentially traverses the addition or multiplication sparse mapping table from the 0th bit, and determines whether the data value of the addition or multiplication sparse mapping table is 1. If the data value is 1, the operation unit spends one cycle to perform the operation of the corresponding group of data. If the data value is 0, the operation unit skips the operation and does not spend a cycle. Therefore, the operation unit needs a maximum of 16 cycles for each round of calculation, that is, when the value of the 16-bit addition sparse mapping table in the addition operation mode or the 16-bit multiplication sparse mapping table in the multiplication operation mode is FFFF in hexadecimal, the operation unit performs the calculation of all 16 groups of data. The operation unit needs a minimum of 0 cycles for each round of calculation, that is, when the value of the 16-bit addition sparse mapping table in the addition operation mode or the 16-bit multiplication sparse mapping table in the multiplication operation mode is 0, the operation unit skips all 16 groups of data.

[0061] The operation unit performs the addition or multiplication operation on the valid input feature sparse compression data and the valid parameter sparse compression data in the minimum 0 cycles to the maximum 16 cycles of each round of calculation, and sends the operation result to the output feature sparse data WDMA. At the end of each round of calculation, the 16-bit addition or multiplication sparse mapping table is sent to the output feature sparse mapping table WDMA as the output feature sparse mapping table.

[0062] The output feature sparse data WDMA is used to write out the output feature sparse compressed data in response to the output feature sparse compressed data write request and according to the start address and size information of the output feature sparse compressed data. The output feature sparse data WDMA writes out 16 numbers at a time and writes out the output feature sparse compressed data in the order of sequentially increasing addresses starting from the start address for valid data from the operation unit.

[0063] The output feature sparse map table WDMA is used to write out the output feature sparse map table in response to the output feature sparse map table write request and according to the start address and size information of the output feature sparse map table. The output feature map table WDMA writes out 16 numbers at a time and writes out the output feature map table in the order of sequentially increasing addresses starting from the start address for valid data from the operation unit.

[0064] The feature of the sparse data computing device further includes reading in sparse compressed data, performing addition or multiplication operation of the sparse compressed data, and generating sparse compressed data.

[0065] The method for computing sparse data is as shown in Figure 4

[0066] (1) Configuration. Read the configuration data required for computation, which mainly includes the start address and size of the input feature sparse compressed data, the start address and size of the parameter sparse compressed data, the start address and size of the output feature sparse compressed data, the start address and size of the input feature sparse map table, the start address and size of the parameter sparse map table, the start address and size of the output feature sparse map table, and configure the modules of the sparse data computing device.

[0067] (2) Read the sparse map table. According to the configuration information, read the input feature sparse map table and the parameter sparse map table.

[0068] For a sparse map table with size (C / 16, H, W), the reading method is to read 1 bit of data at a time and read the data in the order of 0, 1, 2-dimensional sequential traversal starting from the start address.

[0069] ​(3) Reading sparse compression data. Reading input feature sparse compression data according to configuration information and input feature sparse map table, reading parameter sparse compression data according to configuration information and parameter sparse map table. The reading method is: starting from the (0, 0, 0) coordinate of the sparse map table, judging whether the value of the data corresponding to the sparse map table is 0. If it is 0, it means that the sparse compression data corresponding to the coordinate does not exist, and the reading of the coordinate data is skipped. If it is 1, it means that the sparse compression data corresponding to the coordinate exists, and the coordinate data is read. The address of the data reading starts from the starting address and is sequentially incremented with the reading process.

[0070] (4) Calculating to obtain an addition sparse map table or a multiplication sparse map table. The maximum is 16 cycles per round. At the beginning of each round of operation, 16 bits of input feature sparse map table and 16 bits of parameter sparse map table are obtained. For addition operation mode, the 16 bits of input feature sparse map table and the 16 bits of parameter sparse map table are operated by "logical or" to generate an addition sparse map table. For multiplication operation mode, the 16 bits of input feature sparse map table and the 16 bits of parameter sparse map table are operated by "logical and" to generate a multiplication sparse map table.

[0071] (5) Performing addition or multiplication operation. According to the value of 16-bit addition sparse map table or 16-bit multiplication sparse map table, selectively performing addition or multiplication operation: the value of 1 of the sparse map table represents that the data is valid and the operation is performed, and the value of 0 of the sparse map table represents that the data is invalid and the calculation is skipped. Therefore, in the minimum 0 cycles and the maximum 16 cycles of each round of calculation, the valid input feature sparse compression data and the valid parameter sparse compression data are subjected to addition or multiplication operation to generate output feature sparse compression data and output feature sparse map table, and output feature sparse compression data write request and output feature sparse map table write request are sent. The output feature sparse map table is the addition or multiplication sparse map table.

[0072] (6) Writing sparse compression data. Responding to the output feature sparse compression data write request, and writing the 16 data to the corresponding address. The writing method is: writing 16 numbers at a time, and writing the data in the order of address increment from the starting address.

[0073] (7) Writing sparse map table. Responding to the output feature sparse map table write request, and writing the sparse map table data to the corresponding address. The writing method is: writing 1 bit of data at a time, and writing the sparse map table in the order of address increment from the starting address.

[0074] In the embodiment, the sparse data refers to tensor data in a 16-degree parallel storage scheme (C / 16, H, W, 16). In specific applications, to adapt to the scale and processing capacity of different tensor processors, the sparse data can be more widely defined as tensor data in an n-degree parallel storage scheme (C / n, H, W, n), where n is an integer multiple of 8.

[0075] Although the present application has been particularly shown and described with respect to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. A sparse data computing apparatus for a neural network tensor processor, the apparatus comprising: The input feature sparse mapping table RDMA, the parameter sparse mapping table RDMA, the input feature data RDMA, the parameter data RDMA, the configuration unit, the operation unit, the output feature data WDMA, the output feature sparse mapping table WDMA, the input feature sparse compression data, the parameter sparse compression data, the input feature sparse mapping table, the parameter sparse mapping table, the configuration data, the output feature sparse compression data and the output feature sparse mapping table are included. The configuration unit is configured to read the configuration data, and configure the start address information and the size information of the input feature sparse mapping table RDMA to the input feature sparse mapping table RDMA, configure the start address and the size information of the parameter sparse mapping table RDMA to the parameter sparse mapping table RDMA, configure the start address and the size information of the input feature sparse compression data to the input feature data RDMA, configure the start address and the size information of the parameter sparse compression data to the parameter data RDMA, configure the start address and the size information of the output feature sparse compression data to the output feature data WDMA, and configure the start address and the size information of the output feature sparse mapping table WDMA to the output feature sparse mapping table WDMA. The input feature sparse mapping table RDMA is configured to read the input feature sparse mapping table according to the start address and the size information of the input feature sparse mapping table. The parameter sparse mapping table RDMA is configured to read the parameter sparse mapping table according to the start address and the size information of the parameter sparse mapping table. The input feature data RDMA is configured to read the input feature sparse compression data according to the input feature sparse mapping table and the start address and the size information of the input feature data. The parameter data RDMA is configured to read the parameter sparse compression data according to the parameter sparse mapping table and the start address and the size information of the parameter data. The operation unit is configured to perform the addition or multiplication operation of the input feature sparse compression data and the parameter sparse compression data. The output feature sparse data WDMA is configured to write the output feature sparse compression data according to the start address and the size information of the output feature sparse compression data in response to the output feature sparse compression data write request. The output feature sparse mapping table WDMA is configured to write the output feature sparse mapping table according to the start address and the size information of the output feature sparse mapping table in response to the output feature sparse mapping table write request.

2. The sparse data computing apparatus for a neural network tensor processor of claim 1, wherein, The sparse data refers to the tensor data in the n-degree parallel storage scheme (C / n, H, W, n), and the data bit width is 8 bits or 16 bits; the original storage scheme (C, H, W) of the tensor data, W represents width, which is the 0th dimension of the data; H represents height, which is the 1st dimension of the data; C represents channel, which is the 2nd dimension of the data. The conversion method of the n-parallel storage scheme (C / n, H, W, n) is: for the original storage scheme (C, H, W) of tensor data, taking the continuous n data in the C direction, storing in a physically continuous manner, and setting the 0th dimension, the length of the 0th dimension is fixed as n; setting W as the 1st dimension, the length is unchanged; setting H as the 2nd dimension, the length is unchanged; setting C / n as the 3rd dimension, C / n is expressed as the length C divided by n and rounded down.

3. The sparse data computing apparatus for a neural network tensor processor of claim 2, wherein, The n is an integer multiple of 8.

4. The sparse data computing apparatus for a neural network tensor processor of claim 2, wherein, The sparse data has at least one sparse feature point; the sparse feature point refers to that in the sparse data, if the 0th dimension of the 1, 2, 3-dimensional coordinates (Z, Y, X) pointed to is all 0, the (Z, Y, X) coordinates are referred to as a sparse feature point of the sparse data.

5. The sparse data computing apparatus for a neural network tensor processor of claim 4, wherein, The sparse data has N sparse feature points, and the number of N is less than or equal to (C / n) * H * W.

6. The sparse data computing apparatus for a neural network tensor processor of claim 4, wherein, The sparse compressed data refers to the sparse data in which all the n numbers of the 0th dimension pointed to by all the sparse feature points are deleted; the sparse mapping table refers to the tensor data with a size of (C / n, H, W) and a data bit width of 1 bit; The sparse mapping table corresponds to the sparse data: each 1-bit data of the sparse mapping table represents whether the values of the n data of the 0th dimension of the corresponding sparse data are all 0.

7. The sparse data computing apparatus for a neural network tensor processor of claim 2, wherein, The operation unit has a maximum of n periods as a round, and each round performs the operation of n groups of input feature sparse compressed data and n groups of parameter sparse compressed data, and each period performs the operation of 1 group of data, and each group of data contains n numbers, which correspond to the n numbers of the 0th dimension of the sparse compressed data; When the operation of a round starts, the operation unit acquires the input feature sparse mapping table RDMA of the continuous n bits from the input feature sparse mapping table RDMA, and acquires the parameter sparse mapping table RDMA of the continuous n bits from the parameter sparse mapping table RDMA; for the addition operation mode, the input feature sparse mapping table of the continuous n bits and the parameter sparse mapping table of the continuous n bits are subjected to the "logical or" operation to generate the addition sparse mapping table; for the multiplication operation mode, the input feature sparse mapping table of the continuous n bits and the parameter sparse mapping table of the continuous n bits are subjected to the "logical and" operation to generate the multiplication sparse mapping table; The n-bit addition or multiplication sparse mapping table corresponds to n groups of input feature sparse compressed data and parameter sparse compressed data: the kth bit of the mapping table corresponds to the kth group of input feature sparse compressed data and the kth group of parameter sparse compressed data, where k is an integer in the interval [0, n). The execution mode of the operation of a round is: starting from the 0th bit, the addition or multiplication sparse mapping table is traversed in sequence, and it is judged whether the data value of the addition or multiplication sparse mapping table is 1: if it is 1, one period is spent to perform the operation of the corresponding group of data; if it is 0, the operation is skipped and no period is spent.

8. A method for sparse data computation for a neural network tensor processor, comprising: The application is applied to the sparse data calculation device for the neural network tensor processor as claimed in any one of claims 2-7, comprising: Step S1, reading the configuration data required for calculation, and configuring each module of the sparse data calculation device; Step S1, reading the configuration data required for calculation, and configuring each module of the sparse data calculation device; Step S2, reading input feature sparse map table and parameter sparse map table according to configuration information; Step S3, reading input feature sparse compression data according to configuration information and input feature sparse map table, and reading parameter sparse compression data according to configuration information and parameter sparse map table; Step S4, calculating to obtain addition sparse map table or multiplication sparse map table: maximum with n cycles as a round, at the beginning of each round operation, obtaining n continuous bits of input feature sparse map table and n continuous bits of parameter sparse map table; for addition operation mode, the n continuous bits of input feature sparse map table and the n continuous bits of parameter sparse map table are operated to generate addition sparse map table; for multiplication operation mode, the n continuous bits of input feature sparse map table and the n continuous bits of parameter sparse map table are operated to generate multiplication sparse map table; Step S5, performing addition or multiplication operation: according to the value of n-bit addition sparse map table or n-bit multiplication sparse map table, selectively performing addition or multiplication operation: the value of 1 of sparse map table represents that the data is valid and the operation is performed, and the value of 0 of sparse map table represents that the data is invalid and the calculation is skipped; Step S6, writing sparse compression data: responding to the output feature sparse compression data write request, and writing n data to the corresponding address; Step S7, writing sparse map table: responding to the output feature sparse map table write request, and writing sparse map table data to the corresponding address.

9. The method for sparse data computation for a neural network tensor processor of claim 8, wherein, In the step S1, the configuration data includes the start address and size of input feature sparse compression data, the start address and size of parameter sparse compression data, the start address and size of output feature sparse compression data, the start address and size of input feature sparse map table, the start address and size of parameter sparse map table, and the start address and size of output feature sparse map table.

Images