Neural network sparse deployment and calculation method based on storage array

Abstract

The invention provides a neural network sparse deployment and calculation method based on a storage array, and belongs to the technical field of neural network quantization and hardware deployment. According to the method, weight sparse information is deployed in situ by using a storage array, and the advantages of structured sparse and unstructured sparse are combined, so that parallel processing of full-stack sparse information such as structured / unstructured sparse and input sparse is realized, the deployment efficiency and energy efficiency of the neural network are improved, and irregularity of the neural network after quantization can be well compatible; the method provided by the invention is theoretically suitable for all memories with read-write functions, and has strong universality.

Description

Technical Field

[0001] The present invention relates to the field of neural network quantization and hardware deployment, and specifically relates to a method for in-situ deployment and calculation of a sparse storage array of a neural network, which is mainly applied to sensor and edge computing platforms. Background Art

[0002] With the continuous development of artificial intelligence and deep learning technologies, deep neural networks have increasingly become the key to improving task performance. However, these models usually require a large amount of computing resources and storage space. When the target application scenario reaches sensor nodes or other extreme edge nodes, there will be a gap of several to dozens of times between the on-chip resources and the number of network parameters. At this time, there is an obvious contradiction between the network model and the limitation of on-chip resources. Edge computing platforms are often restricted by many factors such as power consumption, area, and heat dissipation, and thus cannot bear the high computing requirements brought by large-scale deep learning models. Neural network quantization technology is a process of converting a deep learning model from a floating-point representation to an integer representation. This technology is very useful when the model is deployed to edge devices because the integer representation requires less memory space and computing resources, thereby improving performance and efficiency. Among them, sparsification is the most direct method for model compression. This method improves the operation speed by skipping unnecessary calculations in network inference and reduces the network size at the same time. Neural network sparsity includes structured sparsity and unstructured sparsity.

[0003] Structured sparsity is usually applied in larger units, such as entire neurons, convolutional kernels, or channels. This can reduce the number of parameters that need to be stored and calculated, and achieve more efficient computing; unstructured sparsity is performed on each individual weight, and can highly flexibly and selectively remove weights that have limited impact on the model performance. By setting specific weights to zero, the storage requirements of the model can be significantly reduced, thereby reducing the occupation of underlying storage and the bandwidth requirements for transmitting data. It is not difficult to see that the two sparsity schemes are applicable to different scenarios and have obvious advantages.

[0004] However, the traditional neural network sparsity schemes still have the following challenges: (1) Neural network sparsity leads to irregularity of weights, making it difficult to deploy and map in a regular storage array. (2) The distribution of unstructured sparsity in the array is uneven, resulting in the need for additional storage space or circuits to process sparse information, which increases the design complexity. (3) Existing work often only optimizes the structured sparsity of weights and lacks hardware designs that optimize the full-chain sparsity. Summary of the Invention

[0005] In view of the problems existing in the prior art, the present invention proposes a neural network sparse deployment and calculation method based on a storage array, which uses the storage array to deploy weight sparse information in-situ, combines the advantages of structured sparsity and unstructured sparsity, realizes parallel processing of full-stack sparse information such as structured / unstructured sparsity and input sparsity, and finally improves the deployment efficiency and energy efficiency of the neural network, and can be well compatible with the irregularity after neural network quantization.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A neural network sparse deployment and calculation method based on a storage array, characterized by comprising the following steps:

[0008] (1) Perform sparse layout of the storage array on the weight data:

[0009] Divide the n-column storage array into an upper part and a lower part. The upper part occupies b rows of storage space and is used to store structured sparse values. The n columns correspond to n structured sparse values, and each structured sparse value corresponds to a column of weight data in the lower part. The structured sparse value represents the number of all-zero columns of weight data between the non-zero column of this weight data and the non-zero column of the next weight data, and satisfies the constraint condition 2 b ≤n≤2 b+1 ;

[0010] The lower part stores unstructured sparse units. Each column of data contains an integer number of unstructured sparse units. Each unstructured sparse unit consists of (1 + a) bit data, where 1 bit is the sparse bit and a bit is the weight data. If the sparse bit is 1, the corresponding a bit of weight data is all zero; if it is 0, the weight data is non-zero;

[0011] (2) Write the weight data with sparse layout into the storage array through writing, and give priority to writing into the unstructured sparse units and then into the structured sparse values;

[0012] (3) Perform sparse coding on the input data: Divide the input data into groups of a bit. The sparse coding of each group of input data is represented as [m, a bit non-zero data group], where m is the number of all-zero groups between the current non-zero group and the previous non-zero group, m is represented by a bit, and m is recorded as the sparse coding value;

[0013] (4) Read the weight data of the storage array column by column, read the input data after sparse coding group by group, and perform the following operations on the weight data of each column: Read the structured sparse value k of the current column and the sparse coding value m of the corresponding input data; if k≥1 or m≥1, skip the calculation of the current column weight and the current group of input data, and assign the result to zero; if k = 0 and m = 0, traverse the unstructured sparse cells of this column for sparse bit identification and multiplication-accumulation calculation with the input data: if the sparse bit is 1, skip the calculation of this cell and assign the result to zero; if the sparse bit is 0, perform multiplication-accumulation calculation on the a-bit weight data and the corresponding a-bit input data;

[0014] (5) Accumulate the calculation results of the unstructured sparse cells at the same positions in each column of the storage array to obtain the final output result.

[0015] Furthermore, the writing method of the unstructured sparse cells in step (1) is to write in row order, and each unstructured sparse cell occupies (1 + a) rows of storage space.

[0016] Furthermore, the sparse coding value m of the first non-zero group in step (3) represents the number of all-zero groups from the start of the input to this non-zero group.

[0017] Furthermore, in step (5), multiple storage arrays are cascaded for parallel calculation, and the multiplication-accumulation results of each array are added.

[0018] Furthermore, for the neural network sparse deployment and calculation method based on the storage array, according to the proportion and distribution law of the zero-containing columns of the weight data, customize and adjust the number of rows b occupied by the structured sparse value, and adjust the unstructured sparse degree by changing the number of bits a of the weight data of the unstructured sparse cells, so as to deepen the sparse degree while ensuring the accuracy.

[0019] Advantages of the present invention:

[0020] The neural network sparse deployment and calculation method based on the storage array of the present invention adopts a deployment strategy combining structured and unstructured sparsity. Theoretically, it is applicable to all memories with read and write functions and has strong universality. This strategy uses the storage array to deploy the weight sparse information in-situ, combines the advantages of structured sparsity and unstructured sparsity, realizes parallel processing of full-stack sparse information such as structured / unstructured sparsity and input sparsity, and finally improves the deployment efficiency and energy efficiency of the neural network and can well accommodate the irregularity after neural network quantization. Description of the drawings

[0021] Figure 1 It is a flowchart of the neural network sparse deployment and calculation method based on the storage array of the present invention;

[0022] Figure 2This is the overall sparse layout diagram of the weight data in the storage array of the present invention;

[0023] Figure 3 This is the sparse layout diagram of the weight data on a 1160*256 scale storage array in an embodiment of the present invention;

[0024] Figure 4 This is the schematic diagram of structured sparsity in an embodiment of the present invention;

[0025] Figure 5 This is the schematic diagram of an unstructured sparse unit in an embodiment of the present invention;

[0026] Figure 6 This is the schematic diagram of the sparse coding of the input data in an embodiment of the present invention. Detailed implementation manners

[0027] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0028] Figure 1 Shown is the flowchart of the neural network sparse deployment and calculation method based on a storage array of the present invention, including the sparse deployment and calculation of weight data and input data. The parameters included in this method are: the structured sparsity value k, the sparse coding value m of the input data, the number of bits a bit of the weight data of the unstructured sparse unit, and each group of input data after sparse coding is represented by (m + a) bit; this method includes the following steps:

[0029] (1) Perform sparse layout of the storage array for the weight data. As Figure 2 shown, divide the storage array of n columns into upper and lower parts. The upper part occupies b rows of storage space and is used to store the structured sparsity value k. Each column of data corresponds to a structured sparsity value, and n structured sparsity values are represented by k0 to k n-1 , which is used to represent the distribution of all-zero columns of the weight data in the lower part. To completely represent all sparse situations and avoid resource waste, it is necessary to satisfy 2 b ≤n≤2 b+1 . The lower part of the array is used to represent unstructured sparsity. Each column of data contains an integer number of unstructured sparse units. Each unstructured sparse unit is composed of (1 + a) bit data. The top 1 bit of data represents the unstructured sparse bit, and the following a bit represents the weight data. If the unstructured sparse bit is 1, it means that the a bit of weight data is all zero. If the unstructured sparse bit is 0, it means that the a bit of weight data is non-zero.

[0030] For example, take b = 8, a = 8, n = 256. As Figure 3As shown, the storage array is divided into upper and lower parts, with scales of 8 * 256 and 1152 * 256 respectively. The upper part of 8 * 256 space is used to store structured sparse values, and the specific representation is as Figure 4 shown, k x and k y are the structured sparse values corresponding to the non-zero columns of the weight data, occupying 8 rows of space, representing the number of all-zero columns of the weight data between the non-zero column of this weight data and the next non-zero column of the weight data. It can be seen from the right part of the figure that k x represents the number of all-zero columns of the weight data from column x to column y; the lower part of 1152 * 256 space is used to represent unstructured sparsity. Each column contains 128 unstructured sparse units, forming 128 * 9 = 1152 rows of weight data with unstructured sparse information. The representation of the unstructured sparse unit is as Figure 5 shown in the left part. Each unstructured sparse unit consists of 9-bit data. The first bit in each group of data is the sparse bit, and the following 8 bits are the weight data. During subsequent calculations, it will be sent to the MAC multiply-accumulate unit together with the 8-bit input data to complete the multiply-accumulate calculation of the weight data and the input data.

[0031] (2) Deploy the weight data after sparse layout to the storage array by writing. First, write the unstructured sparse units. Write each unstructured sparse unit in row form. Each unstructured sparse unit occupies (1 + a) rows of storage space. After writing one row, continue to write the next row until the unstructured sparse unit space is full. Finally, write the structured sparse values to realize the deployment of the weight data on the storage array. Wait for data input.

[0032] (3) Perform sparse coding processing on the input data. Input it in groups of a bits in sequence. The data after sparse processing is represented as [m, a-bit non-zero data], where m is the sparse coding value, representing the number of all-zero groups between this non-zero group and the previous non-zero group. m is also represented by a bits;

[0033] For example, take a = 8. As Figure 6 shown, the original input data is input in groups of 8 bits in sequence. The data after sparse processing is represented as [m, 8-bit non-zero data group], where m represents the number of all-zero groups between this non-zero group and the previous non-zero group, and is also represented by 8 bits. For example, there are 3 8-bit all-zero groups between 0111_1000 and 1111_1111, so it is represented as [3, 1111_1111], and the binary representation is [0000_0011, 1111_1111]; if two non-zero groups are adjacent, m is represented as 0; for the first non-zero group, m represents the number of all-zero groups from the start of input to the appearance of the non-zero group.

[0034] (4) Read the input data after sparse processing in groups, and read the weight data in the storage array column by column. Each column of weight data corresponds to a group of input data. For a group of input data, first read its sparse coding value m. For the weight data of a column in the storage array, first read the structured sparse value k of this column of weight data (such as Figure 4 the k in x , k y value); determine whether the m and k values meet the condition: greater than or equal to 1. If one of the m or k values meets the condition, it means that this group of input data is all zero or this column of weight data is all zero, and the result after multiplication and accumulation calculation is zero, then skip the calculation of this column of weight and this group of input data, and assign the result to zero; if both m and k are 0, it means that both this column of weight data and this group of input data are non-zero values, then traverse the unstructured sparse units of this column for sparse bit recognition and multiplication and accumulation calculation with the input data: first read the sparse bit of the first group of unstructured sparse units. If the sparse bit is 1, it means that this unit is a bit all-zero data, and the calculation of the corresponding position is skipped and the result is assigned to zero; if the sparse bit is 0, it means that this unit is a bit non-zero weight data, and send the a-bit weight data of this unit and the a-bit input data corresponding to this column to the multiplication and accumulation unit MAC for calculation; then read the sparse bit of the second group of unstructured sparse units. Similarly, if the sparse bit is 1, skip the calculation of this unit and assign the result to zero; if the sparse bit is 0, send the weight data of this unit and the input data corresponding to this column to the multiplication and accumulation unit MAC for calculation; the weight data of the unstructured sparse units in the same column corresponds to the same group of input data. Sequentially perform sparse bit recognition and multiplication and accumulation calculation with the input data for each group of unstructured sparse units in this column until the calculation of all unstructured sparse units in this column is completed; repeat this step to complete the calculation of all weight data and input data in the storage array;

[0035] (5) Further process the calculation results, and accumulate the calculation results of the unstructured sparse units at the same position in each column of the storage array. To understand more intuitively, it is to accumulate the calculation result of the first unit in the first column with the calculation results of the first units in the 2nd, 3rd, 4th, and until the last column, and accumulate the calculation result of the second unit in the first column with the calculation results of the second units in the 2nd, 3rd, 4th, and until the last column, and so on, until the last unit in each column is accumulated. After all calculations are completed, the final calculation result is obtained.

[0036] Based on the embodiments of the present invention, multiple storage arrays can be cascaded for parallel calculation, and then the multiplication and accumulation results are added to achieve high-parallel calculation; and according to the proportion and distribution law of the zero-containing columns of the weight data, the number of rows occupied by the structured sparse values can be customized and adjusted, and by changing the number of bits a of the unstructured sparse units, the degree of unstructured sparsity can be adjusted to deepen the degree of sparsity while ensuring the accuracy.

[0037] Although the present invention has been disclosed above in preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make many possible changes and modifications to the technical solution of the present invention, or modify it into equivalent embodiments with equivalent changes, without departing from the scope of the technical solution of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A neural network sparse deployment and computing method based on a storage array, characterized in that, It includes the following steps: (1) Perform sparse layout of the storage array for the weight data: Divide the n-column storage array into an upper part and a lower part. The upper part occupies b rows of storage space and is used to store structured sparse values. The n columns correspond to n structured sparse values, and each structured sparse value corresponds to a column of weight data in the lower part. The structured sparse value represents the number of all-zero columns of weight data between the non-zero column of this weight data and the non-zero column of the next weight data, and satisfies Constraint 2 b ≤n≤2 b+1 ; The lower part stores unstructured sparse cells. Each column of data contains an integer number of unstructured sparse cells. Each unstructured sparse cell consists of (1 + a) bit data, where 1 bit is the sparse bit and a bit is the weight data. If the sparse bit is 1, the corresponding a bit weight data is all zero. If it is 0, the weight data is a non-zero value; (2) Write the weight data with sparse layout into the storage array through deployment. First, write into the unstructured sparse cells, and then write into the structured sparse values; (3) Perform sparse coding on the input data: Divide the input data into groups of a bit. Each group of input data sparse coding is represented as [m, a bit non-zero data group], where m is the number of all-zero groups between the current non-zero group and the previous non-zero group. m is represented by a bit number and m is denoted as the sparse coding value; (4) Read the weight data of the storage array column by column, read the input data after sparse coding group by group, and perform the following operations on the weight data of each column: Read the structured sparse value k of the current column and the sparse coding value m of the corresponding group of input data; If k ≥ 1 or m ≥ 1, skip the calculation of the weight of the current column and the input data of the current group, and assign the result to zero. If k = 0 and m = 0, traverse the unstructured sparse cells of this column for sparse bit identification and multiplication-accumulation calculation with the input data: If the sparse bit is 1, skip the calculation of this cell and assign the result to zero; If the sparse bit is 0, perform multiplication-accumulation calculation on the a bit weight data and the corresponding a bit input data; (5) Accumulate the calculation results of the unstructured sparse cells at the same positions in each column of the storage array to obtain the final output result.

2. The method according to claim 1, wherein In the step (1), the writing method of the unstructured sparse cells is to write in row order, and each unstructured sparse cell occupies (1 + a) rows of storage space.

3. The method according to claim 1, characterized in that In the step (3), the sparse coding value m of the first non-zero group represents the number of all-zero groups from the start of the input to this non-zero group.

4. The method according to claim 1, wherein In the step (5), parallel calculation is performed by cascading multiple storage arrays, and the multiplication-accumulation results of each array are added.

5. The method according to claim 1, characterized in that According to the proportion and distribution law of the zero-containing columns of the weight data, customize and adjust the number of rows b occupied by the structured sparse values, and adjust the degree of non-structured sparsity by changing the number of bits a of the weight data of the unstructured sparse cells.

Images