Sparse neural network processor and method based on mixed-level precision calculation
By designing a sparse neural network processor based on mixed-level precision operations, using branch storage and decoding of sparse weights and activation values, the problem that neural network processors cannot support mixed-level precision calculations is solved, and efficient calculations and low power consumption are achieved .
Patent Information
- Application Number
- CN202110534230.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-17
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-05-17
AI Technical Summary
Neural network processors cannot support mixed-level accuracy calculations and only perform calculations on non-zero weights, resulting in difficulty in maximizing the calculation amount and power consumption.
A sparse neural network processor based on mixed-level precision operations is designed, including weight cache module, activation value cache module, activation value index module and multiplication and accumulation operation module. By compressing branch storage and decoding of sparse weights and activation values, it realizes Non-zero weighted data operation and mixed-level accuracy calculation under high parallelism.
It realizes the maximization of computing volume and power consumption while ensuring accuracy, makes full use of network sparsity, and improves computing efficiency and throughput.
Smart Images

Figure CN113962376B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of deep learning technology, and in particular to a sparse neural network processor and method based on mixed-level precision operations. Background Art
[0002] In order to reduce the computational workload and storage burden of neural networks, pruning the network to remove redundant parameters is a very effective way to accelerate neural networks. Pruning introduces a large amount of sparse weight data into the network model. To take advantage of the sparsity of the network, it is necessary to skip the randomly distributed zero-weight data when executing the network model. Generally, the activation values and weights of neural networks are stored continuously in the cache and are also read continuously when read. Ordinary neural network processors cannot skip zero-weight data and only calculate non-zero weight data.
[0003] In order to minimize the amount of calculation and reduce power consumption while ensuring accuracy, different layers of the neural network will use different data precision (data bit width) to participate in the calculation. The traditional neural network processor can process a single model precision and does not support mixed-level precision calculation. Therefore, the present application aims to provide a neural network processor that can load and operate non-zero weight data under high parallelism and support mixed-level precision calculation. Summary of the invention
[0004] In order to solve the technical problems that the neural network processor can process models with a single precision, does not support mixed-level precision calculations, and is difficult to simultaneously calculate only non-zero weights, the present application discloses a sparse neural network processor and method based on mixed-level precision operations through the following embodiments.
[0005] In a first aspect, the present application discloses a sparse neural network processor based on mixed-level precision operations, comprising: a weight cache module, an activation value cache module, an activation value index module, a multiplication-accumulation operation module and an output cache module, wherein the multiplication-accumulation operation module comprises a multiplication-accumulation operation unit and a register, and the precision of the multiplication-accumulation operation unit is a preset first fixed bit width;
[0006] The weight cache module is used to obtain the compressed sparse weights and store the compressed sparse weights in rows, wherein the data width of each row is the first fixed bit width; it is also used to decode the compressed sparse weights during the convolution operation, output the decoded non-zero weight values to the multiplication and accumulation operation unit in sequence by row, and output the decoded non-zero weight position information to the activation index module in sequence by row;
[0007] The activation value cache module is used to store the activation values of the neural network in rows, wherein the data width stored in each row is the first fixed bit width; and is also used to read the activation values during the convolution operation and output them to the activation index module in sequence by row;
[0008] The activation value index module is used to select the activation value of each received row according to the non-zero weight position information of each received row, and output the selected activation value to the multiplication and accumulation operation unit in sequence by row;
[0009] The multiplication-accumulation operation unit is used to perform a multiplication-accumulation operation on each row of activation values and each row of non-zero weight values received, and obtain a plurality of multiplication-accumulation operation results; and is also used to determine whether to perform a shift operation on each of the multiplication-accumulation operation results, and if the determination result is no, directly output the multiplication-accumulation operation result to the register, and if the determination result is yes, perform a shift on the multiplication-accumulation operation result and then output it to the register;
[0010] The register is used to temporarily store the multiple multiplication-accumulation operation results, and output the multiple multiplication-accumulation results to the output buffer module after the final convolution operation is completed.
[0011] Optionally, the multiplication-accumulation-addition operation unit is used to perform the following steps in the process of determining whether to perform a shift operation on each of the multiplication-accumulation-addition operation results:
[0012] Obtaining a non-zero weight value series corresponding to a target multiplication-accumulation operation result, wherein the target multiplication-accumulation operation result is any of the multiplication-accumulation operation results, and the non-zero weight value series is a row sequence corresponding to the non-zero weight value;
[0013] Obtaining an activation value series corresponding to a target multiplication-accumulation operation result, wherein the activation value series is a row sequence corresponding to the activation value;
[0014] comparing the non-zero weight value series with the activation value series;
[0015] If the non-zero weight value level is consistent with the activation value level, obtaining the judgment result is no;
[0016] If the non-zero weight value level is inconsistent with the activation value level, the judgment result is obtained as yes.
[0017] Optionally, when the multiplication-accumulation-addition operation unit performs a shift on the multiplication-accumulation-addition operation result and then outputs it to the register, it is configured to perform the following steps:
[0018] Obtaining a shift number according to the non-zero weight value level, the activation value level and the first fixed bit width;
[0019] Based on the shift number, the target multiplication-accumulation operation result is left-shifted and then output to the register.
[0020] Optionally, acquiring the shift number according to the non-zero weight value level, the activation value level and the first fixed bit width includes:
[0021] According to the following formula, the shift number is obtained:
[0022] S = (|ab| × (w / 2));
[0023] Among them, S represents the shift number, a represents the non-zero weight value level, b represents the activation value level, and w represents the first fixed bit width.
[0024] Optionally, the compressed sparse weights include a value, a channel difference, and a position difference corresponding to each non-zero weight;
[0025] The channel difference is used to represent the index difference between input channels with non-zero weights;
[0026] The position difference is used to represent the relative position of the non-zero weights within an input channel.
[0027] Optionally, when the weight cache module decodes the compressed sparse weight, it is used to perform the following steps:
[0028] Obtaining a value, a channel difference, a position difference, and a compressed address corresponding to a target non-zero weight, wherein the target non-zero weight is any one of the non-zero weights;
[0029] Obtaining a one-dimensional coordinate value of the target non-zero weight, wherein the one-dimensional coordinate value of the target non-zero weight is a cumulative result of a position difference corresponding to the target non-zero weight and a position difference corresponding to a first non-zero weight, wherein the first non-zero weight includes all non-zero weights whose addresses after compression are smaller than the target non-zero weight;
[0030] According to the one-dimensional coordinate value and the convolution kernel size, obtaining the two-dimensional coordinate value of the target non-zero weight;
[0031] Obtain a channel cumulative value, where the channel cumulative value is the cumulative result of the channel difference corresponding to the target non-zero weight and the channel difference corresponding to the first non-zero weight;
[0032] According to the two-dimensional coordinate value and the channel accumulated value, obtaining the three-dimensional coordinate value of the target with non-zero weight;
[0033] The target non-zero weight position information is obtained, where the target non-zero weight position information includes the two-dimensional coordinate value and the three-dimensional coordinate value.
[0034] The second aspect of the present application discloses a sparse neural network processing method based on mixed-level precision calculation, which is applied to the sparse neural network processor based on mixed-level precision calculation as described in the first aspect of the present application, and the method comprises:
[0035] The weight cache module obtains the compressed sparse weights, and stores the compressed sparse weights in rows, wherein the data width stored in each row is the first fixed bit width;
[0036] The activation value cache module stores activation values of the neural network in rows, wherein the data width stored in each row is the first fixed bit width;
[0037] During the convolution operation:
[0038] The weight cache module decodes the compressed sparse weights, outputs the decoded non-zero weight values to the multiplication-accumulation operation unit in row order, and outputs the decoded non-zero weight position information to the activation index module in row order;
[0039] The activation value cache module reads the activation value and outputs it to the activation index module in row order;
[0040] The activation value index module selects the activation value of each received row according to the non-zero weight position information of each received row, and outputs the selected activation value to the multiplication and accumulation operation unit in sequence by row;
[0041] The multiplication-accumulation operation unit performs a pairwise multiplication-accumulation operation on each row of activation values and each row of non-zero weight values received to obtain a plurality of multiplication-accumulation operation results; and, determines whether to perform a shift operation on each of the multiplication-accumulation operation results, and if the determination result is no, directly outputs the multiplication-accumulation operation result to the register, and if the determination result is yes, performs a shift on the multiplication-accumulation operation result and then outputs it to the register;
[0042] The register temporarily stores the multiple multiplication-accumulation operation results, and outputs the multiple multiplication-accumulation results to the output buffer module after the final convolution operation is completed.
[0043] The third aspect of the present application discloses a computer device, comprising:
[0044] Memory for storing computer programs;
[0045] A processor is used to implement the steps of the sparse neural network processing method based on mixed-level precision calculation as described in the second aspect of the present application when executing the computer program.
[0046] The fourth aspect of the present application discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is processed and executed, the steps of the sparse neural network processing method based on mixed-level precision operations as described in the second aspect of the present application are implemented.
[0047] The present application provides a sparse neural network processor based on mixed-level precision operations, including: a weight cache module, an activation value cache module, an activation value index module, a multiplication-accumulation operation module and an output cache module. The multiplication-accumulation operation module includes a multiplication-accumulation operation unit and a register. The precision of the multiplication-accumulation operation unit is a preset first fixed bit width. The weight cache module stores compressed sparse weights. During each convolution calculation, the weight cache module first decodes and then outputs non-zero weight values and non-zero weight position information. The activation value index module outputs the corresponding activation value according to the non-zero weight position information. The multiplication-accumulation unit adopts a fixed-precision operation mode and combines registers to realize the processing of variable-precision data. The sparse neural network processor provided by the present application can not only load and operate non-zero weight data under high parallelism, but also support mixed-level precision calculations. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0049] Figure 1 A schematic diagram of the structure of a sparse neural network processor based on mixed-level precision calculation disclosed in an embodiment of the present application;
[0050] Figure 2 A schematic diagram of the structure of another sparse neural network processor based on mixed-level precision calculation disclosed in an embodiment of the present application;
[0051] Figure 3 This is a schematic diagram of the sparse weight compression result disclosed in the embodiment of this application;
[0052] Figure 4 A schematic diagram of three coordinates with non-zero weights disclosed in an embodiment of the present application;
[0053] Figure 5 A schematic diagram of selecting an activation value for an activation value index module disclosed in an embodiment of the present application;
[0054] Figure 6 A schematic diagram of data storage of a weight cache module and an activation value cache module disclosed in an embodiment of the present application;
[0055] Figure 7This is a schematic diagram of the weight cache module and the activation value cache module disclosed in the embodiment of the present application for 8-bit data storage when the first fixed bit width is 4 bits. DETAILED DESCRIPTION
[0056] In order to solve the technical problems that the neural network processor can process models with a single precision, does not support mixed-level precision calculations, and is difficult to simultaneously calculate only non-zero weights, the present application discloses a sparse neural network processor and method based on mixed-level precision operations through the following embodiments.
[0057] See also Figure 1 and Figure 2 The first embodiment of the present application discloses a sparse neural network processor based on mixed-level precision operation, including: a weight cache module, an activation value cache module, an activation value index module, a multiplication-accumulation operation module and an output cache module, wherein the multiplication-accumulation operation module includes a multiplication-accumulation operation unit and a register. The precision of the multiplication-accumulation operation unit is a preset first fixed bit width.
[0058] The weight cache module is used to obtain compressed sparse weights and store the compressed sparse weights in rows, wherein the data width stored in each row is the first fixed bit width.
[0059] The weight cache module is also used to decode the compressed sparse weights during the convolution operation, output the decoded non-zero weight values row by row to the multiplication and accumulation operation unit, and output the decoded non-zero weight position information row by row to the activation index module.
[0060] In one implementation, the sparse neural network processor based on mixed-level precision operations provided in this embodiment also includes a weight compression module, which is used to pre-encode and compress the sparse weight data, and output the compressed sparse weights to the weight cache module. The compressed sparse weights include the value, channel difference and position difference corresponding to each non-zero weight.
[0061] The compression result records the relative position of non-zero weights through "channel difference" and "position difference". Among them, the channel difference is used to represent the index difference between the input channels of non-zero weights. If the channel difference = 1, it means that the position stores the first non-zero weight in a certain input channel, and the channel difference = 0 means that the position stores the non-zero weight in the same input channel. The position difference is used to represent the relative position of the non-zero weight in an input channel. By accumulating the value of the position difference, the one-dimensional coordinate of the non-zero weight can be obtained. In addition, according to the order in which each non-zero weight is located in the convolution kernel before compression, each non-zero weight obtained after compression corresponds to an address. As an example, see Figure 2, for the weight data of the two-channel convolution kernel, the encoding obtains the compression result shown in the table on the right.
[0062] It should be noted that see Figure 4 , non-zero weights include three types of coordinates: one-dimensional coordinate (x), two-dimensional coordinate (x, y) and three-dimensional coordinate (x, y, ch), where ch represents the channel. In order to simplify the decoding circuit, the starting values of "x" and "y" are both 0, and the starting value of "ch" is 1.
[0063] In one implementation, when the weight cache module decodes the compressed sparse weight, it is used to perform the following steps:
[0064] Step 11, obtaining the value, channel difference, position difference and compressed address corresponding to the target non-zero weight, wherein the target non-zero weight is any one of the non-zero weights.
[0065] Step 12, obtain the one-dimensional coordinate value of the target non-zero weight, the one-dimensional coordinate value of the target non-zero weight is the cumulative result of the position difference corresponding to the target non-zero weight and the position difference corresponding to the first non-zero weight, and the first non-zero weight includes all non-zero weights whose compressed addresses are smaller than the target non-zero weight.
[0066] Step 13, obtaining the two-dimensional coordinate value of the target non-zero weight according to the one-dimensional coordinate value and the convolution kernel size.
[0067] In actual operation, the convolution kernel size (length * width) of the commonly used neural network model is between 1*1 and 7*7, so the convolution kernel size supported by this embodiment is also 1*1 to 7*7.
[0068] Assume that the one-dimensional coordinate value is 1D_loc, the width of the convolution kernel is NKX, and a parameter is preset as j. The value range of j is: 0 ≤j<7 (set according to the convolution kernel size).
[0069] When calculating the two-dimensional coordinate value of the target non-zero weight, the one-dimensional coordinate value 1D_loc is compared with (j*NKX). When j*NKX≤1D_loc<(j+1)*NKX is satisfied, the calculation is performed using the following formula:
[0070] x=1D_loc-j*NKX;
[0071] y = j;
[0072] By calculating the above formula, the two-dimensional coordinate value (x, y) of the target non-zero weight can be obtained.
[0073] Step 14, obtaining a channel cumulative value, wherein the channel cumulative value is the cumulative result of the channel difference corresponding to the target non-zero weight and the channel difference corresponding to the first non-zero weight. Specifically, the channel cumulative value is the value of ch.
[0074] Step 15, obtaining the three-dimensional coordinate value (x, y, ch) of the target with non-zero weight according to the two-dimensional coordinate value and the channel accumulated value.
[0075] Step 16, obtaining the target non-zero weight position information, wherein the target non-zero weight position information includes the two-dimensional coordinate value (x, y) and the three-dimensional coordinate value (x, y, ch).
[0076] The following combination Figure 3 The example of , details how the weight cache module decodes the compressed sparse weights.
[0077] by Figure 3 The bold non-zero weight "14" is used as an example for explanation. The value of the non-zero weight is "14", the address is "1", the position difference is "1", the channel difference is "0", and the convolution kernel size NKX=3.
[0078] First, the position differences are accumulated to obtain the one-dimensional coordinate value 1D_loc of the non-zero weight: 1D_loc=2+1=3.
[0079] Secondly, when 1*NKX≤3<(1+1)*NKX is satisfied, j=1 is calculated, and the values of x and y are calculated, x=1D_loc-j*NKX=3-1*3=0, y=j=1, and the two-dimensional coordinate value (x,y)=(0,1) is obtained.
[0080] Next, the channel differences are accumulated to obtain ch=1+0=1.
[0081] Finally, the three-dimensional coordinate value (x, y, ch) = (0, 1, 1) is determined.
[0082] The activation value cache module is used to store the activation values of the neural network in rows, wherein the data width stored in each row is the first fixed bit width.
[0083] The activation value cache module is also used to read the activation value during the convolution operation and output it to the activation index module in sequence by row.
[0084] The activation value index module is used to select the activation value of each received row according to the non-zero weight position information of each received row, and output the selected activation value to the multiplication and accumulation operation unit in sequence by row.
[0085] Next, we will explain how to select the corresponding activation value based on the non-zero weight position information.
[0086] See also Figure 5 In (a), when the two-dimensional coordinate value of a non-zero weight is obtained by decoding, the activation value associated with the non-zero weight during the convolution operation is determined accordingly ( Figure 5 The diagonal shaded part corresponding to (a) is the activation value associated with the non-zero weight).
[0087] See also Figure 5 In (b), after the activation value associated with the non-zero weight is read out, the value of the non-zero weight and the activation value can be calculated. In the figure, "MAC" is a multiplication and accumulation unit. The third coordinate "ch" of the three-dimensional coordinate value can enable the non-zero weight to further select the activation value associated with the non-zero weight in the input channel dimension.
[0088] See also Figure 5 In (c), assuming that the convolution kernel has 4 channels, the 3D coordinate "3" in the figure indicates that the non-zero weight is in the third channel, based on which the activation value of the third channel can be selected.
[0089] This embodiment utilizes the sparsity of weights to continuously store the activation values of the neural network in an activation value cache module, and sequentially reads them according to the convolution transport mode during the calculation process; the sparse weights are stored in the weight cache module after being encoded using the encoding method (the weight cache module also stores the value and position information of non-zero weights), and the sparse weights are decoded in real time during the calculation process to obtain the numerical value and position information of the non-zero weights, and the corresponding activation value is read according to the decoded position information, and then the read activation value and the decoded weight value are jointly output to the multiplication and addition operation unit.
[0090] The multiplication-accumulation operation unit is used to perform pairwise multiplication-accumulation operations on each row of activation values and each row of non-zero weight values received to obtain multiple multiplication-accumulation operation results.
[0091] The multiplication-accumulation operation unit is also used to determine whether to perform a shift operation on each of the multiplication-accumulation operation results, and if the judgment result is no, directly output the multiplication-accumulation operation results to the register; if the judgment result is yes, shift the multiplication-accumulation operation results before outputting them to the register.
[0092] Assume that the fixed bit width (i.e., the first fixed bit width) of the multiplication and accumulation unit is w bits. When the accuracy of a certain layer of the network is w bits, this layer is calculated according to the normal storage mode and calculation mode; if the accuracy of a certain layer of the network is nw bits (n is a positive integer, n>1), at this time, the processor splits the activation value and the compressed sparse weight into n w bit data rows and stores them in the activation value cache module and the weight cache module.
[0093] like Figure 6 As shown in the figure, each row in the activation value cache module and the weight cache module only stores w bits of data. During the convolution calculation process, the n w bits of data are read row by row, and then the w bits of activation data and w bits of weight data obtained each time are sent to the multiplication and accumulation unit for two-by-two multiplication and accumulation calculations. Then, the result of each w bit operation is temporarily stored in the register through the shift operation, and finally the result of the nw bit operation is stored in the output cache.
[0094] The shift operation is triggered only when the network level precision is nw bit (n>1). Before the multiplication and accumulation operation result is output to the register, the multiplication and accumulation operation unit will determine whether a shift operation needs to be performed, which is specifically performed through the following steps:
[0095] Step 21, obtaining the non-zero weight value series corresponding to the target multiplication-accumulation operation result, wherein the target multiplication-accumulation operation result is any of the multiplication-accumulation operation results, and the non-zero weight value series is the row sequence corresponding to the non-zero weight value.
[0096] Step 22, obtaining the activation value level corresponding to the target multiplication-accumulation operation result, wherein the activation value level is the row sequence corresponding to the activation value.
[0097] In one operation mode, when nw bit data is stored in a cache module (including a weight cache module and an activation value cache module) in rows, w bits are stored in each row, the lowest w bit level of the data is set to 1, the second lowest w bit level is set to 2, and so on, then the nw bit data has a total of n levels of w bit data.
[0098] Step 23, compare the non-zero weight value series with the activation value series. If the non-zero weight value series is consistent with the activation value series, execute step 14; if the non-zero weight value series is inconsistent with the activation value series, execute step 15.
[0099] Step 24, if the judgment result is no, then the target multiplication-accumulation operation result is directly output to the register.
[0100] Step 25, if the judgment result is yes, the target multiplication-accumulation operation result is shifted and then output to the register.
[0101] Furthermore, when the multiplication-accumulation-addition operation unit performs a shift on the target multiplication-accumulation-addition operation result and then outputs it to the register, it is used to perform the following steps:
[0102] A shift number is obtained according to the number of non-zero weight value levels, the number of activation value levels and the first fixed bit width.
[0103] Based on the shift number, the target multiplication-accumulation operation result is left-shifted and then output to the register.
[0104] The step of obtaining the shift number according to the number of non-zero weight value levels, the number of activation value levels and the first fixed bit width comprises:
[0105] According to the following formula, the shift number is obtained:
[0106] S = (|ab| × (w / 2)).
[0107] Among them, S represents the number of shifts, a represents the number of non-zero weight values, b represents the number of activation values, and w represents the first fixed bit width.
[0108] In actual operation, suppose that in a certain multiplication and accumulation operation, a is the non-zero weight value series and b is the activation value series. When a=b, no shift operation is required; when a≠b, after multiplying the w bit non-zero weight and the activation value, the result is shifted left by (|ab| ×(w / 2)) bits, and then the shifted result is temporarily stored in the register. After n*n operations, that is, after all n w bits of weight and activation are calculated, the result of the register will be output to the output buffer.
[0109] For example, the first fixed bit width is 4 bits, that is, the calculation precision of the multiplication and accumulation unit is 4 bits. Then, by changing the calculation mode, it can support processing of three levels of precision, namely 4 bits, 8 bits and 16 bits. For 8-bit and 16-bit data, it is split into 2 and 3 4-bit data and stored in the cache module. During the calculation process, the data is read from the cache line by line, so that the purpose of using one 4-bit to process three precisions is achieved.
[0110] For example, for 8-bit data, Figure 7 As shown in the figure, the low and high 4 bits of the activation value and weight are stored in two rows and read twice. The multiplication and accumulation unit first performs multiplication calculation on the low 4 bits of the activation value and weight, and then outputs it to the register. Then, the high 4 bits and low 4 bits of the activation value and weight are multiplied and shifted left by 2 bits and stored in the register (after adding to the existing product and then storing it). Finally, the high 4 bits of the activation value and weight are multiplied and shifted left by 4 bits and stored in the register (after adding to the existing product and then storing it).
[0111] The register is used to temporarily store the multiple multiplication-accumulation operation results, and output the multiple multiplication-accumulation results to the output buffer module after the final convolution operation is completed.
[0112] The present application provides a sparse neural network processor based on mixed-level precision operations, including: a weight cache module, an activation value cache module, an activation value index module, a multiplication-accumulation operation module and an output cache module. The multiplication-accumulation operation module includes a multiplication-accumulation operation unit and a register. The precision of the multiplication-accumulation operation unit is a preset first fixed bit width. The weight cache module stores compressed sparse weights. During each convolution calculation, the weight cache module first decodes and then outputs non-zero weight values and non-zero weight position information. The activation value index module outputs the corresponding activation value according to the non-zero weight position information. The multiplication-accumulation unit adopts a fixed-precision operation mode and combines registers to realize the processing of variable-precision data. The sparse neural network processor provided by the present application can not only load and operate non-zero weight data under high parallelism, but also support mixed-level precision calculations.
[0113] The sparse neural network processor provided in this application includes a hardware-friendly sparse weight encoding and decoding method, which fully utilizes the sparsity of the network by skipping the calculation of randomly distributed zero weights, thereby achieving the effect of reducing the amount of calculation and increasing the throughput.
[0114] At the same time, this sparse neural network processor supports the processing of multi-precision data by changing the storage mode and calling the variable precision processing unit (register), so as to achieve the purpose of using a fixed bit width operation unit to process multiple levels of precision, greatly improving the utilization efficiency of the operation unit.
[0115] The second embodiment of the present application discloses a sparse neural network processing method based on mixed-level precision calculation, which is applied to the sparse neural network processor based on mixed-level precision calculation as described in the first embodiment of the present application, and the method includes:
[0116] The weight cache module obtains the compressed sparse weights and stores the compressed sparse weights in rows, wherein the data width stored in each row is a first fixed bit width.
[0117] The activation value cache module stores the activation values of the neural network in rows, wherein the data width stored in each row is the first fixed bit width.
[0118] Assuming that the first fixed bit width is w bits, if the accuracy of a certain layer of the network is w bits, this layer is calculated according to the normal storage mode and calculation mode; if the accuracy of a certain layer of the network is nw bits (n is a positive integer, n>1), at this time, the processor splits the activation value and the compressed sparse weight into n w bit data branches and stores them in the activation value cache module and the weight cache module.
[0119] During the convolution operation:
[0120] The weight cache module decodes the compressed sparse weights, outputs the decoded non-zero weight values to the multiplication and accumulation operation unit in row sequence, and outputs the decoded non-zero weight position information to the activation index module in row sequence.
[0121] The activation value cache module reads the activation value and outputs it to the activation index module in row order.
[0122] The activation value index module selects the activation value of each received row according to the non-zero weight position information of each received row, and outputs the selected activation value to the multiplication and accumulation operation unit in sequence by row.
[0123] The multiplication-accumulation operation unit performs a multiplication-accumulation operation on each row of activation values and each row of non-zero weight values received, and obtains a plurality of multiplication-accumulation operation results. And, it is determined whether to perform a shift operation on each of the multiplication-accumulation operation results, and if the result of the determination is no, the multiplication-accumulation operation result is directly output to the register, and if the result of the determination is yes, the multiplication-accumulation operation result is shifted and then output to the register.
[0124] The register temporarily stores the multiple multiplication-accumulation operation results, and outputs the multiple multiplication-accumulation results to the output buffer module after the final convolution operation is completed.
[0125] The present application provides a sparse neural network processing method based on mixed-level precision operations. During each convolution calculation, the weight cache module first performs decoding, and then outputs the non-zero weight value and non-zero weight position information. The activation value index module outputs the corresponding activation value according to the non-zero weight position information. The multiplication and accumulation unit adopts a fixed-precision operation mode, changes the storage mode of the weight cache module and the activation value cache module, and combines registers to realize the processing of variable-precision data. The sparse neural network processing method provided by the present application can load and operate non-zero weight data under high parallelism, and support mixed-level precision calculations.
[0126] The third embodiment of the present application discloses a computer device, including:
[0127] Memory, used to store computer programs.
[0128] A processor is used to implement the steps of the sparse neural network processing method based on mixed-level precision calculation as described in the second embodiment of the present application when executing the computer program.
[0129] The fourth embodiment of the present application discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is processed and executed, the steps of the sparse neural network processing method based on mixed-level precision operations as described in the second embodiment of the present application are implemented.
[0130] The present application is described in detail above in conjunction with specific implementation methods and exemplary examples, but these descriptions cannot be understood as limiting the present application. Those skilled in the art understand that, without departing from the spirit and scope of the present application, a variety of equivalent replacements, modifications or improvements can be made to the technical solution of the present application and its implementation methods, all of which fall within the scope of the present application. The scope of protection of the present application shall be subject to the attached claims.
Claims
1. A sparse neural network processor based on mixed-level precision computing, characterized in that: include: A weight cache module, an activation value cache module, an activation value index module, a multiplication-accumulation operation module and an output cache module, wherein the multiplication-accumulation operation module includes a multiplication-accumulation operation unit and a register, and the precision of the multiplication-accumulation operation unit is a preset first fixed bit width; The weight cache module is used to obtain compressed sparse weights and store the compressed sparse weights in rows, wherein the data width stored in each row is the first fixed bit width; It is also used to decode the compressed sparse weights during the convolution operation, output the decoded non-zero weight values to the multiplication and accumulation operation unit in row order, and output the decoded non-zero weight position information to the activation value index module in row order; The activation value cache module is used to store the activation values of the neural network in rows, wherein the data width stored in each row is the first fixed bit width; and is also used to read the activation values during the convolution operation and output them to the activation value index module in sequence by row; The activation value index module is used to select the activation value of each received row according to the non-zero weight position information of each received row, and output the selected activation value to the multiplication and accumulation operation unit in sequence by row; The multiplication-accumulation operation unit is used to perform a pairwise multiplication-accumulation operation on each row of activation values and each row of non-zero weight values received to obtain a plurality of multiplication-accumulation operation results; It is also used to determine whether to perform a shift operation on each of the multiplication-accumulation-addition operation results, and if the determination result is no, directly output the multiplication-accumulation-addition operation results to the register; if the determination result is yes, shift the multiplication-accumulation-addition operation results before outputting them to the register; The register is used to temporarily store the multiple multiplication-accumulation operation results, and output the multiple multiplication-accumulation operation results to the output buffer module after the final convolution operation is completed.
2. The sparse neural network processor based on mixed-level precision computing according to claim 1, characterized in that: The multiplication-accumulation-addition operation unit is used to perform the following steps in the process of determining whether to perform a shift operation on each of the multiplication-accumulation-addition operation results: Obtaining a non-zero weight value series corresponding to a target multiplication-accumulation operation result, wherein the target multiplication-accumulation operation result is any of the multiplication-accumulation operation results, and the non-zero weight value series is a row sequence corresponding to the non-zero weight value; Obtaining an activation value series corresponding to a target multiplication-accumulation operation result, wherein the activation value series is a row sequence corresponding to the activation value; comparing the non-zero weight value series with the activation value series; If the non-zero weight value level is consistent with the activation value level, obtaining the judgment result is no; If the non-zero weight value level is inconsistent with the activation value level, the judgment result is obtained as yes.
3. The sparse neural network processor based on mixed-level precision computing according to claim 2, characterized in that: When the multiplication-accumulation-addition operation unit performs a shift on the multiplication-accumulation-addition operation result and then outputs it to the register, the multiplication-accumulation-addition operation unit is used to perform the following steps: Obtaining a shift number according to the non-zero weight value level, the activation value level and the first fixed bit width; Based on the shift number, the target multiplication-accumulation operation result is left-shifted and then output to the register.
4. The sparse neural network processor based on mixed-level precision computing according to claim 3, characterized in that: The acquiring the shift number according to the non-zero weight value level, the activation value level and the first fixed bit width comprises: According to the following formula, the shift number is obtained: S = (|ab| × (w / 2)); Among them, S represents the number of shifts, a represents the number of non-zero weight values, b represents the number of activation values, and w represents the first fixed bit width.
5. The sparse neural network processor based on mixed-level precision computing according to claim 1, characterized in that: The compressed sparse weights include the value, channel difference and position difference corresponding to each non-zero weight; The channel difference is used to represent the index difference between input channels with non-zero weights; The position difference is used to represent the relative position of the non-zero weights within an input channel.
6. The sparse neural network processor based on mixed-level precision computing according to claim 5, characterized in that: When the weight cache module decodes the compressed sparse weight, it is used to perform the following steps: Obtaining a value, a channel difference, a position difference, and a compressed address corresponding to a target non-zero weight, wherein the target non-zero weight is any one of the non-zero weights; Obtaining a one-dimensional coordinate value of the target non-zero weight, wherein the one-dimensional coordinate value of the target non-zero weight is a cumulative result of a position difference corresponding to the target non-zero weight and a position difference corresponding to a first non-zero weight, wherein the first non-zero weight includes all non-zero weights whose addresses after compression are smaller than the target non-zero weight; According to the one-dimensional coordinate value and the convolution kernel size, obtaining the two-dimensional coordinate value of the target non-zero weight; Obtain a channel cumulative value, where the channel cumulative value is a cumulative result of a channel difference corresponding to the target non-zero weight and a channel difference corresponding to the first non-zero weight; According to the two-dimensional coordinate value and the channel accumulated value, obtaining the three-dimensional coordinate value of the target with non-zero weight; The target non-zero weight position information is obtained, where the target non-zero weight position information includes the two-dimensional coordinate value and the three-dimensional coordinate value.
7. A sparse neural network processing method based on mixed-level precision calculation, characterized in that: Applied to a sparse neural network processor based on mixed-level precision computing as claimed in any one of claims 1 to 6, the method comprising: The weight cache module obtains the compressed sparse weights, and stores the compressed sparse weights in rows, wherein the data width stored in each row is the first fixed bit width; The activation value cache module stores activation values of the neural network in rows, wherein the data width stored in each row is the first fixed bit width; During the convolution operation: The weight cache module decodes the compressed sparse weights, outputs the decoded non-zero weight values to the multiplication-accumulation operation unit in row order, and outputs the decoded non-zero weight position information to the activation index module in row order; The activation value cache module reads the activation value and outputs it to the activation index module in row order; The activation value index module selects the activation value of each received row according to the non-zero weight position information of each received row, and outputs the selected activation value to the multiplication and accumulation operation unit in sequence by row; The multiplication-accumulation operation unit performs a multiplication-accumulation operation on each row of activation values and each row of non-zero weight values received, and obtains a plurality of multiplication-accumulation operation results; and determines whether to perform a shift operation on each of the multiplication-accumulation operation results, and if the determination result is no, directly outputs the multiplication-accumulation operation result to the register, and if the determination result is yes, performs a shift on the multiplication-accumulation operation result and then outputs it to the register; The register temporarily stores the multiple multiplication-accumulation operation results, and outputs the multiple multiplication-accumulation operation results to the output buffer module after the final convolution operation is completed.
8. A computer device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the sparse neural network processing method based on mixed-level precision calculation as described in claim 7 when executing the computer program.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is processed and executed, the steps of the sparse neural network processing method based on mixed-level precision calculations as described in claim 7 are implemented.
Citation Information
Patent Citations
Low-power architecture for sparse neural network
CN110073370A
Data analysis processing method and device
CN111553471A
