A neural network weight encoding method and hardware system based on matrix sharing

Abstract

The present invention discloses a neural network weight encoding method and hardware system based on a shared matrix, and belongs to the technical field of neural network algorithm hardware implementation. The method aims to solve the problems of the current weight encoding method, such as the large number of storage and computing devices required, the large size of the storage and computing array, and the high overhead. The method uses a shared matrix to realize the encoding of the weight parameters in the convolution kernel of the neural network in pairs. Compared with the traditional method of encoding each weight parameter independently, the number of storage and computing devices in the storage and computing array is reduced. Under the premise of ensuring the accuracy of the neural network, the efficiency of the use of storage and computing devices is improved, and the network overhead is reduced. At the same time, the encoding method of the present invention has good compatibility and is not limited by the type of non-volatile storage and computing devices. Compared with the existing encoding methods, the encoding method based on a shared matrix is ​​more suitable for large-scale and ultra-large-scale storage and computing architectures.

Description

Technical Field

[0001] The invention relates to a neural network weight encoding method and a hardware system based on matrix sharing, belonging to the technical field of neural network algorithm hardware implementation. Background Art

[0002] With the development of deep learning, neural networks have been increasingly widely used in many fields such as image recognition, speech recognition, and natural language processing. However, as the network architecture becomes more and more complex, the amount of data transmission and computation in neural networks has increased dramatically. In addition, computing and data transmission consume a lot of power, making it difficult to deploy neural network applications on hardware devices.

[0003] In recent years, the integrated storage and computing architecture for neural network computing has received widespread attention and research. The basic idea is to store some simple but data-intensive logical computing functions in the memory by storing weight mapping in the storage and computing array, so as to reduce the amount of data transmitted and the transmission distance between the memory and the processor.

[0004] The operating principle of the classic non-volatile storage array is as follows Figure 1 To calculate the two multiplication matrices The values ​​of the weight matrix W are stored in the storage array in the form of conductance, and then the matrix The value of is input into the input end of the storage and computing array in the form of voltage, and the calculation result in the form of current is obtained from the output end of the storage and computing array, completing the whole process of storage and computing integration.

[0005] The weight parameters in a neural network are usually divided into positive and negative. In order to express the weight parameters in the storage array, positive and negative matrices are often used. The encoded values ​​are stored in the positive and negative matrices respectively, and then the output results of the two matrices are subtracted, and the difference between the matrices is used to represent the value of the weight parameter. Figure 2 As shown in the figure, the current encoding method processes each weight parameter independently and maps it to positive and negative matrices, which means that at least two storage and computing devices are required to represent each weight. In some application scenarios that require high precision, multiple positive matrix devices and multiple negative matrix devices are used to represent one weight, and the average device overhead for each weight parameter is even higher.

[0006] Today, simple neural networks are unable to meet task requirements, so the structure of neural networks has become more complex and the scale has continued to expand. As a result, the application scenarios of the storage-computing integrated architecture have become more extensive and complex. The realization of large-scale and ultra-large-scale neural networks in the hardware field is the only way for the development of the storage-computing integrated architecture. With the increase in the depth and parameters of the neural network, the number of devices and array size required in the hardware implementation process will inevitably become larger and larger. The large array size brings a series of problems including cost, chip area, parasitic parameters, and testability. The difficulty of hardware implementation increases exponentially with the increase in the number of devices. Summary of the invention

[0007] In view of the problems that the existing neural network weight encoding methods require a large number of storage and computing devices, have a large storage and computing array size, and have high overhead when implemented in neural network hardware, the present invention provides a neural network weight parameter encoding method and hardware system based on matrix sharing.

[0008] The first object of the present invention is to provide a neural network weight parameter encoding method based on matrix sharing, characterized in that the method comprises:

[0009] Step 1: Fixed-point processing of the weight parameters in the trained neural network convolution kernel;

[0010] Step 2: Group the convolution kernels in the neural network. When the number of convolution kernels is even, group all the convolution kernels in pairs. When the number of convolution kernels is odd, take out any convolution kernel and group the remaining convolution kernels in pairs.

[0011] Step 3: Calculate the actual encoding value, including:

[0012] The weight parameters of any convolution kernel taken out in step 2 are independently encoded. After encoding, each weight parameter outputs two actual encoding values ​​and maps them to the positive matrix and the negative matrix.

[0013] For the convolution kernels grouped in pairs in step 2: two weight parameters at the same position in the convolution kernels of the same group are encoded in pairs, and three actual encoding values ​​are output after encoding every two weight parameters, and mapped and stored in three matrices;

[0014] Step 4: Concatenate the matrices storing the actual coding values ​​to obtain the encoded neural network weight matrix.

[0015] Optional, use Represents two weight parameters at the same position of the same set of convolution kernels. The encoding is performed in pairs, and the encoding results are mapped and stored in three matrices, wherein the three matrices are Positive matrix, Positive matrix and shared negative matrix, the pairwise encoding process includes:

[0016] S1: Input weight parameters

[0017] S2: Judgment

[0018] when season

[0019] when season

[0020] S3:

[0021] S4: Judgment

[0022] when hour, Keep the original value unchanged;

[0023] when season

[0024] S5: Output data

[0025] in, is the weight parameter The actual encoding value after encoding;

[0026] Weight Parameters Two coded values and Jointly determine the weight parameters Two coded values and Joint decision, where is a shared negative coded value;

[0027] Mapping storage to Positive matrix, Mapping storage to Positive matrix, The mapping is stored in the shared negative matrix.

[0028] Optional, use Represents two weight parameters at the same position of the same set of convolution kernels. The encoding is performed in pairs, and the encoding results are mapped and stored in three matrices, wherein the three matrices are Negative matrix, Negative matrix and shared positive matrix, the pairwise encoding process includes:

[0029] S1: Input weight parameters

[0030] S2: Judgment

[0031] when make

[0032] when make

[0033] S3:

[0034] S4: Judgment

[0035] when hour, Keep the original value unchanged;

[0036] when hour,

[0037] S5: Output data

[0038] in, is the weight parameter The actual encoding value after encoding;

[0039] Weight Parameters Two coded values and Jointly determine the weight parameters Two coded values and Joint decision, where is a shared positive coding value;

[0040] Mapping storage to Negative matrix, Mapping storage to Negative matrix, The mapping is stored into a shared positive matrix.

[0041] The second object of the present invention is to provide a neural network chip, comprising a storage and computing array composed of multiple storage and computing devices, each weight value in the neural network weight matrix is ​​written one by one into the corresponding storage and computing device, characterized in that the neural network weight matrix is ​​obtained using the above-mentioned neural network weight parameter encoding method.

[0042] Optionally, the storage and computing device is a non-volatile storage and computing device.

[0043] Optionally, the precision of the storage and computing device is 8 bits.

[0044] Optionally, the precision of the storage and computing device is 4 bits.

[0045] The third object of the present invention is to provide a computing device, including a memory and a processor, wherein the memory stores computer executable instructions, and is characterized in that when the instructions are executed by the processor, the above-mentioned neural network weight parameter encoding method is executed.

[0046] The fourth object of the present invention is to provide a neural network hardware system, characterized in that the neural network system includes: the neural network chip and / or computing device provided by the present invention.

[0047] The present invention also provides the application of the above-mentioned neural network weight parameter encoding method and / or the above-mentioned neural network chip and / or the above-mentioned computing device and / or the above-mentioned neural network hardware system in the field of neural network technology.

[0048] The beneficial effects of the present invention are:

[0049] When implementing the hardware of the neural network, the convolution kernels in the neural network are grouped in pairs, and the weight parameters at the same position of the convolution kernel are encoded in pairs by sharing the matrix. After the two weight parameters are encoded, only three storage and computing devices are needed to realize the storage and computing, which reduces the number of storage and computing devices required for the storage and computing array. At the same time, the encoding method of the present invention is not limited by the type of non-volatile storage and computing devices and has good compatibility. Compared with the existing method of independently encoding each weight parameter and storing the encoded value in two storage and computing devices, the present invention uses at least 1.5 storage and computing devices for each weight parameter on average, saving nearly 1 / 4 of the device overhead for the storage and computing array. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 Schematic diagram of the operating principle of the non-volatile storage and computing array.

[0052] Figure 2 Schematic diagram of the storage and computation array structure of the current encoding method.

[0053] Figure 3 It is a schematic diagram of the shared device structure of the present invention.

[0054] FIG. 4( a ) is a schematic diagram of the algorithm encoding process in the entire process of mapping from the algorithm to the storage and calculation array of the present invention.

[0055] FIG. 4( b ) is a schematic diagram of the process of writing the coded value into the storage and computing array during the entire process of mapping from the algorithm to the storage and computing array in the present invention.

[0056] Figure 5 The present invention is a flowchart of the encoding method of the present invention. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0058] The encoding method proposed in the present invention is oriented to the weight parameters in the neural network. In practical applications, the neural network is first trained to obtain the final trained neural network convolution kernel, and then the weight parameters in the convolution kernel are encoded using the encoding method provided by the present invention.

[0059] Embodiment 1:

[0060] This embodiment provides a neural network encoding method based on matrix sharing, which is applied to a neural network. The encoding process includes:

[0061] Step 1: The weight parameters in the trained neural network convolution kernel are fixed-point processed. The fixed-point method in this embodiment is a linear transformation method;

[0062] Table 1 is the convolution kernel weight parameter matrix after fixed-point processing in this embodiment;

[0063] Table 1 Weight matrix after fixed-point processing

[0064]

[0065] Step 2: Group the convolution kernels processed in step 1, rearrange them, and process the weight parameters in pairs;

[0066] Table 2 is the rearranged matrix after grouping;

[0067] Table 2 Rearrangement matrix

[0068] Convolution kernel 1 rearrangement 5 15 -7 -9 Convolution kernel 2 rearrangement 12 -3 -7 11

[0069] Step 3: Calculate the actual code value; This embodiment selects a shared negative matrix according to Figure 5 The encoding flow chart encodes the rearrangement matrix;

[0070] The weight parameters corresponding to the uplink and downlink are processed in pairs, and the two share a storage and computing device for initial encoding to obtain the initial encoding matrix in Table 3.

[0071] Table 3 Initial coding matrix

[0072] Convolution kernel 1 positive matrix 5 15 0 0 Shared Negative Matrix 0 0 7 9 Convolution kernel 2 positive matrix 12 -3 0 20

[0073] The matrix of the initial encoding is judged. Since the physical values ​​in the actual storage array have no distinction between positive and negative values, when there is a negative encoding value, an offset is introduced into the encoding of this group, and the encoding values ​​of the other groups remain unchanged, and the actual encoding matrix of Table 4 is obtained.

[0074] Table 4 Actual coding matrix

[0075] Convolution kernel 1 positive matrix 5 18 0 0 Shared Negative Matrix 0 3 7 9 Convolution kernel 2 positive matrix 12 0 0 20

[0076] Step 4: Concatenate the actual encoding matrices in Table 4 to obtain the encoded neural network weight matrix.

[0077] In order to verify the effectiveness and low device cost of the encoding method of the present invention, a series of experiments were conducted. During the experiment, the encoding method of the present invention was applied to different neural networks and verified using the MINIST handwritten digit data set to observe the accuracy of network inference after encoding and the device cost required for the storage and computing array in the hardware implementation.

[0078] In the first experiment, a small-scale neural network was designed, which includes 3 convolutional layers and 1 fully connected layer. In the first convolutional layer, the convolution kernel size is 5*5*1*7, and no padding is added; in the second convolutional layer, the convolution kernel size is 3*3*7*7, and padding is added; in the third convolutional layer, the convolution kernel size is 3*3*7*7, and padding is added; the fourth layer is a fully connected layer, with 63 input neurons and 10 output neurons.

[0079] Next, the encoding method of the present invention is applied to a neural network with deeper layers and larger scale. The new neural network has a total of 8 layers, including 5 convolutional layers and 3 fully connected layers. In the first convolutional layer, the convolution kernel size is 7*7*1*16, and no padding is added; in the second convolutional layer, the convolution kernel size is 5*5*16*32, and padding is added; in the third convolutional layer, the convolution kernel size is 3*3*32*128, and padding is added; in the fourth convolutional layer, the convolution kernel size is 3*3*128*128, and padding is added; in the fifth convolutional layer, the convolution kernel size is 3*3*128*64, and padding is added. The last three layers are fully connected layers for classification, and the number of neurons in each layer is 1024, 512 and 10 respectively.

[0080] The experimental results are summarized in Table 5. The application of the encoding method of the present invention in two neural networks can effectively verify the effectiveness of the encoding method. At the same time, the horizontal comparison of the two networks can show the proportional relationship between the optimization effect of the encoding method and the network scale.

[0081] Table 5 Summary of experimental results

[0082] Network Name Neural Network 1 Neural Network 2 Number of layers 5 8 Number of parameters 1,718 1,065,098 Accuracy 96.71％ 98.94％ Number of original storage and computing devices 3,436 2,130,196 Number of existing computing devices 2,639 1,595,328 Save money 23.2％ 25.0％

[0083] According to the experimental results given in Table 5, it can be seen that after applying the encoding method of the present invention, the target network still performs well in terms of inference accuracy, verifying the effectiveness of the encoding method. At the same time, by comparing the number of storage and computing devices required before and after the application of the encoding method of the present invention, it can be found that the encoding method has the characteristic of significantly reducing device overhead. Therefore, the encoding method of the present invention can solve the problem of needing to arrange a large number of circuit devices in the process of neural network hardware implementation while ensuring network accuracy, and effectively reduce development costs and design difficulties.

[0084] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A neural network weight parameter encoding method based on matrix sharing, characterized in that: The method comprises: Step 1: Fixed-point processing of the weight parameters in the trained neural network convolution kernel; Step 2: Group the convolution kernels in the neural network. When the number of convolution kernels is even, group all the convolution kernels in pairs. When the number of convolution kernels is odd, take out any convolution kernel and group the remaining convolution kernels in pairs. Step 3: Calculate the actual encoding value, including: The weight parameters of any convolution kernel taken out in step 2 are independently encoded. After encoding, each weight parameter outputs two actual encoding values ​​and maps them to the positive matrix and the negative matrix. For the convolution kernels grouped in pairs in step 2: two weight parameters at the same position in the convolution kernels of the same group are encoded in pairs, and three actual encoding values ​​are output after encoding every two weight parameters, and mapped and stored in three matrices; Step 4: Concatenate the matrices storing the actual coding values ​​to obtain the encoded neural network weight matrix; use , Represents two weight parameters at the same position of the same set of convolution kernels. , Perform pairwise encoding, and map the encoding results to three matrices; The three matrices are Positive matrix, Positive matrix and shared negative matrix, the pairwise encoding process includes: S1: Input weight parameters , ; S2: Judgment : when season , ; when season , ; S3： ； S4: Judgment : when hour, , , Keep the original value unchanged; when season , , ; S5: Output data , , ; in, , , is the weight parameter , The actual encoding value after encoding; Weight Parameters Two coded values and Jointly determine the weight parameters Two coded values and Joint decision, where is a shared negative coded value; Mapping stored to Positive matrix, Mapping stored to Positive matrix, The mapping is stored in the shared negative matrix; Or the three matrices are Negative matrix, Negative matrix and shared positive matrix, the pairwise encoding process includes: S1: Input weight parameters , ; S2: Judgment : when ,make , ; when ,make , ; S3： ； S4: Judgment : when hour, , , Keep the original value unchanged; when hour, , , ; S5: Output data , , ; in, , , is the weight parameter , The actual encoding value after encoding; Weight Parameters Two coded values and Jointly determine the weight parameters Two coded values and Joint decision, where is a shared positive coding value; Mapping stored to Negative matrix, Mapping stored to Negative matrix, The mapping is stored into a shared positive matrix.

2. A neural network chip, comprising a storage and computing array composed of a plurality of storage and computing devices, wherein each weight value in a neural network weight matrix is ​​written into a corresponding storage and computing device one by one, characterized in that: The neural network weight matrix is ​​obtained using the neural network weight parameter encoding method as described in claim 1.

3. The chip according to claim 2, characterized in that: The storage and computing device is a non-volatile storage and computing device.

4. The chip according to claim 3, characterized in that: The precision of the storage and computing device is 8 bits.

5. The chip according to claim 3, characterized in that: The precision of the storage and computing device is 4 bits.

6. A computing device comprising a memory and a processor, wherein the memory stores computer executable instructions, characterized in that: When the instruction is executed by the processor, the neural network weight parameter encoding method as described in claim 1 is executed.

7. A neural network hardware system, characterized in that: The neural network hardware system includes: a neural network chip as described in any one of claims 2-5 and / or a computing device as described in claim 6.

8. Application of the neural network weight parameter encoding method as described in claim 1 and / or the neural network chip as described in any one of claims 2-5 and / or the computing device as described in claim 6 and / or the neural network hardware system as described in claim 7 in the field of neural network technology.

Images