Network quantization and mapping system for memory-calculation integrated memristor chip

Abstract

The invention relates to a network quantization and mapping system for a memory-calculation integrated memristor chip. The network quantization and mapping system comprises a quantization module and a mapping module, the quantization module is used for carrying out weight quantization on the deep neural network and sending a quantized result to the mapping module; and the mapping module is used for deploying and mapping the quantized result to the memristor chip, and performing reasoning calculation on the deep convolutional neural network through the memristor chip. The memristor chip-oriented computing speed and deployment mapping efficiency of the neural network are improved by quantizing the weight and activation of the neural network into low bit width.

Description

Technical Field

[0001] The present invention relates to the field of software optimization methods for integrated storage and computing chips based on memristors, and in particular to a network quantization and mapping system for integrated storage and computing memristor chips. Background Art

[0002] Deep neural networks have achieved tremendous success in fields such as computer vision, natural language processing, and speech recognition. However, this often comes with expensive computational and storage requirements. Improving the computational efficiency and reducing storage requirements of deep neural networks has become a pressing issue. Traditionally, neural network weights and activations have been stored using 32-bit floating-point numbers, which incurs significant computational resource consumption. Quantization involves representing model parameters that originally used 64 or 32 bits using 8, 4, or even fewer bits, thereby reducing hardware storage requirements and computational complexity. While memristor chips struggle to process floating-point data, quantized networks offer greater efficiency. Summary of the Invention

[0003] The purpose of the present invention is to provide a network quantization and mapping system for a storage and computing integrated memristor chip to solve the problems existing in the above-mentioned prior art. The present invention improves the computing speed and deployment mapping efficiency of the neural network for the memristor chip by quantizing the weights and activations of the neural network to a low bit width.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A network quantization and mapping system for a memory-computing integrated memristor chip, comprising: a quantization module and a mapping module;

[0006] The quantization module is used to quantize the weights of the deep neural network and send the quantized results to the mapping module;

[0007] The mapping module is used to map the quantized results onto the memristor chip and perform inference calculations on the deep convolutional neural network through the memristor chip.

[0008] Optionally, the quantization module includes: a bit convolution kernel, a weighted low bit width quantization unit;

[0009] The bit convolution kernel is used to binarize the weights and input activations of the convolution layer and then perform convolution calculations using bit operations;

[0010] The weight low-bit-width quantization unit is used to construct a straight-through estimator, and adopt the straight-through estimator to binarize the weight to obtain the low-bit-width weight, that is, the quantized result.

[0011] Optionally, in the bit convolution kernel, using bit operations to complete convolution calculation includes:

[0012] Binarize the weights and input activations of the convolutional layers and deploy the network on a memristor chip.

[0013] When deployed in the memristor chip for forward transmission, the dot product of two bit vectors x and y is calculated by bitwise operation using a preset expression;

[0014] Wherein, the preset expression is:

[0015] x·y=bitcount(&(x,y)),x i ,y i ∈{0, 1}

[0016] Among them, bitcount is the vector dot product operation, & is the bitwise AND operation, x i is the i-th element of vector x, y i is the i-th element of vector y.

[0017] Optionally, the through estimator takes a real number input r i ∈[0, 1] is quantized into k-bit digital output r o ∈[0, 1];

[0018] The straight-through estimator is:

[0019]

[0020] Among them, Forward is the neural network forward reasoning label, r o is the quantized result, k is the number of quantized bits, round is the rounding algorithm, and backward is the neural network back propagation. For the reverse result, For r i Find the integral, For r o Find the integral, r i The real number passed into the estimator to quantize the neural network parameters.

[0021] Optionally, using the direct estimator to binarize the weights includes:

[0022] Forward:r o =sign(r i )×E F (|r i |)

[0023]

[0024] Among them, sign(r i ) is to find the sign function, E F (|r i |) is to find the expected function.

[0025] Optionally, the mapping module includes: a translation unit, a weight matrix splitting unit, and a mapping unit;

[0026] The translation unit is used to translate the quantized result so that the weight matrix becomes a positive number;

[0027] The weight matrix splitting unit is used to split and map the translated result onto different cross arrays;

[0028] The mapping unit is used to fill the mapped weight matrix and map the weight to the conductance of the memristor through a preset mapping formula.

[0029] Optionally, the weight matrix splitting unit splits and maps the translated result onto different cross arrays, including:

[0030] If the number of columns of the cross array is less than the number of columns of the weight matrix, the weight matrix is divided into multiple groups by column, and each group is mapped to a cross array;

[0031] If the number of rows in the crossbar array is smaller than the number of rows in the weight matrix, the weight matrix is divided into multiple groups by row, and each group is mapped to one crossbar array.

[0032] Optionally, the preset mapping formula is:

[0033]

[0034] G=G min +G s ×w

[0035] Among them, G s is the conductivity value after treatment, G Max is the maximum conductance value, G min is the minimum conductance value, is the memristor device parameter, and G is the final conductance value.

[0036] The beneficial effects of the present invention are:

[0037] This invention uses a quantization module to quantize the weights of a deep neural network, sending the quantized results to a mapping module. The mapping module then maps the quantized results onto a memristor chip, which then performs inference calculations on the deep convolutional neural network. By quantizing the weights and activations of the neural network to a low bit width, this invention improves the computational speed and deployment efficiency of the neural network on the memristor chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 Schematic diagram of splitting and mapping convolution kernels onto multiple cross arrays according to an embodiment of the present invention; (a) is column splitting and (b) is row splitting;

[0040] Figure 2 A schematic diagram of zero padding of the weight matrix according to an embodiment of the present invention;

[0041] Figure 3 This is a weight mapping flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] This embodiment proposes a network quantization and mapping system for a memory-computing integrated memristor chip, including: a quantization module and a mapping module;

[0045] The quantization module is used to quantize the weights of the deep neural network and send the quantized results to the mapping module;

[0046] The mapping module is used to deploy and map the quantized results onto the memristor chip, and perform inference calculations on the deep convolutional neural network through the memristor chip.

[0047] Specifically, the purpose of this embodiment is to provide a network quantization key module for a deep learning storage and computing integrated chip and a mapping deployment method for a memristor chip architecture, which mainly includes two parts: quantization and mapping.

[0048] Furthermore, the quantization module includes: a bit convolution kernel, a weighted low bit width quantization unit;

[0049] The bit convolution kernel is used to binarize the weights and input activations of the convolution layer and then perform convolution calculations using bit operations;

[0050] The weight low-bit-width quantization unit is used to construct a straight-through estimator, and the straight-through estimator is used to binarize the weights to obtain low-bit-width weights, that is, the quantized results.

[0051] Furthermore, in the bit convolution kernel, the convolution calculation is completed by using bit operations including:

[0052] Binarize the weights and input activations of the convolutional layers and deploy the network on a memristor chip.

[0053] When deployed in the memristor chip for forward transmission, the dot product of two bit vectors x and y is calculated by bitwise operation using a preset expression;

[0054] Furthermore, the direct estimator takes the real input r i ∈[0, 1] is quantized into k-bit digital output r o ∈[0, 1].

[0055] Specifically, in this embodiment, the quantization module is designed as follows: first, the weights of the deep neural network are quantized through this module, and then the quantized results are given to the mapping module to perform mapping deployment of the deep neural network towards the memristor chip.

[0056] (1) Bit convolution kernel design

[0057] In this embodiment, the weights and input activations of the convolutional layer are first binarized before the network is deployed in the memristor chip. When deployed in the memristor chip for the forward pass, the dot product of the two bit vectors x and y is calculated by bitwise operation using Equation (1). Therefore, the most computationally expensive convolution can be performed using the bit operation kernel, where bitcount calculates the number of 1s in the bit vector.

[0058] x·y=bitcount(&(x,y)),x i ,y i ∈{0,1} (1)

[0059] The 1-bit convolution shown in formula (1) can also be used to calculate the convolution of low-bit-width fixed-point integers, assuming that x is an M-bit fixed-point integer, that is, And y is a K-bit fixed-point integer, that is in and are both bit vectors, then the convolution of x and y can be calculated by formula (2):

[0060]

[0061] In the above equation, the computational complexity is O(MK), which is proportional to the bit width of x and y.

[0062] (2) Direct-through estimator design

[0063] The straight-through estimator (STE) is designed to solve the problem that back propagation cannot be chained due to the non-differentiable nature of the quantization function. For a 1-bit quantization function, i.e., the sign function, its expression is as shown in Equation (3):

[0064]

[0065] Among them The weight parameter represents the quantization. Since it is a non-differentiable function, its gradient cannot be directly calculated by the chain rule. STE solves this problem. STE sets the gradient of the input to a threshold function equal to its output gradient, regardless of the actual derivative of the threshold function itself. The following STE is defined using Bernoulli sampling as shown in Equations (4) and (5):

[0066] Forward:q~bernoulli(p)(4)

[0067]

[0068] Where c represents the target function. Since sampling from the Bernoulli distribution is not a differentiable function, The definition of is not clear, so the chain rule cannot be used to construct the backward pass directly from the forward pass. However, since the expected value of q is equal to p, this embodiment can use the well-defined gradient As In other words, STE provides a custom

[0069] The STE used in this embodiment is quantize k , which takes a real number as input i ∈[0, 1] is quantized into k-bit digital output r o ∈[0,1]. The STE is defined as follows (6) (7):

[0070]

[0071] It is obvious by construction that the output r of the quantized STE o is a real number that can be represented by k bits. In addition, since r ois a k-bit fixed-point integer, so by using the fixed-point integer dot product in Equation (2), the dot product of two sequences of such k-bit real numbers can be efficiently calculated and then appropriately scaled.

[0072] (3) Low-bitwidth quantization of weights

[0073] In this embodiment, STE is used to binarize the weights. The weights are binarized by the following STE:

[0074] Forward:r o = sign(r i )×E F (|r i |) (8)

[0075]

[0076] Where sign(r i ) is the sign function. Scale factor E F (|r i |) is the average of the absolute values of each output channel of the weights. The rationale is that introducing this scaling factor will increase the value range of the weights while still being able to utilize the bit convolution kernel.

[0077] Furthermore, the mapping module includes: a translation unit, a weight matrix splitting unit, and a mapping unit;

[0078] A translation unit is used to translate the quantized result so that the weight matrix becomes a positive number;

[0079] The weight matrix splitting unit is used to split and map the translated results onto different cross arrays;

[0080] The mapping unit is used to fill the mapped weight matrix and map the weight to the conductance of the memristor through a preset mapping formula.

[0081] Furthermore, the weight matrix splitting unit splits and maps the weight matrix W onto different cross arrays, including:

[0082] If the number of columns of the cross array is less than the number of columns of the weight matrix, the weight matrix is divided into multiple groups by column, and each group is mapped to a cross array;

[0083] If the number of rows in the crossbar array is smaller than the number of rows in the weight matrix, the weight matrix is divided into multiple groups by row, and each group is mapped to one crossbar array.

[0084] Specifically, in this embodiment, the mapping of the network to the memristor after quantization includes:

[0085] The mapping of quantized deep neural networks is mainly aimed at deep convolutional neural networks. The network parameters are quantized and the quantized parameters are deployed and mapped to the memristor chip, thereby realizing the inference calculation of the deep convolutional neural network by the memristor chip.

[0086] Deep Neural Networks (DNNs) are structured as a series of sequentially cascaded convolutional and fully connected layers. The output of the previous layer is fed into the input of the next layer. There are two primary operations in DNNs: convolution (CONV) and matrix-vector multiplication (MVM). The mapping of memristor arrays primarily implements the aforementioned modules: For CONV operations, features are extracted from the input feature map (FM) using multiple 3D kernels (C, K, K, N). The output FM is generated by sliding the 3D kernels across the input feature map (FM). To perform a CONV operation using an RRAM array, the input FM (C, H, W) and kernels (C, K, K, N) are mapped to arrays of input voltages (H×W, C×K×K) and conductances (C×K×K, N). Similarly, for MVM operations, the input vectors (H) and matrices (H, W) are mapped to input voltages (H) and conductances (H, W). When the size of the memristor array is smaller than the size of the kernel or weight, the kernel and weight will be mapped to the multi-array. The result can be expressed as the output current:

[0087]

[0088] (1) Weight Mapping

[0089] This embodiment implements a method for mapping a weight matrix to the memristor conductance of a memristor crossbar array: W represents a weight matrix of size H×W, where H represents the number of input channels, W represents the number of output channels, and B W Indicates the bit width of the weight, N R , N C represents the number of rows and columns of the memristor crossbar array.

[0090] First, the weight matrix W is quantized from floating point to B W Bit.

[0091]

[0092] Since some items in the weight matrix are negative and the conductance can only be integers, the weight matrix is made positive by translating the weight W.

[0093]

[0094] For each weight w, this embodiment uses N cThe memristor unit is used to represent this weight. The bit width of each unit is B_c. That is, w i The value of the weight represented by the i-th memristor unit of weight w. For example, if two 4-bit memristor units are used to represent an 8-bit weight, the first memristor unit is used to represent the first 4 bits of the weight, and the second memristor unit is used to represent the last 4 bits of the weight.

[0095] 1.for i=1:N c do:

[0096] 2.

[0097] 3.w i =w / / b i

[0098] 4.w=w%b i

[0099] 5.end for

[0100] By mapping each weight to N c units, the weight matrix W becomes H×N c *W. Since the size of the memristor crossbar array is fixed, if the size of the weight matrix is larger than the size of the memristor crossbar array, the weight matrix needs to be split and mapped to different crossbar arrays, mainly including column splitting and row splitting:

[0101] 1) Column Splitting

[0102] The columns of different cross arrays represent their corresponding convolution output channels. Therefore, if the number of columns of the cross array (N C ) is less than the number of columns in the weight matrix (N c *W), the output channels need to be divided into X groups of memristor crossbar arrays, as shown in equations (12) and Figure 1 As shown in (a). The input features will be sent to each cross array in a group, and the weight matrix needs to be divided into X by column C The groups are mapped to different memristor crossbar arrays, such as Figure 1 (a) shows the same input features will be input into each cross array of a group. C The calculation formula is shown in formula (12).

[0103]

[0104] 2) Row Splitting

[0105] Figure 1 (b) shows the basic process of row splitting. If the number of rows in a cross array (N R) is less than the number of rows H of the weight matrix, then the elements of a Conv kernel will be divided into group, as shown in equation (13):

[0106]

[0107] Each group will be mapped to a memristor crossbar array. The input feature map is also divided into X R Group and send to the input interface of the corresponding crossbar array. Some outputs are obtained from different crossbar arrays, so the adder tree needs to be cascaded after the crossbar array group to accumulate all X R The output of the group is combined and the final output is obtained.

[0108] If X R *N R >H or X C *N C >N c *W, then the weight matrix needs to be zero-filled so that the filled matrix size is (X R *N R ,X C *N C ).like Figure 2 As shown, the weight matrix size is 784×500, and the size of the memristor array is 112*128. It can be obtained that the weight matrix needs to be divided into 8 groups by row and 4 groups by column, so the weight matrix needs to be filled to 896×512.

[0109] Then use the formula:

[0110]

[0111] G=G min +G s ×w (15)

[0112] Map the weight w to the conductance G of the memristor, where B C is the bit width of the memristor unit, G Max ,G min are the maximum and minimum values of the memristor conductance, respectively.

[0113] In summary, the weight mapping process is as follows Figure 3 As shown, firstly, the neural network is trained by software such as Theano to obtain the weight matrix, and then the weight is converted to B W Integer, the weight is converted into a non-negative number through shift transformation, and then the weight is mapped to the memristor unit. After that, the weight matrix is split into rows and columns as needed, and then the weight matrix is mapped to the conductance value of the memristor unit.

[0114] Based on the quantitative deployment of the memristor array in this embodiment, the energy consumption on the memristor is much lower than the quantitative deployment of the same network on the CPU and GPU platforms, as shown in Table 1 below.

[0115] Table 1

[0116]

[0117] In addition to the deployment comparison on the above-mentioned different hardware platforms, a comparison was also made with other methods. As shown in Table 2 below, the quantization strategy adopted by other methods is a simple rounding and truncation method, and the mapping strategy is an overall mapping method. The method of this embodiment is a specially designed quantization strategy and a fine-grained row and column mapping strategy, so the accuracy of the present invention is significantly higher.

[0118] Table 2

[0119]

[0120] Furthermore, this embodiment calculates the resource status of the memristor chip after mapping, as shown in Table 3 below. Different crossbar array sizes in the table represent different memristor chip sizes, and other indicators are actual test results. Compared with the current main memristors, all of them have good performance.

[0121] Table 3

[0122]

[0123] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A network quantization and mapping system for a memory-computing integrated memristor chip, characterized by: include: Quantization module and mapping module; The quantization module is used to quantize the weights of the deep neural network and send the quantized results to the mapping module; The mapping module is used to map the quantized results onto the memristor chip and perform inference calculations on the deep convolutional neural network through the memristor chip.

2. The network quantization and mapping system for the integrated storage and computing memristor chip according to claim 1 is characterized in that: The quantization module includes: a bit convolution kernel and a weighted low-bit-width quantization unit; The bit convolution kernel is used to binarize the weights and input activations of the convolution layer and then perform convolution calculations using bit operations; The weight low-bit-width quantization unit is used to construct a straight-through estimator, and adopt the straight-through estimator to binarize the weight to obtain the low-bit-width weight, that is, the quantized result.

3. The network quantization and mapping system for the integrated storage and computing memristor chip according to claim 2 is characterized in that: In the bit convolution kernel, the convolution calculation is completed by using bit operations, including: Binarize the weights and input activations of the convolutional layers and deploy the network on a memristor chip. When deployed in the memristor chip for forward transmission, the dot product of two bit vectors x and y is calculated by bitwise operation using a preset expression; Wherein, the preset expression is: x·y=bitcount(&(x,y)),x i ,y i ∈{0,1} Among them, bitcount is the vector dot product operation, & is the bitwise AND operation, x i is the i-th element of vector x, y i is the i-th element of vector y.

4. The network quantization and mapping system for the integrated storage and computing memristor chip according to claim 2, characterized in that: The straight-through estimator takes a real number as input r i ∈[0, 1] is quantized into k-bit digital output r o ∈[0, 1]; The straight-through estimator is: Among them, Forward is the neural network forward reasoning label, r o is the quantized result, k is the number of quantized bits, round is the rounding algorithm, and backward is the neural network back propagation. For the reverse result, For r i Find the integral, For r o Find the integral, r i The real number passed into the estimator to quantize the neural network parameters.

5. The network quantization and mapping system for the integrated storage and computing memristor chip according to claim 4 is characterized in that: Using the direct estimator to binarize the weights includes: Forward:r o =sign(r i )×E F (|r i |) Among them, sign(r i ) is to find the sign function, E F (|r i |) is to find the expected function.

6. The network quantization and mapping system for the integrated storage and computing memristor chip according to claim 1, characterized in that: The mapping module includes: a translation unit, a weight matrix splitting unit, and a mapping unit; The translation unit is used to translate the quantized result so that the weight matrix becomes a positive number; The weight matrix splitting unit is used to split and map the translated result onto different cross arrays; The mapping unit is used to fill the mapped weight matrix and map the weight to the conductance of the memristor through a preset mapping formula.

7. The network quantization and mapping system for the integrated storage and computing memristor chip according to claim 6, characterized in that: The weight matrix splitting unit splits and maps the translated result onto different cross arrays, including: If the number of columns of the cross array is less than the number of columns of the weight matrix, the weight matrix is divided into multiple groups by column, and each group is mapped to a cross array; If the number of rows in the crossbar array is smaller than the number of rows in the weight matrix, the weight matrix is divided into multiple groups by row, and each group is mapped to one crossbar array.

8. The network quantization and mapping system for the integrated storage and computing memristor chip according to claim 6, characterized in that: The preset mapping formula is: G=G min +G s ×w Among them, G s is the conductivity value after treatment, G Max is the maximum conductance value, G min is the minimum conductance value, is the memristor device parameter, G is the final conductance value, and w is the weight.

Images