A convolutional neural network quantization method based on in-memory computing circuit

Abstract

The application discloses a convolutional neural network quantification method based on in-memory computing circuit, and belongs to the field of convolutional neural network quantification. A quantification network model is designed for the activation value and weight of a convolutional layer by using a convolutional neural network; 1000 samples are selected as a quantification data set, 50 samples are grouped as a group, and the samples are divided into 20 groups; a weight quantification coefficient is calculated and fixed to the quantification network model; an activation value quantification coefficient is calculated; a quantification network model for calculating an ivc coefficient is designed according to a data flow process of the in-memory computing circuit, 100 samples are selected as quantification samples, and the ivc coefficient is calculated; the weight and offset of the convolutional layer are converted into an integer form, the ivc coefficient is mapped to a conductance value of a resistor in the circuit, the activation value of the convolutional layer, the weight quantification coefficient, the ivc coefficient and the next layer convolutional activation value quantification coefficient are fused, and the fused values are converted into a*2 b form, a quantification post-convolution network is designed to infer a network model in the in-memory computing circuit, and quantification deployment of the convolutional neural network in the in-memory computing circuit is realized.

Description

Technical Field

[0001] This invention relates to the field of convolutional neural network quantization technology, and in particular to a convolutional neural network quantization method based on in-memory computing circuits. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, convolutional neural networks have been applied to scenarios such as intelligent robots, autonomous driving, intelligent security, and smart homes. However, traditional artificial intelligence chips are designed based on the "von Neumann" architecture, with separate data and computing units. Data exchange between memory and processor is carried out through a data bus, which causes a lot of time and power consumption overhead for data transmission, resulting in the "memory wall" and "power wall" problems.

[0003] In-memory computing circuits (chips) deploy data directly within the processor's computing units, avoiding repeated data retrieval from storage devices, thus reducing power consumption and improving processing speed. However, in-memory computing circuits suffer from limitations, including low computing power, small memory size, and suboptimal characteristics of resistive devices. Furthermore, to reduce power consumption and area, analog-to-digital converters are typically designed to be 8-bit, only able to retrieve 8 bits of data. Therefore, when deploying convolutional neural network models on in-memory computing circuits, floating-point data needs to be converted to fixed-point data.

[0004] Quantization is the most common method for deploying convolutional neural network models on AI chips. There are two main types of quantization techniques: one is quantization-aware training, which inserts pseudo-quantization nodes on the basis of a pre-trained model and then performs micro-training. This method is effective for quantizing activation values ​​and weights, but cannot quantize the IVC coefficients before the analog-to-digital converter circuit. The other is post-training quantization, which uses a pre-trained model and a quantized dataset to quantize floating-point activation values, weights, and other parameters, converting them into fixed-point data. This method only focuses on the similarity of the data before and after quantization, without considering the similarity of the output values ​​of the convolutional computation units after quantization. Summary of the Invention

[0005] The purpose of this invention is to provide a quantization method for convolutional neural networks based on in-memory computing circuits, so as to solve the problem of quantization deployment of convolutional neural networks on in-memory computing circuits.

[0006] To address the aforementioned technical problems, this invention provides a convolutional neural network quantization method based on in-memory computing circuits, comprising:

[0007] Step S1: Using a pre-trained convolutional neural network model, design a quantization network model for the activation values ​​and weights of the convolutional layers. The weights are quantized uniformly and symmetrically through each channel, and the activation values ​​are quantized uniformly and symmetrically through each layer.

[0008] Step S2: Select 1000 samples as the quantization dataset. These data cover all types of training set objects. Randomly select 50 samples as a group, divide them into 20 groups, and feed them into the quantization network model in sequence.

[0009] Step S3: Calculate the weight quantization coefficients of each channel of each convolutional layer using the maximum value method, and fix the results into the quantized network model;

[0010] Step S4: Combine the fully floating-point convolutional network inference model, use the mean squared error method to calculate the activation value quantization coefficients, calculate for each convolutional layer, and then average the quantization coefficients calculated from 20 sets of data to obtain the activation value quantization coefficients for each convolutional layer.

[0011] Step S5: For the IVC coefficients in the in-memory computing circuit, the convolution operation in the data flow processing of the in-memory computing circuit is simulated by software. Combined with the activation value and weight quantization results, a quantization network model for calculating the IVC coefficients is designed.

[0012] Step S6: Select 100 samples containing various types of training set objects as the IVC coefficient quantization dataset, randomly select 5 samples as a group, divide into 20 groups of data, and feed them into the IVC coefficient quantization network model in sequence.

[0013] Step S7: Calculate the IVC coefficients using the KL divergence method, performing the calculation layer by layer. Then, average the IVC coefficients calculated from the 20 sets of quantized data to obtain the IVC coefficients corresponding to each convolutional layer.

[0014] Step S8: Convert the convolutional layer weights and offsets to integer form, map the IVC coefficients to the conductance values ​​of the resistors in the circuit, and fuse the convolutional layer activation values, weight quantization coefficients, IVC coefficients, and the quantization coefficients of the next layer's convolutional activation values, and convert them into a*2. b The form is given, where a and b are integers. The design of the quantized convolutional network in the in-memory computing circuit inference network model is carried out to realize the quantized deployment of the convolutional neural network in the in-memory computing circuit.

[0015] In one embodiment, step S1 includes:

[0016] In designing a quantized network model, the activation values ​​and weights of each convolutional layer are floating-point data. First, the quantization coefficients are calculated, and then multiplied by the quantization coefficient S. x Weight S w Quantized to Int8 data type;

[0017] Then perform a convolution operation, and add the offset b. i *(S w *S x The value of ) is then divided by the coefficient (S). w *S xThe dequantization operation outputs floating-point data, which is used as the activation value for the next convolutional layer. At the same time, a parallel fully floating-point convolutional network inference model is designed to calculate the mean square error with the output of the convolutional layer of the quantized network, which is used to select the optimal quantization coefficients for the activation values.

[0018] In one implementation, step S3 includes:

[0019] First, divide each channel in the weight into two parts: greater than 0 and less than 0. Select the part with more data to calculate the absolute value, take the maximum value as the cut-off value, calculate the quantization coefficient, and fix the pre-calculated quantization coefficient into the quantization network model.

[0020] In one embodiment, step S4 includes:

[0021] First, using the histogram statistical method, multiple sets of cut-off values ​​were selected for calculating the quantization coefficient of the activation value, and the quantization coefficient was calculated.

[0022] Then, the convolutional inference process of this layer is completed separately, and multiple sets of floating-point data convolution results are output. After that, the mean square error is calculated with the convolution results of the fully floating-point inference network.

[0023] Finally, the quantization coefficient corresponding to the minimum mean squared error is used as the activation value quantization coefficient for that convolutional layer.

[0024] In one implementation, step S5 includes:

[0025] In the quantized network model for calculating IVC coefficients, the Int8 activation values ​​input to the first convolutional layer are calculated by multiplying floating-point activation values ​​by the quantization coefficients. The Int8 activation values ​​of other convolutional layers are the output values ​​of the previous convolutional layer. The Int8 weights are stored in the in-memory computing circuit. The Int8 activation values ​​are divided into 8 1-bit values, which are then multiplied and accumulated with the weights to obtain 8 sets of Int16 data. The IVC quantization coefficients are then calculated first. Then quantize to Int8, then dequantize to Int16 data, then shift and add the 8 sets of data, plus the offset. Finally, multiply by the coefficient. Output Int8 data as the activation value for the next convolutional layer.

[0026] In one implementation, step S8 includes:

[0027] In the in-memory computation circuit inference network model, convolutional networks map the IVC coefficients to the circuit's resistance and conductance values ​​G. i The activation values, weight quantization coefficients, IVC coefficients, and activation value quantization coefficients of the next layer convolution are fused into a coefficient. And convert it to a*2 bThe form is given, where a and b are integers, and the offset b is... i Multiply Convert to integer form to enable quantized deployment of convolutional neural networks in in-memory computing circuits.

[0028] This invention provides a quantization method for convolutional neural networks based on in-memory computing circuits. When calculating the quantization coefficients of activation values ​​and IVC coefficients, the selected quantization dataset covers all types of objects in the training set, enabling the quantized network inference model to have better generalization ability. During activation value quantization, the mean squared error is calculated between the convolution results of the quantized model and the results of the fully floating-point network model to select the optimal activation value quantization coefficients, fully considering the impact of quantization on the convolution results, making the quantized convolution results closer to the true values. By first quantizing the activation values ​​and weights, and then designing a quantized network model to calculate the IVC coefficients, the IVC coefficients are calculated layer by layer. Compared to previous statistical methods that select IVC coefficients, all convolutional layers use the same IVC coefficients, greatly improving the accuracy of the quantized network model. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a convolutional neural network quantization method based on in-memory computing circuits provided by the present invention;

[0030] Figure 2 This is a schematic diagram of the activation value and weight quantization network model provided by the present invention;

[0031] Figure 3 This is a schematic diagram of the activation value quantization coefficient calculation process provided by the present invention;

[0032] Figure 4 This is a schematic diagram of the network model for the IVC coefficient calculation process provided by the present invention;

[0033] Figure 5 This is a schematic diagram of the network inference process after IVC coefficient quantization provided by the present invention;

[0034] Figure 6 This is a schematic diagram of the in-memory computation circuit inference process of the quantized network provided by the present invention. Detailed Implementation

[0035] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a further detailed explanation of the convolutional neural network quantization method based on in-memory computing circuits proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of this invention.

[0036] Example 1

[0037] This invention provides a quantization method for convolutional neural networks based on in-memory computing circuits, the process of which is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0038] (1) Based on the trained convolutional neural network model, design a quantized network model for the convolutional activation values ​​and weights. The quantized network model for the convolutional layers i and i+1 is as follows: Figure 2 As shown, the upper part is the inference process of the fully floating-point data network, the lower part is the inference process of the quantized data network, and the middle part is the calculation of the mean square error of the convolution results of fully floating-point and quantized data in the same convolution layer, which is used to select the optimal activation value quantization coefficient.

[0039] (2) In order to make the activation value quantization coefficient applicable to all types of input samples, 1000 samples were selected as the quantization dataset. The quantization data can cover all types of characteristics of the training set objects. 50 samples were randomly assigned as a group, and a total of 20 groups of data were input into the quantization network model in sequence to calculate the activation value quantization coefficient.

[0040] (3) The weight values ​​are not affected by the quantized data and remain unchanged after the network model training is completed. First, calculate the quantization coefficients of the weight values ​​and fix them in the quantized network model. The weights are quantized using the channel-wise uniform symmetric maximum quantization method. First, calculate the quantization coefficients. The weight values ​​of the i-th convolutional layer are:

[0041]

[0042] Where ch is the number of channels in the convolutional layer, and n is the weight of each channel. This is the k-th weight value of the j-th channel.

[0043] The number of weight values ​​greater than or equal to 0 and less than 0 in the j-th channel are counted as follows: At the same time, the maximum weight value greater than or equal to 0 is counted. Minimum weight value less than 0

[0044] like Then the channel intercept value like The channel cutoff value is then... like The channel cutoff value is then...

[0045] Calculate the quantization coefficient of the weight of the j-th channel:

[0046]

[0047] in, is the weight quantization coefficient of the j-th channel of the i-th convolutional layer, and b is the bit width of the quantized data.

[0048] Similarly, calculate the quantization coefficients of all convolutional layer weights and fix them in the quantized network model. During inference, the weights of the i-th convolutional layer are quantized:

[0049]

[0050] Here, clip represents the truncation operation, and round represents the rounding operation. i W represents the floating-point weights of the i-th convolutional layer. i The data is the quantized weights of the i-th layer convolution. For weight w i The corresponding quantization coefficient, where b is the bit width of the quantized data.

[0051] (4) The activation values ​​are quantized using the uniform symmetric mean square error method layer by layer. The calculation process of the quantization coefficients of the activation values ​​of the i-th convolutional layer is as follows: Figure 3 As shown, for the input activation values, multiple cutoff values ​​are selected using the histogram method to calculate quantization coefficients. These quantization coefficients are then used for quantization, convolved with the weighted quantized data, and an offset is added. After dequantization, each floating-point result is output. The mean squared error is calculated by convolving each of these floating-point results with the full floating-point result. The quantization coefficient corresponding to the minimum mean squared error is selected as the optimal activation value quantization coefficient. The specific calculation process is as follows:

[0052] Given a set of quantized data, obtain the activation values ​​of the i-th convolutional layer:

[0053] x i =[x 1 ,x 2 ,x 3 ,…,x k ,…,x N ]

[0054] Where N is the total number of activation values ​​in the i-th convolutional layer.

[0055] Count the number of activation values ​​greater than 0 and less than 0, N. r and N l If N r <N l Then the activation value x will be inverted. i =-x i .

[0056] Select activation values ​​greater than 0, and calculate the maximum and minimum values ​​among them:

[0057] x max =max(x k ), k = 1, 2, 3, ..., N

[0058] x min =min(xk ), k = 1, 2, 3, ..., N

[0059] The interval [x min ,x max Divided into 2048 equal parts, each part has a length of bin:

[0060]

[0061] The center point of each bin interval from 128 to 2048 is selected as the cutoff value:

[0062] β l =x min +(l-0.5)*bin,l=128,129,…,2048

[0063] Where, β l It is the median value of the l-th bin interval.

[0064] Calculate the cutoff value β l Corresponding quantization coefficients:

[0065]

[0066] in, Let be the l-th quantization coefficient of the activation weights of the i-th convolutional layer, and b be the bit width of the quantized data. The quantization process is as follows:

[0067]

[0068] Where clip represents the truncation operation, round represents the rounding operation, and x i The value is a floating-point activation value of the i-th convolutional layer. For x i Quantified data.

[0069] Perform a convolution operation with the weighted quantized values, add an offset, then perform dequantization to output the result. Meanwhile, the output of the same convolutional layer of the all-floating-point network is calculate and The mean squared error between them is: mse(l), where l = 128, 129, ..., 2048. The index value l corresponding to the minimum value in mse(l) is calculated. m :

[0070] l m =argmin(mse(l))

[0071] Therefore, the optimal quantization coefficient for the activation value of the i-th convolutional layer is: The convolution result of the quantization coefficient Used as the activation value for the next convolutional layer, the quantization coefficients for the next convolutional layer are calculated, and the quantization coefficients for the activation values ​​of all convolutional layers are calculated in sequence.

[0072] Input all groups of quantized samples in sequence to obtain 20 groups of quantization coefficients. Calculate the average of the 20 groups of activation value quantization coefficients for the same convolutional layer to obtain the final activation value quantization coefficients for that convolutional layer.

[0073] (5) After calculating the activation value and weight quantization coefficient, the activation value of the first convolution layer is multiplied by its quantization coefficient and quantized into Int8 type data. The activation value of other convolution layers is the result of the previous convolution layer multiplied by the coefficient. Quantized to Int8 type data, weights multiplied by weight quantization coefficients. Quantizing the data into Int8 bytes and combining it with the data flow format of the in-memory computing circuit's computing unit, a network model for calculating the IVC coefficients is designed, such as... Figure 4 As shown, the Int8 weights are fixed in the in-memory computing circuit's calculation unit. The Int8 activation values ​​are divided into 8 groups of 1-bit data and sequentially input into the calculation unit. The 1-bit activation values ​​and 8-bit weights are multiplied and accumulated, outputting 8 groups of Int16 type data. The KL divergence method is used to calculate the IVC coefficients. Quantize 8 sets of Int16 data into Int8 data. The quantization process is as follows:

[0074]

[0075] Here, `clip` represents the truncation operation, and `round` represents the rounding operation. i Y represents the Int16 data before quantization in the i-th convolutional layer (ivc). i Let b be the Int8 data after quantization in the i-th convolutional layer (ivc), where b is the bit width of the quantized data, which is 8 bits here.

[0076] Next, the data is dequantized to Int16 using the IVC coefficients. The dequantization process is as follows:

[0077]

[0078] Here, round represents the floor function. i The quantized Int8 data for the i-th layer IVC coefficients. This is the Int16 data after IVC dequantization.

[0079] Next, 8 sets of Int16 data The data is sequentially fed into a shift accumulator for shifting and addition to complete the convolution operation, and then the offset B is added. i Multiply by the coefficient Output Int8 data as the activation value data for the next convolution, and start calculating the IVC coefficients of the next convolution layer. Complete the calculation of IVC coefficients for all convolution layers in sequence.

[0080] (6) Select 100 samples that best cover the characteristics of the training set as IVC quantization data. Randomly divide the samples into groups of 5, and input these 20 groups of data into the IVC coefficient calculation network model to obtain 20 groups of IVC quantization coefficients. Calculate the average of the 20 groups of IVC coefficients from the same convolutional layer, and use this average as the final IVC coefficient calculation result. This completes the calculation of the IVC quantization coefficients. The inference process of the convolutional network after quantizing the IVC coefficients of the i-th convolutional layer is as follows: Figure 5 As shown, the difference from the network for calculating IVC coefficients is that an 8-bit analog-to-digital converter (ADC) circuit is added after IVC quantization, and the IVC dequantization operation is moved to after adding the offset.

[0081] (7) Figure 6 As shown To make the inference data stream of the quantized network more closely resemble the processing of the in-memory computing circuit, the IVC quantization coefficients are mapped to the resistance and conductance values ​​G of the IVC circuit. i ,Right now:

[0082]

[0083] in, Quantize the IVC coefficients of the i-th convolutional layer.

[0084] offset b i Convert to integer form, that is:

[0085]

[0086] The activation value, weight quantization coefficient, IVC coefficient, and activation value quantization coefficient of the next convolutional layer are fused together. And convert it to a*2 b In this form, a and b are integer data; therefore, multiply by a coefficient. The operation becomes multiplying by an integer 'a' and then shifting by abs(b) times, where if b is greater than 0, the shift is to the right, and if b is less than 0, the shift is to the left. That is:

[0087]

[0088] in, These represent the activation value, weight quantization coefficient, and IVC coefficient of the i-th convolutional layer, respectively. The activation values ​​of the (i+1)th convolutional layer are quantized.

[0089] Weight multiplied by weight quantization factor The data is quantized into Int8 and pre-stored in the computing unit of the in-memory computing circuit, enabling the quantization deployment of the convolutional neural network in the in-memory computing circuit.

[0090] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A quantization method for convolutional neural networks based on in-memory computing circuits, characterized in that, include: Step S1: Using a pre-trained convolutional neural network model, design a quantization network model for the activation values ​​and weights of the convolutional layers. The weights are quantized uniformly and symmetrically through each channel, and the activation values ​​are quantized uniformly and symmetrically through each layer. Step S2: Select 1000 samples as the quantization dataset. These data cover all types of objects in the training set. Randomly select 50 samples as a group, divide them into 20 groups, and feed them into the quantization network model in sequence. Step S3: Calculate the weight quantization coefficients of each channel of each convolutional layer using the maximum value method, and fix the results into the quantized network model; Step S4: Combine the fully floating-point convolutional network inference model, use the mean squared error method to calculate the activation value quantization coefficients, calculate for each convolutional layer, and then average the quantization coefficients calculated from 20 sets of data to obtain the activation value quantization coefficients for each convolutional layer. Step S5: For the IVC coefficients in the in-memory computing circuit, the convolution operation in the data flow processing of the in-memory computing circuit is simulated by software. Combined with the activation value and weight quantization results, a quantization network model for calculating the IVC coefficients is designed. Step S6: Select 100 samples containing various types of training set objects as the IVC coefficient quantization dataset, randomly select 5 samples as a group, divide into 20 groups of data, and feed them into the IVC coefficient quantization network model in sequence. Step S7: Calculate the IVC coefficients using the KL divergence method, performing the calculation layer by layer. Then, average the IVC coefficients calculated from the 20 sets of quantized data to obtain the IVC coefficients corresponding to each convolutional layer. Step S8: Convert the convolutional layer weights and offsets to integer form, map the IVC coefficients to the conductance values ​​of the resistors in the circuit, and fuse the convolutional layer activation values, weight quantization coefficients, IVC coefficients, and the quantization coefficients of the next layer's convolutional activation values, and convert them into... Form, among which, , For integers, a quantized convolutional network in-memory computation circuit inference network model is designed to realize the quantized deployment of convolutional neural networks in in-memory computation circuits; Step S1 includes: In designing a quantized network model, the activation values ​​and weights of each convolutional layer are floating-point data. First, the quantization coefficients are calculated, and then the activation values ​​are multiplied by the quantization coefficient S. x Weight multiplied by quantization coefficient S w Quantized to Int8 data type; Then perform a convolution operation, and add the offset b. i *(S w *S x The value of ) is then divided by the coefficient (S). w *S x The dequantization operation outputs floating-point data, which is used as the activation value for the next convolutional layer. At the same time, a parallel fully floating-point convolutional network inference model is designed to calculate the mean square error with the output of the convolutional layer of the quantized network, which is used to select the optimal quantization coefficient for the activation value.

2. The convolutional neural network quantization method based on in-memory computing circuits as described in claim 1, characterized in that, Step S3 includes: First, divide each channel in the weight into two parts: greater than 0 and less than 0. Select the part with more data to calculate the absolute value, take the maximum value as the cut-off value, calculate the quantization coefficient, and fix the pre-calculated quantization coefficient into the quantization network model.

3. The convolutional neural network quantization method based on in-memory computing circuits as described in claim 2, characterized in that, Step S4 includes: First, using the histogram statistical method, multiple sets of cut-off values ​​were selected for calculating the quantization coefficient of the activation value, and the quantization coefficient was calculated. Then, the convolutional inference process of this layer is completed separately, and multiple sets of floating-point data convolution results are output. After that, the mean square error is calculated with the convolution results of the fully floating-point inference network. Finally, the quantization coefficient corresponding to the minimum mean squared error is used as the activation value quantization coefficient for that convolutional layer.

4. The convolutional neural network quantization method based on in-memory computing circuits as described in claim 3, characterized in that, Step S5 includes: In the quantized network model for calculating IVC coefficients, the Int8 activation values ​​input to the first convolutional layer are calculated by multiplying floating-point activation values ​​by the quantization coefficients. The Int8 activation values ​​of other convolutional layers are the output values ​​of the previous convolutional layer. The Int8 weights are stored in the in-memory computing circuit. The Int8 activation values ​​are divided into 8 1-bit values, which are then multiplied and accumulated with the weights to obtain 8 sets of Int16 data. The IVC quantization coefficients are then calculated first. Then quantize to Int8, then dequantize to Int16 data, then shift and add the 8 sets of data, plus the offset. Finally, multiply by the coefficient. Output Int8 data as the activation value for the next convolutional layer.

5. The convolutional neural network quantization method based on in-memory computing circuits as described in claim 4, characterized in that, Step S8 includes: In the in-memory computation circuit inference network model, convolutional networks map the IVC coefficients to the circuit's resistance and conductance values ​​G. i The activation values, weight quantization coefficients, IVC coefficients, and activation value quantization coefficients of the next layer convolution are fused into a coefficient. And transformed into Form, among which, , For integer shaping, offset Multiply Convert to integer form to enable quantized deployment of convolutional neural networks in in-memory computing circuits.

Images