Neural network data processing method and apparatus

Abstract

Embodiments of the present application provide a neural network data processing method and device, wherein the method comprises: obtaining a convolution feature map to be input into an i-th layer convolution layer in a neural network model; quantizing a convolution point in the convolution feature map into a plurality of four-bit data; splitting each four-bit data into four single-bit data; inputting each single-bit data in the four single-bit data into a memory-computing array in sequence, the memory-computing array being configured to perform a convolution calculation corresponding to the i-th layer convolution layer to obtain a convolution result quantization value output by the memory-computing array; wherein the neural network model is obtained based on pre-training and re-training, in the pre-training process, a target output of the i-th layer convolution layer is an algorithm result corresponding to the i-th layer convolution layer, and in the re-training process, the target output of the i-th layer convolution layer is the convolution result quantization value output by the memory-computing array. Embodiments of the present application can reduce power consumption of a computer device.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a neural network data processing method and apparatus. Background Technology

[0002] In the era of the Internet of Things (IoT), intelligent visual sensing devices are ubiquitous, and visual sensing tasks are shifting from the cloud to terminal devices with processing and computing capabilities. The localization of intelligent visual algorithms from the cloud to the terminal for information processing places higher demands on the power consumption, computing power, and storage capabilities of traditional devices. From a power consumption perspective, compared to traditional digital neural network accelerators, in-memory computing can reduce energy consumption of intelligent visual sensing systems while increasing computing power by reducing the cost of analog-to-digital conversion and data transfer. Regarding computing and storage, current mainstream neural networks typically have millions of operations per second (GOPS) and millions of parameters, thus requiring a series of lightweight processing measures for neural network algorithms, including low-bit-width quantization, pruning, and compression.

[0003] Traditional neural network quantization methods based on in-memory computation obtain quantization results with high bit widths through fine-tuning after training, but cannot avoid the huge overhead brought by the analog-to-digital converter (ADC) in the in-memory array. Summary of the Invention

[0004] This invention provides a neural network data processing method and apparatus to address the shortcomings of high power consumption and large area of ​​high-resolution ADCs in memory computing arrays used in existing in-memory computing chips. This method can reduce the ADC resolution to reduce the ADC area and power consumption while maintaining the accuracy of the neural network.

[0005] In a first aspect, the present invention provides a neural network data processing method, applied to a neural network model including at least one convolutional layer, comprising:

[0006] Obtain the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, where N is the total number of convolutional layers and N is an integer;

[0007] The convolution points in the convolution feature map are quantized into multiple four-bit data.

[0008] Each of the four-bit data is split into four single-bit data;

[0009] Each of the four single-bit data is sequentially input into the storage array, which is used to perform convolution calculations corresponding to the i-th convolutional layer, and obtain the quantized value of the convolution result output by the storage array.

[0010] The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array.

[0011] Optionally, quantizing the convolution points in the convolutional feature map into multiple four-bit data includes:

[0012] Based on the maximum value in the convolutional feature map, the convolutional feature map is normalized to obtain a normalized convolutional feature map.

[0013] Multiple normalized convolution points are obtained based on the normalized convolution feature map;

[0014] The data points in the normalized convolution points are converted into corresponding integer values, and the integer values ​​range from [-7, 7].

[0015] Each of the integer values ​​is converted into its corresponding four-bit data.

[0016] Optionally, converting the data points in the normalized convolution points into corresponding integer values ​​includes:

[0017] The data points in the normalized convolution points are numerically expanded and rounded based on a preset factor to obtain the corresponding integer values.

[0018] Optionally, the neural network model is obtained by training through the following steps:

[0019] The initial neural network model is pre-trained to obtain a pre-trained neural network model;

[0020] The pre-trained neural network model is retrained, and during the retraining process, error compensation is performed on the target output of the convolutional layer in the pre-trained neural network model layer by layer.

[0021] The error compensation for the target output includes:

[0022] The quantization result corresponding to the algorithm result of the i-th convolutional layer is added to the pre-obtained error value to obtain the target output corresponding to the i-th convolutional layer. The error value is the difference between the quantization result output by the in-memory array after completing the convolution operation and the quantization result corresponding to the algorithm result of the i-th convolutional layer.

[0023] Optionally, the error value is obtained through the following steps:

[0024] The convolution points used during training are input into the algorithm corresponding to the i-th convolutional layer to obtain the algorithm result corresponding to the i-th convolutional layer;

[0025] The convolution points used during training are quantized into corresponding four-bit data, and the four-bit data is split into four single-bit data. The four single-bit data are input into the in-memory array model to obtain the quantization result output by the in-memory array model after completing the convolution operation. The in-memory array model is pre-built.

[0026] The algorithm result corresponding to the i-th convolutional layer is quantized to obtain the quantization result corresponding to the algorithm result corresponding to the i-th convolutional layer.

[0027] The error value is obtained by subtracting the quantization result output by the in-memory array model from the quantization result corresponding to the algorithm result of the i-th convolutional layer.

[0028] Optionally, during the pre-training process, the method further includes:

[0029] Obtain the convolutional feature map to be input to the i-th convolutional layer;

[0030] Based on the maximum value in the convolutional feature map to be input, the convolutional feature map to be input is normalized to obtain a normalized convolutional feature map to be input.

[0031] Based on the normalized convolutional feature map to be input, the first convolution point is obtained;

[0032] The data points in the first convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the second convolution point corresponding to the first convolution point. The preset weight is the ratio of the maximum value to the preset multiple.

[0033] Each of the second convolutional points is input into the i-th convolutional layer.

[0034] Optionally, during the pre-training process, the method further includes:

[0035] Obtain the convolution result output by the i-th convolutional layer;

[0036] Based on the maximum value in the convolution result, the convolution result is normalized to obtain a normalized convolution result;

[0037] Based on the normalized convolution result, multiple third convolution points are obtained;

[0038] The data points in the third convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the fourth convolution point corresponding to the third convolution point. The preset weight is the ratio of the maximum value to the preset multiple.

[0039] Each of the fourth convolutional points is input into the next layer structure of the i-th convolutional layer.

[0040] In a second aspect, the present invention also provides a neural network data processing apparatus, applied to a neural network model including at least one convolutional layer, comprising:

[0041] The acquisition unit is used to acquire the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, and N is the total number of convolutional layers, where N is an integer;

[0042] The first splitting unit is used to quantize the convolution points in the convolution feature map into multiple four-bit data;

[0043] The second splitting unit is used to split each of the four-bit data into four single-bit data;

[0044] The transmission unit is used to sequentially input each of the four single-bit data into the storage array, and the storage array is used to perform convolution calculations corresponding to the i-th convolutional layer to obtain the quantized value of the convolution result output by the storage array.

[0045] The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array.

[0046] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the neural network data processing methods described above.

[0047] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the neural network data processing method as described above.

[0048] The neural network data processing method and apparatus provided by the present invention reduce the area and power consumption of the ADC in the in-memory computing array by using low bit width quantization convolution results; and by retraining, the neural network model is adapted to the output results of the in-memory computing array, which alleviates the problem of the sharp decline in the accuracy of the neural network model deployed on the in-memory computing chip and improves the robustness of the neural network model. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart of the neural network data processing method provided in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of a convolutional feature map provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram illustrating the error generated by ADC quantization of the in-memory array provided in an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the multiplication, accumulation, and quantization operations of the convolution point format provided in this embodiment of the invention;

[0054] Figure 5 This is a schematic diagram of four-bit data formation provided in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of target output error compensation provided in an embodiment of the present invention;

[0056] Figure 7 This is a schematic diagram of the structure of a neural network data processing device provided in an embodiment of the present invention;

[0057] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0059] The following is combined Figures 1-6 This invention describes a neural network data processing method provided in an embodiment of the invention.

[0060] Figure 1 This is a flowchart of the neural network data processing method provided in an embodiment of the present invention, such as... Figure 1 As shown, the neural network data processing method provided in this embodiment of the invention is applied to a neural network model including at least one convolutional layer, and includes:

[0061] Step 110: Obtain the convolutional feature map to be input to the i-th convolutional layer in the neural network model, i = 1, 2, ..., N, where N is the total number of convolutional layers and N is an integer;

[0062] Specifically, the neural network model in this embodiment of the invention is a neural network model with at least one convolutional layer, also known as a convolutional neural network. It should be understood that this embodiment of the invention does not limit the type of convolution and neural network model. For example, the convolution can be 1D convolution, 2D convolution, 3D convolution, fully connected, transposed convolution, channel-separable convolution, or dilated convolution neural network operator; the neural network model can be a directly connected neural network, a U-shaped neural network, a residual structure network, a recurrent neural network, a multilayer perceptron (MLP), a Transformer network, or an attention mechanism neural network model, etc.

[0063] Convolutional feature maps can be one-dimensional, two-dimensional, or three-dimensional.

[0064] Step 120: Quantize the convolution points in the convolution feature map into multiple four-bit data;

[0065] Specifically, the number of convolution points in the convolutional feature map is multiple, and these multiple convolution points can be obtained through translation. Figure 2 This is a schematic diagram of a convolutional feature map provided in an embodiment of the present invention, such as... Figure 2 As shown, the depth of the convolutional feature map is 16, and the size of a convolution point is 3×3×16. In the diagram, a smallest square in the cross-section represents one unit (also called data or data point). For example, squares 1, 2, 3, 7, 8, 9, 13, 14, and 15, combined with the 3×3×16 units shown in the depth diagram, constitute one convolution point. Each time the map is shifted one unit to the right, squares 2, 3, 4, 8, 9, 10, 14, 15, and 16, combined with the 3×3×16 units shown in the depth diagram, constitute a new convolution point. When the map reaches the rightmost end, it shifts down and returns to the leftmost end, and so on, to obtain multiple convolution points.

[0066] Taking a convolution point as an example, with 3×3×16 units as one convolution point, each convolution point contains 3×3×16 floating-point data. One floating-point data can be quantized into one four-bit data. One convolution point yields 3×3×16 four-bit data. Quantization refers to the technique of using lower-precision data for computation and memory access.

[0067] Step 130: Split each of the four-bit data into four single-bit data;

[0068] For example, data 5 can be represented by 4 bits 0101, which can be broken down into four single-bit data: 0, 1, 0, 1.

[0069] Step 140: Each of the four single-bit data is sequentially input into the storage array, which is used to perform convolution calculations corresponding to the i-th convolutional layer, and obtain the quantized value of the convolution result output by the storage array.

[0070] Specifically, the convolution point is split into four single-bit data points, and these four single-bit data points are input into the in-memory compute array (IMA) for convolution operations. The IMA, also known as an in-memory compute chip or in-memory computing integrated chip, is used to perform convolution calculations for the i-th convolutional layer. After multiplication and accumulation, the IMA quantizes the result using its ADC, shifts the bits, and adds the result to obtain the quantized convolution result.

[0071] The convolution result has a data range of (-144, +144) before quantization. It needs to be quantized using a 9-bit ADC to ensure the correctness of the result. The quantization range of the 9-bit ADC is (-256, +255), but the area and power consumption costs are too high. Figure 3 This is a schematic diagram illustrating the error generated by ADC quantization of the in-memory array provided in an embodiment of the present invention, as shown below. Figure 3 As shown, this embodiment of the invention uses a 4-bit ADC to quantize the convolution result, quantizing the convolution result in the range (-144, +144) to the interval (-8, +7). However, if a single-bit convolution result is directly quantized using a 4-bit ADC during neural network model training, the accuracy of the neural network model cannot be improved. Therefore, the neural network model in this embodiment of the invention is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. That is, the data used in the pre-training process is floating-point data, obtaining a neural network model without ADC quantization (hereinafter referred to as the pre-trained neural network model). The accuracy of the neural network model is guaranteed through the pre-training process.

[0072] Meanwhile, splitting the 4-bit activation value into four single-bit data points and performing convolution on each introduces another problem: the same convolution result may be quantized into different values ​​before each ADC quantization. For example... Figure 3 As shown, the convolution results corresponding to the two single-bit data in cases 1 and 2 are the same after shifting and adding. However, the results of quantizing the convolution results corresponding to the single-bit data by the ADC and then shifting and adding are different. Yet, the algorithm (i.e., the algorithm corresponding to the i-th convolutional layer) always quantizes the same value for this convolution result, thus generating an error. Since the representation range of a 4-bit ADC is (-8, +7), and the error is on the same order of magnitude as this range, the accuracy reduction caused by this error is particularly significant. Therefore, in the neural network model of this embodiment, during retraining, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the in-memory array. This embodiment retrains the pre-trained neural network model so that the target output of the i-th convolutional layer is the quantized value of the convolution result output by the in-memory array, thereby adapting the trained neural network model to the quantized value of the convolution result output by the in-memory array. This ensures that the accuracy of the neural network model after low-bit-width quantization does not significantly decrease compared to the accuracy of the pre-trained neural network model.

[0073] Figure 4 This is a schematic diagram of the multiplication, accumulation, and quantization operations of the convolution point format provided in this embodiment of the invention, as shown below. Figure 4 As shown, during pre-training, the result of convolving 3×3 digits 5 with the convolution kernel is 25, and the quantization value corresponding to the algorithm result is 2. During retraining, the four single-bit data 0, 1, 0, and 1 are multiplied and accumulated with the same convolution kernel respectively, finally obtaining the unquantized convolution result 0505, and the ADC-quantized convolution result output by the in-memory array is 0101. In the pre-training process, the algorithm result 25 corresponding to the i-th convolutional layer is the target output of the pre-trained network model to ensure network accuracy. In the retraining process, the quantized value of the convolution result output by the in-memory array (0101 in this embodiment) is used as the target output, so that the neural network model can adapt to the output of the in-memory array and ensure network accuracy.

[0074] The neural network data processing method provided in this invention reduces the area and power consumption of the ADC in the in-memory computing array by using low bit width quantized convolution results; and by retraining, it adapts the neural network model to the output results of the in-memory computing array, alleviating the problem of the sharp decline in the accuracy of the neural network model deployed on the in-memory computing chip and improving the robustness of the neural network model.

[0075] Optionally, step 120, quantizing the convolution points in the convolutional feature map into multiple four-bit data, includes:

[0076] Step 121: Based on the maximum value in the convolutional feature map, normalize the convolutional feature map to obtain a normalized convolutional feature map.

[0077] For example, taking a convolutional feature map of size 16x32x32 as an example, firstly, find the maximum value among the 16x32x32 data points, and then divide all 16x32x32 data points by this maximum value to obtain a normalized convolutional feature map. The normalized convolutional feature map contains 16x32x32 data points.

[0078] Step 122: Obtain multiple normalized convolution points based on the normalized convolution feature map;

[0079] The method for obtaining convolution points, as described above, will not be repeated here.

[0080] Step 123: Convert the data points in the normalized convolution points into corresponding integer values, wherein the integer values ​​range from [-7, 7].

[0081] Optionally, converting the data points in the normalized convolution points into corresponding integer values ​​includes:

[0082] The data points in the normalized convolution points are numerically expanded and rounded based on a preset factor to obtain the corresponding integer values.

[0083] Specifically, in neural network models, the data in convolution points is usually floating-point data, which remains floating-point data after normalization. The floating-point data is mapped to integer values ​​for conversion to the corresponding four-bit data.

[0084] Optionally, the preset multiplier can be 7x.

[0085] In one possible implementation, the process of quantizing the data points in the convolutional feature map into integers can be represented by a first formula, as shown below:

[0086]

[0087] Where, x in The input represents a single data point from the convolutional points. The input is a floating-point number, x. q，out The output is the integer value corresponding to the input, Q. k (x in (T, K) represents the first mapping rule, where T represents the maximum value in the convolutional feature map, K represents the target number of bits, and round represents the rounding operation. This represents a normalized value.

[0088] Optionally, in this embodiment of the invention, K = 4, and the first formula can be expressed as:

[0089]

[0090] For example, when When the value is any value between 1.5 and 2.4, the integer result is ( The corresponding integer values ​​are all 2.

[0091] T represents the maximum value in the convolutional feature map, which is the value obtained by traversing x. in The maximum value in the range, T's role is normalization, therefore The range is (-1, +1). The range is (-7, 7), so when performing a round operation, you can get an integer in [-7, +7].

[0092] Step 124: Convert each of the integer values ​​into corresponding four-bit data.

[0093] Figure 5 This is a schematic diagram of four-bit data formation provided in an embodiment of the present invention. Figure 5 For example, the size of a convolution point is 3×3×16. A convolution point contains 3×3×16 floating-point data. So, inputting a convolution point will output the corresponding 3×3×16 integer values. An integer value is converted into a four-bit data. A convolution point will output the corresponding 3×3×16 four-bit data.

[0094] The neural network data processing method provided in this embodiment of the invention maps floating-point numbers to integers through a first mapping rule, and then quantizes them into four-bit data. The four-bit data can be split into four single-bit data for convolution operation, thereby reducing the power consumption and area of ​​the ADC in the in-memory array.

[0095] Optionally, the neural network model is obtained by training through the following steps:

[0096] Step 010: Pre-train the initial neural network model to obtain a pre-trained neural network model;

[0097] Step 020: Retrain the pre-trained neural network model. During the retraining process, error compensation is performed on the target output of the convolutional layers in the pre-trained neural network model layer by layer.

[0098] Steps 010 and 020 are described below.

[0099] Step 010: Pre-train the initial neural network model to obtain a pre-trained neural network model.

[0100] Optionally, this step trains a pre-trained neural network model without using ADC quantization, and then retrains the model after adding errors.

[0101] For example, if the size of the input convolutional feature map is (16, 32, 32), then 16×32×32, a total of 16384 four-bit data points can be obtained. First, the four-bit data input to the convolutional layer is split into four single-bit data points. For a 3×3 convolutional kernel, the theoretical maximum value of the single-bit convolution result is 3×3×16, which is 144. Similarly, the minimum value is -144.

[0102] In application, the convolution points in the convolution feature map need to be quantized to a four-bit width. Four-bit width quantization refers to scaling the data in the range of (-144, +144) to (-8, +7). The quantization interval of the four single bits is (-8, +7). After shifting and adding, the final convolution result is in the range of (-64, +56). Data in the range of (-64, +56) can be represented by seven bits.

[0103] During pre-training, the data format is floating-point numbers. However, during application, the data input to the memory array is single-bit data split from 4-bit data. The difference between floating-point data and 4-bit data may affect network accuracy. For example, if the floating-point data is 3.41235464 during training, but the data is quantized to 3 during application, the significant difference between the training and application data will affect network accuracy.

[0104] To ensure accuracy, a regular 2 is used at this time. 4 The system uses floating-point numbers to represent four bits of data. Therefore, four-bit data splitting cannot be performed during pre-training. On the other hand, a regular number of floating-point numbers are needed to represent the final convolution result. During training, it is not possible to quantize specific single-bit data. Instead, the result of performing convolution operations on four-bit data is simulated.

[0105] During training, floating-point data is used to simulate the input of the i-th convolutional layer as four-bit data:

[0106] Optionally, during the pre-training process, the method further includes:

[0107] Obtain the convolutional feature map to be input to the i-th convolutional layer;

[0108] Based on the maximum value in the convolutional feature map to be input, the convolutional feature map to be input is normalized to obtain a normalized convolutional feature map to be input.

[0109] Based on the normalized convolutional feature map to be input, the first convolution point is obtained;

[0110] The data points in the first convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the second convolution point corresponding to the first convolution point. The preset weight is the ratio of the maximum value to the preset multiple.

[0111] Each of the second convolutional points is input into the i-th convolutional layer.

[0112] Specifically, please refer to the above introduction on integer conversion; it will not be repeated here. The second formula represents:

[0113]

[0114] Where, x y,out x represents the integer value corresponding to the data point in the first convolution point. in2 T1 represents the data point in the first convolutional point, T2 represents the maximum value in the convolutional feature map, K represents the target number of bits, and round represents the rounding operation. Represents the normalized value. This indicates the preset weights.

[0115] Optionally, in this embodiment of the invention, K = 4, and the second formula can be expressed as:

[0116]

[0117] T2 represents the maximum value in the convolutional feature map, which is the value obtained by traversing x. in2 The maximum value in the range, T is used for normalization, therefore The range is (-1, +1). The range is (-7, 7), so when performing a round operation, we can obtain integers in the range [-7, +7]. During training, each of these integer values ​​needs to be multiplied by a preset weight to restore the original x. in The range, then x y，out The input is fed into the i-th convolutional layer for convolutional estimation.

[0118] In the neural network data processing method provided in this embodiment of the invention, four-bit data is simulated using floating-point data to avoid x in2 With x y，out The difference is too large, which makes x y，out It can simulate four-bit data to ensure network accuracy.

[0119] In the analog circuitry of the in-memory array, the input consists of four single-bit data points, each of which is convolved. After convolution, a shift and add operation is performed to obtain the final convolution result; Y = x y，out1 ×(-8)+xy，out2 ×4+x y，out3 ×2+x y，out4 The data range of x is in the range of (-8, +7) after ADC quantization, so the range of Y can be obtained as (-56, 64).

[0120] Since the training process cannot break down data into single bits for convolution, but instead directly obtains the final convolution result after convolution, it is necessary to simulate the range of Y during training. That is, during training, floating-point data is used to simulate the result after performing convolution operations on four-bit data.

[0121] Optionally, during the pre-training process, the method further includes:

[0122] Obtain the convolution result output by the i-th convolutional layer;

[0123] Based on the maximum value in the convolution result, the convolution result is normalized to obtain a normalized convolution result;

[0124] Based on the normalized convolution result, multiple third convolution points are obtained;

[0125] The data points in the third convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the fourth convolution point corresponding to the third convolution point. The preset weight is the ratio of the maximum value to the preset multiple.

[0126] Each of the fourth convolutional points is input into the next layer structure of the i-th convolutional layer.

[0127] Specifically, for the data processing of the convolution results, please refer to the above introduction on the data processing of convolution feature maps, which will not be repeated here.

[0128] In the neural network data processing method provided in this embodiment of the invention, the result of convolution operation after simulating four-bit data is avoided from being too different from the convolution result after data processing (second convolution point) and the convolution result without data processing. At the same time, the second convolution point can simulate the result of convolution operation after four-bit data is performed, thus ensuring network accuracy.

[0129] like Figure 4 As shown, Figure 4This demonstrates the specific principles of multiply-accumulate and quantization operations at both the algorithmic and in-memory computation circuit levels. At the algorithmic level, convolving 3×3 digits of 5 with a convolution kernel results in 25. After quantization, the output convolution result is 2. In the in-memory computation circuit, four bits (0101) represent the digit 5. Multiplying and accumulating these bits with the same 3×3 convolution kernel yields unquantized (0505) and ADC-quantized (0101) convolution results, respectively. After shifting and adding, the unquantized and ADC-quantized convolution results are 25 and 5, respectively. Clearly, whether at the algorithmic or in-memory computation circuit level, the convolution result of the 3×3 convolution points after multiply-accumulate operations but before quantization is the same—both are 25. The error arises because the actual quantization result of the ADC deviates from the ideal quantization result of the algorithm. Therefore, when the neural network model trained using the algorithmic quantization result performs forward inference, the accuracy drops sharply after in-memory computation modeling, from 82.6% to 10%, indicating complete misclassification.

[0130] In summary, the first optimal model is obtained by training the neural network model using the ideal quantization result, which serves as the basis for subsequent retraining.

[0131] Step 020: Retrain the pre-trained neural network model. During the retraining process, error compensation is performed on the target output of the convolutional layers in the pre-trained neural network model layer by layer.

[0132] Optionally, the error value is obtained through the following steps:

[0133] The convolution points used during training are input into the algorithm corresponding to the i-th convolutional layer to obtain the algorithm result corresponding to the i-th convolutional layer;

[0134] The convolution points used during training are quantized into four-bit data, and the four-bit data is split into four single-bit data. The four single-bit data are then input into the in-memory array model to obtain the quantization result output by the in-memory array model. The in-memory array model is pre-built.

[0135] The algorithm result corresponding to the i-th convolutional layer is quantized to obtain the quantization result corresponding to the algorithm result corresponding to the i-th convolutional layer.

[0136] The error value is obtained by subtracting the quantization result output by the in-memory array model from the quantization result corresponding to the algorithm result of the i-th convolutional layer.

[0137] Specifically, during training, the data input to the i-th convolutional layer (i.e., the convolutional points used during training) consists of 16 equally spaced floating-point data points within the interval (-1, +1). The convolutional points used during training are input into the algorithm corresponding to the i-th convolutional layer to obtain the algorithm result corresponding to the i-th convolutional layer.

[0138] Optionally, for quantizing the convolution points used during training into four-bit data and splitting the four-bit data into four single-bit data, the four-bit data can be recovered by multiplying the (-1, +1) data (the convolution points used during training) by 7 and taking the integer part. Then, the four-bit data is split into single bits.

[0139] The in-memory computing array model is obtained by modeling the functionality of the in-memory computing array. Optionally, the in-memory computing array can be modeled using the Python language to obtain the in-memory computing array model.

[0140] The four single-bit data are respectively input into the in-memory array model, and the in-memory array model performs the convolution operation corresponding to the i-th convolutional layer to obtain the quantization result output by the in-memory array model.

[0141] The algorithm result corresponding to the i-th convolutional layer is quantized to obtain the quantization result corresponding to the algorithm result.

[0142] Figure 6 This is a schematic diagram of target output error compensation provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the quantization result output by the in-memory array model is subtracted from the quantization result corresponding to the algorithm result of the i-th convolutional layer to obtain the error value between the in-memory array and the ideal result of the algorithm.

[0143] It should be understood that, in the embodiments of the present invention, the order of "inputting the convolution points used during training into the algorithm corresponding to the i-th convolutional layer" and "quantizing the convolution points used during training into four-bit data, splitting the four-bit data into four single-bit data, and inputting the four single-bit data into the in-memory array model" is not limited.

[0144] The error compensation for the target output includes:

[0145] The quantization result corresponding to the algorithm result of the i-th convolutional layer is added to the pre-obtained error value to obtain the target output corresponding to the i-th convolutional layer. The error value is the difference between the quantization result output by the storage array and the quantization result corresponding to the algorithm result of the i-th convolutional layer.

[0146] like Figure 6As shown, the quantization result corresponding to the algorithm result of the i-th convolutional layer is added to the pre-obtained error value and used as the target output of the i-th convolutional layer. In this way, the target output of the convolutional layer during retraining matches the quantization result of the actual output of the storage array.

[0147] For error compensation of the target output of the convolutional layer in the pre-trained neural network model, it means that error compensation is performed on only one convolutional layer at a time, and after the current convolutional layer is compensated, the next convolutional layer is compensated.

[0148] It should be understood that error values ​​are calculated separately for each convolutional layer, and error values ​​are calculated under various training input conditions, so that the neural network model can obtain compensation for various convolutional points and improve network accuracy.

[0149] The neural network data processing method provided in this invention uses a pre-trained neural network model as a foundation, adding errors layer by layer until an optimal final neural network model is obtained. This training method can resolve the errors caused by low-bit-width quantization in in-memory arrays while maintaining a certain level of accuracy in the neural network model.

[0150] The neural network data processing method provided by this invention uses simulated in-memory array modeling and algorithmic quantization to calculate the relative error. It then retrains the neural network model by adding this error layer by layer. This ensures that the accuracy of the low-bit-width quantized neural network model does not significantly decrease compared to the original model without added error. First, a normal pre-trained neural network model is trained, but the convolution results are not quantized. Afterward, each layer calculates and adds error, and retraining is performed based on the previous optimal neural network model. Quantization-aware training uses floating-point data during training, and inference after training uses a floating-point to integer mapping. During this process, the error also exhibits a mapping relationship.

[0151] In one possible implementation, the neural network model is used in the field of image processing, such as image annotation, image classification, or image recognition, and the data processing method of the neural network model includes:

[0152] Image data to be processed;

[0153] The image data to be processed is input into the neural network model;

[0154] For the i-th convolutional layer in the neural network model:

[0155] Obtain the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, where N is the total number of convolutional layers and N is an integer;

[0156] The convolution points in the convolution feature map are quantized into multiple four-bit data.

[0157] Each of the four-bit data is split into four single-bit data;

[0158] Each of the four single-bit data is sequentially input into the storage array, which is used to perform convolution calculations corresponding to the i-th convolutional layer, and obtain the quantized value of the convolution result output by the storage array.

[0159] The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array.

[0160] Optionally, the neural network model can be a residual neural network (ResNet).

[0161] It should be understood that neural network models maintain the spatial structure of the image data to be processed during the learning process. The convolutional feature map and the original image have a spatial correspondence, and the specific corresponding position and size can be measured by the receptive field.

[0162] For data processing of neural network model structures other than convolutional layers, please refer to relevant technologies; details will not be elaborated here.

[0163] The neural network data processing method provided in this invention reduces the area and power consumption of the ADC in the in-memory computing array by using low bit width quantization of the convolution results; and by retraining, it adapts the neural network model to the output results of the in-memory computing array, alleviating the problem of the sharp decline in the accuracy of the neural network model deployed on the in-memory computing chip and improving the robustness of the neural network model; furthermore, the convolution feature map can reflect image features, and by processing the convolution feature map, the quantization of the neural network model based on in-memory computing can be realized to make it suitable for deep neural network models for image processing.

[0164] In one embodiment, the neural network data processing method provided by this invention quantizes a neural network model deployed in an in-memory computing circuit architecture. Before performing convolution operations in the in-memory computing circuit architecture, the convolution feature map needs to be decomposed into the form of convolution points, that is, multiply-accumulate operations are performed after each sliding of the convolution kernel. For multi-channel convolution operations, low bit width quantization often reduces accuracy.

[0165] like Figure 2As shown, this invention quantizes the convolution points of the convolutional layer into four bits, and then splits each convolution point into four single-bit data points for convolution operations. After multiplication and accumulation, the data is quantized by an ADC, shifted, and added to obtain the quantized convolution result. This method of using in-memory computation to complete the convolution operation introduces quantization errors, which are unavoidable in hardware design. This embodiment of the invention compensates for the network before deploying the neural network model to the in-memory computing array to maintain the inference accuracy of the neural network model.

[0166] The following describes a method for retraining a neural network model with progressively added errors for an in-memory array architecture, provided by an embodiment of the present invention:

[0167] This invention proposes a method for compensating for errors generated during in-memory computation at the algorithm level: a retraining method that adds errors layer by layer. This method of adding errors layer by layer for in-memory computation is applied during neural network model training, such as... Figure 3 As shown, the same single-bit convolution result may have multiple different shift-add results after single-bit ADC quantization in different layers. First, an optimal neural network model is trained without considering in-memory computation error. Based on this model, the error between the two different quantization results of the real-time algorithm and array modeling is calculated and added to the quantization result of the algorithm. The neural network is then retrained to obtain the optimal neural network, and so on.

[0168] This invention first models the four-bit splitting, multiplication-accumulation, and shift-add operations in the in-memory computing circuit using software, enabling the neural network to calculate the aforementioned errors during training. The calculated errors are then added to the algorithm results and converted into floating-point numbers to ensure the integer mapping relationship during the final forward inference.

[0169] The neural network data processing method provided in this invention alleviates the problem of a sharp decline in the accuracy of neural networks deployed on in-memory computing chips by adding errors layer by layer for retraining, thereby improving the robustness of the neural network model. While ensuring accuracy, it uses low bit width quantization of convolution results, thereby reducing the area and power consumption of the ADC in the in-memory computing chip.

[0170] The neural network data processing apparatus provided by the present invention is described below. The neural network data processing apparatus described below can be referred to in correspondence with the neural network data processing method described above.

[0171] Figure 7 This is a schematic diagram of a neural network data processing device provided in an embodiment of the present invention. The neural network data processing device provided in this embodiment of the present invention can be applied to neural networks including at least one convolutional layer, such as... Figure 7 As shown, the neural network data processing apparatus provided in this embodiment of the invention includes:

[0172] The acquisition unit 710 is used to acquire the convolutional feature map to be input to the i-th convolutional layer in the neural network, where i = 1, 2, ..., N, N is the total number of convolutional layers, and N is an integer.

[0173] The first splitting unit 720 is used to quantize the convolution points in the convolution feature map into multiple four-bit data;

[0174] The second splitting unit 730 is used to split each of the four-bit data into four single-bit data;

[0175] The transmission unit 740 is used to sequentially input each of the four single-bit data into the storage array, the storage array is used to perform convolution calculations corresponding to the i-th convolutional layer, and obtain the quantized value of the convolution result output by the storage array;

[0176] The neural network is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array.

[0177] Optionally, the first splitting unit 720 is used to quantize the convolution points in the convolution feature map into multiple four-bit data, including:

[0178] The first splitting unit 720 is used to normalize the convolutional feature map based on the maximum value in the convolutional feature map to obtain a normalized convolutional feature map.

[0179] The first splitting unit 720 is used to obtain multiple normalized convolution points based on the normalized convolution feature map;

[0180] The first splitting unit 720 is used to convert the data points in the normalized convolution points into corresponding integer values, wherein the integer values ​​range from [-7, 7].

[0181] The first splitting unit 720 is used to convert each of the integer values ​​into corresponding four-bit data.

[0182] Optionally, the first splitting unit 720 is used to convert the data points in the normalized convolution points into corresponding integer values, including:

[0183] The first splitting unit 720 is used to numerically expand and round the data points in the normalized convolution points based on a preset factor to obtain the corresponding integer values.

[0184] Optionally, the device further includes a training unit;

[0185] The training unit is used to pre-train the initial neural network to obtain a pre-trained neural network;

[0186] The training unit is used to retrain the pre-trained neural network, and during the retraining process, error compensation is performed on the target output of the convolutional layer in the pre-trained neural network layer by layer.

[0187] The training unit, used for target output error compensation, includes:

[0188] The training unit is used to add the quantization result corresponding to the algorithm result of the i-th convolutional layer to the pre-obtained error value, and use it as the target output corresponding to the i-th convolutional layer. The error value is the difference between the quantization result output by the in-memory array after completing the convolution operation and the quantization result corresponding to the algorithm result of the i-th convolutional layer.

[0189] Optionally, the training unit is used to input the convolution points used during training into the algorithm corresponding to the i-th convolutional layer to obtain the algorithm result corresponding to the i-th convolutional layer;

[0190] The training unit is used to quantize the convolution points used during training into four-bit data, split the four-bit data into four single-bit data, input the four single-bit data into the in-memory array model, and obtain the quantization result output by the in-memory array model. The in-memory array model is pre-built.

[0191] The training unit is used to quantize the algorithm result corresponding to the i-th convolutional layer to obtain the quantization result corresponding to the algorithm result corresponding to the i-th convolutional layer.

[0192] The training unit is used to subtract the quantization result output by the in-memory array model from the quantization result corresponding to the algorithm result of the i-th convolutional layer to obtain the error value.

[0193] Optionally, during the pre-training process, the training unit is further configured to:

[0194] Obtain the convolutional feature map to be input to the i-th convolutional layer;

[0195] Based on the maximum value in the convolutional feature map to be input, the convolutional feature map to be input is normalized to obtain a normalized convolutional feature map to be input.

[0196] Based on the normalized convolutional feature map to be input, the first convolution point is obtained;

[0197] The data points in the first convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the second convolution point corresponding to the first convolution point. The preset weight is the ratio of the maximum value to the preset multiple.

[0198] Each of the second convolutional points is input into the i-th convolutional layer.

[0199] Optionally, during the pre-training process, the training unit is further configured to:

[0200] Obtain the convolution result output by the i-th convolutional layer;

[0201] Based on the maximum value in the convolution result, the convolution result is normalized to obtain a normalized convolution result;

[0202] Based on the normalized convolution result, multiple third convolution points are obtained;

[0203] The data points in the third convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the fourth convolution point corresponding to the third convolution point. The preset weight is the ratio of the maximum value to the preset multiple.

[0204] Each of the fourth convolutional points is input into the next layer structure of the i-th convolutional layer.

[0205] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions from the memory 730 to execute a neural network data processing method applied to a neural network including at least one convolutional layer, including:

[0206] Obtain the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, where N is the total number of convolutional layers and N is an integer;

[0207] The convolution points in the convolution feature map are quantized into multiple four-bit data.

[0208] Each of the four-bit data is split into four single-bit data;

[0209] Each of the four single-bit data is sequentially input into the storage array, which is used to perform convolution calculations corresponding to the i-th convolutional layer, and obtain the quantized value of the convolution result output by the storage array.

[0210] The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array.

[0211] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer being able to execute a neural network data processing method provided by the above methods, applied to a neural network including at least one convolutional layer, comprising:

[0213] Obtain the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, where N is the total number of convolutional layers and N is an integer;

[0214] The convolution points in the convolution feature map are quantized into multiple four-bit data.

[0215] Each of the four-bit data is split into four single-bit data;

[0216] Each of the four single-bit data is sequentially input into the storage array, which is used to perform convolution calculations corresponding to the i-th convolutional layer, and obtain the quantized value of the convolution result output by the storage array.

[0217] The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array.

[0218] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a neural network data processing method provided by the methods described above, applied to a neural network including at least one convolutional layer, comprising:

[0219] Obtain the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, where N is the total number of convolutional layers and N is an integer;

[0220] The convolution points in the convolution feature map are quantized into multiple four-bit data.

[0221] Each of the four-bit data is split into four single-bit data;

[0222] Each of the four single-bit data is sequentially input into the storage array, which is used to perform convolution calculations corresponding to the i-th convolutional layer, and obtain the quantized value of the convolution result output by the storage array.

[0223] The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array.

[0224] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0225] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0226] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A neural network data processing method, characterized in that, Applied to neural network models that include at least one convolutional layer, including: Obtain the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, and N is the total number of convolutional layers, where N is an integer; the neural network model is deployed on an in-memory computing chip. The convolution points in the convolution feature map are quantized into multiple four-bit data. Each of the four-bit data is split into four single-bit data; Each of the four single-bit data is sequentially input into the in-memory computing array, which is used to perform convolution calculations corresponding to the i-th convolutional layer to obtain the quantized value of the convolution result output by the in-memory computing array; the in-memory computing array is an in-memory computing chip. The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array. The neural network model was obtained through training the following steps: The initial neural network model is pre-trained to obtain a pre-trained neural network model; The pre-trained neural network model is retrained, and during the retraining process, error compensation is performed on the target output of the convolutional layer in the pre-trained neural network model layer by layer. Error compensation for the target output includes: The quantization result corresponding to the algorithm result of the i-th convolutional layer is added to the pre-obtained error value to obtain the target output corresponding to the i-th convolutional layer. The error value is the difference between the quantization result output by the in-memory array after completing the convolution operation and the quantization result corresponding to the algorithm result of the i-th convolutional layer.

2. The neural network data processing method according to claim 1, characterized in that, The step of quantizing the convolution points in the convolutional feature map into multiple four-bit data includes: Based on the maximum value in the convolutional feature map, the convolutional feature map is normalized to obtain a normalized convolutional feature map. Multiple normalized convolution points are obtained based on the normalized convolution feature map; The data points in the normalized convolution points are converted into corresponding integer values, and the integer values ​​range from [-7, 7]. Each of the integer values ​​is converted into its corresponding four-bit data.

3. The neural network data processing method according to claim 2, characterized in that, The step of converting the data points in the normalized convolution points into corresponding integer values ​​includes: The data points in the normalized convolution points are numerically expanded and rounded based on a preset factor to obtain the corresponding integer values.

4. The neural network data processing method according to claim 1, characterized in that, The error value is obtained through the following steps: The convolution points used during training are input into the algorithm corresponding to the i-th convolutional layer to obtain the algorithm result corresponding to the i-th convolutional layer; The convolution points used during training are quantized into corresponding four-bit data, and the four-bit data is split into four single-bit data. The four single-bit data are input into the in-memory array model to obtain the quantization result output by the in-memory array model after completing the convolution operation. The in-memory array model is pre-built. The algorithm result corresponding to the i-th convolutional layer is quantized to obtain the quantization result corresponding to the algorithm result corresponding to the i-th convolutional layer. The error value is obtained by subtracting the quantization result output by the in-memory array model from the quantization result corresponding to the algorithm result of the i-th convolutional layer.

5. The neural network data processing method according to claim 3, characterized in that, During the pre-training process, the method further includes: Obtain the convolutional feature map to be input to the i-th convolutional layer; Based on the maximum value in the convolutional feature map to be input, the convolutional feature map to be input is normalized to obtain a normalized convolutional feature map to be input. Based on the normalized convolutional feature map to be input, the first convolution point is obtained; The data points in the first convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the second convolution point corresponding to the first convolution point. The preset weight is the ratio of the maximum value to the preset multiple. Each of the second convolutional points is input into the i-th convolutional layer.

6. The neural network data processing method according to claim 3, characterized in that, During the pre-training process, the method further includes: Obtain the convolution result output by the i-th convolutional layer; Based on the maximum value in the convolution result, the convolution result is normalized to obtain a normalized convolution result; Based on the normalized convolution result, multiple third convolution points are obtained; The data points in the third convolution point are converted into corresponding integer values, and each integer value is multiplied by a preset weight to obtain the fourth convolution point corresponding to the third convolution point. The preset weight is the ratio of the maximum value to the preset multiple. Each of the fourth convolutional points is input into the next layer structure of the i-th convolutional layer.

7. A neural network data processing device, characterized in that, Applied to neural network models that include at least one convolutional layer, including: The acquisition unit is used to acquire the convolutional feature map to be input to the i-th convolutional layer in the neural network model, where i = 1, 2, ..., N, and N is the total number of convolutional layers, where N is an integer; the neural network model is deployed on an in-memory computing chip. The first splitting unit is used to quantize the convolution points in the convolution feature map into multiple four-bit data; The second splitting unit is used to split each of the four-bit data into four single-bit data; A transmission unit is used to sequentially input each of the four single-bit data into the in-memory computing array, which is used to perform convolution calculations corresponding to the i-th convolutional layer to obtain the quantized value of the convolution result output by the in-memory computing array; the in-memory computing array is an in-memory computing chip. The neural network model is obtained based on pre-training and retraining. During the pre-training process, the target output of the i-th convolutional layer is the algorithm result corresponding to the i-th convolutional layer. During the retraining process, the target output of the i-th convolutional layer is the quantized value of the convolution result output by the storage array. The neural network model was obtained through training the following steps: The initial neural network model is pre-trained to obtain a pre-trained neural network model; The pre-trained neural network model is retrained, and during the retraining process, error compensation is performed on the target output of the convolutional layer in the pre-trained neural network model layer by layer. Error compensation for the target output includes: The quantization result corresponding to the algorithm result of the i-th convolutional layer is added to the pre-obtained error value to obtain the target output corresponding to the i-th convolutional layer. The error value is the difference between the quantization result output by the in-memory array after completing the convolution operation and the quantization result corresponding to the algorithm result of the i-th convolutional layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the neural network data processing method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the neural network data processing method as described in any one of claims 1 to 6.

Images