Neural network computing method and neural network computing device

Abstract

A neural network computing method, device and system. According to the method, after receiving first input data, a first neural network calculation can be performed on the first input data according to first weights stored in a computing array, wherein the first weights are weights of a first neural network layer in a neural network system, the first weights include a plurality of weight values, each weight value in the plurality of weight values is represented by N-bit numbers with the same weight, and N is greater than or equal to 2. The above method can reduce the error of the neural network calculation result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a neural network computing method and a neural network computing device. Background Technology

[0002] Neural networks are tools for realizing artificial intelligence, characterized by high computational demands and memory-intensive processing of input data. One method to improve the processing efficiency of neural networks is to deploy them using an in-memory computing architecture. This method leverages the characteristic that the weights of neural networks remain constant during computation, pre-writing the neural network weights into the computing storage medium, thus simultaneously storing and calculating the neural network weights. This reduces the time and energy consumption associated with data exchange and computation.

[0003] In in-memory computing architectures, the crossbar is the core module for implementing storage and computation. A crossbar is typically a multiply-accumulate array built with non-volatile storage media, containing several rows and columns. Rows can also be called word lines, and columns can be called bit lines. The intersections of rows and columns are the computing units, usually composed of non-volatile memory (NVM) devices. During neural network computation, weight values ​​are pre-stored in the NVM devices in the form of electrical conductance. When input data enters the crossbar as voltage, the voltage flows through the corresponding conductance to form currents that converge in the same column. The sum of these currents can be used to represent the accumulated result of the product of the input data and the neural network weights.

[0004] The weights of a neural network typically need to be stored in an NVM device after being encoded in binary. The NVM device uses different conductance values ​​to represent different weights. However, due to imperfections in the manufacturing process of NVM devices, the conductance values ​​cannot accurately represent the weights, leading to errors in the current output by the cross-array, which in turn causes errors in the neural network calculation results output by the cross-array. Given that the manufacturing process cannot be improved in the short term, reducing the errors in the neural network calculation results is a problem that needs to be solved. Summary of the Invention

[0005] This application provides a neural network calculation method that can reduce the error of neural network calculation results when the manufacturing process cannot be improved temporarily.

[0006] In a first aspect, a neural network computation method is provided, comprising: receiving first input data; performing a first neural network computation on the first input data according to a first weight stored in a computation array, wherein the first weight is the weight of a first neural network layer in a neural network system, the first weight includes a plurality of weight values, and each of the plurality of weight values ​​is represented by an N-digit number with the same weight, where N is greater than or equal to 2.

[0007] In weight values ​​obtained using non-uniform encoding (such as binary encoding), the weights of each significant bit are different. When there are errors in the storage units of the computation array, the errors of the more significant bits (such as the most significant bit) will be amplified, causing the actual weight value to deviate significantly from the theoretical weight value. In the neural network computation method provided in this embodiment of the invention, the computation array stores weight values ​​obtained using uniform encoding. In weight values ​​obtained using uniform encoding, the weights of each significant bit are the same, which is equivalent to the absence of most significant and least significant bits. Therefore, even if there are device errors in the storage units of the computation array, the errors of each significant bit will not be amplified. Therefore, the above-described neural network computation method can reduce the error of the neural network computation result.

[0008] Optionally, the plurality of weight values ​​includes a first weight value, and the N-digit number representing the first weight value includes a first value and a second value, wherein the first value is greater than the second value, the first value is stored in a first memory unit of the computing array, and the second value is stored in a second memory unit of the computing array, wherein the device error of the first memory unit is less than the device error of the second memory unit. In practical applications, the memory units in the computing array may include novel non-volatile memories such as resistive random-access memory (ReRAM), which has the advantage of integrating storage and computing.

[0009] In weight values ​​obtained using non-uniform encoding methods (such as binary encoding), the fixed relative positions of the significant bits may force higher-weight significant bits (such as the most significant bit) to be stored in memory units with larger errors. This can cause the actual weight values ​​to deviate significantly from the theoretical weight values, ultimately leading to larger errors in the neural network calculation results. In contrast, in weight values ​​obtained using uniform encoding methods, since the weights of each significant bit are the same, their relative positions can be flexibly adjusted. Appropriate values ​​can be stored in memory units with larger errors, thus reducing the negative impact of memory unit errors on the actual weight values.

[0010] Optionally, the N bits used to represent the first weight value are stored in the N storage units of the computing array, and the method further includes: when performing neural network calculation on the first input data based on the first weight value, inputting the output results of the N storage units storing the first weight value into an accumulator for accumulation calculation; when the accumulation number of the accumulator reaches N, controlling the accumulator to output the calculation result.

[0011] In uniformly encoded weight values, each effective bit has the same weight. Neural network chips can obtain the cumulative output result by adding the output results of each column of the computation array storing uniformly encoded weight values, without needing to shift the output results of each column, thus reducing the workload of the neural network chip.

[0012] Optionally, it further includes: receiving second input data; performing a second neural network calculation on the second input data according to the second weights stored in the computing array, wherein the second weights are the weights of the second neural network layer in the neural network system, the second weights include multiple weight values, each of the multiple weight values ​​is represented by M bits with the same weight, M is greater than or equal to 2, and M is not equal to N.

[0013] The second weight has a different effective bit length than the first weight, indicating that their precisions differ. In a uniformly encoded weight value, each effective bit has the same weight. The neural network chip can obtain the accumulated output by adding the outputs of each column of the computation array storing the uniformly encoded weight values, without needing to shift the outputs of each column. Therefore, uniformly encoded weight values ​​of different lengths can be used in different neural network layers, saving hardware overhead while meeting precision requirements.

[0014] Optionally, the first weight is obtained by encoding the first initial weight according to a uniform coding method, wherein the uniform coding method includes multi-level cell (MLC) uniform coding.

[0015] Since each MLC memory cell can represent multiple values, sampling MLC memory cells for uniform encoding can reduce the number of memory cells used compared to using single-level cell (SLC) memory cells for uniform encoding.

[0016] In a second aspect, a neural network computing device is provided, comprising: a receiving port for receiving first input data; and a computing array for performing a first neural network calculation on the first input data based on first weights stored in the computing array, wherein the first weights are weights of a first neural network layer in a neural network system, the first weights include multiple weight values, and each of the multiple weight values ​​is represented by an N-digit number with the same weight, where N is greater than or equal to 2.

[0017] The aforementioned neural network computing device can be a neural network chip or a module within a neural network chip. In weight values ​​obtained using non-uniform encoding (such as binary encoding), the weights of each significant bit are different. When there are errors in the storage units of the computing array, the error of the more significant bit (such as the most significant bit) will be amplified, causing the actual weight value to deviate significantly from the theoretical weight value. In the neural network computing device provided in this embodiment of the invention, the computing array stores weight values ​​obtained using a uniform encoding method. In weight values ​​obtained using uniform encoding, the weights of each significant bit are the same, equivalent to the absence of most significant and least significant bits. Therefore, even if there are device errors in the storage units of the computing array, the errors of each significant bit will not be amplified. Thus, the aforementioned neural network computing method can reduce the error of the neural network computing results. Optionally, the plurality of weight values ​​includes a first weight value, and the N-digit number representing the first weight value includes a first value and a second value. The first value is greater than the second value. The first value is stored in a first storage unit of the computing array, and the second value is stored in a second storage unit of the computing array. The device error of the first storage unit is less than the device error of the second storage unit.

[0018] In practical applications, in neural network computing devices, the in-memory units in the computing array can have new types of non-volatile memories such as ReRAM, which integrates storage and computing.

[0019] In weight values ​​obtained using non-uniform encoding methods (such as binary encoding), the fixed relative positions of the significant bits may force higher-weight significant bits (such as the most significant bit) to be stored in memory units with larger errors. This can cause the actual weight values ​​to deviate significantly from the theoretical weight values, ultimately leading to larger errors in the neural network calculation results. In contrast, in weight values ​​obtained using uniform encoding methods, since the weights of each significant bit are the same, their relative positions can be flexibly adjusted. Appropriate values ​​can be stored in memory units with larger errors, thus reducing the negative impact of memory unit errors on the actual weight values.

[0020] Optionally, the N bits representing the first weight value are stored in the N memory units of the computing array. The chip further includes an accumulator, which is used to: input the output results of the N memory units storing the first weight value into the accumulator for accumulation calculation when performing neural network calculation on the first input data based on the first weight value; and control the accumulator to output the calculation result when the accumulation number of the accumulator reaches N.

[0021] In uniformly encoded weight values, each effective bit has the same weight. Neural network chips can obtain the cumulative output result by adding the output results of each column of the computation array storing uniformly encoded weight values, without needing to shift the output results of each column, thus reducing the workload of the neural network chip.

[0022] Optionally, the device further includes an analog-to-digital converter (ADC), which is connected to the computing array and the accumulator respectively, for converting the analog output results of the N memory units into digital output results, and sending the digital output results to the accumulator for accumulation calculation.

[0023] Optionally, the receiving port is further configured to: receive second input data; the computing array is further configured to: perform a second neural network calculation on the second input data according to the second weights stored in the computing array, wherein the second weights are the weights of the second neural network layer in the neural network system, the second weights include multiple weight values, each of the multiple weight values ​​is represented by M bits with the same weight, M is greater than or equal to 2, and M is not equal to N.

[0024] The second weight has a different effective bit length than the first weight, indicating that their precisions differ. In a uniformly encoded weight value, each effective bit has the same weight. The neural network chip can obtain the accumulated output by adding the outputs of each column of the computation array storing the uniformly encoded weight values, without needing to shift the outputs of each column. Therefore, uniformly encoded weight values ​​of different lengths can be used in different neural network layers, saving hardware overhead while meeting precision requirements.

[0025] Optionally, the first weight is obtained by encoding the first initial weight according to a uniform coding method, wherein the uniform coding method includes MLC uniform coding.

[0026] Since each MLC memory cell can represent multiple values, sampling MLC memory cells for uniform encoding can reduce the number of memory cells used compared to using SLC memory cells for uniform encoding.

[0027] Thirdly, a neural network computing device is provided, including a unit for performing any of the methods in the first aspect.

[0028] Fourthly, a neural network computing system is provided, including a processor and a neural network chip as described in any one of the second aspects.

[0029] Fifthly, a computer program product is provided, the computer program product comprising: computer program code, which, when run by a device for training an intelligent agent, causes the device to perform any of the methods in the first aspect.

[0030] In a sixth aspect, a computer-readable medium is provided, the computer-readable medium storing program code, the program code including instructions for performing any of the methods in the first aspect. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of a multilayer neural network provided in this application;

[0032] Figure 2 This is a schematic diagram of a cross array provided in this application;

[0033] Figure 3 This is a schematic diagram of a neural network calculation method provided in this application;

[0034] Figure 4 This is a schematic diagram of a uniform coding method provided in this application;

[0035] Figure 5 This is a schematic diagram of another uniform coding method provided in this application;

[0036] Figure 6 This is a schematic diagram of neural network computation based on MLC NVM device and uniform coding provided in this application;

[0037] Figure 7 This is a schematic diagram of a priority mapping algorithm provided in this application;

[0038] Figure 8 This is a schematic diagram of an accumulator provided in this application;

[0039] Figure 9 This is a schematic diagram of a multi-precision configuration of a neural network layer using an accumulator, as provided in this application.

[0040] Figure 10 This is a schematic diagram of a neural network computing device provided in this application;

[0041] Figure 11This is a schematic diagram of a neural network computing system provided in this application. Detailed Implementation

[0042] To facilitate understanding of the technical solution of this application, the concepts involved in this application will be briefly introduced first.

[0043] Artificial neural networks (ANNs), also known simply as neural networks (NNs) or neural network-like systems, are mathematical or computational models in machine learning and cognitive science that mimic the structure and function of biological neural networks (the central nervous system of animals, especially the brain) to estimate or approximate functions. Artificial neural networks can include convolutional neural networks (CNNs), deep neural networks (DNNs), multilayer perceptrons (MLPs), and other types of neural networks.

[0044] A neural network can include multiple neural network layers, which can include convolutional layers, pooling layers, fully connected layers, etc. Figure 1 This shows a portion of a neural network layer. For example... Figure 1 As shown, the neural network 100 may include a first layer 102, a second layer 104, a third layer 106, a fourth layer 108, a fifth layer 110, and so on up to an nth layer 112. The first layer 102 can perform convolution operations; the second layer 104 can perform pooling operations on the output data of the first layer 102; the third layer 106 can perform convolution operations on the output data of the second layer 104; the fourth layer 108 can perform convolution operations on the output of the third layer 106; and the fifth layer 110 can perform summation operations on the output data of the second layer 104 and the output data of the fourth layer 108, and so on. It is understood that... Figure 1 This is just a simple example and explanation of neural network layers, and does not restrict the specific operations of each neural network layer. For example, the fourth layer 108 can also be a pooling operation, and the fifth layer 110 can also be a convolution operation or a pooling operation, or other neural network operations.

[0045] Below is a brief introduction to the method of performing neural network computation using cross arrays.

[0046] The following shows a neural network weight applicable to this application, which is a j-row k-column matrix, where each element represents a weight value.

[0047]

[0048] Neural network weights can be configured in a cross array before neural network computation. For example, each element in a matrix is ​​configured in a corresponding storage unit in the cross array, thus enabling multiplication and addition operations between the input data and the neural network weights through the cross array.

[0049] Cross-arrays are typically multiply-accumulate arrays constructed from non-volatile storage media such as resistive random-access memory (ReRAM). The structure of a cross-array is as follows: Figure 2 As shown. A cross array can include multiple storage units, such as G. 1,1 G 2,1 The memory cells are located at the intersection of rows and columns. If a cross array includes 1000 rows and 1000 columns, then the number of memory cells in the cross array is one million. In this application, the weight value W can be... 0,0 Configure to Figure 2 G 1,1 In the middle, the weight value W 1,0 Configure to Figure 2 G 2,1 Medium, etc. Each weight value corresponds to a storage unit, and each weight value is stored in the storage unit in the form of conductivity. During neural network computation, input data is transmitted through word lines of a cross-array (such as...). Figure 2 The input port 204 shown is used to input the cross array. Input data can be represented by voltages (such as V1, V2, V3, and V4), allowing the input data to be multiplied by the weight values ​​stored in the memory unit. The resulting calculation is expressed as current from the output of each column of the cross array (e.g., ...). Figure 2 The output port 206 shown is the output port.

[0050] The current value of each column output can be calculated using the formula I = GV. For example, the current value of the first column output is denoted as I1, and I1 = V1 * G. 1,1 +V2*G 2,1 +V3*G 3,1 +V4*G 4,1 .

[0051] Because the number of conductance values ​​of the NVM devices constituting the memory computing unit is limited, the weight values ​​of the neural network need to be stored in multiple memory computing units in a row. Taking the binary weight value 10 as an example, the binary encoding result of this weight value is 1010. If each NVM device has two conductance values, then this weight value needs to be stored in four NVM devices. Here, the weight value 10 can be called the initial weight value, and the weight value 1010 can be called the encoded weight value.

[0052] Due to manufacturing limitations, the conductivity values ​​of NVM devices may contain errors, leading to errors in the stored weight values. For example, 1, 0, 1, 0 are the ideal values ​​represented by the four NVM devices storing the encoded weight values. In a non-ideal situation, the values ​​represented by these four NVM devices might be 1.1, 0.1, 0.9, and 0.1. In this case, the actual weight value represented by these four NVM devices would be 11.1 (1.1*8+0.1*4+0.9*2+0.1*1).

[0053] In the example above, although the error of the value 1.1 is only 0.1, this value is the most significant bit (MSB) of the binary encoding result, which actually represents the value 8.8, resulting in an error of 0.8. Therefore, the error of the NVM device storing the value 1.1 is amplified by a factor of 8. The rightmost value 0.1 is the least significant bit (LSB) of the binary encoding result, and the error of the NVM device is the actual error of this significant bit. It can be seen that, with the same NVM device error, the higher the number of significant bits in each value of the encoded weight, the larger the actual error of the encoded weight, and the larger the error of the neural network calculation result; conversely, the lower the number of significant bits in each value of the encoded weight, the smaller the actual error of the encoded weight, and the smaller the error of the neural network calculation result.

[0054] Based on this, this application provides a neural network calculation method that can reduce the error of neural network calculation results when the manufacturing process cannot be improved in the short term.

[0055] like Figure 3 As shown, method 300 can be executed by a neural network chip containing a computing array, and method 300 includes:

[0056] S310 receives the first input data.

[0057] The first input data can be data from an image recognition scenario, or data from a voice wake-up, voiceprint recognition, or other scenarios. This application does not limit the specific content of the first input data.

[0058] The first input data can be data processed by a digital-to-analog converter (DAC) and can be represented by voltage. Subsequently, the neural network chip inputs the first input data into the computing array, that is, it applies voltage to the computing array and performs the following steps.

[0059] S320, perform a first neural network calculation on the first input data according to the first weight stored in the computing array, wherein the first weight is the weight of the first neural network layer in the neural network system, the first weight includes multiple weight values, and each of the multiple weight values ​​is represented by an N-digit number with the same weight, where N is greater than or equal to 2.

[0060] The first weight is a matrix containing multiple weight values. The following uses the first weight value as an example to illustrate the storage format of each weight value in the first weight.

[0061] The first weight value is the encoded weight value obtained after uniform encoding of the initial weight value. For example, if the initial weight value is the decimal number 10, the first weight value obtained after uniform encoding of 10 can be 1111111111, where each "1" has the same weight; the first weight value obtained after uniform encoding of 10 can also be 22222 or 33310, where each value has the same weight. Equal weights can also be understood as the MSB and LSB of the encoded weight value being equal.

[0062] Below, taking binary encoding as an example, we will use formulas to demonstrate the beneficial effects of uniform encoding compared to non-uniform encoding.

[0063] When using binary NVM devices for uniform encoding, each NVM device can represent two conductance states (i.e., two resistance states), meaning each NVM device can represent the values ​​0 and 1. Assume the uniform encoding weights w... i The required bit length is N, and the value of each bit is g. i,j g i,j ∈{0,1}, corresponding to the high resistance state (HRS) and low resistance state (LRS) of the device, respectively. Then...

[0064]

[0065] Assume the computational array has R rows, and the voltage V in each row is... i From the output data x i Therefore, by summing the N columns of current values, we can obtain...

[0066]

[0067] As can be seen from formula (2), the weight values ​​obtained by uniform coding can correctly realize the vector inner product operation.

[0068] Assume that the resistance of the NVM device follows a log-normal distribution: R = R0 * e θWhere θ ~ N(0,σ); then the conductance of the NVM device also follows a log-normal distribution, i.e., Where θ ~ N(0,σ). Assume N SLCNVM devices (G i0 If the weight value w is encoded using ∈{0,1}, then the variance of the actual weight value w′ is...

[0069]

[0070] When using binary encoded weight values ​​w, N′ SLC NVM devices (G) are required. i0 ∈{0,1}), the variance of the actual weight value w′ is

[0071]

[0072] Where N = 2 N′ -1.

[0073] Comparison D u (w′) and D b (w′) indicates that in D(e -θ Under the same conditions, the variance of uniform encoding is smaller than that of binary encoding. Therefore, the results calculated based on the neural network weights obtained by uniform encoding are more accurate.

[0074] Figure 4 This is another example demonstrating that uniform coding has better technical performance than non-uniform coding.

[0075] Assuming the initial weight value ranges from 0 to 15, 4 bits are required when using binary representation; while 15 bits are required when using an SLCNVM device for uniform encoding. The results of encoding the decimal value "10" using uniform encoding based on an SLC NVM device and binary encoding are as follows: Figure 4 As shown.

[0076] Ideally, each SLC NVM device has no error, and the results of the two encoding methods mentioned above are the same.

[0077] With the same error variance, the result of binary encoding is 11.1, while the result of uniform encoding is 10.3. It can be seen that, under non-ideal conditions, uniform encoding can represent more accurate neural network weight values ​​than binary encoding, thus obtaining more accurate neural network calculation results.

[0078] Besides using SLC NVM devices for uniform coding, MLC NVM devices can also be used. If a K-bit MLC NVM device (i.e., one MLC NVM device has 2...) KIf the conductance states are uniformly encoded, then the MLCNVM device can represent values ​​greater than or equal to 0 and less than (2 ohms). K Any value among integers from -1.

[0079] Uniform encoding of weight values ​​ranging from 0 to R using SLC NVM devices requires R SLC NVM devices, while uniform encoding of weight values ​​ranging from 0 to R using K-bit MLC NVM devices requires... MLC NVM devices, among which This indicates rounding up. Therefore, uniform encoding using MLC NVM devices consumes fewer components, and the number of components is directly proportional to the chip's power consumption and area. Thus, using MLC NVM devices for uniform encoding can reduce the power consumption and area of ​​neural network chips.

[0080] Since each MLC NVM device can represent multiple values, using MLC NVM devices for uniform encoding offers greater flexibility compared to using SLC NVM devices for uniform encoding.

[0081] Figure 5 An example of uniform coding using SLC NVM devices and MLC NVM devices is shown.

[0082] For the decimal value "10", the result obtained by uniform encoding using SLC NVM devices is "1111111111", which occupies ten SLC NVM devices. If each MLC NVM device can represent an integer from 0 to 3, then there are multiple encoding results when uniform encoding using MLC NVM devices, and each encoding result only uses five MLC NVM devices.

[0083] Because uniform coding using MLC NVM devices offers greater flexibility, an optimal coding result can be obtained through theoretical derivation.

[0084] If N MLC NVM devices are used to represent the weight value w, and the ideal conductance of each MLC NVM device is G, then If the actual conductivity value G′=G·e -θ The actual weight value The variance of the actual weight values ​​is

[0085]

[0086] According to Cauchy's inequality, we get... When all G kWhen the values ​​are equal, the equality sign holds, and D(w′) is minimized. Therefore, the preferred uniform encoding method is to make the values ​​of each significant bit of the encoding result as equal as possible. For example, when using five 2-bit MLC NVM devices to uniformly encode the decimal number 8, a preferred encoding method is "22211".

[0087] The following describes the method for neural network computation based on MLC NVM devices and uniform coding provided in this application.

[0088] Assuming the computing array is 2*5 in size and consists of 2-bit MLC NVM devices, the computing array can store two weight values ​​with a range of 0 to 15. Figure 6 The right side shows the calculation results under ideal conditions, i.e., the theoretical output; Figure 6 The left side shows the calculation results under non-ideal conditions.

[0089] The input data are decimal values ​​10 and 7, which are encoded into binary as 1010 and 0111. The weight values ​​stored in the MLC NVM device are decimal values ​​7 and 5, which are encoded into 22111 and 11111 after uniform encoding. Under non-ideal conditions, the MLC NVM device cannot accurately represent integers between 0 and 3, and the actual weight values ​​will have some error, such as... Figure 6 As shown in the left-hand diagram.

[0090] Figure 6 Each shaded area represents one cycle of input. In the first cycle, the 0s and 1s input to the calculation array are multiplied and added with the weight values, outputting five columns of values. These five columns of values ​​are then processed by the ADC and input to the accumulator. The accumulator adds these five columns of values, yielding an output of 5.2. Following the same process, the accumulator outputs for the next three cycles are 12, 5.2, and 6.9, respectively. Since the input data is binary encoded, the outputs for these four cycles need to be shifted and accumulated by a shift accumulator, resulting in a final value of 105.2.

[0091] Comparing the final result 105.2 with the theoretical output 105, we can see that: because the weights of each value in the uniform encoding are equal, it is equivalent to the absence of MSB (Maximum Sum By), avoiding the amplification effect of device errors on MSB, thus reducing the error of the neural network calculation result; on the other hand, different devices have positive and negative errors, which are also partially offset by the accumulator. Compared with neural network calculation based on binary encoding, neural network calculation based on MLC NVM devices and uniform encoding can reduce the error of the neural network calculation result.

[0092] In addition to using uniform encoding to encode the weight values ​​as described above, a hybrid encoding method can also be used. For example, uniform encoding can be used to encode the weight values ​​in the first neural network layer, while non-uniform encoding can be used to encode the weight values ​​in other neural network layers.

[0093] As can be seen from the above embodiments, in the weight values ​​obtained using non-uniform encoding (such as binary encoding), the weights of each significant bit are different. When there is an error in the NVM device, the error of the significant bit with a higher weight (such as the MSB) will be amplified, causing the actual weight value to deviate significantly from the theoretical weight value. In addition, because the relative positions of each significant bit are fixed, the significant bit with a higher weight (such as the MSB) may be forced to be stored on the NVM device with a larger error, which will also cause the actual weight value to deviate significantly from the theoretical weight value. Ultimately, this leads to a larger error in the neural network calculation results.

[0094] In the weight values ​​obtained using uniform encoding, each significant bit has the same weight, which is equivalent to the absence of MSB and LSB. Therefore, the error of each significant bit value is not amplified. Furthermore, since each significant bit has the same weight, their relative positions can be flexibly adjusted. Appropriate values ​​can be stored on NVM devices with larger errors to reduce the negative impact of NVM device errors on the actual weight values. Therefore, method 300 can reduce the error of neural network calculation results even when manufacturing processes cannot be improved temporarily.

[0095] As mentioned above, in the weight value obtained using uniform coding, each significant bit has the same weight. Therefore, the relative positions of each significant bit can be flexibly adjusted. For example, for the decimal value 7, there are two uniform coding results: 33100 and 03013. These two coding results represent the same value. This characteristic of uniform coding can be used to improve the accuracy of the actual weight value.

[0096] Assuming the resistance of an NVM device follows a log-normal distribution, the conductance of the NVM device can be expressed as G = G0·e -θ The output current of a series of NVM devices It is the input voltage; when When unchanged, It is positively correlated with G, that is, the lower the resistance (or the higher the conductance), the greater the current generated, and the more severely it is affected by device errors.

[0097] To reduce the negative impact of non-ideal factors, this application provides a priority mapping algorithm, which improves the accuracy of weight values ​​by prioritizing mapping states with lower mapping resistance to NVM devices with smaller errors.

[0098] The steps of the priority mapping algorithm are as follows.

[0099] Step 1: Test the error of all resistor states for each NVM device.

[0100] Step 2: For each resistor state, sort the errors of the N NVMs used to represent a weight value in that resistor state in ascending order of error to obtain an index.

[0101] For example, five 2-bit MLC NVM devices are used to represent a weight value. Each MLC NVM device can use four resistance states to represent the values ​​0, 1, 2, and 3. Among them, LRS can represent 3, middle resistance state (MRS)2 can represent 2, MRS1 can represent 1, and HRS can represent 0. The order of these four resistance states is HRS>MRS1>MRS2>LRS. The correspondence between resistance states and values ​​is shown in Table 1.

[0102] Table 1

[0103] HRS MRS1 MRS2 LRS

[0104] The indices of the five 2-bit MLC NVM devices are ①②③④⑤, and the test and classification results of these five 2-bit MLC NVM devices are shown in Table 2.

[0105] Table 2

[0106] LRS 5 2 1 4 3 MRS2 2 5 4 1 3 MRS1 2 5 4 3 1

[0107] In Table 2, columns 2 through 6 represent the error of each state of the NVM device, from ① to ⑤.

[0108] For example, in the second column, 5 indicates that the LRS error of NVM device ① ranks fifth among the five NVM devices (i.e., the largest error), 2 in the third row indicates that the MRS2 error of NVM device ① ranks second among the five NVM devices, and 2 in the fourth row indicates that the MRS1 error of NVM device ① ranks second among the five NVM devices.

[0109] For example, in the fifth column, 4 indicates that the LRS error of NVM device ④ ranks fourth among the five NVM devices, 1 indicates that the MRS1 error of NVM device ④ ranks first among the five NVM devices (i.e., the smallest error), and 3 indicates that the MRS2 error of NVM device ④ ranks third among the five NVM devices.

[0110] Step 3: Initialize all NVM devices to HRS (corresponding to 0).

[0111] Step 4: Starting from LRS, write the uniformly encoded weight values ​​into the NVM device with the smallest error according to the index.

[0112] like Figure 7 As shown, for the decimal value 10, the result after uniform encoding is 33310. If the priority mapping algorithm is used, according to Table 1 and the principle of "prioritizing mapping the state with lower resistance value", 3 can be mapped to the NVM device first. According to Table 2, the indices of the NVM devices with smaller LRS error are ②③⑤. Therefore, the resistance values ​​of the NVM devices corresponding to these three indices can be set to LRS to represent 3.

[0113] Next, MRS2 mapping is performed; since there is no value corresponding to MRS2 for 33310, this step can be skipped.

[0114] Next, the mapping of MRS1 is performed. According to Table 1, the value corresponding to MRS1 is 1. According to Table 2, the order of MRS1 errors of NVM devices is ⑤①④③②, with the errors increasing sequentially from left to right. Since ⑤ has already been mapped to the value 3, it is necessary to select one from the remaining NVM devices. Among the remaining NVM devices, the NVM device with index ① indicates the smallest MRS1 error and has not been mapped to any other values. Therefore, the value 1 in 33310 can be mapped to the NVM device with index ①, thus completing the mapping of the uniformly encoded weight value to the NVM device.

[0115] Without the priority mapping algorithm, the three 3s in 33310 would be mapped to three NVM devices with indices ①②③, and the NVM device with index ① would have the largest LRS error; the 1s in 33310 would be mapped to the NVM device with index ④, and the NVM device with index ④ would also have a large MRS1 error, resulting in a large overall error for 33310. Therefore, the priority mapping algorithm provided in this application utilizes the characteristic that the relative positions of each effective bit in the uniform coding result can be flexibly adjusted, thereby improving the accuracy of the actual weight value.

[0116] The aforementioned priority mapping method can also be described as follows: N values ​​representing a weight value include a first value and a second value. The first value (e.g., 3 in the example above) is greater than the second value (e.g., 1 in the example above). The first value is stored in a first storage unit of the computing array, and the second value is stored in a second storage unit of the computing array. The error in storing the first value in the first storage unit is less than the error in storing the first value in the second storage unit, and the error in storing the second value in the first storage unit is less than the error in storing the second value in the second storage unit. That is, when the error in storing both values ​​in a storage unit is small, the larger value is preferentially stored in that storage unit.

[0117] Furthermore, because uniform coding allows for flexible adjustment of the relative positions of each valid bit, when an NVM device experiences a stuck-at-fault (SAF) failure, the NVM device with the specific SAF failure can be mapped to a specific state. By adjusting the states of other NVM devices, the weight values ​​represented by multiple NVM devices can be kept constant. For example, if three 2-bit MLC NVM devices represent a decimal weight value of 6 using 321, and the NVM device representing 2 experiences a SAF0 failure (i.e., stuck in state 0), the other two NVM devices can be mapped to 33, and the faulty NVM device can be mapped to an invalid value, thus solving the SAF problem.

[0118] The previous section introduced a neural network computation method within a single neural network layer. The following section introduces a neural network computation method applicable to multiple neural network layers, as provided in this application.

[0119] In neural networks, different neural network layers have different accuracy requirements. The following characteristics of uniformly encoded weights can be used to save hardware costs by using uniformly encoded weights of different lengths in different neural network layers while meeting the accuracy requirements.

[0120] In uniformly encoded weight values, each effective bit has the same weight. Neural network chips can obtain the cumulative output result by adding the output results of each column of the computation array storing uniformly encoded weight values ​​without shifting the output results of each column. Therefore, uniformly encoded weight values ​​of different lengths can be used in different neural network layers.

[0121] Can be used Figure 8 The accumulator 800 shown performs addition operations on the output results of each column. The operation flow of the accumulator 800 is as follows.

[0122] Step 01: Initialization. Counter 801 and sum register 802 are reset. The number of storage units used to store a weight value is written into configuration register 803. For example, for a weight value consisting of N values, the value written to register 803 is N.

[0123] Step 02: Accumulator 800 can be connected to ADC. Each time the ADC converts and outputs a data, it is input into accumulator 800 for accumulation. The result of the accumulation is stored in register 802, and at the same time, the count value of counter 801 is incremented by 1.

[0124] Step 03: When the count value in counter 801 reaches N, it indicates that accumulator 800 has accumulated the output results of N columns of NVM devices. Then comparator 804 outputs an enable signal, outputs the data in register 802 to output unit 805, and then resets counter 801 and register 802 through reset signal.

[0125] Step 04: Repeat steps 02 and 03 until the output results of all columns of the calculation array have been accumulated.

[0126] Figure 9 It demonstrates how to use accumulators (such as accumulator 800) to achieve multi-precision configuration of neural network layers.

[0127] Assume the computing array is 2*12 in size and consists of 24 2-bit MLC NVM devices. Figure 9 The neural network layer shown in the left figure uses 6 NVM devices to represent a weight value. Figure 9 The neural network layer shown in the right figure uses four NVM devices to represent one weight value. Therefore, the computation array shown in the left figure can store four weight values ​​(9, 16, 4, 8), while the computation array shown in the right figure can store six weight values ​​(7, 5, 10, 1, 4, 2). Different shaded areas in the figures represent different periods. Because the ADC is time-division multiplexed, each period can only convert one column. For a 12-column computation array, it takes 12 periods to convert the current output from all columns into digital signals. Figure 9 In the left and right diagrams, the accumulator is configured in different states. In the left diagram, the accumulator accumulates 6 data points at a time, while in the right diagram, the accumulator accumulates 4 data points at a time. Therefore, in the 6th and 12th cycles, the left diagram outputs two results, 13 and 24, which are the results of multiplying and adding the two weight values ​​in one row of the calculation array with the first input data (11). In the 4th, 8th, and 12th cycles, the right diagram outputs three results, 8, 9, and 12, which are the results of multiplying and adding the three weight values ​​in one row of the calculation array with the second input data (10).

[0128] exist Figure 9In the illustrated embodiment, a 2*12 computation array is also used. The computation array shown in the left figure can store 4 weights, while the computation array shown in the right figure can store 6 weights. Different precision outputs for different neural network layers are calculated through the configuration of the accumulator. Traditional neural network computation architectures do not support multi-precision configurations. To meet the 6-bit precision requirement in the left figure, the right figure also requires 6 NVM devices to represent a single weight value, meaning the computation array shown in the right figure can only store 4 weights. However, in the neural network computation architecture based on uniform coding, the multi-precision configurable design allows the right figure to store 6 weights, thereby reducing hardware overhead.

[0129] This application can also be combined with existing technologies to achieve better technical results, as shown in Table 3.

[0130] Table 3

[0131] Ideal situation 98.13％ 99.23％ 89.75％ 93.42％ binary 64.11％ 66.51％ 23.01％ 10.66％ Binary + DVA 94.23％ 97.23％ 42.52％ 16.62％ Uniform coding 91.73％ 78.17％ 20.12％ 12.47％ Uniform coding + DVA 97.24％ 97.66％ 42.91％ 18.83％ Uniform coding + priority mapping algorithm 96.21％ 95.76％ 65.44％ 81.40％ Uniform coding + priority mapping algorithm + DVA 98.07％ 98.78％ 86.65％ 87.94％

[0132] As can be seen from Table 3, combining the uniform coding and priority mapping algorithm provided in this application with the device variation aware (DVA) technology in the prior art achieves the best results in several neural network application scenarios.

[0133] The foregoing has detailed examples of the neural network computation methods provided in this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0134] This application can divide the neural network computing device into functional units based on the above method example. For example, each function can be divided into its own functional unit, or two or more functions can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0135] Figure 10This is a schematic diagram of the structure of a neural network computing device 1000 provided in this application. The neural network computing device 1000 includes a receiving port 1001 and a computing array 1002.

[0136] Receive port 1001 is used to receive the first input data;

[0137] The computing array 1002 is used to perform a first neural network calculation on the first input data based on the first weights stored in the computing array. The first weights are the weights of the first neural network layer in the neural network system. The first weights include multiple weight values, and each of the multiple weight values ​​is represented by an N-digit number with the same weight, where N is greater than or equal to 2.

[0138] Optionally, the plurality of weight values ​​includes a first weight value, and the N-digit number representing the first weight value includes a first value and a second value. The first value is greater than the second value. The first value is stored in a first storage unit of the computing array, and the second value is stored in a second storage unit of the computing array. The device error of the first storage unit is less than the device error of the second storage unit.

[0139] Optionally, the N bits representing the first weight value are stored in the N memory units of the computing array 1002. The device 1000 further includes an accumulator, which is used to: input the output results of the N memory units storing the first weight value into the accumulator for accumulation calculation when performing neural network calculation on the first input data based on the first weight value; and control the accumulator to output the calculation result when the number of accumulations of the accumulator reaches N.

[0140] Optionally, the device 1000 further includes an ADC, which is connected to the computing array 1002 and the accumulator respectively, for converting the analog output results of the N memory units into digital output results, and sending the digital output results to the accumulator for accumulation calculation.

[0141] Optionally, the receiving port 1001 is further configured to: receive second input data; the computing array 1002 is further configured to: perform a second neural network calculation on the second input data according to the second weights stored in the computing array 1002, wherein the second weights are the weights of the second neural network layer in the neural network system, the second weights include multiple weight values, each of the multiple weight values ​​is represented by M bits with the same weight, M is greater than or equal to 2, and M is not equal to N.

[0142] Optionally, the first weight is obtained by encoding the first initial weight according to a uniform coding method, wherein the uniform coding method includes MLC uniform coding.

[0143] The specific manner in which the device 1000 performs the neural network calculation method and the beneficial effects thereof can be found in the relevant descriptions in the method embodiments.

[0144] This application also provides a neural network computing system 1100, such as... Figure 11 As shown. Figure 11 The dashed lines in the diagram indicate that the unit or module is optional. Device 1100 can be used to implement the methods described in the above method embodiments, and device 1100 can be a terminal device, a server, or a chip.

[0145] The apparatus 1100 includes one or more processors 1101, which can support the implementation of the methods in the method embodiments. The processor 1101 can be a general-purpose processor or a special-purpose processor. For example, the processor 1101 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gate, transistor logic devices, or discrete hardware components.

[0146] The processor 1101 can be used to control the device 1100, execute software programs, and process data from the software programs. The device 1100 may also include a communication unit 1105 for inputting (receiving) and outputting (transmitting) signals.

[0147] For example, device 1100 may be a chip, communication unit 1105 may be the input and / or output circuit of the chip, or communication unit 1105 may be the communication interface of the chip, and the chip may be a component of terminal equipment, network equipment or other electronic equipment.

[0148] For example, device 1100 may be a terminal device or a server, and communication unit 1105 may be a transceiver of the terminal device or the server, or communication unit 1105 may be a transceiver circuit of the terminal device or the server.

[0149] The device 1100 may include one or more memories 1102 storing a program 1104. The program 1104 can be executed by a processor 1101 to generate instructions 1103, causing the processor 1101 to execute the method described in the above method embodiments according to the instructions 1103. Optionally, the memory 1102 may also store data (such as the various weights and input data described above). Optionally, the processor 1101 may also read data stored in the memory 1102, which may be stored at the same memory address as the program 1104, or the data may be stored at a different memory address than the program 1104.

[0150] The processor 1101 and memory 1102 can be configured separately or integrated together, for example, integrated on the system on chip (SOC) of the terminal device.

[0151] The device 1100 may also include a computing array 1106. The data output by the processor 1101 is processed by the DAC 1107 and then loaded onto the computing array 1106 as input data for the computing array 1106. Subsequently, the computing array 1106 performs neural network calculations on the input data and pre-configured neural network weights to obtain preliminary calculation results. These preliminary calculation results are processed by the sample and hold unit (S+H) 1108, the ADC 1109, and the accumulator 1110 and then output as the final calculation result.

[0152] This application also provides a computer program product that, when executed by processor 1101, implements the methods described in any of the method embodiments of this application.

[0153] The computer program product can be stored in memory 1102, for example, program 1104. Program 1104 is finally converted into an executable object file that can be executed by processor 1101 after processing such as preprocessing, compilation, assembly and linking.

[0154] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the methods described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

[0155] The computer-readable storage medium is, for example, memory 1102. Memory 1102 can be volatile memory or non-volatile memory, or memory 1102 can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0156] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and technical effects of the above-described apparatus and equipment can be referred to the corresponding processes and technical effects in the foregoing method embodiments, and will not be repeated here.

[0157] In the several embodiments provided in this application, the systems, apparatuses, and methods disclosed can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The apparatus embodiments described above are merely illustrative; the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components can be combined or integrated into another system. Furthermore, the coupling between units or components can be direct coupling or indirect coupling, including electrical, mechanical, or other forms of connection.

[0158] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0159] Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and / or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this paper generally indicates that the preceding and following related objects have an "or" relationship.

[0160] In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A neural network computation method, characterized in that, include: Receive the first input data; The first input data is computed using a first neural network based on the first weights stored in the computing array. The first weights are the weights of the first neural network layer in the neural network system. The first weights include multiple weight values, and each of the multiple weight values ​​is represented by an N-digit number with the same weight, where N is greater than or equal to 2. The plurality of weight values ​​include a first weight value, and the N-digit number representing the first weight value includes a first value and a second value. The first value is greater than the second value. The first value is stored in a first storage unit of the computing array, and the second value is stored in a second storage unit of the computing array. The device error of the first storage unit is less than the device error of the second storage unit.

2. The method according to claim 1, characterized in that, The method further includes storing the N bits representing the first weight value in N memory units of the computing array, and the method also includes: When performing neural network calculations on the first input data based on the first weight value, the output results of the N storage units storing the first weight value are input into the accumulator for accumulation calculation; When the accumulator accumulates N times, the accumulator is controlled to output the calculation result.

3. The method according to claim 1 or 2, characterized in that, Also includes: Receive the second input data; The second input data is computed using the second weights stored in the computing array. The second weights are the weights of the second neural network layer in the neural network system. The second weights include multiple weight values, each of which is represented by M bits with the same weight, where M is greater than or equal to 2 and M is not equal to N.

4. The method according to claim 1 or 2, characterized in that, The first weight is obtained by encoding the first initial weight according to a uniform coding method, which includes multi-layer coding (MLC) uniform coding.

5. A neural network computing device, characterized in that, include: The receiving port is used to receive the first input data. A computing array is used to perform a first neural network calculation on the first input data based on a first weight stored in the computing array, wherein the first weight is the weight of the first neural network layer in the neural network system, the first weight includes multiple weight values, and each of the multiple weight values ​​is represented by an N-digit number with the same weight, where N is greater than or equal to 2; The plurality of weight values ​​include a first weight value, and the N-digit number representing the first weight value includes a first value and a second value. The first value is greater than the second value. The first value is stored in a first storage unit of the computing array, and the second value is stored in a second storage unit of the computing array. The device error of the first storage unit is less than the device error of the second storage unit.

6. The apparatus according to claim 5, characterized in that, The N-digit number representing the first weight value is stored in N memory units of the computing array. The device further includes an accumulator, which is used for: When performing neural network calculations on the first input data based on the first weight value, the output results of the N storage units storing the first weight value are input into the accumulator for accumulation calculation; When the accumulator accumulates N times, the accumulator is controlled to output the calculation result.

7. The apparatus according to claim 6, characterized in that, The device further includes: An analog-to-digital converter (ADC) is connected to both the computing array and the accumulator. The ADC is used to convert the analog output results of the N memory computing units into digital output results and send the digital output results to the accumulator for accumulation calculation.

8. The apparatus according to any one of claims 5 to 7, characterized in that: The receiving port is also used to: receive second input data; The computing array is further configured to: perform a second neural network calculation on the second input data according to the second weights stored in the computing array, wherein the second weights are the weights of the second neural network layer in the neural network system, the second weights include multiple weight values, each of the multiple weight values ​​is represented by M bits with the same weight, M is greater than or equal to 2, and M is not equal to N.

9. The apparatus according to any one of claims 5 to 7, characterized in that, The first weight is obtained by encoding the first initial weight according to a uniform coding method, which includes multi-layer coding (MLC) uniform coding.

10. A neural network computing system, characterized in that, include: The processor is used to send the first input data; The neural network computing device according to any one of claims 5 to 9, wherein the neural network computing device is connected to the processor and is configured to receive input data from the processor and perform a first neural network calculation on the first input data.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 4.

Images