A computing precision compensation architecture and method for a deep learning storage-computing integrated chip

Abstract

The application discloses a kind of computing precision compensation architecture and method for deep learning storage and calculation integrated chip, in the process of neural network model to memristor array mapping, only share row resources between different network layers, do not share column resources, there is different degree of resource surplus in column direction according to the difference of neural network scale;Based on the surplus resource after neural network model mapping, the compensation architecture is trained, and the memristor array corrects the output current through the input voltage mode.The application is aimed at the comprehensive influence of different noises, and aimed at the noise situation actually generated in the actual use process of the memristor chip, basically without increasing additional hardware consumption, avoids the precision loss problem caused by the inaccuracy of noise model when adding noise model in offline training, and improves the network precision of neural network model in actual memristor chip.

Description

Technical Field

[0001] This invention relates to the field of reliability design technology for in-memory computing chips based on memristor materials, and specifically to a computational accuracy compensation architecture and method for deep learning in-memory computing chips. Background Technology

[0002] With the continuous development of deep learning networks, the requirements for speed and power consumption in neural network computing are increasing. Traditional mainstream hardware platforms are mainly CPUs and GPUs, both based on the von Neumann computing architecture. Deep neural network algorithms are data-intensive, data-stream computing processes, generating a large amount of data transfer between processing and storage units. Due to the "memory wall" problem, limited data transfer bandwidth significantly affects the computing speed of neural networks. In-memory computing can effectively reduce data communication costs and is an important research direction for neural network computing architectures. Memristors, compared to traditional memories, have higher integration and energy efficiency and are suitable for in-memory computing, making them the best devices for realizing in-memory computing chips.

[0003] Due to technological limitations, current memristor computing arrays exhibit various types of noise when running deep neural network calculations, such as hard errors, process deviations, read noise, write noise, and voltage drops. These various noises superimpose within the memristor array, collectively affecting the output results. Analyzing individual noises during offline training cannot resolve the accuracy loss in actual calculations, while online training write operations are costly and significantly impact memristor lifespan and reliability. Summary of the Invention

[0004] To overcome the above technical problems, the present invention aims to provide a computational accuracy compensation architecture and method for deep learning in-memory computing chips. This architecture addresses the combined effects of different noises and the actual noise generated by memristor chips during actual use (the characteristics of noise vary at different life stages of memristor use). It does not increase additional hardware costs and avoids the accuracy loss caused by inaccurate noise models when adding noise models during offline training, thereby improving the network accuracy of neural network models in actual memristor chips.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A computational precision compensation architecture for deep learning in-memory computing chips, in the process of mapping a neural network model to a memristor array, different network layers only share row resources and do not share column resources. In the column direction, there are different degrees of resource surplus depending on the scale of the neural network.

[0007] Based on the residual resource training compensation architecture after the neural network model mapping, the memristor array corrects the output current by means of input voltage.

[0008] The memristor array contains three types of noise: random write noise, interconnect resistance noise, and SAF noise. Random noise is unavoidably generated when writing data to the memristor elements, and this noise is represented by Gaussian noise. In the simulation, different standard deviations represent different levels of noise.

[0009] Interconnect noise is caused by the resistance on the connection lines between the cells in the memristor array. The resistance on the connection lines introduces non-ideal values ​​into the memristor array, which is interconnect noise.

[0010] According to Kirchhoff's voltage law, the equivalent conductance of memristor elements located in the lower left part of the memristor array is relatively close to the ideal value, while the memristor elements in the upper right are more severely affected. As process dimensions shrink, interconnect resistance gradually increases, increasing interference with the current in the array. Since memristor arrays are calculated based on current, the increase in interconnect resistance also gradually increases its impact on calculation accuracy. Some memristor elements may get stuck in high or low conductance states, namely Stuck-At-0 (SA0) and Stuck-At-1 (SA1). The noise caused by this is called Stuck-At-Fault (SAF) noise. This noise is caused by immature manufacturing processes and individual memristor element failures, and is therefore difficult to avoid. SA0 is usually caused by a short circuit, which will cause the memristor element to always be in a high conductance state, while SA1 is caused by component damage or circuit breakage, which will cause this position to always be in a low conductance state. If the parameters of the neural network are directly deployed on a memristor array affected by SAF noise, the ability of SAF noise to affect the computational accuracy is closely related to the randomness of SAF noise.

[0011] The neural network includes convolutional layers and fully connected layers;

[0012] For a convolutional layer (C, K, K, N), the size after mapping to the memristor array is (CxKxK, N). Adding KxK rows of input data to the memristor array corresponds to adding a KxK convolutional kernel to each output channel of the convolutional layer in the network model. For a fully connected layer (H, W), the size after mapping to the memristor array is still (H, W). Adding M rows of input data to the memristor array corresponds to adding a parameter matrix of size (M, W) to the fully connected layer.

[0013] The neural network model is LeNeT-5, VGG8, or ResNet.

[0014] The memristor array includes memristor array 1, memristor array 2, memristor array 3 and memristor array 4;

[0015] The memristor array 1 contains all parameters of the first convolutional layer and some parameters of the second convolutional layer. The two layers of parameters are distributed side by side in the row direction to avoid data output crossing in the column direction. The remaining memristor units are in an idle state.

[0016] The memristor array 2 contains the remaining parameters of the second convolutional network. When the second convolutional layer is calculated, the calculation outputs of memristor array 1 and memristor array 2 are added together to obtain the final calculation result. The matrix parameters of the third fully connected layer are placed together by memristor array 3 and memristor array 4, which are the same as those of the second convolutional layer. The final calculation result is obtained by adding the outputs of memristor array 3 and memristor array 4.

[0017] The first convolutional layer does not have any compensation channels added, and the remaining memristor cells in the column direction of its corresponding memristor resources remain idle. The second convolutional layer adds compensation channels, and the size of the compensation channels of the convolutional layer is related to the size of the original convolutional layer of the second layer. They are evenly distributed in memristor array 1 and memristor array 2, and placed in the same column as the original convolutional kernel. When the second convolution is calculated, the output of the compensation channels and the output of the original convolutional kernel are naturally superimposed in the same memristor array. The third fully connected layer is distributed in two memristor arrays, memristor array 3 and memristor array 4. Each memristor array adds one compensation channel, and the placement position is also in the same column as the original matrix parameters.

[0018] A compensation method for a computational precision compensation architecture for in-memory computing chips for deep learning includes the following steps:

[0019] First, design and train the original network model, which is LeNeT-5, VGG8 or ResNet. Determine the parameter scale of each layer. Before mapping the original network model to the memristor array, calculate the remaining resources of each convolutional layer and fully connected layer in the network model based on the size of the memristor array.

[0020] When the number of remaining memristor cells available for compensation channels in a network layer is insufficient to add compensation channels in each memristor array, the option is to either abandon adding compensation channels in that network layer or obtain compensation channel space by allocating more hardware resources to that network layer.

[0021] After the original network model completes the mapping, the compensated channel weights are trained.

[0022] For convolutional layers, the input to the compensation channel is generated from the inputs of all the original network models belonging to the same output channel; for fully connected layers, the input to the compensation channel is generated from the inputs of a portion of the original network models located in the same memristor array.

[0023] For hardware and software co-training, the initialization, calculation, and training of the compensation channel are completed offline. The input of the original network model is fed into the memristor array, and the output of the memristor array is used as an intermediate value. At the same time, the input of the compensation channel is generated offline based on the input of the original network model. After being calculated with the compensation channel weights, it is added to the intermediate value calculated by the memristor array to generate the final calculation result, which is used for the next neural network operation. The weight update only applies to the offline compensation channel weights.

[0024] After training is completed, the compensation channel weights are written into the compensation channel space reserved by the memristor array. For pure online training, the initialized weight values ​​are first written into the compensation channel space reserved by the memristor array, and the output values ​​of each memristor array directly include the compensation channel calculation results.

[0025] When the network size is small and a single network layer only requires a single memristor array, when adding compensation channels, the scale of the compensation channels for each layer needs to be considered before mapping the original network model to the memristor array. For convolutional layers, the compensation channel scale is (1, K, K, N), that is, the number of input channels is 1 and the number of output channels is N. The number of output channels is equal to the number of output channels of the convolutional layer in the original network model. After mapping to the memristor array, the array size occupied is (KxK, N), that is, the number of input rows required by the memristor array is KxK and the number of output columns is N. The number of output columns of the compensation channels is the same as the number of output columns of the convolutional layer in the original network model. The position allocated in the memristor array is above the convolutional layer of the original network model, and it is written into the idle memristor cells in the N columns, without occupying the memristor resources outside the N columns.

[0026] When the network is large and a single network layer requires multiple memristor arrays for mapping, the compensation channels of the convolutional layer are distributed as evenly as possible among the memristor arrays. The convolutional kernels (3, 3, 3, 4) of the original network model are mapped to a size of (27, 4) and are allocated to memristor arrays 1 and 2. The convolutional kernels (1, 3, 3, 4) of the corresponding compensation channels are mapped to a size of (9, 4). Memristor array 1 is allocated (5, 4) and memristor array 2 is allocated (4, 4). For convolutional layers, when it is necessary to increase the number of compensation channels, (1, K, K, N) is used as the basic unit. The number of input channels can be increased to any integer value, while the other parameters remain unchanged. The input values ​​of the newly added compensation channels are generated by changing the selector configuration.

[0027] When the network size is very large, the number of memristor arrays is represented by a two-dimensional vector (H_, V_), where H_ represents the number of memristor arrays required in the input row direction. A value of H_ of 2 indicates that the fully connected matrix is ​​horizontally divided into two matrices. The output values ​​of memristor arrays 3 and 4 need to be added together to obtain the final calculation result.

[0028] V_ represents the number of memristor arrays required in the output column direction. When the number of output columns of a single memristor array is less than the number of output channels of a network layer, the parameter matrix corresponding to the network layer needs to be vertically divided into multiple matrices and distributed to multiple memristor arrays. The output values ​​of each memristor array need to be concatenated to obtain the final calculation result. For example, if the number of memristor arrays required for a network layer is (2,3), it means that the network layer needs a total of 2x3=6 memristor arrays.

[0029] For a convolutional layer of size (C, K, K, N), the minimum size of the compensation channels is (1, K, K, N), which maps to (KxK, N) in the memristor array. The number of input rows for its compensation channels is kxk. If the number of memristor arrays required for this convolutional layer is (H_conv, V_conv), and the compensation channels are evenly distributed across the memristor arrays, the number of input rows required for a compensation channel in a single memristor array is... That is, the result of (KxK) / H_conv is rounded up only if the number of free input rows in a single memristor array is greater than 1. Only then can the design requirements of the compensation channel be met;

[0030] For a fully connected layer, the required number of memristor arrays is (H_fc, V_fc). Each array reserves zero or more rows of memristor cells as compensation channels. The total number of compensation channels M can be an integer.

[0031] If the remaining number of rows / columns is insufficient to add compensation channels in each memristor array, compensation channels can be added in a portion of the memristor array, or compensation channel space can be obtained by allocating more hardware resources to the network layer. After reserving compensation channels, the original network model is mapped based on the remaining memristor array resources.

[0032] The present invention has the following beneficial effects:

[0033] The compensation architecture training of this invention is based on the actual memristor chip calculation output, which can avoid the accuracy loss problem caused by the inaccuracy of the noise model in offline model training. In offline model training, noise is added to simulate the memristor chip calculation, which reduces the accuracy drop after the neural network model is mapped to the memristor chip. However, since the noise model cannot accurately simulate the complex noise environment in the actual memristor chip, the model training effect of adding noise offline is limited.

[0034] The number of parameters in the compensation architecture of this invention is much smaller than the number of parameters in the original network model, which effectively reduces the number of weight updates during online training and alleviates the new noise problem caused by write operations in memristor chips. Attached image description:

[0035] Figure 1 This is a diagram of the overall architecture of the present invention.

[0036] Figure 2 This diagram illustrates the mapping of a neural network model to a memristor array when there is no compensation channel.

[0037] Figure 3 This diagram illustrates the mapping of the compensation channel to the memristor array after the compensation channel is added.

[0038] Figure 4 This is a flowchart of the compensation channel weight training process of the present invention.

[0039] Figure 5 This is a schematic diagram illustrating the effect of the present invention on the LeNeT-5 network model based on the MNIST dataset.

[0040] Figure 6 This is a schematic diagram illustrating the effect of the present invention on the VGG8 network model based on the CIFAR10 dataset. Detailed Implementation

[0041] The present invention will be further described in detail below with reference to the embodiments.

[0042] refer to Figure 1 The compensation architecture described in this invention is applied to memristor-based computing chips. During the mapping process from a neural network model to a memristor array, different network layers only share row resources and do not share column resources. Therefore, in the column direction, there are different degrees of resource surplus depending on the scale of the neural network.

[0043] Figure 1 Memristor array 1 contains all parameters of the first convolutional layer and some parameters of the second convolutional layer. The parameters of the two layers are distributed side by side in the row direction to avoid data intersection at the output end in the column direction. The remaining memristor units are in an idle state. Memristor array 2 contains the remaining parameters of the second convolutional network. When calculating the second convolutional layer, the calculation outputs of memristor array 1 and memristor array 2 are added together to obtain the final calculation result. Memristor array 3 and memristor array 4 jointly place the matrix parameters of the third fully connected layer, which is the same as the second convolutional layer. The outputs of memristor array 3 and memristor array 4 are added together to obtain the final calculation result.

[0044] After the original network model is mapped, the compensation channel is designed based on the remaining row / column resources of each network layer in the column direction. Figure 1The first convolutional layer does not have a compensation channel added, and the remaining memristor cells in the column direction of its corresponding memristor resources remain idle. The second convolutional layer adds a compensation channel, and the size of the compensation channel of the convolutional layer is related to the size of the original convolutional layer of the second layer. It is distributed as evenly as possible in memristor array 1 and memristor array 2, and placed in the same column as the original convolutional kernel. When the second convolution is calculated, the output of the compensation channel in the same memristor array is naturally superimposed with the output of the original convolutional kernel. The third fully connected layer is distributed to two memristor arrays, memristor array 3 and memristor array 4. The size of the compensation channel of the fully connected layer is flexibly designed, with one compensation channel added to each memristor array, and placed in the same column as the original matrix parameters.

[0045] In neural network computation, the input to the compensation channel is generated based on the input to the original network model of the same layer. Figure 1 The calculation steps for a 28x28 input image in a memristor computing chip incorporating the compensation architecture of this invention are as follows:

[0046] For the first convolutional layer without compensation channels, the input image is sampled to obtain a 3x3 receptive field each time. After format conversion, several column vectors of length 3x3 = 9 are generated. These vectors are input to memristor array 1, and after calculation by the memristor array, a 26x26x3 feature map is obtained. After pooling, a 13x13x3 feature map is obtained.

[0047] For the second convolutional layer containing the compensation channel, the feature map is sampled to obtain a 3x3x3 multidimensional vector each time, and a 3x3 compensation channel input is generated based on the sampling. For example, when the selector is set to the maximum value, the position at coordinate (1,1) in the original feature map sampling has values ​​of 1, 0, and 3, respectively. Then, the position at coordinate (1,1) in the compensation channel input has a value of 3. After generating the compensation channel input, the original feature map sampling is converted into several 3x3x3 = 27 column vectors, which are input into the input channels of the original network model convolutional layer in memristor arrays 1 and 2. The compensation channel input is converted into several 3x3 = 9 column vectors, which are input into the compensation channel part in memristor arrays 1 and 2. After the current superposition of the original convolutional channel and the compensation channel inside the memristor array and the addition of the calculation output of memristor arrays 1 and 2, an 11x11x4 feature map is obtained. After pooling, a 6x6x4 feature map is obtained and converted into a column vector of length 144, which is used as the original network model input of the fully connected layer.

[0048] For a fully connected layer containing compensation channels, the input of each compensation channel is generated from a portion of the original network model inputs located within the same memristor array. For memristor array 3, if the number of original network model inputs is 100 and the selector is set to the maximum value, then the compensation channel input value in memristor array 3 is the maximum value among the 100 original network model inputs. The remaining 44 inputs in the original network model are allocated to memristor array 4, so the compensation channel input value in memristor array 4 is the maximum value among the 44 original network model inputs. After generating the compensation channel input values, the compensation channel inputs are concatenated with the original network model inputs to form a new input vector, which is then fed into memristor arrays 3 and 4 respectively. After the current superposition of the original fully connected channels and compensation channels within the memristor arrays and the sum of the calculated outputs of memristor arrays 3 and 4, the final result is obtained.

[0049] refer to Figure 2 Assuming a small network size, where each network layer requires only a single memristor array, traditional methods, when mapping the original neural network model to a memristor array, do not add compensation channels; instead, they directly determine the memristor array resources used based on the parameters of each layer of the original network model. (Reference) Figure 2 The left side shows a schematic diagram of the convolutional layer mapping. The size of the convolutional layer is represented as (C, K, K, N), where C represents the number of input channels, K represents the kernel size, and N represents the number of output channels. After mapping to the memristor array, the occupied array size is (CxKxK, N), that is, the required number of memristor input rows is CxKxK, and the number of output columns is N. The remaining memristor units belonging to the N columns are idle, and other memristor units not belonging to the N columns can be used for other network layers.

[0050] refer to Figure 2 The diagram on the right shows the mapping of a fully connected layer. A weight matrix of size (H, W) is mapped to a memristor array of size (H, W). That is, the number of memristor input rows required is H, and the number of output columns is W. The remaining memristor cells belonging to columns W are idle, and other memristor cells not belonging to columns W can be used for other network layers.

[0051] like Figure 3 As shown, assuming a small network size, where a single network layer requires only a single memristor array, when adding compensation channels designed in this invention, the scale of each layer's compensation channel needs to be considered before mapping the original network model to the memristor array. (Reference) Figure 3The left side shows a schematic diagram of the convolutional layer compensation channel. For a convolutional layer, the compensation channel size is (1, K, K, N), meaning there is 1 input channel and N output channels. The number of output channels is equal to the number of output channels in the convolutional layer of the original network model. After mapping to the memristor array, the array size occupied is (KxK, N), meaning the required number of input rows in the memristor array is KxK and the number of output columns is N. The number of output columns of the compensation channel is the same as the number of output columns of the convolutional layer of the original network model. The position allocated in the memristor array is above the convolutional layer of the original network model, and it is written into the idle memristor cells in the N columns, without occupying the memristor resources outside the N columns.

[0052] In practical memristor chips, when the network size is large and a single network layer requires multiple memristor arrays for mapping, the compensation channels of the convolutional layer are distributed as evenly as possible across the memristor arrays, as shown in the reference... Figure 1 In the second convolutional layer, the convolutional kernels (3, 3, 3, 4) of the original network model are mapped to a size of (27, 4) and allocated to memristor arrays 1 and 2. The convolutional kernels (1, 3, 3, 4) of the corresponding compensation channels are mapped to a size of (9, 4). (5, 4) are allocated in memristor array 1 and (4, 4) are allocated in memristor array 2. The compensation architecture described in this invention is flexible and can increase or decrease the number of compensation channels according to different situations. For convolutional layers, when it is necessary to increase the number of compensation channels, (1, K, K, N) is used as the basic unit. The number of input channels can be increased to any integer value, while the other parameters remain unchanged. The input values ​​of the newly added compensation channels are generated by changing the selector configuration.

[0053] Make the number of compensation channel input rows in the two memristor arrays as balanced as possible.

[0054] refer to Figure 3 The right side shows a schematic diagram of the compensation channel for the fully connected layer. For the fully connected layer, the compensation channel size is (M, W), where M can be any integer value, i.e., the number of input channels is M and the number of output channels is W. The number of output channels is equal to the number of output channels in the fully connected layer of the original network model. After mapping to the memristor array, the size is (M, W), i.e., the number of required memristor input rows is M and the number of required memristor output columns is W. The number of output columns of the compensation channel is the same as the number of output columns in the fully connected layer of the original network model. The position allocated in the memristor array is above the fully connected layer of the original network model, and it is written into the idle memristor cells in column W, without occupying the memristor resources outside column W.

[0055] In practical memristor chips, when the network scale is large and a single network layer requires multiple memristor arrays for mapping, the number of compensation channels in the fully connected layer can be set based on the required number of memristor arrays, ensuring that each memristor array contains the same number of compensation channels as much as possible. (See reference...) Figure 1The fully connected layer in the memory has a fully connected matrix of size (144,10), which is divided into two arrays of (100,10) and (44,10) and distributed in memristor arrays 3 and 4. Each memristor array 3 and 4 has a row of compensation channels (1,10), so the compensation channel size of the fully connected layer is (2,10).

[0056] The compensation architecture described in this invention is flexible and can increase or decrease the number of compensation channels according to different situations. For fully connected layers, M in the compensation channel scale (M, W) can be any integer value, as long as W remains unchanged. The input values ​​of newly added compensation channels are generated by changing the selector configuration.

[0057] refer to Figure 4 The compensation method for reducing the impact of noise on neural network accuracy in memristor-based in-memory computing chips, as described in this invention, includes the following steps:

[0058] First, design and train the original network model, such as the common LeNeT-5, VGG8 or ResNet, and determine the parameter scale of each layer. Before mapping the original network model to the memristor array, calculate the remaining resources of each convolutional layer and fully connected layer in the network model based on the size of the memristor array.

[0059] When the network is very large, the required number of memristor arrays is generally represented by a two-dimensional vector (H_, V_), where H_ represents the number of memristor arrays required in the input row direction, as shown in the reference. Figure 1 In a fully connected layer, H_ is 2, indicating that the fully connected matrix is ​​horizontally divided into two matrices. The output values ​​of memristor arrays 3 and 4 need to be added to obtain the final calculation result. V_ represents the number of memristor arrays required in the output column direction. When the number of output columns of a single memristor array is less than the number of output channels of a network layer, the parameter matrix of that network layer needs to be vertically divided into multiple matrices and distributed to multiple memristor arrays. The output values ​​of each memristor array need to be concatenated to obtain the final calculation result. For example, if a network layer requires (2,3) memristor arrays, it means that the network layer needs a total of 2 x 3 = 6 memristor arrays.

[0060] For a convolutional layer of size (C, K, K, N), the minimum size of the compensation channels is (1, K, K, N), which maps to (KxK, N) in the memristor array. The number of input rows for its compensation channels is kxk. If the number of memristor arrays required for this convolutional layer is (H_conv, V_conv), and the compensation channels are evenly distributed across the memristor arrays, the number of input rows required for a compensation channel in a single memristor array is... That is, the result of (KxK) / H_conv is rounded up only if the number of free input rows in a single memristor array is greater than 1. Only then can the design requirements of the compensation channel be met.

[0061] In practice, the size of the memristor array and the number of parameters of the original network model are fixed and cannot be changed. After the calculation of the memristor array resources occupied by the original network model is completed, the number of memristor units that can be used for compensation channels in the remaining resources of each network layer is also fixed. When the number of memristor units that can be used for compensation channels in a certain network layer is insufficient to add compensation channels in each memristor array, it is possible to choose to give up adding compensation channels in that network layer, or to obtain compensation channel space by allocating more hardware resources to that network layer.

[0062] For a fully connected layer, the required number of memristor arrays is (H_fc, V_fc). Each array can reserve zero or more rows of memristor cells as compensation channels. The total number of compensation channels M can be an integer.

[0063] If the remaining number of rows / columns is insufficient to add compensation channels in each memristor array, compensation channels can be selectively added in some memristor arrays, or compensation channel space can be obtained by allocating more hardware resources to the network layer; after reserving compensation channels, the original network model is mapped based on the remaining memristor array resources.

[0064] After the original network model is mapped, the compensation channel weights are trained. For convolutional layers, the input to the compensation channel is generated from the inputs of all original network models belonging to the same output channel. For fully connected layers, the input to the compensation channel is generated from the inputs of some original network models located in the same memristor array. For hardware-software co-training, the initialization, calculation, and training of the compensation channels are completed offline. The input of the original network model is fed into the memristor array, and the output of the memristor array is used as an intermediate value. At the same time, the compensation channel input is generated offline based on the input of the original network model. After corresponding calculations are performed with the compensation channel weights, it is added to the intermediate value calculated by the memristor array to generate the final calculation result, which is used for the next neural network operation. Weight updates only apply to the offline compensation channel weights. After training, the compensation channel weights are written into the compensation channel space reserved by the memristor array. For purely online training, the initialized weight values ​​are first written into the compensation channel space reserved by the memristor array. The output values ​​of each memristor array directly contain the compensation channel calculation results. The compensation channel or the original network model can be updated according to actual needs.

[0065] refer to Figure 5 and 6Compared to neural network computation based on memristor arrays, the offline compensation channel training has lower hardware noise and higher computational accuracy. By adding simulated memristor noise to the offline compensation channel training, the difference between the accuracy of the compensation channel training and the accuracy of the neural network after writing to the memristor array can be reduced.

Claims

1. A computational accuracy compensation system for in-memory computing chips for deep learning, characterized in that, During the mapping of a neural network model to a memristor array, different network layers only share row resources and do not share column resources. Depending on the size of the neural network, there are varying degrees of resource surplus in the column direction. Based on the residual resource training compensation system after neural network model mapping, the memristor array corrects the output current by means of input voltage; The memristor array includes memristor array 1, memristor array 2, memristor array 3 and memristor array 4; The memristor array 1 includes all parameters of the first convolutional layer and some parameters of the second convolutional layer. The two layers of parameters are distributed side by side in the row direction, and the remaining memristor cells are in an idle state. The memristor array 2 contains the remaining parameters of the second convolutional network. When calculating the second convolutional layer, the calculation outputs of memristor array 1 and memristor array 2 are added together to obtain the final calculation result. The matrix parameters of the third fully connected layer are placed together by memristor array 3 and memristor array 4, which are the same as those of the second convolutional layer. The final calculation result is obtained by adding the outputs of memristor array 3 and memristor array 4. The first convolutional layer does not have a compensation channel added, and the remaining memristor cells in the column direction of its corresponding memristor resources remain idle. The second convolutional layer adds compensation channels, the size of which is related to the size of the original convolutional layer in the second layer. These channels are evenly distributed across memristor array 1 and memristor array 2, and placed in the same column as the original convolutional kernel. When the second convolution is calculated, the output of the compensation channel and the output of the original convolutional kernel are naturally superimposed in the same memristor array. The third fully connected layer is distributed across two memristor arrays, memristor array 3 and memristor array 4. Each memristor array adds one compensation channel, which is also placed in the same column as the original matrix parameters.

2. The computational accuracy compensation system for in-memory computing chips for deep learning according to claim 1, characterized in that, The neural network includes convolutional layers and fully connected layers; For a convolutional layer (C, K, K, N), the size after mapping to the memristor array is (CxKxK, N). Adding KxK rows of input data to the memristor array corresponds to adding a KxK convolutional kernel to each output channel of the convolutional layer in the network model. For a fully connected layer (H, W), the size remains (H, W) after mapping to the memristor array. Adding M rows of input data to the memristor array corresponds to adding a parameter matrix of size (M, W) to the fully connected layer.

3. The compensation method for a computational accuracy compensation system for a deep learning in-memory computing chip as described in any one of claims 1-2, characterized in that, Includes the following steps: First, the original network model is designed and trained, and the parameter scale of each layer is determined. Before the original network model is mapped to the memristor array, the remaining resources of each convolutional layer and fully connected layer in the original network model are calculated based on the size of the memristor array. When the number of remaining memristor cells available for compensation channels in a network layer is insufficient to add compensation channels in each memristor array, the option is to either abandon adding compensation channels in that network layer or obtain compensation channel space by allocating more hardware resources to that network layer. After the original network model completes the mapping, the compensated channel weights are trained. For convolutional layers, the input to the compensation channel is generated from the inputs of all the original network models belonging to the same output channel; for fully connected layers, the input to the compensation channel is generated from the inputs of a portion of the original network models located in the same memristor array. For hardware and software co-training, the initialization, calculation and training of the compensation channel are completed offline; The input of the original network model is fed into the memristor array, and the output of the memristor array is used as an intermediate value. At the same time, a compensation channel input is generated offline based on the input of the original network model. After the compensation channel weights are calculated accordingly, the result is added to the intermediate value calculated by the memristor array to generate the final calculation result, which is used for the next step of neural network operation. After training is complete, the compensation channel weights are written into the compensation channel space reserved in the memristor array; For pure online training, the initialized weight values ​​are written into the compensation channel space reserved by the memristor array, and the output value of each memristor array directly includes the calculation result of the compensation channel.

4. The compensation method for a computational accuracy compensation system for a deep learning in-memory computing chip according to claim 3, characterized in that, When the network size is small and a single network layer only requires a single memristor array, when adding compensation channels, the scale of the compensation channels for each layer is considered before mapping the original network model to the memristor array. For convolutional layers, the compensation channel scale is (1, K, K, N), that is, the number of input channels is 1 and the number of output channels is N. The number of output channels is equal to the number of output channels of the convolutional layer in the original network model. After mapping to the memristor array, the array size occupied is (KxK, N), that is, the number of input rows required by the memristor array is KxK and the number of output columns is N. The number of output columns of the compensation channel is the same as the number of output columns of the convolutional layer in the original network model. The position allocated in the memristor array is above the convolutional layer of the original network model, and it is written into the idle memristor cells in the N columns, without occupying the memristor resources outside the N columns.

5. The compensation method for a computational accuracy compensation system for a deep learning in-memory computing chip according to claim 3, characterized in that, When the network is large and a single network layer requires multiple memristor arrays to complete the mapping, the compensation channels of the convolutional layer should be distributed as evenly as possible among the memristor arrays. For convolutional layers, when it is necessary to increase the number of compensation channels, the basic unit is (1, K, K, N). The number of input channels can be increased to any integer value, while the other parameters remain unchanged. The input values ​​of the newly added compensation channels are generated by changing the selector configuration.

6. The compensation method for a computational accuracy compensation system for a deep learning in-memory computing chip according to claim 3, characterized in that, When the network size is very large, the number of memristor arrays is represented by a two-dimensional vector (H_, V_), where H_ represents the number of memristor arrays required in the input row direction. A value of H_ of 2 indicates that the fully connected matrix is ​​horizontally divided into two matrices. The output values ​​of memristor arrays 3 and 4 need to be added together to obtain the final calculation result. V_ represents the number of memristor arrays required in the output column direction. When the number of output columns of a single memristor array is less than the number of output channels of a certain network layer, the parameter matrix corresponding to that network layer needs to be vertically divided into multiple matrices and distributed to multiple memristor arrays. The output values ​​of each memristor array need to be concatenated to obtain the final calculation result.

7. The compensation method for a computational accuracy compensation system for a deep learning in-memory computing chip according to claim 3, characterized in that, For a convolutional layer of size (C, K, K, N), the minimum size of the compensation channels is (1, K, K, N), which maps to (KxK, N) in the memristor array. The number of input rows for its compensation channels is kxk. If the number of memristor arrays required for this convolutional layer is (H_conv, V_conv), and the compensation channels are evenly distributed across the memristor arrays, the number of input rows required for a compensation channel in a single memristor array is... That is, the result of (KxK) / H_conv is rounded up, only when the number of free input rows in a single memristor array is greater than Only then can the design requirements of the compensation channel be met; For a fully connected layer, the required number of memristor arrays is (H_fc, V_fc). Each array reserves zero or more rows of memristor cells as compensation channels. The total number of compensation channels M can be an integer. If the remaining number of rows / columns is insufficient to add compensation channels in each memristor array, compensation channels can be added in a portion of the memristor array, or compensation channel space can be obtained by allocating more hardware resources to the network layer. After reserving compensation channels, the original network model is mapped based on the remaining memristor array resources.

Images