Method and electronic device for deploying neural network in analog memory computing NPU

Abstract

The invention provides a method and electronic equipment for deploying a neural network in an analog memory computing (NPU), the method comprises the following steps: a plurality of storage units form a memory bank, an ACiM arranges storage areas for storing weights by taking the memory bank as a unit according to the number of input channels, and the method comprises the following steps: remodeling the weight of an operator for deploying the neural network, selecting weights of a preset number of input channels to be inserted into other input channels; wherein the type of the operator is a calculation-intensive type; a neural network is deployed in the computing NPU in the simulation memory, meanwhile, the boundary of a storage unit in the NPU is kept to change by taking 2N bytes as a unit, and the setting of the weight is kept to follow 2M bytes of the boundary of a memory bank of hardware to which the NPU belongs; wherein N belongs to [1, X], M belongs to [1, Y], and X and Y belong to positive integers. According to the method, the effect of preferably deploying the neural network in the computing NPU in the simulation memory can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of neural network deployment technology, and in particular to a method and electronic device for deploying a neural network in an analog in-memory computing NPU. Background Art

[0002] With the widespread use of artificial intelligence applications in various mobile smart devices such as smartphones, smart watches, and smart bracelets, supporting end-side artificial intelligence reasoning has become an important feature of mobile devices. End-side reasoning eliminates the delay caused by network data transmission and enhances the security and privacy of user data. However, as the demand for mobile device intelligence continues to grow, the need for efficient end-side reasoning of large-scale models with a large number of parameters and high bandwidth requirements has become increasingly important. In particular, the inefficient performance of existing general-purpose processors such as CPUs and GPUs is gradually failing to meet these growing demands. Computing in Memory (CiM, a new computing architecture) breaks through the memory wall problem of the von Neumann architecture and provides new opportunities for end-side reasoning of large-scale models. CiM helps achieve bandwidth advantages for AI reasoning by accelerating the matrix multiplication operations of neural networks through near-memory computing architectures. In order to strike a balance between bandwidth and energy efficiency, CiM on mobile devices mainly utilizes analog CiM (Analog Computing in Memory, AciM) rather than digital CiM (Digital Computing in Memory, DCiM). However, due to various sources of defects, such as the unstable nature of analog circuits, limited dynamic range, insufficient effective resolution of Digital-to-Analog Converter (DAC), and narrow perception margin of Analog-to-Digital Converter (ADC), using ACiM will lead to a decrease in computational accuracy. However, most existing solutions focus on hardware design or some software-level methods that are highly hardware-dependent. Existing characterization studies do not understand the computational characteristics of ACiM. How the computational error performance pattern evolves on ACiM and the reasons behind it are not fully explored.

[0003] Figure 2 The ACiM principle diagram in the prior art is shown as follows: Figure 2As shown, the storage elements are located at the intersections of metal connections in the crossbar array. In this setup, matrix multiplication operations are performed in an analog manner using Kirchhoff's laws and Ohm's law. Voltage vectors are applied to the rows of the crossbar switch array, and current is measured and accumulated along each column. In general, the voltage represents the input value, while the conductance of the storage element represents the weight. ACiM has higher theoretical performance and energy efficiency. Specifically, ACiM requires only one calculation to generate an element in a matrix multiplication operation, while DCiM requires multiple bit-by-bit calculations to generate entries, resulting in higher latency and longer dwell time for the same input. Therefore, the device community for mobile devices pays more attention to ACiM.

[0004] Currently, most existing research focuses on hardware-level design, such as adding error correction modules and bit slicing mechanisms. The bit slicing mechanism utilizes more memory cells on ACiM to represent some weight values. For example, an INT8 / UINT8 weight value can initially be represented by one memory cell, while the bit slicing mechanism can use 4 or 5 memory cells to represent the INT8 / UINT8 weight value. By doing so, the number of perception levels of the ADC can be significantly reduced from 28 to 22 or 2. Therefore, the perception edge of the ADC can be greatly widened.

[0005] However, existing research focuses on hardware-level design. Adding error correction modules will increase the complexity and cost of chip design. Bit slicing mechanisms will reduce the scalability and computing density of chips. Both go against the original design intent of ACiM and offset the design advantages of ACiM.

[0006] Existing software-based solutions usually modify the parameters and / or weights of the original model, such as adding hardware-induced noise patterns into the model's training phase.

[0007] However, this noise-aware retraining approach is not universal. Each version of the model deployed on each ACiM chip needs to be retrained. For retraining or fine-tuning, datasets and labels are necessary. However, privacy concerns make people reluctant to upload personal data for cloud training and only accept end-side training. This means that noise-aware models in the cloud are not universally applicable to end-side usage scenarios, because further adaptation or retraining on the end-side will affect the effectiveness of the noise-aware parameters in the model. However, it should be noted that regardless of the model, the pattern of the added noise used for retraining is the same, so the computational characteristics of ACiM are consistent. Summary of the invention

[0008] In view of this, an embodiment of the present invention provides a method and electronic device for deploying a neural network in an analog in-memory computing NPU to eliminate or improve one or more defects existing in the prior art.

[0009] One aspect of the present invention provides a method for deploying a neural network in an analog in-memory computing NPU, wherein a plurality of storage units form a memory bar, and ACiM arranges a storage area for storing weights in memory bars according to the number of input channels, and the method comprises the following steps: reshaping the weights of operators used to deploy the neural network, selecting the weights of a preset number of input channels and interspersing them with other input channels; wherein the type of the operator is computationally intensive; deploying the neural network in the analog in-memory computing NPU, while keeping the boundaries of the storage units in the NPU at 2 N The weight size is set to follow the memory bank boundary of the NPU hardware. M Bytes; where N∈[1,X], M∈[1,Y], X and Y are positive integers.

[0010] In some embodiments of the present invention, the method of selecting weights of a preset number of input channels and interspersing them into other input channels includes: selecting a preset number of input channels, extracting the weights of the selected input channels; keeping the weight values ​​and the total number of weights unchanged, and inserting the extracted weights into unselected input channels.

[0011] In some embodiments of the present invention, in the step of deploying a neural network in a simulated in-memory computing NPU, the neural network is burned into a development board adapted to the memory conditions of AciM.

[0012] In some embodiments of the present invention, the weights of the operators used to deploy the neural network are evenly distributed in all activated memory bars, and preset scheduling instructions are used to control the number of repeated activations of the memory bars.

[0013] In some embodiments of the present invention, in the step of reshaping the weights of the operators used to deploy the neural network, a 32-bit bias value located in the middle area is set to avoid the frequency of occurrence of repeated extreme values ​​including 0 exceeding a preset range according to preset instructions.

[0014] In some embodiments of the present invention, the analog in-memory computing NPU includes a DAC component, and in the ACiM scenario, the inferred input voltage includes negative voltages with different direction meanings.

[0015] In some embodiments of the present invention, the weights of the operators used to deploy the neural network include negative weights, and the negative weights are implemented at the hardware level by a differential circuit, which uses two conductive elements to represent the weights, one representing the positive part and the other representing the negative part.

[0016] Corresponding to the above method, the present invention also provides an electronic device, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the electronic device implements the steps of the method described in any one of the above embodiments.

[0017] Corresponding to the above method, the present invention also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method described in any one of the above embodiments are implemented.

[0018] Corresponding to the above method, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any one of the above embodiments when executed by a processor.

[0019] The method and electronic device for deploying a neural network in an analog in-memory computing NPU proposed in the present invention can deploy the neural network in an analog in-memory computing NPU by reshaping the weights of the operators used to deploy the neural network and setting the storage boundaries in the NPU, thereby achieving the preferred effect of deploying the neural network in the analog in-memory computing NPU.

[0020] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0021] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0023] Figure 1 Flow chart of a method for deploying a neural network in an analog in-memory computing NPU according to an embodiment of the present invention.

[0024] Figure 2 Schematic diagram of ACiM principle in the prior art.

[0025] Figure 3 It is a line graph of the MSE changes of the convolution operator with 7 convolution kernel sizes, 1 step size, and different numbers of input and output channels.

[0026] Figure 4 Schematic diagram of the storage planning of weights on ACiM.

[0027] Figure 5 The MSE change line graph of the convolution operator with 8 output channels, 4 input channels, 1 step size and different convolution kernel sizes.

[0028] Figure 6 Schematic diagram of the principle of interleaving weights to minimize the number of input channels.

[0029] Figure 7 A schematic diagram of the structure of the computer equipment included in the system. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0031] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0032] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0033] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0034] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0035] In order to overcome the problems existing in the prior art, the present invention proposes a method and electronic device for deploying neural networks in an analog in-memory computing NPU. The method combines the analysis of multiplication and addition operations to reveal the patterns and reasons behind the computational characteristics on ACiM. Typical multiplication and addition operators (such as convolution and fully connected operators) are fully experimented from several aspects such as channel and kernel size. The experiments show that the computational error of ACiM is highly correlated with the use of storage units on ACiM. Based on this important insight, this scheme adjusts the memory configuration of weights and improves the computational accuracy loss of ACiM. The revealed features can serve as a guide to help design more computationally reliable operators on ACiM or manage memory on ACiM. Finally, several key insights for the actual deployment of artificial intelligence models on ACiM are proposed in detail.

[0036] In order to reveal the patterns and laws of computational errors on ACiM, how they evolve and the reasons behind them, this program conducted a large number of experiments on each parameter of the convolution operator (such as the number of input channels, the number of output channels, the size of the convolution kernel, and the step size), and found that the computational error of ACiM is highly correlated with the use of storage units on ACiM. And using this important discovery, the memory configuration of the weights was adjusted, which essentially alleviated the problem of ACiM's computational accuracy loss. Finally, this program also proposed several key insights from other aspects for the actual deployment of artificial intelligence models on ACiM.

[0037] Figure 1 The flowchart of the method for deploying a neural network in an analog in-memory computing NPU in one embodiment of the present invention is shown in FIG. A plurality of storage units form a memory bar, and ACiM arranges the storage area for storing weights in memory bars according to the number of input channels. The method includes:

[0038] Step S110: reshape the weights of operators used to deploy the neural network, and select weights of a preset number of input channels to be interspersed with other input channels; wherein the type of the operator is computationally intensive.

[0039] In the specific implementation process, the number of channels is reduced as much as possible through interspersing, but at the same time the total number of original weights is kept unchanged, and the value of the weight is not modified. In this way, the storage unit can be configured more reasonably and the configuration of the storage unit has stronger effectiveness.

[0040] Step S120: deploying the neural network in the simulated in-memory computing NPU while keeping the boundaries of the storage units in the NPU at 2 N The weight size is set to follow the memory bank boundary of the NPU hardware. M Bytes; where N∈[1,X], M∈[1,Y], X and Y are positive integers.

[0041] The method and electronic device for deploying a neural network in an analog in-memory computing NPU proposed in the present invention can deploy the neural network in the analog in-memory computing NPU by reshaping the weights of the operators used to deploy the neural network and setting the storage boundaries in the NPU, thereby achieving the preferred effect of deploying the neural network in the analog in-memory computing NPU.

[0042] In some embodiments of the present invention, the weights of a preset number of input channels are selected and interspersed into other input channels, including: (1) selecting a preset number of input channels and extracting the weights of the selected input channels; and (2) keeping the weight values ​​and the total number of weights unchanged, and inserting the extracted weights into unselected input channels.

[0043] By adopting this embodiment of the invention, the number of channels can be reduced and the weights can be kept unchanged, thereby improving the effect of model deployment.

[0044] In some embodiments of the present invention, in the step of deploying a neural network in a simulated in-memory computing NPU, the neural network is burned into a development board adapted to the memory conditions of AciM.

[0045] By adopting this embodiment of the invention, a neural network can be deployed in a simulated in-memory computing NPU on a development board from scratch.

[0046] In some embodiments of the present invention, the weights of the operators used to deploy the neural network are evenly distributed in all activated memory bars, and preset scheduling instructions are used to control the number of repeated activations of the memory bars.

[0047] By adopting this embodiment of the invention, it is possible to avoid the repeated accumulation of negative effects of simulation problems that may be caused by repeated activation.

[0048] In some embodiments of the present invention, in the step of reshaping the weights of operators used to deploy a neural network, a 32-bit bias value located in the middle area is set to prevent the occurrence frequency of repeated extreme values ​​including 0 from exceeding a preset range according to preset instructions.

[0049] By adopting this embodiment of the invention, it is possible to avoid reasoning delays and possible calculation errors by avoiding a large number of repeated extreme values ​​including 0.

[0050] In some embodiments of the present invention, the analog in-memory computing NPU includes a DAC component, and in the ACiM scenario, the inferred input voltage includes negative voltages with different direction meanings.

[0051] When adopting the embodiment of the invention, only negative voltage is included in the ACiM scenario, and negative values ​​are not included. Negative digital input is avoided to ensure the effect of the deployed neural network.

[0052] In some embodiments of the present invention, the weights of the operators used to deploy the neural network include negative weights, and the negative weights are implemented at the hardware level by a differential circuit, and the differential circuit uses two conductive elements to represent the weights, one representing the positive part and the other representing the negative part.

[0053] By adopting this embodiment of the invention, the negative weight can be represented by calculating the difference between two conductive elements.

[0054] The following is an analysis of the process of obtaining the solution.

[0055] Table 1 shows the MSE of the convolution operator with 8 input channels, 7 convolution kernel sizes, 1 step size, and different output channel numbers of 1, 2, 4, and 8 respectively.

[0056] Output channel number 1 2 4 8 MSE 9.38 10.09 9.30 10.17

[0057] Table 2 MSE of the convolution operator with 8 output channels, 7 convolution kernel sizes, 1 stride, and 1, 2, 4, and 8 different input channels.

[0058] Input Channel 1 2 4 8 MSE 1.28 1.62 3.76 10.17

[0059] Figure 3 It is a line graph of the MSE changes of the convolution operator with 7 convolution kernel sizes, 1 step size, and different numbers of input and output channels.

[0060] In Table 1, Table 2 and Figure 3 In the above, we can observe that: (1) when the input channel is fixed, the MSE remains almost unchanged as the output channel increases. The slight fluctuation of MSE may be caused by some random errors. (2) Figure 3 With the same computational workload, the MSE in Table 2 is much better than that in Table 1. For this reason, it can be inferred that the computational error performance is closely related to the relationship between the input channel and the output channel.

[0061] In order to better demonstrate this specific relationship, the GEMM operator is selected in one embodiment of the present invention for experiments, so as to focus only on the input channel and the output channel and exclude the influence of other parameters. N , N∈[3,10]. It is found that these 8 sets of experiments show very similar results. Due to limited space, only four sets of experimental results are shown in Table 3. It can be clearly observed that under the same input channel, no matter how the output channel changes, the pattern and law of the calculation error are very similar. The calculation error only gets worse with the increase of the number of input channels. Here, it can be inferred that the calculation error performance is closely related to the input channel.

[0062] Table 3 MSE of GEMM operators with 128 or 512 output channels and different input channels, the number of input channels is 2 N , N∈[3,10].

[0063] Input Channel 8 16 32 64 128 256 512 1024 8Output Channel 1.33 3.74 5.91 10.21 17.16 34.14 62.93 252.43 64Output Channel 1.47 2.74 4.88 10.57 18.73 34.93 72.61 256.36 128 Output Channel 1.56 3.43 5.79 10.11 19.37 38.94 64.16 265.36 512Output Channel 1.73 3.15 5.93 11.17 19.13 39.48 61.17 267.95

[0064] The essence of the multiplication-addition operator is the accumulation of the product of the input and the weight. With this in mind, the number of input channels and the number of output channels will affect the number of weights, and thus the memory area required to store the weights. The use of the memory area is based on a storage unit called a memory cell. Multiple storage cells form a memory bar. In fact, the overall calculation error depends largely on the memory bar. It can be understood that ACiM arranges the storage area for storing weights in units of memory bars according to the number of input channels, such as Figure 4 As shown, Figure 4 Schematic diagram of the storage plan for weights on ACiM. Each input channel matches a memory bank. This does not violate the convolution calculation process, because the weights of different output channels at the same height and width coordinates will be multiplied by the same input, but the weights of different input channels will not.

[0065] Even if a memory cell is not fully utilized, it is still fully activated. The more weights to store, the more memory cells are needed on the ACiM. Due to the inherent imperfections of analog circuits, larger memory areas activated on the ACiM will lead to more severe effects of parasitic currents and other issues, thus degrading computational error performance. However, some important clues about the boundaries of memory cells and methods to expand the capacity of memory bars are still missing here. Figure 5 This is a line graph of the MSE changes of the convolution operator with 8 output channels, 4 input channels, 1 step size and different convolution kernel sizes. Figure 5 It can be seen that the MSE has a clear sudden increase between kernel sizes 3 and 5, 7 and 9, 15 and 17, and 31 and 33 in the line graph. It can be concluded that the boundaries of the new memory cells increase in a staircase pattern instead of increasing linearly with the number of weights. In each step of the staircase, the MSE increases with the increase in kernel size due to the increase in memory area usage. This phenomenon will be explained later. The boundaries of the memory cells increase in a staircase pattern of 2 N The memory capacity is expanded by the size of the new storage unit at a time.

[0066] Since there is currently no hardware interface available on ACiM. In theory, reshaping the weights of the operator can verify the effectiveness of the reasonable configuration of the storage unit. It is more feasible to reduce the input channels as much as possible. But at the same time, the number of weights cannot be reduced and the value of the weights cannot be modified to ensure the ideal calculation process and results. Figure 6 A schematic diagram of the principle of interleaving weights to minimize the number of input channels. Figure 6 An intuitive way to reshape the weights is shown. By interleaving the weights from some input channels with other input channels, the number of weights is not reduced. It is also possible to obtain the correct computation process while interleaving the inputs and adjusting the stride (in Figure 6 Example in, the step size needs to be adjusted to 2). Table 4 shows that interleaving the weights in some input channels into other input channels can gain some benefits in MSE, thereby improving the calculation error performance. However, it is also observed that there is little improvement in MSE performance when the output channel is 8. Before interleaving, the weights are 7, 14, 28, and 56 bytes when the output channels are 1, 2, 4, and 8, respectively. After interleaving, the weights are 14, 28, 56, and 112 bytes when the output channels are 1, 2, 4, and 8. In the case of 8 output channels, only 112 bytes of weights have little improvement in MSE performance. This phenomenon is related to the memory bank capacity. If the weights occupy more than 64 bytes of memory in each input channel, a new memory bank will be activated. The weights will be evenly distributed in all activated memory banks. For the case where the interleaved weight size exceeds 64 bytes, this leads to similar MSE performance before and after interleaving. Therefore, the maximum boundary of the storage unit should be 32 bytes. That is, X is 5 and the boundary of the storage unit is 2 N The format of bytes changes, N∈[1,5]. That is, the maximum capacity of a memory stick is 64 bytes. More experimental results are shown in Table 5, and the same conclusion can be drawn.

[0067] Table 4 shows the MSE comparison before and after interleaving weights.

[0068] Configuration Before interleaving After interleaving 1o8i7k1s_2_1o4i14k2s 9.38 6.83(27.2％↓) 2o8i7k1s_2_2o4i14k2s 10.09 6.87(31.9％↓) 4o8i7k1s_2_4o4i14k2s 9.3 6.63(28.7％↓) 8o8i7k1s_2_8o4i14k2s 10.17 9.59 2o16i7k1s_2_2o8i14k2s 17.8 14.24(20％↓) 4o16i7k1s_2_4o8i14k2s 18.11 14.13(22％↓) 8o16i7k1s_2_8o8i14k2s 17.96 17.1

[0069] In Table 4, 1o8i7k1s_2_1o4i14k2s indicates that the convolution operator with 1 output channel, 8 input channels, 7 convolution kernel sizes and 1 stride is adjusted to a convolution operator with 1 output channel, 4 input channels, 14 convolution kernel sizes and 2 strides through interleaving.

[0070] Table 5 shows the MSE comparison before and after interleaving weights.

[0071]

[0072]

[0073] In Table 5, 8o8i7k1s_2_8o4i14k2s indicates that the convolution operator with 8 output channels, 8 input channels, 7 convolution kernel sizes and 1 stride is adjusted to a convolution operator with 8 output channels, 4 input channels, 14 convolution kernel sizes and 2 strides through interleaving.

[0074] In actual application scenarios, operators can be divided into two types, namely memory-intensive and compute-intensive. In general, convolution operators are usually compute-intensive, while GEMM operators are memory-intensive. On general-purpose processors, both types of operators can be deployed at will. Developers of AI models do not need to care about the operator type. The AI ​​deployment framework and operating system will arrange computing resources. However, in the ACiM scenario, the weights of the operators need to be burned into non-volatile memory in advance. This determines that in the presence of dynamic scheduling on general-purpose processors, the memory usage on ACiM when inferring the model is unchangeable. Under the same computing workload shown in Tables 2 and 3, the MSE performance of the convolution operator is better than that of the GEMM operator.

[0075] When deploying AI models on general-purpose processors, the weight size of the operator is usually not the main choice for tuning, but the model structure is. As the weight size increases, the calculation error performance will not suddenly increase. But in the ACiM scenario, the weight size is more important than the model structure. The number of input channels of the operator should be reduced as much as possible. More input channels means more memory banks are activated. More activated memory banks mean a larger activation storage area, which will lead to a decrease in calculation error performance. In addition, the setting of weight size should follow the boundaries of the memory banks2 M bytes, M∈[1,Y], to make better use of the memory cells and bars. Y is a positive integer. In the experimental setting of this scheme, the maximum capacity of the memory bar is 64 bytes. If the product of the output channel and the weight size exceeds 64 bytes, a new memory bar will be activated, equivalent to twice the number of input channels. One thing to note is that the maximum capacity of the memory bar should be different on different ACiM devices and needs to be carefully re-measured.

[0076] As mentioned above, computational workload will have a negative impact on MSE performance. The larger the computational workload, the more severe the MSE performance degradation. Excluding the impact of the number of input channels and the number of output channels, a small step size will also have a negative impact on MSE performance. Repeated activation is the key reason behind the computational workload. Repeated activation causes the negative impact of the simulation problem to accumulate repeatedly. According to Formula 1, a larger step size will lead to a smaller output size, and so does the number of simulation calculations. Each simulation calculation means a repeated activation of the memory area storing the weights. On the other hand, when the step size is fixed, the number of simulation calculations is negatively linearly related to the convolution kernel size. However, the total activation size of the memory area is exponentially related to the convolution kernel size. Simply put, assume that the stride is 1 and the padding is 0. The output size is Input_size-Kernel_size+1. The total activation memory area size is Output_size×Kernel_size, which is equal to -Kernel_size2+(Input_size+1)×Kernel_size. When the convolution kernel size is equal to (Input_size+1) / 2, the maximum value of the total activation memory area size can be obtained. In practical applications, the convolution kernel size is much smaller than (Input_size+1) / 2. Therefore, it can be observed that Figure 5 In each step, the MSE increases with the increase of the convolution kernel size due to the increase in memory area usage.

[0077]

[0078] Formula 1 shows the relationship between the output size of the convolution operator and the input size, convolution kernel size, stride and padding.

[0079] Different weight values ​​will have completely different MSE performance. In physics, negative numbers only represent directions, not specific values. Therefore, negative weights will be implemented by differential circuits. Differential circuits use two conductive elements to represent weights, one for the positive part and the other for the negative part. By calculating the difference between the two conductive elements, the representation of negative weights can be achieved. In Table 6, the patterns and regularities are difficult to infer. Some patterns can still be found by referring to the actual output data respectively. There may be a nonlinear relationship between conductance and applied voltage. This nonlinearity may cause calculation errors under different weight values, especially at extreme values ​​such as -128 or 127. However, due to truncation, the MSE performance at extreme values ​​is relatively good. For 0, it is represented by a very small conductance, so a small current that can be easily mishandled is generated. Therefore, the 0 value is susceptible. Worse, this nonlinearity also causes a mismatch in the mapping between the actual weight value and the burned-in conductance. Due to the 32-bit representation of the bias, small bias values ​​are easily overwhelmed. Therefore, a large number of repeated extreme values ​​including 0 should be avoided. In addition, the bias value should be placed near the middle area. Fortunately, in real applications, repeated extreme values ​​of weights and biases are very rare.

[0080] Table 6 Comparison of MSE of convolution operators with 4 output channels, 8 input channels, 7 convolution kernel sizes, 1 step size and different weight setting values ​​under different temperature settings.

[0081] Weight Value -128 -64 -1 0 1 64 128 Random Normal Temperature(24°) 15.02 511.34 10.92 59 57.37 1307.58 13.06 9.3 High Temperature(53°) 13.72 510.89 10.34 58.36 57.37 1307.85 13.06 9.27

[0082] The values ​​of resistance and conductivity may drift with temperature changes. This drift may cause instability and errors in the calculation results. Considering the actual output data in Table 6 respectively, it can be observed that the actual output value oscillates around a certain value. However, this oscillation is relatively small. In actual application scenarios, the temperature under normal working conditions of mobile devices has little effect on the calculation results.

[0083] In the ACiM scenario, the voltage represents the input value and the current represents the result of the calculation. For convenience, this voltage is called the II (inference input) voltage and this current is called the OI (inference output) current. In order to convert the digital input value into an analog signal that ACiM can understand, the DAC is a necessary component. The II voltage has no negative value. In physics, negative voltage only represents the direction, not the value. Therefore, ACiM can easily accept and process digital inputs without negative values. However, digital inputs with negative values ​​require more effort to handle. Negative digital inputs do represent the value. The OI current corresponding to the negative digital input cannot be simply accumulated with other currents in the same column of the crossbar array. Like the negative voltage, the negative current also represents only the direction. In an accumulated manner, changing one current component in the circuit path is very expensive and therefore completely unacceptable. In order to correctly and simply handle negative digital inputs, ACiM will handle negative and non-negative digital inputs separately. Negative digital inputs are 0 in the positive part and vice versa. The 0 value is represented by a very small voltage and therefore will produce a small current that can be easily mishandled. Therefore, the 0 value is susceptible. These two OI current parts are calculated in the digital domain after passing through the ADC. This approach is equivalent to doubling the input size, which will lead to worse inference latency and predictable computational error performance. In order to save energy, the OI current is a few milliamps (mA). This results in a narrow perception margin of the ADC [9,27] and a high error conversion rate. In summary, it is recommended that operators inferred on ACiM should try to avoid negative digital inputs, and inference after the ReLU activation function may be a more effective method.

[0084] There are two implementation paths for the above solution: (1) adjusting the operator in the existing model; (2) designing the operator and model structure based on the above method.

[0085] The hardware required to implement this solution can use a commercial artificial intelligence development board with ACiM technology. After the model adapted to ACiM's memory conditions is burned into the development board, reasoning can be performed.

[0086] The method and electronic device proposed in this solution can deploy the neural network in the simulated in-memory computing NPU by reshaping the weights of the operators used to deploy the neural network and setting the storage boundaries in the NPU, thereby achieving the effect of preferably deploying the neural network in the simulated in-memory computing NPU.

[0087] This scheme conducts experiments on the number of input channels, output channels, convolution kernel size, and step size of the convolution operator, as well as the number of input channels and output channels of the fully connected operator to understand the computational characteristics of ACiM's multiplication and addition operations, and reveals how the computational error patterns and rules evolve on ACiM and the reasons behind them: multiple storage units form a memory bar, and ACiM arranges the storage area for storing weights in memory bars according to the number of input channels, and the boundaries of the storage units are 2 N The capacity of the memory stick is expanded by the size of the new storage unit at a time.

[0088] During the design phase, the correlation between the computational error patterns and regularities described above and the memory usage on ACiM was verified experimentally. The revealed features can be used as guidelines to design more reliable computational operators for ACiM or to manage memory usage on ACiM. In addition, this scheme also proposes a simple and effective method for adjusting the memory configuration of weights: by interspersing the weights in some input channels with other input channels, and after interspersing the inputs and adjusting the stride, the correct computational process can be obtained, and the computational accuracy loss problem of ACiM is essentially alleviated.

[0089] This proposal proposes several key insights for the actual deployment of artificial intelligence models on ACiM in detail: prefer to use computationally intensive operators, follow (1) to minimize the number of input channels of operators, repeated activation will cause the negative effects of simulation problems to accumulate repeatedly, a large number of repeated extreme values ​​including 0 should be avoided, the temperature under normal working conditions of mobile devices has little effect on the calculation results, and operators inferred on ACiM should try to avoid negative digital inputs. It reveals how the patterns and laws of calculation errors on ACiM evolve and the reasons behind them. And using this important discovery, a method for adjusting the memory configuration of weights is provided, which essentially alleviates the problem of ACiM's calculation accuracy loss. Finally, several key insights for the actual deployment of artificial intelligence models on ACiM can be drawn from other aspects to help better design ACiM-friendly operators or models and better utilize ACiM.

[0090] Corresponding to the above method, the present invention also provides an electronic device, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device implements the steps of the method described above.

[0091] Figure 7This is a schematic diagram of the computer equipment included in the system. Figure 7 The computer device 00 includes: a processor 01, a memory 02, and a computer program stored in the memory 02 and executable on the processor 01. When the processor 01 executes the computer program, the human factor data server access control method provided in the above method embodiment is implemented.

[0092] Among them, the processor 01 is connected to the memory 02, such as through a bus 03. The processor 01 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in combination with the contents disclosed in this application. The processor 01 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, etc. The bus 03 may include a path to transmit information between the above components. The bus 03 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 130 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The memory 02 is used to store a computer program corresponding to the human factor data server access control method of the above embodiment of the present application, and the computer program is controlled and executed by the processor 01. The processor 01 is used to execute the computer program stored in the memory 02 to implement the content shown in the above method embodiment.

[0093] Corresponding to the above method, the present invention also provides a computer-readable storage medium, on which a computer program / instruction is stored, and when the computer program / instruction is executed by a processor, the steps of the method described in any one of the above embodiments are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0094] Corresponding to the above method, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any one of the above embodiments when executed by a processor.

[0095] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0096] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0097] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for deploying a neural network in an analog in-memory computing NPU, characterized in that: A plurality of storage units form a memory bar, and ACiM arranges the storage area for storing weights in memory bars according to the number of input channels. The method comprises: Reshape the weights of operators used to deploy the neural network, and select the weights of a preset number of input channels to be interspersed with other input channels; wherein the type of the operator is computationally intensive; Deploy neural networks in an emulated in-memory computing NPU while keeping the memory cell boundaries in the NPU at 2 N The weight size is set to follow the memory bank boundary of the NPU hardware. M Bytes; where N∈[1,X], M∈[1,Y], X and Y are positive integers.

2. The method according to claim 1, characterized in that The step of selecting the weights of a preset number of input channels and interspersing them with other input channels includes: Select a preset number of input channels and extract the weights of the selected input channels; Keeping the weight values ​​and the total number of weights unchanged, the extracted weights are inserted into the unselected input channels.

3. The method according to claim 1, characterized in that In the step of deploying a neural network in a simulated in-memory computing NPU, the neural network is burned into a development board adapted to the memory conditions of ACiM.

4. The method according to claim 1, characterized in that: The weights of the operators used to deploy the neural network are evenly distributed in all activated memory bars, and the number of repeated activations of the memory bars is controlled by using preset scheduling instructions.

5. The method according to claim 1, characterized in that: In the step of reshaping the weights of the operators used to deploy the neural network, a 32-bit bias value located in the middle area is set, and the frequency of occurrence of repeated extreme values ​​including 0 is prevented from exceeding a preset range according to preset instructions.

6. The method according to claim 1, characterized in that The analog in-memory computing NPU includes a DAC component, and in the ACiM scenario, the inference input voltage includes negative voltages with different direction meanings.

7. The method according to claim 6, characterized in that The weights of the operators used to deploy the neural network include negative weights, which are implemented at the hardware level by a differential circuit, which uses two conductive elements to represent the weights, one representing the positive part and the other representing the negative part.

8. An electronic device comprising a processor, a memory and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions. When the computer program / instructions are executed, the electronic device implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images