Methods and electronic devices for deploying neural networks in analog in-memory computing NPUs

Abstract

This invention provides a method and electronic device for deploying a neural network in an analog in-memory computing (NPU). The method includes: multiple storage units forming a memory module; an ACiM (Acoustic Computing Machine) arranging storage areas for storing weights in units of memory modules according to the number of input channels; the method further includes: reshaping the weights of operators used for deploying the neural network; selecting a preset number of input channel weights and interleaving them into other input channels; wherein the operator type is computationally intensive; deploying the neural network in the analog in-memory computing (NPU) while maintaining the boundaries of the storage units in the NPU at 2... N The changes are made in bytes, while maintaining the weight settings within the memory boundaries of the NPU's hardware. M Bytes; where N∈[1,X], M∈[1,Y], and X and Y are positive integers. The invention achieves the optimal effect of deploying neural networks in an analog in-memory computing NPU.

Description

Technical Field

[0001] This invention relates to the field of neural network deployment technology, and more particularly to a method and electronic device for deploying neural networks in an analog in-memory computing NPU. Background Technology

[0002] With the widespread application of artificial intelligence in various mobile smart devices (such as smartphones, smartwatches, and smart bands), supporting edge AI inference has become a crucial function for mobile devices. Edge inference eliminates latency caused by network data transmission and enhances the security and privacy of user data. However, with the continued growth in demand for mobile device intelligence, the need for efficient edge inference of large-scale models with numerous parameters and high bandwidth requirements is becoming increasingly important. In particular, the inefficient performance of existing general-purpose processors (such as CPUs and GPUs) is gradually failing to meet these growing demands. In-memory computing (a novel computing architecture, CiM) overcomes the memory wall problem of the von Neumann architecture, providing new opportunities for edge inference of large-scale models. By accelerating matrix multiplication operations in neural networks through near-memory computing architecture, CiM helps to achieve bandwidth advantages for AI inference. To achieve a balance between bandwidth and energy efficiency, CiM on mobile devices primarily utilizes Analog Computing in Memory (AciM) rather than Digital Computing in Memory (DCiM). However, due to various sources of defects, such as the instability of analog circuits, limited dynamic range, insufficient effective resolution of digital-to-analog converters (DACs), and narrow perception edges of analog-to-digital converters (ADCs), utilizing ACiM leads to a decrease in computational accuracy. However, most existing solutions focus on hardware design or some highly hardware-dependent software-level methods. Existing characterization studies do not fully understand the computational properties of ACiM. Currently, there is a lack of sufficient exploration into how computational error performance patterns evolve on ACiM and the underlying reasons.

[0003] Figure 2 This is a schematic diagram of the ACiM principle in the prior art. The ACiM principle is illustrated as follows: Figure 2As shown, the storage elements are located at the intersections of the metal connections in the cross-connect array. In this setup, matrix multiplication is performed analogically using Kirchhoff's laws and Ohm's law. A voltage vector is applied across the rows of the cross-connect array, and current is measured and accumulated along each column. Typically, the voltage represents the input value, while the conductance of the storage element represents the weight. ACiM offers higher theoretical performance and energy efficiency. Specifically, ACiM generates an element in a matrix multiplication operation with only one computation, while DCiM requires multiple bit-by-bit computations to generate an entry, resulting in higher latency and longer dwell time for the same input. Therefore, the device community for mobile devices is more focused on ACiM.

[0004] Currently, most existing research focuses on hardware-level design, such as adding error correction modules and bit slicing mechanisms. Bit slicing utilizes more memory units on the ACiM to represent weight values. For example, an INT8 / UINT8 weight value might initially be represented by a single memory unit, while bit slicing can use 4 or 5 memory units. By doing so, the number of sensing levels of the ADC can be significantly reduced from 2^8 to 2^2 or 2^3. Therefore, the sensing edge of the ADC can be greatly widened.

[0005] However, existing research focusing on hardware-level design suggests that adding error correction modules would increase chip design complexity and cost. Bit slicing mechanisms would reduce chip scalability and computational density. Both of these contradict the original design intent of ACiM and negate its design advantages.

[0006] Existing software-based solutions typically modify the parameters and / or weights of the original model, for example, by adding hardware-induced noise patterns to the model's training phase.

[0007] However, this noise-aware retraining approach is not universally applicable. Each version of the model deployed on each ACiM chip requires retraining. For retraining or fine-tuning, datasets and labels are necessary. However, privacy concerns discourage users from uploading personal data for cloud training, favoring edge-based training. This means that cloud-based noise-aware models are not universally suitable for edge-based use cases, as further adaptation or retraining on the edge will affect the effectiveness of the noise-aware parameters in the model. It should be noted, however, that the pattern of added noise used for retraining remains the same regardless of the model, thus the computational characteristics of ACiM are consistent. Summary of the Invention

[0008] In view of this, embodiments of the present invention provide a method and electronic device for deploying neural networks in an analog in-memory computing NPU to eliminate or improve one or more defects present in the prior art.

[0009] One aspect of the present invention provides a method for deploying a neural network in an analog in-memory computing (NPU), wherein multiple storage units form a memory module, and ACiM arranges storage areas for storing weights in units of memory modules according to the number of input channels. The method includes the following steps: reshaping the weights of operators used to deploy the neural network; selecting a preset number of input channel weights and interleaving them into other input channels; wherein the type of operator is computationally intensive; deploying the neural network in the analog in-memory computing (NPU) while maintaining the boundaries of the storage units in the NPU at 2... N The changes are made in bytes, while maintaining the weight settings within the memory boundaries of the NPU's hardware. M Bytes; where N∈[1,X], M∈[1,Y], and X and Y are positive integers.

[0010] In some embodiments of the present invention, the step of selecting a preset number of input channels and interleaving the weights of other input channels includes: selecting a preset number of input channels and extracting the weights of the selected input channels; keeping the weight values ​​and the total number of weights unchanged, and inserting the extracted weights into the unselected input channels.

[0011] In some embodiments of the present invention, in the step of deploying a neural network in an analog in-memory computing NPU, the neural network is burned into a development board adapted to the memory conditions of AciM.

[0012] In some embodiments of the present invention, the weights of the operators used to deploy the neural network are evenly distributed across all activated memory modules, and the number of times the memory modules are repeatedly activated is controlled by a preset scheduling instruction.

[0013] In some embodiments of the present invention, in the step of reshaping the weights of the operators used to deploy the neural network, a bias value in the middle region represented by 32 bits is set to avoid the frequency of repeated extreme values, including 0, from exceeding a preset range according to preset instructions.

[0014] In some embodiments of the present invention, the analog in-memory computing NPU includes a DAC component, and the inference input voltage in the ACiM scenario includes negative voltages with different directional implications.

[0015] In some embodiments of the present invention, the weights of the operators used to deploy the neural network include negative weights, which are implemented at the hardware level by a differential circuit that uses two conductive elements to represent the weights, one representing the positive portion and the other representing the negative portion.

[0016] Corresponding to the above methods, the present invention also provides an electronic device, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the electronic device implements the steps of any of the methods described in the above embodiments.

[0017] In accordance with the above methods, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method as described in any of the above embodiments.

[0018] Corresponding to the above methods, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any of the above embodiments.

[0019] The method and electronic device for deploying neural networks in an analog in-memory computing NPU proposed in this invention can set the memory boundary in the NPU by reshaping the weights of the operators used to deploy the neural network, thereby achieving the preferred effect of deploying neural networks in an analog in-memory computing NPU.

[0020] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0021] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0023] Figure 1 This is a flowchart of a method for deploying a neural network in a simulated in-memory computing NPU according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the ACiM principle in the existing technology.

[0025] Figure 3 Line graph showing the MSE variation of the convolution operator with 7 convolution kernel sizes, 1 stride, and different numbers of input and output channels.

[0026] Figure 4 This is a schematic diagram of the storage planning for weights on ACiM.

[0027] Figure 5 Line graph showing the MSE variation of convolution operators with 8 output channels, 4 input channels, 1 stride, and different kernel sizes.

[0028] Figure 6 This is a schematic diagram illustrating the principle of interleaving weights to minimize the number of input channels.

[0029] Figure 7 This is a schematic diagram of the computer equipment included in the system. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0031] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0032] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0033] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0034] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0035] To overcome the problems of existing technologies, this invention proposes a method and electronic device for deploying neural networks in an analog in-memory computing NPU. This method combines the analysis of multiplication and addition operations to reveal the patterns and underlying reasons for computational characteristics on ACiM. Extensive experiments were conducted on typical multiplication and addition operators (such as convolution and fully connected operators) from several aspects, including channel and kernel size. Experiments show that the computational error of ACiM is highly correlated with the use of memory units on ACiM. Based on this important insight, this scheme adjusts the memory configuration of weights and improves the computational accuracy loss of ACiM. The revealed features can serve as guidance to help design more computationally reliable operators or manage memory on ACiM. Finally, several key insights for the practical deployment of artificial intelligence models on ACiM are presented in detail.

[0036] To uncover the patterns and underlying causes of computational errors on ACiM, this approach conducted extensive experiments on every parameter of the convolution operator (e.g., number of input channels, number of output channels, kernel size, stride). The results revealed a high correlation between ACiM computational errors and the use of memory units on ACiM. Leveraging this important finding, the memory configuration of the weights was adjusted, fundamentally mitigating the computational accuracy loss problem of ACiM. Finally, this approach also offers several key insights for the practical deployment of artificial intelligence models on ACiM.

[0037] Figure 1 This is a flowchart illustrating a method for deploying a neural network in an analog in-memory computing NPU according to an embodiment of the present invention. Multiple storage units form memory modules, and ACiM arranges storage weights in memory module units based on the number of input channels. The method includes:

[0038] Step S110: Reshape the weights of the operators used to deploy the neural network, and select a preset number of input channel weights to interleave with other input channels; wherein the type of the operator is computationally intensive.

[0039] In the actual implementation process, the number of channels is reduced as much as possible by interleaving, while keeping the total number of weights unchanged and the weight values ​​unchanged. This allows for a more reasonable configuration of storage units, making the configuration of storage units more effective.

[0040] Step S120: Deploy the neural network in the simulated in-memory computing NPU while maintaining the boundaries of the memory cells in the NPU at 2... N The changes are made in bytes, while maintaining the weight settings within the memory boundaries of the NPU's hardware. M Bytes; where N∈[1,X], M∈[1,Y], and X and Y are positive integers.

[0041] The method and electronic device for deploying neural networks in an analog in-memory computing NPU proposed in this invention can set the memory boundary in the NPU by reshaping the weights of the operators used to deploy the neural network, thereby achieving the preferred effect of deploying neural networks in an analog in-memory computing NPU.

[0042] In some embodiments of the present invention, the selection of a preset number of input channel weights interleaved into other input channels includes: (1) selecting a preset number of input channels and extracting the weights of the selected input channels; (2) keeping the weight values ​​and the total number of weights unchanged, and inserting the extracted weights into the unselected input channels.

[0043] By employing this embodiment of the invention, the number of channels can be reduced while maintaining the same weights, thereby improving the effectiveness of model deployment.

[0044] In some embodiments of the present invention, in the step of deploying a neural network in an analog in-memory computing NPU, the neural network is burned into a development board adapted to the memory conditions of AciM.

[0045] Using this embodiment of the invention, it is possible to deploy neural networks from scratch on a development board in an analog in-memory computing NPU.

[0046] In some embodiments of the present invention, the weights of the operators used to deploy the neural network are evenly distributed across all activated memory modules, and the number of times the memory modules are repeatedly activated is controlled by a preset scheduling instruction.

[0047] By employing this embodiment of the invention, the negative impacts of repeated activation that may lead to simulation problems can be avoided from accumulating repeatedly.

[0048] In some embodiments of the present invention, in the step of reshaping the weights of operators used to deploy neural networks, a bias value in the middle region represented by 32 bits is set to avoid the frequency of repeated extreme values, including 0, exceeding a preset range according to preset instructions.

[0049] By employing this embodiment of the invention, it is possible to avoid delays in reasoning and potential calculation errors by avoiding a large number of repetitive extreme values, including 0.

[0050] In some embodiments of the present invention, the analog in-memory computing NPU includes a DAC component, and the inference input voltage in the ACiM scenario includes negative voltages with different directional meanings.

[0051] Using this embodiment of the invention, only negative voltages are included in the ACiM scenario, and negative values ​​are not included. Negative digital inputs should be avoided to ensure the effectiveness of the deployed neural network.

[0052] In some embodiments of the present invention, the weights of the operators used to deploy the neural network include negative weights, which are implemented at the hardware level by a differential circuit that uses two conductive elements to represent the weights, one representing the positive part and the other representing the negative part.

[0053] Using this embodiment of the invention, negative weights can be represented by calculating the difference between two conductive elements.

[0054] The following section analyzes the process of obtaining this solution.

[0055] Table 1 shows the MSE of the convolution operator with 8 input channels, 7 convolution kernel size, 1 stride, and 1, 2, 4, and 8 different output channels.

[0056] Number of output channels 1 2 4 8 MSE 9.38 10.09 9.30 10.17

[0057] Table 2 shows the MSE of the convolution operator with 8 output channels, 7 convolution kernel size, 1 stride, and 1, 2, 4, and 8 different input channels.

[0058] Number of input channels 1 2 4 8 MSE 1.28 1.62 3.76 10.17

[0059] Figure 3 Line graph showing the MSE variation of the convolution operator with 7 convolution kernel sizes, 1 stride, and different numbers of input and output channels.

[0060] In Table 1, Table 2 and Figure 3 In the study, it can be observed that: (1) When the input channels are fixed, the MSE remains almost unchanged as the output channels increase. The slight fluctuations in MSE may be due to some random errors. (2) In Figure 3 Under the same computational workload, the MSE in Table 2 is much better than that in Table 1. Therefore, it can be inferred that the computational error performance is closely related to the relationship between the input and output channels.

[0061] To better illustrate this specific relationship, in one embodiment of the invention, the LeGEMM operator was selected for experimentation to focus only on the input and output channels and exclude the influence of other parameters. Eight sets of experiments were conducted with different combinations of input and output channels, each with 2 input channels. N N∈[3, 10]. These eight experimental groups showed very similar results. Due to space limitations, only four experimental groups are shown in Table 3. It can be clearly observed that, with the same input channels, the patterns and rules of computational error are very similar regardless of the output channels. The computational error only worsens with the increase in the number of input channels. Here, it can be inferred that the computational error performance is highly dependent on the number of input channels.

[0062] Table 3 shows the MSE of GEMM operators with 128 or 512 output channels and different input channels, with 2 input channels respectively. N , N∈[3,10].

[0063] Input Channel 8 16 32 64 128 256 512 1024 8 Output Channel 1.33 3.74 5.91 10.21 17.16 34.14 62.93 252.43 64 Output Channel 1.47 2.74 4.88 10.57 18.73 34.93 72.61 256.36 128 Output Channel 1.56 3.43 5.79 10.11 19.37 38.94 64.16 265.36 512 Output Channel 1.73 3.15 5.93 11.17 19.13 39.48 61.17 267.95

[0064] The multiply-add operator is essentially the sum of the product of the input and the weights. Considering this, both the number of input channels and the number of output channels affect the number of weights, thus affecting the memory area needed to store them. The use of memory area is based on storage units called Memory Cells. Multiple storage units form Memory Bars. In fact, the overall computational error largely depends on the memory bars. It can be understood that ACiM arranges the storage areas for storing weights in units of memory bars according to the number of input channels, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the storage plan for weights on ACiM. Each input channel is matched with one memory block. This does not violate the convolution calculation process, because the weights of different output channels at the same height and width coordinates will be multiplied by the same input, but the weights of different input channels will not.

[0065] Even if a memory cell is not fully utilized, it will still be fully activated. The more weights to store, the more memory cells are needed on the ACiM. Due to the inherent limitations of analog circuits, a larger activated memory region on the ACiM will lead to a more severe impact from parasitic currents and other problems, thus reducing computational error performance. However, some important clues regarding memory cell boundaries and methods for expanding memory module capacity are still lacking. Figure 5 Line graphs showing the MSE variation of convolution operators with 8 output channels, 4 input channels, 1 stride, and different kernel sizes. Figure 5 It can be seen that MSE exhibits a significant abrupt increase between kernel sizes 3 and 5, 7 and 9, 15 and 17, and 31 and 33 in the line graph. It can be concluded that the boundaries of new memory cells increase in a stepped pattern, rather than linearly with the number of weights. Within each step, MSE increases with kernel size due to the increased memory region utilization. This will be explained later. The boundaries of memory cells increase in a stepwise manner with increasing kernel size. N The changes are made in bytes, where N∈[1,X] (X is a positive integer, which will be discussed later). The capacity of the memory module is expanded one at a time by the size of the new storage unit.

[0066] Currently, there is no available hardware interface on ACiM. Theoretically, reshaping the operator weights can verify the effectiveness of a reasonable memory configuration. Minimizing the number of input channels is more feasible. However, the number of weights cannot be reduced, and their values ​​cannot be modified, to ensure an ideal computational process and results. Figure 6 This is a schematic diagram illustrating the principle of interleaving weights to minimize the number of input channels. Figure 6 This demonstrates an intuitive method for reshaping weights. By interleaving weights from some input channels into others, the number of weights is not reduced. Correct calculations can be achieved while interleaving inputs and adjusting the stride (in...). Figure 6 In the example, the step size needs to be adjusted to 2). Table 4 shows that interleaving weights from some input channels into other input channels can provide some benefit to MSE, thereby improving computational error performance. However, it was also observed that when there are 8 output channels, the MSE performance is almost unimproved. Before interleaving, the weights were 7, 14, 28, and 56 bytes for output channels 1, 2, 4, and 8, respectively. After interleaving, the weights were 14, 28, 56, and 112 bytes for output channels 1, 2, 4, and 8. With only 112 bytes of weights for output channel 8, the MSE performance is almost unimproved. This phenomenon is related to memory capacity. If the weights occupy more than 64 bytes of memory in each input channel, a new memory module will be activated. The weights will be evenly distributed across all activated memory modules. For cases where the interleaved weight size exceeds 64 bytes, this results in similar MSE performance before and after interleaving. Therefore, the maximum boundary of the storage unit should be 32 bytes. That is, with X = 5, the boundary of the storage unit is 2 bytes. N The form of bytes varies, N∈[1, 5]. That is to say, the maximum capacity of a memory stick is 64 bytes. More experimental results are shown in Table 5, which leads to the same conclusion.

[0067] Table 4 shows a comparison of MSE before and after the interleaving weights.

[0068] Configuration information Before interleaving After interleaving 1o8i7k1s_2_1o4i14k2s 9.38 6.83(27.2％↓) 2o8i7k1s_2_2o4i14k2s 10.09 6.87(31.9％↓) 4o8i7k1s_2_4o4i14k2s 9.3 6.63(28.7％↓) 8o8i7k1s_2_8o4i14k2s 10.17 9.59 2o16i7k1s_2_2o8i14k2s 17.8 14.24(20％↓) 4o16i7k1s_2_4o8i14k2s 18.11 14.13(22％↓) 8o16i7k1s_2_8o8i14k2s 17.96 17.1

[0069] In Table 4, 1o8i7k1s_2_1o4i14k2s represents the convolution operator with 1 output channel, 8 input channels, 7 kernel size, and 1 stride, which is adjusted to 1 output channel, 4 input channels, 14 kernel size, and 2 stride by interleaving.

[0070] Table 5 shows a comparison of MSE before and after the interleaving weights.

[0071]

[0072]

[0073] In Table 5, 8o8i7k1s_2_8o4i14k2s represents the convolution operator with 8 output channels, 8 input channels, 7 kernel size, and 1 stride, which is adjusted to 8 output channels, 4 input channels, 14 kernel size, and 2 stride through interleaving.

[0074] In practical applications, operators can be categorized into two types: memory-intensive and computationally-intensive. Generally, convolution operators are computationally intensive, while GEMM operators are memory-intensive. On general-purpose processors, both types of operators can be deployed arbitrarily. Developers of AI models do not need to concern themselves with operator types; the AI ​​deployment framework and operating system will allocate computational resources. However, in the ACIM scenario, operator weights need to be pre-programmed into non-volatile memory. This means that, given dynamic scheduling on general-purpose processors, memory usage on ACIM during model inference is immutable. Under the same computational workload shown in Tables 2 and 3, the MSE performance of convolution operators is superior to that of GEMM operators.

[0075] When deploying AI models on general-purpose processors, operator weight size is usually not a primary tuning consideration, but model structure is. Increasing weight size doesn't necessarily lead to a sudden increase in computational error performance. However, in ACiM scenarios, weight size is more important than model structure. The number of input channels for operators should be minimized. More input channels mean more memory modules are activated. More activated memory modules mean larger activated storage areas, which will lead to a decrease in computational error performance. Furthermore, weight size settings should adhere to the memory module boundaries. M The memory capacity is M ∈ [1, Y], to make fuller use of storage units and bars. Y is a positive integer. In the experimental setup of this scheme, the maximum capacity of the memory bar is 64 bytes. If the product of the output channel and weight size exceeds 64 bytes, a new memory bar will be activated, equivalent to twice the number of input channels. It should be noted that the maximum capacity of the memory bar should be different on different ACiM devices and needs to be carefully remeasured.

[0076] As mentioned above, computational workload negatively impacts MSE performance. The greater the computational workload, the more severe the performance degradation. Even excluding the influence of the number of input and output channels, small strides also negatively affect MSE performance. Repeated activation is a key reason behind the computational workload. Repeated activation leads to the cumulative negative effects of simulation problems. According to Equation 1, a larger stride results in a smaller output size, and the same applies to the number of simulation calculations. Each simulation calculation means repeatedly activating the memory region of the stored weights. On the other hand, when the stride is fixed, the number of simulation calculations is negatively linearly related to the kernel size. However, the total activated size of the memory region is exponentially related to the kernel size. Simply put, assuming a stride of 1 and padding of 0, the output size is Input_size - Kernel_size + 1. The total activated memory region size is Output_size × Kernel_size, which equals -Kernel_size² + (Input_size + 1) × Kernel_size. The maximum size of the total activated memory region can be obtained when the kernel size equals (Input_size + 1) / 2. In practical applications, the kernel size is much smaller than (Input_size + 1) / 2. Therefore, it can be observed that... Figure 5 The pattern is as follows. In each step, the MSE increases with the increase of the convolution kernel size due to the increase in memory region utilization.

[0077]

[0078] Formula 1 shows the relationship between the output size of the convolution operator and the input size, kernel size, stride, and padding.

[0079] Different weight values ​​will result in completely different MSE performance. In physics, negative numbers only represent direction, not specific values. Therefore, negative weights will be implemented using differential circuits. Differential circuits use two conductive elements to represent the weight, one representing the positive part and the other the negative part. The negative weight representation can be achieved by calculating the difference between the two conductive elements. In Table 6, the patterns and regularities are difficult to deduce. Some patterns can still be found by referring to the actual output data separately. There may be a nonlinear relationship between conductance and applied voltage. This nonlinearity may cause calculation errors under different weight values, especially at extreme values ​​such as -128 or 127. However, due to truncation, the MSE performance at extreme values ​​is relatively good. For 0, it is represented by a very small conductance, thus generating a small current that is easily mishandled. Therefore, the value of 0 is susceptible to influence. Worse still, this nonlinearity can also lead to a mapping mismatch between the actual weight value and the burned-in conductance. Due to the 32-bit representation of the bias, small bias values ​​are easily overwhelmed. Therefore, a large number of repetitive extreme values, including 0, should be avoided. In addition, the bias value should be placed near the middle region. Fortunately, in real-world applications, repeated extreme values ​​of weights and biases are very rare.

[0080] Table 6 compares the MSE of convolution operators with 4 output channels, 8 input channels, 7 kernel size, 1 stride, and different weight settings under different temperature settings.

[0081] Weight Value -128 -64 -1 0 1 64 128 Random Normal Temperature (24°C) 15.02 511.34 10.92 59 57.37 1307.58 13.06 9.3 High Temperature (53°C) 13.72 510.89 10.34 58.36 57.37 1307.85 13.06 9.27

[0082] The values ​​of resistance and conductivity may drift with temperature changes. This drift can lead to instability and errors in the calculation results. Considering the actual output data in Table 6, it can be observed that the actual output values ​​oscillate around a certain value. However, this oscillation is relatively small. In practical applications, the temperature under normal operating conditions of mobile devices has little impact on the calculation results.

[0083] In the ACiM scenario, voltage represents the input value, and current represents the calculation result. For convenience, this voltage is called the II (Inference Input) voltage, and the current is called the OI (Inference Output) current. A DAC is a necessary component to convert digital input values ​​into analog signals that the ACiM can understand. II voltages do not have negative values. In physics, negative voltages only represent direction, not value. Therefore, the ACiM can easily accept and process digital inputs without negative values. However, digital inputs with negative values ​​require more effort to process. Negative digital inputs do represent that value. The OI current corresponding to a negative digital input cannot simply accumulate with other currents in the same column of the cross array. Like negative voltages, negative currents only represent direction. Changing a current component in the circuit path in an accumulative manner is very expensive and therefore completely unacceptable. To properly and simply process negative digital inputs, the ACiM processes negative and non-negative digital inputs separately. Negative digital inputs are 0 in the positive part, and vice versa. A 0 value is represented by a very small voltage, thus producing a small current that is easily mishandled. Therefore, 0 values ​​are susceptible to interference. These two OI current components are computed in the digital domain after passing through the ADC. This approach is equivalent to doubling the input size, which leads to worse inference latency and predictable computational errors. To conserve energy, the OI current is only a few milliamperes (mA). This results in a narrow sensing edge for the ADC [9,27] and a high error conversion rate. In conclusion, it is recommended that operators inference on the ACiM should avoid negative digital inputs as much as possible, and inference after the ReLU activation function may be a more effective approach.

[0084] The above scheme has two implementation paths: (1) adjust the operators in the existing model; (2) design the operator and model structure based on the above method.

[0085] The hardware required to implement this solution can be a commercial AI development board using ACiM technology. After burning the model, which is compatible with ACiM memory conditions, into the development board, inference can be performed.

[0086] The method and electronic device proposed in this solution can set the storage boundaries in the NPU by reshaping the weights of the operators used to deploy the neural network, thereby achieving the optimal effect of deploying the neural network in the analog in-memory computing NPU.

[0087] This scheme conducted experiments on four aspects—the number of input channels, the number of output channels, the kernel size, and the stride—of the convolution operator, as well as two aspects—the number of input channels and the number of output channels—of the fully connected operator, to understand the computational characteristics of multiply-accumulate operations in ACiM. It also revealed how the computational error patterns and rules evolve in ACiM and the underlying reasons: multiple storage units form a Memory Bar, and ACiM arranges storage weights in units of memory bars according to the number of input channels. The boundaries of the storage units are 2... N The memory module's capacity is expanded in bytes, where N∈[1,X] (X is a positive integer that may vary for each ACiM device), with the capacity of the memory module increasing by the size of the new storage unit at a time.

[0088] During the design phase, experiments verified the correlation between the computational error patterns and laws described above and memory usage on ACiM. The revealed characteristics can serve as guidelines for designing more reliable computational operators for ACiM or managing memory usage on ACiM. Furthermore, this scheme proposes a simple and effective method for adjusting weights in memory configuration: by interleaving weights from some input channels into other input channels, and after interleaving inputs and adjusting the step size, a correct computation process can be obtained, fundamentally mitigating the computational accuracy loss problem of ACiM.

[0089] This proposal details several key insights for practically deploying AI models on ACiM: a preference for computationally intensive operators; minimizing the number of input channels for operators as described in (1); repeated activation leading to the cumulative negative impact of simulation problems; avoiding numerous repetitive extreme values, including 0; minimal impact of temperature on computational results under normal operating conditions of mobile devices; and minimizing negative numerical inputs for operators used in inference on ACiM. It reveals how the patterns and laws of computational errors on ACiM evolve and the underlying causes. Utilizing this important finding, a method for adjusting memory configuration to weights is provided, which essentially alleviates the computational accuracy loss problem of ACiM. Finally, several other key insights for practically deploying AI models on ACiM can be derived to help design better ACiM-friendly operators or models and better utilize ACiM.

[0090] Corresponding to the above method, the present invention also provides an electronic device, which includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device performs the steps of the method as described above.

[0091] Figure 7This is a schematic diagram of the computer equipment included in the system. See also... Figure 7 The computer device 00 includes: a processor 01, a memory 02, and a computer program stored on the memory 02 and executable on the processor 01. When the processor 01 executes the computer program, it implements the human factors data server access control method provided in the above method embodiments.

[0092] The processor 01 is connected to the memory 02, such as via a bus 03. The processor 01 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 01 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The bus 03 may include a pathway for transmitting information between the aforementioned components. The bus 03 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 130 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The text uses only a single thick line to represent a bus, but this does not imply that there is only one bus or one type of bus. Memory 02 stores a computer program corresponding to the human factors data server access control method described in the above embodiments of this application. This computer program is executed under the control of processor 01. Processor 01 executes the computer program stored in memory 02 to implement the content shown in the aforementioned method embodiments.

[0093] Corresponding to the methods described above, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method as described in any of the above embodiments. The computer-readable storage medium may be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0094] Corresponding to the above methods, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any of the above embodiments.

[0095] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0096] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0097] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0098] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for deploying neural networks in an analog in-memory computing NPU, characterized in that, Multiple storage units form a memory module. ACiM arranges storage regions with storage weights in units of memory modules according to the number of input channels. The method includes: The weights of the operators used to deploy the neural network are reshaped, and the weights of a preset number of input channels are interleaved into other input channels; wherein, the type of the operator is computationally intensive, and the weights of the operators used to deploy the neural network are evenly distributed in all activated memory modules, and the number of times the memory modules are repeatedly activated is controlled by a preset scheduling instruction; Deploying neural networks in a simulated in-memory computing NPU while maintaining the boundaries of memory cells within the NPU at 2... N The changes are made in bytes, while maintaining the weight settings within the memory boundaries of the NPU's hardware. M Bytes; where N∈[1,X], M∈[1,Y], and X and Y are positive integers; In the step of reshaping the weights of the operators used to deploy the neural network, a 32-bit bias value located in the middle region is set to avoid the frequency of repeated extreme values, including 0, exceeding a preset range according to preset instructions. The step of selecting a preset number of input channels and interleaving their weights into other input channels includes: selecting a preset number of input channels and extracting the weights of the selected input channels; keeping the weight values ​​and the total number of weights unchanged, and inserting the extracted weights into the unselected input channels.

2. The method according to claim 1, characterized in that, In the step of deploying a neural network in a simulated in-memory computing NPU, the neural network is burned into a development board adapted to the memory conditions of ACiM.

3. The method according to claim 1, characterized in that, The analog in-memory computing NPU includes a DAC component, and in the ACiM scenario, the inference input voltage contains negative voltages with different directional meanings.

4. The method according to claim 3, characterized in that, The weights of the operators used to deploy the neural network include negative weights, which are implemented at the hardware level by a differential circuit that uses two conductive elements to represent the weights, one representing the positive part and the other representing the negative part.

5. An electronic device, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the electronic device implements the steps of the method as described in any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 4.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 4.

Images