Full-simulation non-multiplication and addition operation method and device oriented to simulation in-memory calculation

Abstract

The invention relates to a full-simulation non-multiplication and addition operation method and device for simulation in-memory calculation, and the method comprises the steps: training a target full model, and obtaining a corresponding weight and a batch normalization parameter, the batch normalization parameter comprising bias information and a zoom factor; based on the simulation in-memory calculation array and the simulation bias array, in combination with the weight and bias information, carrying out multiplication and addition operation and batch normalized bias calculation; generating a target reference voltage of the ReLU-ADC based on a global resistance ladder voltage divider or an analog bias array in combination with a scaling factor, and setting a DAC minimum output voltage thereof; and based on the target reference voltage and the minimum output voltage of the DAC, the ReLU-ADC performs full-simulation non-multiplication-addition operation on the bias multiplication-addition simulation calculation result. Therefore, the problems that in a traditional CIM architecture, a mode of completing non-multiplication and addition operation depending on a digital domain has significant bottlenecks in key performance indexes such as speed, power consumption and area, and the requirements of resource-limited scenes such as an intelligent sensor are difficult to meet are solved.

Description

Technical Field

[0001] This application relates to the field of analog in-memory computing technology, and particularly to a fully analog non-multiply-and-accumulate operation method and device for analog in-memory computing. Background Art

[0002] With the continuous increase in the scale and complexity of Convolutional Neural Network (CNN) models, the traditional architecture has gradually exposed the following bottlenecks: First, frequent data transmission leads to the "memory wall" problem, and the insufficient memory bandwidth severely limits the computing efficiency. Second, the energy consumption problem caused by high-frequency data transmission poses a challenge to resource-constrained application scenarios (such as edge devices and intelligent sensors). Finally, to meet the computing requirements of large-scale networks, the traditional architecture needs to deploy a large number of storage and computing units, significantly increasing the chip area and cost.

[0003] To address these challenges, the Computing-in-Memory (CIM) architecture has emerged. By directly embedding the computing function into the storage unit, CIM enables data processing to be completed inside the memory, thereby significantly reducing data transmission, latency, and power consumption. At the same time, CIM utilizes the parallelism of analog computing to efficiently implement the Multiply-and-Accumulate (MAC) operation at the hardware level, accelerating the CNN inference process. In addition, through the integrated design of storage and computing, CIM optimizes the utilization efficiency of hardware resources, significantly reducing the chip area, and showing obvious advantages in resource-constrained scenarios such as intelligent sensors.

[0004] Although CIM performs excellently in MAC operations, the computing of CNN not only includes MAC operations but also non-multiply-and-accumulate operations, such as Batch Normalization (BN) and activation functions (such as Rectified Linear Unit (ReLU)). These operations are an indispensable part of CNN. However, existing CIM architectures usually rely on digital circuits to complete fully analog non-multiply-and-accumulate (Non-MAC) operations. This design pattern destroys the full analog computing integrity of the CIM architecture, resulting in frequent data conversion between the analog domain and the digital domain, which not only introduces additional latency and energy consumption but also requires adding a large number of digital computing circuits, thus significantly increasing resource consumption such as system complexity, power consumption, and area.

[0005] Therefore, how to efficiently implement Non-MAC operations in the analog domain has become an important challenge in building a truly fully analog CIM architecture.

[0006] Specifically, the prior art can use a current-mode multiply-accumulate calculation method. Each multiplication result is represented as a current, and the currents are accumulated to achieve the effect of summation. To avoid excessive current, the prior art can adopt a method of segmented summation and then accumulation. During the actual execution process, as Figure 1 shown, the prior art can perform MAC calculations in each sub-block to obtain a current, and after conversion by an analog-to-digital conversion unit, a partial sum is obtained; the partial sums of multiple sub-blocks are accumulated in an accumulation summation unit to obtain a final result.

[0007] Some prior art can also use grouped capacitors applicable to the CIM architecture to implement a DAC-free digital-to-analog conversion scheme. As Figure 2 shown, taking 4 bits as an example, the prior art designs group a row of capacitors in a ratio of 1:2:4:8 and connect them to different bit positions of the input data respectively; among them, the lowest bit position is connected to S capacitors, the second lowest bit position is connected to 2S capacitors, and so on, the third bit is connected to 4S capacitors, and the highest bit is connected to 8S capacitors, thus forming a capacitance ratio relationship; during the charge and discharge phase, each capacitor stores charge according to the bit weight value of the input signal, and then by paralleling all the capacitors for charge sharing, a unified voltage output value is obtained, realizing the digital-to-analog conversion of the input data; this method effectively reduces the design complexity and power consumption of the DAC through a simple hardware implementation method.

[0008] In addition, the prior art can also use a charge accumulator based on the C2C structure to achieve 2's complement signed multi-bit charge accumulation. As shown in Figure 3, this design uses the C2C principle to complete the step-by-step coupling and accumulation of charges, where the capacitance value of the sampling capacitor is C and the capacitance value of the coupling capacitor is 2C. This scheme makes full use of the characteristics of the C2C structure, which not only meets the requirements of multi-bit calculations in deep networks but also takes into account the efficiency and reliability of hardware design.

[0009] However, although the traditional CIM architecture can efficiently complete MAC operations in the analog domain, Non-MAC operations (such as batch normalization and activation functions) still rely on digital domain processing. This design results in frequent digital-to-analog and analog-to-digital conversions during the calculation process, significantly increasing the hardware resource requirements and system complexity.

[0010] Specifically, as Figure 4As shown in the figure, since input data is usually transmitted to the CIM architecture in digital form, it is first necessary to convert the digital signal into an analog signal through a DAC (Digital Audio Compress, digital-to-analog converter) to perform MAC operations in the analog domain. However, since non-MAC operations can only be implemented in the digital domain, the MAC operation results need to be converted back into digital signals through an ADC (Analog-to-Digital Converter), and then operations such as batch normalization and activation functions are completed. Finally, when the processed digital signal is output to the next layer, it needs to be converted back into an analog signal through a DAC again to enter the next round of MAC calculations. This frequent signal conversion process greatly increases the number of DACs and ADCs and the hardware overhead, and at the same time has a series of negative impacts on system performance.

[0011] First of all, digital-to-analog conversion and analog-to-digital conversion not only increase the latency but also significantly increase the power consumption, thus reducing the overall energy efficiency of the system. Secondly, since the efficiency of digital computing is usually lower than that of analog computing, completing non-MAC operations in the digital domain further slows down the computing speed. Finally, to support frequent signal conversion and digital computing operations, the system needs to deploy a large number of DACs, ADCs, and digital processing units, resulting in a significant increase in the consumption of area resources and further reducing the area efficiency of the hardware.

[0012] In summary, the traditional CIM architecture's mode of relying on the digital domain to complete non-MAC operations has significant bottlenecks in key performance indicators such as speed, power consumption, and area, and it is difficult to meet the requirements of resource-constrained scenarios such as intelligent sensors, and it urgently needs to be solved. Summary of the Invention

[0013] This application provides a fully analog non-multiply-accumulate operation method and device for analog in-memory computing to solve the problems that the traditional CIM architecture's mode of relying on the digital domain to complete non-multiply-accumulate operations has significant bottlenecks in key performance indicators such as speed, power consumption, and area, and it is difficult to meet the requirements of resource-constrained scenarios such as intelligent sensors.

[0014] The first aspect embodiment of the present application provides a fully analog non-multiply-add operation method for analog in-memory computing, including the following steps: training a target full model to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and scaling factors; based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and the bias information, perform the multiply-add operation and the bias calculation of batch normalization in the target full model to obtain a corresponding bias multiply-add analog calculation result; based on a preset global resistor ladder voltage divider or the analog bias array, and in combination with the scaling factor, generate a target reference voltage of a preset ReLU-ADC, and set the minimum output voltage of the DAC of the ReLU-ADC; based on the target reference voltage and the minimum output voltage of the DAC, enable the ReLU-ADC to perform a fully analog non-multiply-add operation on the bias multiply-add analog calculation result, where the fully analog non-multiply-add operation includes a scaling function operation, a ReLU activation function operation, and a quantization operation.

[0015] Optionally, in an embodiment of the present application, the step of based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and the bias information, perform the multiply-add operation and the bias calculation of batch normalization in the target full model to obtain a corresponding bias multiply-add analog calculation result includes: paralleling a preset plurality of analog in-memory computing units to construct the analog in-memory computing array, and inputting the output result of the previous layer and the weights into the plurality of analog in-memory computing units to perform a multiply-add operation in an analog in-memory computing manner to obtain an analog multiply-add calculation result; constructing the analog bias array through the plurality of analog in-memory computing units, and inputting the bias information into the plurality of analog in-memory computing units to obtain a target analog bias signal; based on the analog in-memory computing array and the analog bias array, perform an analog addition operation on the analog multiply-add calculation result and the analog bias signal to obtain the bias multiply-add analog calculation result.

[0016] Optionally, in an embodiment of the present application, the step of based on a preset global resistor ladder voltage divider or the analog bias array, and in combination with the scaling factor, generate a target reference voltage of a preset ReLU-ADC includes: determining the global maximum reference voltage of the ReLU-ADC based on the scaling factor; determining a target reference voltage range based on the minimum output voltage of the DAC and the global maximum reference voltage, and generating a plurality of equally spaced reference voltage values according to the target reference voltage range and the global resistor ladder voltage divider; selecting any one of the plurality of equally spaced reference voltage values by a local selector in the ReLU-ADC according to a preset selection signal, and determining the target reference voltage according to the selected reference voltage value.

[0017] Optionally, in an embodiment of the present application, based on the preset global resistance ladder voltage divider or the analog bias array, and in combination with the scaling factor, to generate the target reference voltage of the preset ReLU-ADC, further includes: inputting the scaling factor into the analog bias array to generate the target reference voltage.

[0018] Optionally, in an embodiment of the present application, based on the target reference voltage and the minimum output voltage of the DAC, the ReLU-ADC performs a full analog non-multiplication and addition operation on the bias multiplication and addition analog calculation result, where the full analog non-multiplication and addition operation includes a scaling function operation, a ReLU activation function operation, and a quantization operation, including: adjusting the quantization range of the preset SAR-ADC by using the target reference voltage in the ReLU-ADC to perform the scaling function operation in the full analog non-multiplication and addition operation; setting the lower limit of the quantization range of the SAR-ADC by using the minimum output voltage of the DAC in the ReLU-ADC to perform the ReLU activation function operation in the full analog non-multiplication and addition operation; performing analog-to-digital conversion and quantization operation on the bias multiplication and addition analog calculation result in the ReLU-ADC in the SAR-ADC manner.

[0019] An embodiment of the second aspect of the present application provides a full analog non-multiplication and addition operation device for analog in-memory computing, including: a training module, configured to train a target full model to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and a scaling factor; a multiplication and addition operation module, configured to perform the multiplication and addition operation and the bias calculation of batch normalization in the target full model based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and the bias information, to obtain a corresponding bias multiplication and addition analog calculation result; a generation module, configured to generate a target reference voltage of a preset ReLU-ADC based on a preset global resistance ladder voltage divider or the analog bias array, and in combination with the scaling factor, and set the minimum output voltage of the DAC of the ReLU-ADC; a non-multiplication and addition operation module, configured to make the ReLU-ADC perform a full analog non-multiplication and addition operation on the bias multiplication and addition analog calculation result based on the target reference voltage and the minimum output voltage of the DAC, where the full analog non-multiplication and addition operation includes a scaling function operation, a ReLU activation function operation, and a quantization operation.

[0020] Optionally, in an embodiment of the present application, the multiply-accumulate operation module includes: a first construction unit configured to parallelize a plurality of preset analog in-memory computing units to construct the analog in-memory computing array, and input the output result of the previous layer and the weight value into the plurality of analog in-memory computing units to perform a multiply-accumulate operation in an analog in-memory computing manner to obtain an analog multiply-accumulate calculation result; a second construction unit configured to construct the analog bias array through the plurality of analog in-memory computing units, and input the bias information into the plurality of analog in-memory computing units to obtain a target analog bias signal; an analog addition unit configured to perform an analog addition operation on the analog multiply-accumulate calculation result and the analog bias signal based on the analog in-memory computing array and the analog bias array to obtain the bias multiply-accumulate analog calculation result.

[0021] Optionally, in an embodiment of the present application, the generation module includes: a first determination unit configured to determine the global maximum reference voltage of the ReLU-ADC based on the scaling factor; a second determination unit configured to determine a target reference voltage range based on the minimum output voltage of the DAC and the global maximum reference voltage, and generate a plurality of equally spaced reference voltage values according to the target reference voltage range and the global resistor ladder voltage divider; a selection unit configured to select any one of the plurality of equally spaced reference voltage values through a local selector in the ReLU-ADC according to a preset selection signal, and determine the target reference voltage according to the selected reference voltage value.

[0022] Optionally, in an embodiment of the present application, the generation module further includes: a third determination unit configured to input the scaling factor into the analog bias array to generate the target reference voltage.

[0023] Optionally, in an embodiment of the present application, the non-multiply-accumulate operation module includes: a scaling unit configured to adjust the quantization range of a preset SAR-ADC in the ReLU-ADC by using the target reference voltage to perform a scaling function operation in the all-analog non-multiply-accumulate operation; a ReLU function implementation unit configured to set the lower limit of the quantization range of the SAR-ADC in the ReLU-ADC by using the minimum output voltage of the DAC to perform a ReLU activation function operation in the all-analog non-multiply-accumulate operation; a quantization unit configured to perform analog-to-digital conversion and quantization operations on the bias multiply-accumulate analog calculation result in the ReLU-ADC in a SAR-ADC manner.

[0024] A third aspect embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the all-analog non-multiplication and addition operation method for analog in-memory computing as described in the above embodiments.

[0025] A fourth aspect embodiment of the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the all-analog non-multiplication and addition operation method for analog in-memory computing as described above.

[0026] A fifth aspect embodiment of the present application provides a computer program product including a computer program, which is executed to implement the above-mentioned all-analog non-multiplication and addition operation method for analog in-memory computing.

[0027] Therefore, the embodiments of the present application have the following beneficial effects:

[0028] The embodiments of the present application can train a target full model to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and scaling factors; based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and bias information, perform multiplication and addition operations and bias calculations for batch normalization in the target full model to obtain corresponding bias multiplication and addition analog calculation results; based on a preset global resistor ladder voltage divider or an analog bias array, and in combination with the scaling factor, generate a target reference voltage for a preset ReLU-ADC and set the minimum output voltage of the DAC of the ReLU-ADC; based on the target reference voltage and the minimum output voltage of the DAC, enable the ReLU-ADC to perform all-analog non-multiplication and addition operations on the bias multiplication and addition analog calculation results, where the all-analog non-multiplication and addition operations include scaling function operations, ReLU activation function operations, and quantization operations. The present application implements Non-MAC operations in the analog domain, thereby eliminating the need for frequent digital-to-analog conversions and significantly improving the performance and efficiency of the CIM architecture. Thus, it solves the problems that the mode of relying on the digital domain to complete non-multiplication and addition operations in the traditional CIM architecture has significant bottlenecks in key performance indicators such as speed, power consumption, and area, and it is difficult to meet the requirements of resource-constrained scenarios such as intelligent sensors.

[0029] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Description of the Drawings

[0030] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0031] Figure 1Schematic diagram of a sub-unit, MAC array, and analog-digital hybrid in-memory computing module with reconfigurable bit width;

[0032] Figure 2 Schematic diagram of a CIM array structure;

[0033] Figure 3 Schematic diagram of a C2C accumulator structure;

[0034] Figure 4 Schematic diagram of the calculation process in a traditional CIM architecture;

[0035] Figure 5 Flowchart of a fully analog non-multiply-accumulate operation method for analog in-memory computing according to an embodiment of the present application;

[0036] Figure 6 Schematic diagram of the calculation process in a fully analog CIM architecture provided by an embodiment of the present application;

[0037] Figure 7 Schematic diagram of a CIM unit structure provided by an embodiment of the present application;

[0038] Figure 8 Schematic diagram of a ReLU-ADC structure provided by an embodiment of the present application;

[0039] Figure 9 Schematic diagram of the structure of a global resistor ladder voltage divider and a DC selector provided by an embodiment of the present application;

[0040] Figure 10 Schematic diagram of the data flow of a non-MAC operation provided by an embodiment of the present application;

[0041] Figure 11 Schematic diagram of the relationship between the voltage domain and the numerical domain during a non-MAC operation provided by an embodiment of the present application;

[0042] Figure 12 Example diagram of a fully analog non-multiply-accumulate operation device for analog in-memory computing according to an embodiment of the present application;

[0043] Figure 13 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application.

[0044] Among them, 10 - fully analog non-multiply-accumulate operation device for analog in-memory computing; 100 - training module, 200 - multiply-accumulate operation module, 300 - generation module, 400 - non-multiply-accumulate operation module; 1301 - memory, 1302 - processor, 1303 - communication interface. Detailed implementation manners

[0045] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation of the present application.

[0046] The following describes a fully analog non-multiplication and addition operation method and device for analog in-memory computing according to an embodiment of the present application. In view of the problems mentioned in the above background art, the present application provides a fully analog non-multiplication and addition operation method for analog in-memory computing. In this method, a target full model is trained to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and scaling factors; based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and bias information, perform multiplication and addition operations and bias calculations for batch normalization in the target full model to obtain corresponding bias multiplication and addition analog calculation results; based on a preset global resistor ladder voltage divider or an analog bias array, and in combination with the scaling factor, generate a target reference voltage for a preset ReLU-ADC, and set the minimum output voltage of the DAC of the ReLU-ADC; based on the target reference voltage and the minimum output voltage of the DAC, enable the ReLU-ADC to perform a fully analog non-multiplication and addition operation on the bias multiplication and addition analog calculation results, where the fully analog non-multiplication and addition operation includes a scaling function operation, a ReLU activation function operation, and a quantization operation. The present application realizes Non-MAC operations in the analog domain, thereby eliminating the need for frequent digital-to-analog conversions and significantly improving the performance and efficiency of the CIM architecture. Thus, it solves the problems that the mode of relying on the digital domain to complete non-MAC operations in the traditional CIM architecture has significant bottlenecks in key performance indicators such as speed, power consumption, and area, and it is difficult to meet the requirements of resource-constrained scenarios such as intelligent sensors.

[0047] Specifically, Figure 5 FIG. is a flowchart of a fully analog non-multiplication and addition operation method for analog in-memory computing provided by an embodiment of the present application.

[0048] As Figure 5 shown, the fully analog non-multiplication and addition operation method for analog in-memory computing includes the following steps:

[0049] In step S501, train a target full model to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and scaling factors.

[0050] In step S502, based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and bias information, perform multiplication and addition operations and bias calculations for batch normalization in the target full model to obtain corresponding bias multiplication and addition analog calculation results.

[0051] In the embodiments of the present application, the target full model can be trained first to obtain corresponding BN parameters such as bias information and scaling factors, and a bias unit (i.e., an analog bias array) is introduced into the CIM array (i.e., an analog in-memory computing array), and the bias calculation in batch normalization is directly processed in the CIM array through the bias unit to obtain the corresponding bias calculation result (i.e., the bias multiplication-addition analog calculation result).

[0052] Optionally, in an embodiment of the present application, based on a preset analog in-memory computing array and an analog bias array, combined with weight values and bias information, the multiplication-addition operation and the bias calculation of batch normalization in the target full model are performed to obtain the corresponding bias multiplication-addition analog calculation result, including: paralleling a preset plurality of analog in-memory computing units to construct an analog in-memory computing array, and inputting the output result of the previous layer and the weight values into the plurality of analog in-memory computing units to perform the multiplication-addition operation in the analog in-memory computing manner to obtain an analog multiplication-addition calculation result; constructing an analog bias array through the plurality of analog in-memory computing units, and inputting the bias information into the plurality of analog in-memory computing units to obtain a target analog bias signal; based on the analog in-memory computing array and the analog bias array, performing an analog addition operation on the analog multiplication-addition calculation result and the analog bias signal to obtain the bias multiplication-addition analog calculation result.

[0053] In the actual execution process, the embodiments of the present application can obtain the parameters of the corresponding BN operation through model training, and after simplifying them into bias information P and scaling factor Q, map them to the analog domain to complete related calculations in the analog domain.

[0054] It should be noted that, as Figure 4 shown, the CIM array in the embodiments of the present application is composed of a plurality of CIM units (i.e., a plurality of analog in-memory computing units), and each CIM unit is used to store analog weight values and perform an analog multiplication operation according to the input value to output an analog signal (such as current or charge), as Figure 6 shown; by paralleling a plurality of CIM units, an accumulation operation can be realized in the analog domain, thereby completing the analog MAC calculation. To further expand the functions of the CIM array, the embodiments of the present application can introduce a bias unit into the CIM array, and the structure of the bias unit is similar to that of a traditional computing unit, as Figure 7 shown.

[0055] Thus, the embodiments of the present application can ensure that the bias unit can stably output a fixed analog value, thereby realizing offset adjustment in the MAC calculation result, so that the bias unit can efficiently perform the bias adjustment function in the analog domain and complete the bias processing in Non-MAC operations.

[0056] In step S503, based on a preset global resistor ladder voltage divider or an analog bias array, and in combination with a scaling factor, a target reference voltage of the preset ReLU-ADC is generated, and the minimum output voltage of the DAC of the ReLU-ADC is set.

[0057] Furthermore, an embodiment of the present application can construct an ADC with scaling and ReLU activation function capabilities, namely, ReLU-ADC, whose structure is as Figure 8 shown; this ReLU-ADC is designed based on a traditional 4-bit successive approximation register (SAR) ADC (i.e., SAR-ADC), and the following improvements are made to it:

[0058] 1. The reference voltage V ref of the ReLU-ADC is programmable, making its range flexibly adjustable;

[0059] 2. The minimum output voltage of the DAC of the ReLU-ADC is set to That is, only the positive part in the voltage domain is retained, realizing the ReLU activation function.

[0060] Thus, the embodiment of the present application uses SAR-ADC to perform analog data conversion and quantization, and realizes the ReLU activation function in the ADC through the minimum output voltage of the DAC, enabling it to have scaling and ReLU activation function capabilities, thereby completing MAC and non-MAC operations in the analog domain, realizing a full analog computing mode, ensuring higher computing efficiency and lower hardware complexity, and solving the problem of analog computing integrity caused by relying on digital circuits in the traditional CIM architecture.

[0061] Optionally, in an embodiment of the present application, based on a preset global resistor ladder voltage divider or an analog bias array, and in combination with a scaling factor, to generate a target reference voltage of the preset ReLU-ADC, including: determining the global maximum reference voltage of the ReLU-ADC based on the scaling factor; determining a target reference voltage range based on the minimum output voltage of the DAC and the global maximum reference voltage, and generating a plurality of equally spaced reference voltage values according to the target reference voltage range and the global resistor ladder voltage divider; selecting any one of the plurality of equally spaced reference voltage values through a local selector in the ReLU-ADC according to a preset selection signal, and determining the target reference voltage according to the selected reference voltage value.

[0062] As a realizable way, an embodiment of the present application can use a resistor ladder voltage divider to generate a reference voltage (i.e., the target reference voltage).

[0063] Specifically, as Figure 9 shown, to achieve a flexibly adjustable reference voltage Vref And to reduce resource overhead, embodiments of the present application can use a global resistor ladder voltage divider to generate 16 equally spaced voltage values, whose range (i.e., the target reference voltage range) is from to the maximum reference voltage of the entire system (i.e., the global maximum reference voltage) V top . Each ReLU-ADC is equipped with a local selector for selecting any one of the 16 reference voltage values as the target reference voltage V ref according to the selection signal K; wherein, the selection signal K is independently programmable, so as to realize the independent selection of V ref .

[0064] Thus, embodiments of the present application use a global resistor ladder voltage divider and the maximum reference voltage V top in the ReLU-ADC to generate a set of equally spaced reference voltage values, and independently select one of the reference voltage values V ref through the local selector signal for quantization, providing flexible reference voltage selection, thus realizing an ADC with programmable range, and being able to adjust the scaling operation, avoiding the digital storage overhead and digital-to-analog conversion overhead required for storing reference voltage values for each ADC in the traditional method.

[0065] Optionally, in an embodiment of the present application, based on a preset global resistor ladder voltage divider or an analog bias array, and combined with a scaling factor, to generate the target reference voltage of the preset ReLU-ADC, further includes: inputting the scaling factor into the analog bias array to generate the target reference voltage.

[0066] In addition, in addition to using a resistor ladder voltage divider, embodiments of the present application can also use a CIM array to generate the target reference voltage V ref ; similar to the working principle of the bias unit, embodiments of the present application can input a predetermined signal and weight value into the CIM array, be able to generate the required voltage value, and use this voltage value as the reference voltage input into the ReLU-ADC.

[0067] Thus, embodiments of the present application make full use of the flexibility and resource sharing characteristics of the CIM array, combine the efficiency and flexibility of analog computing, not only reduce the requirements for digital circuits and complex hardware designs and hardware complexity, but also improve the overall system efficiency, thereby being able to further reduce the hardware cost while improving the flexibility and reliability of reference voltage generation.

[0068] In step S504, based on the target reference voltage and the minimum output voltage of the DAC, the ReLU-ADC performs a full analog non-multiplication and addition operation on the bias multiplication and addition analog calculation result, where the full analog non-multiplication and addition operation includes a scaling function operation, a ReLU activation function operation, and a quantization operation.

[0069] Optionally, in an embodiment of the present application, based on the target reference voltage and the minimum output voltage of the DAC, the ReLU-ADC performs all-analog non-multiplication and addition operations on the bias multiplication and addition analog calculation results, where the all-analog non-multiplication and addition operations include scaling function operations, ReLU activation function operations, and quantization operations, including: adjusting the quantization range of the preset SAR-ADC by using the target reference voltage in the ReLU-ADC to perform the scaling function operation in the all-analog non-multiplication and addition operations; setting the lower limit of the SAR-ADC quantization range by using the minimum output voltage of the DAC in the ReLU-ADC to perform the ReLU activation function operation in the all-analog non-multiplication and addition operations; performing analog-to-digital conversion and quantization operations on the bias multiplication and addition analog calculation results in the ReLU-ADC in the SAR-ADC manner.

[0070] In the actual execution process, in the ReLU-ADC, the embodiment of the present application performs a scaling operation through a programmable reference voltage (i.e., the target reference voltage) and integrates the scaling operation into the ADC to be implemented in the analog domain, thereby supporting free adjustment of the quantization range of the SAR-ADC and supporting different scaling factors for batch normalization to achieve the scaling function; secondly, the embodiment of the present application can also use the minimum output voltage of the DAC in the ReLU-ADC to set the lower limit of the SAR-ADC quantization range to achieve the ReLU activation function; in addition, the embodiment of the present application can also perform analog-to-digital conversion and quantization on the bias multiplication and addition analog calculation results in the ReLU-ADC in the SAR-ADC manner.

[0071] Thus, the embodiment of the present application directly processes non-multiplication and addition operations in the analog domain through the ReLU-ADC and the bias unit, which has higher computational parallelism and efficiency compared to traditional digital circuit solutions, greatly improving the processing speed.

[0072] It should be noted that the output voltage V of the DAC DAC can be expressed as:

[0073]

[0074] wherein, V DAC represents the target DAC output voltage; D[b] represents the SAR logic signal of the b-th bit; b represents the b-th bit; V ref represents the target reference voltage; represents the inverse signal of the SAR logic of the b-th bit; V DD represents the power supply voltage.

[0075] As shown above, the minimum input value of the ADC is and its quantization step is determined by the programmable V refControl is such that the ReLU-ADC integrates ADC quantization, ReLU activation, and adjustable scaling functions.

[0076] Embodiments of the present application can directly calculate non-MAC operations (BN, ReLU, and quantization) in the analog domain through the ReLU-ADC and the bias unit. The following details and explains the non-MAC operation process from a mathematical perspective. The mathematical expression of the non-MAC operation process is:

[0077]

[0078] Among them, Quant(X) rounds X to the nearest integer, and clip(X) limits X between 0 and 15. In addition, s, μ, σ, γ, and β are BN parameters obtained through model training, and after conversion, they become the bias P and the scaling factor Q respectively.

[0079] Figure 10 Shows the non-MAC data stream of the embodiments of the present application. The bias X+P is directly implemented in the CIM array through the bias unit, while the remaining operations are integrated into the ReLU-ADC. Figure 11 Shows the relationship between non-MAC calculation in the voltage domain and the numerical domain; by assigning different numerical values to the same voltage before and after the ADC, the scaling of X×Q is achieved; before conversion, V DD represents the maximum numerical output N after MAC max , and the V of the numerical value ref is defined as After conversion, from to V ref the voltage is quantized between 0 and 15, simulating the scaling multiplication and the ADC has built-in clip(·) and Quant(·) functions.

[0080] For a given coefficient Q, the ideal reference voltage is However, in a CNN, each output channel usually requires a unique scaler Q, so an independent V ref needs to be provided for each channel. To minimize hardware resources, embodiments of the present application propose a learnable global highest reference voltage V top , and the local reference voltage V ref is selected as the closest available value among 16 equally spaced values from to V top , thus approximating the ideal

[0081] Thus, by combining the programmable bias units in the CIM array, the embodiments of the present application can complete the complete calculation of Non-MAC operations such as batch normalization, ReLU activation function, and quantization in the analog domain without relying on the digital circuits in the traditional architecture, avoiding the calculation method relying on digital circuits in the traditional architecture, reducing the number of digital-to-analog converters (DAC and ADC), and reducing power consumption and area.

[0082] According to the all-analog non-multiply-accumulate operation method for analog in-memory computing proposed by the embodiments of the present application, a target full model is trained to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and scaling factors; based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and bias information, to perform the multiply-accumulate operation and the bias calculation of batch normalization in the target full model, obtaining the corresponding bias multiply-accumulate analog calculation result; based on a preset global resistor ladder voltage divider or an analog bias array, and in combination with the scaling factor, to generate the target reference voltage of a preset ReLU-ADC, and set the minimum output voltage of the DAC of the ReLU-ADC; based on the target reference voltage and the minimum output voltage of the DAC, enabling the ReLU-ADC to perform all-analog non-multiply-accumulate operations on the bias multiply-accumulate analog calculation result, where the all-analog non-multiply-accumulate operations include scaling function operations, ReLU activation function operations, and quantization operations. The present application realizes Non-MAC operations in the analog domain, thereby eliminating the frequent digital-to-analog conversion requirements and significantly improving the performance and efficiency of the CIM architecture.

[0083] Secondly, an all-analog non-multiply-accumulate operation device for analog in-memory computing proposed by the embodiments of the present application is described with reference to the accompanying drawings.

[0084] Figure 12 It is a block diagram of the all-analog non-multiply-accumulate operation device for analog in-memory computing according to the embodiments of the present application.

[0085] As Figure 12 shown, the all-analog non-multiply-accumulate operation device 10 for analog in-memory computing includes: a training module 100, a multiply-accumulate operation module 200, a generation module 300, and a non-multiply-accumulate operation module 400.

[0086] Among them, the training module 100 is used to train a target full model to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and scaling factors.

[0087] The multiply-accumulate operation module 200 is used to perform the multiply-accumulate operation and the bias calculation of batch normalization in the target full model based on a preset analog in-memory computing array and an analog bias array, and in combination with the weights and bias information, obtaining the corresponding bias multiply-accumulate analog calculation result.

[0088] A generation module 300 is configured to generate a target reference voltage of a preset ReLU-ADC based on a preset global resistor ladder voltage divider or an analog bias array, in combination with a scaling factor, and set the minimum output voltage of the DAC of the ReLU-ADC.

[0089] A non-multiply-add operation module 400 is configured to perform a full analog non-multiply-add operation on the bias multiply-add analog calculation result by the ReLU-ADC based on the target reference voltage and the minimum output voltage of the DAC, where the full analog non-multiply-add operation includes a scaling function operation, a ReLU activation function operation, and a quantization operation.

[0090] Optionally, in an embodiment of the present application, the multiply-add operation module 200 includes: a first construction unit, a second construction unit, and an analog addition unit.

[0091] The first construction unit is configured to connect in parallel a preset plurality of analog in-memory computing units to construct an analog in-memory computing array, and input the output result and weights of the previous layer into the plurality of analog in-memory computing units to perform a multiply-add operation in an analog in-memory computing manner to obtain an analog multiply-add calculation result.

[0092] The second construction unit is configured to construct an analog bias array through the plurality of analog in-memory computing units and input bias information into the plurality of analog in-memory computing units to obtain a target analog bias signal.

[0093] The analog addition unit is configured to perform an analog addition operation on the analog multiply-add calculation result and the analog bias signal based on the analog in-memory computing array and the analog bias array to obtain a bias multiply-add analog calculation result.

[0094] Optionally, in an embodiment of the present application, the generation module 300 includes: a first determination unit, a second determination unit, and a selection unit.

[0095] The first determination unit is configured to determine the global maximum reference voltage of the ReLU-ADC based on the scaling factor.

[0096] The second determination unit is configured to determine a target reference voltage range based on the minimum output voltage of the DAC and the global maximum reference voltage, and generate a plurality of equally spaced reference voltage values according to the target reference voltage range and the global resistor ladder voltage divider.

[0097] The selection unit is configured to select any one of the plurality of equally spaced reference voltage values by a local selector in the ReLU-ADC according to a preset selection signal, and determine the target reference voltage according to the selected reference voltage value.

[0098] Optionally, in an embodiment of the present application, the generation module 300 further includes: a third determination unit, configured to input a scaling factor into the analog bias array to generate a target reference voltage.

[0099] Optionally, in an embodiment of the present application, the non-multiplication and addition operation module 400 includes: a scaling unit, a ReLU function implementation unit, and a quantization unit.

[0100] Among them, the scaling unit is configured to adjust the quantization range of a preset SAR-ADC in the ReLU-ADC by using the target reference voltage to perform a scaling function operation in the all-analog non-multiplication and addition operation.

[0101] The ReLU function implementation unit is configured to set the lower limit of the SAR-ADC quantization range by using the minimum output voltage of the DAC in the ReLU-ADC to perform a ReLU activation function operation in the all-analog non-multiplication and addition operation.

[0102] The quantization unit is configured to perform analog-to-digital conversion and quantization operations on the bias multiplication and addition analog calculation result in the ReLU-ADC in the SAR-ADC manner.

[0103] It should be noted that the foregoing explanation of the embodiment of the all-analog non-multiplication and addition operation method for analog in-memory computing is also applicable to the all-analog non-multiplication and addition operation device for analog in-memory computing in this embodiment, and will not be elaborated here.

[0104] An all-analog non-multiplication and addition operation device for analog in-memory computing according to an embodiment of the present application includes a training module 100, configured to train a target full model to obtain the weights and batch normalization parameters of the target full model, where the batch normalization parameters include bias information and a scaling factor; a multiplication and addition operation module 200, configured to perform multiplication and addition operations and bias calculations for batch normalization in the target full model based on a preset analog in-memory computing array and an analog bias array, in combination with the weights and bias information, to obtain a corresponding bias multiplication and addition analog calculation result; a generation module 300, configured to generate a target reference voltage of a preset ReLU-ADC based on a preset global resistor ladder voltage divider or an analog bias array, in combination with the scaling factor, and set the minimum output voltage of the DAC of the ReLU-ADC; a non-multiplication and addition operation module 400, configured to enable the ReLU-ADC to perform an all-analog non-multiplication and addition operation on the bias multiplication and addition analog calculation result based on the target reference voltage and the minimum output voltage of the DAC, where the all-analog non-multiplication and addition operation includes a scaling function operation, a ReLU activation function operation, and a quantization operation. The present application realizes Non-MAC operations in the analog domain, thereby eliminating the need for frequent digital-to-analog conversions and significantly improving the performance and efficiency of the CIM architecture.

[0105] Figure 13Schematic diagram of the structure of the electronic device provided by the embodiments of the present application. The electronic device may include:

[0106] A memory 1301, a processor 1302, and a computer program stored on the memory 1301 and executable on the processor 1302.

[0107] When the processor 1302 executes the program, it implements the all-analog non-multiplication and addition operation method for analog in-memory computing provided in the above embodiments.

[0108] Further, the electronic device further includes:

[0109] A communication interface 1303 for communication between the memory 1301 and the processor 1302.

[0110] The memory 1301 is used to store a computer program executable on the processor 1302.

[0111] The memory 1301 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0112] If the memory 1301, the processor 1302, and the communication interface 1303 are implemented independently, the communication interface 1303, the memory 1301, and the processor 1302 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 13 only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0113] Optionally, in a specific implementation, if the memory 1301, the processor 1302, and the communication interface 1303 are integrated on a chip, the memory 1301, the processor 1302, and the communication interface 1303 may communicate with each other through an internal interface.

[0114] The processor 1302 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0115] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned all-analog non-multiplication and addition operation method for analog in-memory computing is implemented.

[0116] The embodiments of the present application also provide a computer program product, including a computer program, and when the computer program is executed, it is used to implement the above-mentioned all-analog non-multiplication and addition operation method for analog in-memory computing.

[0117] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or N embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0118] In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0119] Any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of the code including one or N executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, and this should be understood by those skilled in the art of the embodiments of the present application.

[0120] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part (electronic device) having one or N wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0121] It should be understood that the various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented using hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0122] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0123] In addition, each functional unit in various embodiments of the present application may be integrated into one processing module, may exist separately as individual physical units, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0124] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A fully analog non-multiplication-addition operation method for analog in-memory computing, characterized in that: The following steps are involved: Training a target full model to obtain weights and batch normalization parameters of the target full model, wherein the batch normalization parameters include bias information and a scaling factor; Based on the preset analog in-memory calculation array and analog bias array, and in combination with the weights and the bias information, multiplication and addition operations in the target full model and batch normalized bias calculations are performed to obtain corresponding bias multiplication and addition simulation calculation results; Based on a preset global resistor ladder divider or the analog bias array, and in combination with the scaling factor, to generate a preset target reference voltage of the ReLU-ADC and set a DAC minimum output voltage of the ReLU-ADC; Based on the target reference voltage and the DAC minimum output voltage, the ReLU-ADC performs a full analog non-multiplication-addition operation on the bias multiplication-addition analog calculation result, wherein the full analog non-multiplication-addition operation includes a scaling function operation, a ReLU activation function function operation and a quantization operation.

2. The method according to claim 1, characterized in that The preset analog in-memory calculation array and analog bias array are combined with the weights and the bias information to perform multiplication and addition operations in the target full model and batch normalized bias calculations to obtain corresponding bias multiplication and addition simulation calculation results, including: Connecting a plurality of preset analog in-memory computing units in parallel to construct the analog in-memory computing array, and inputting the output results of the previous layer and the weights into the plurality of analog in-memory computing units, performing multiplication and addition operations in an analog in-memory computing manner, and obtaining analog multiplication and addition calculation results; Constructing the analog bias array through the multiple analog in-memory computing units, and inputting the bias information into the multiple analog in-memory computing units to obtain a target analog bias signal; Based on the analog in-memory calculation array and the analog bias array, an analog addition operation is performed on the analog multiplication-addition calculation result and the analog bias signal to obtain the biased multiplication-addition analog calculation result.

3. The method according to claim 1, characterized in that The preset global resistor ladder voltage divider or the analog bias array is combined with the scaling factor to generate a preset target reference voltage of the ReLU-ADC, including: Based on the scaling factor, determining a global maximum reference voltage of the ReLU-ADC; Determining a target reference voltage range based on the DAC minimum output voltage and the global maximum reference voltage, and generating a plurality of equally spaced reference voltage values ​​according to the target reference voltage range and the global resistor ladder divider; A local selector in the ReLU-ADC selects any one reference voltage value from the multiple equidistant reference voltage values ​​according to a preset selection signal, and determines the target reference voltage according to the selected reference voltage value.

4. The method according to claim 1, characterized in that: The preset global resistor ladder voltage divider or the analog bias array is combined with the scaling factor to generate a preset target reference voltage of the ReLU-ADC, and further includes: The scaling factor is input into the analog bias array to generate the target reference voltage.

5. The method according to claim 1, characterized in that Based on the target reference voltage and the DAC minimum output voltage, the ReLU-ADC performs a full analog non-multiplication-addition operation on the bias multiplication-addition analog calculation result, wherein the full analog non-multiplication-addition operation includes a scaling function operation, a ReLU activation function function operation and a quantization operation, including: In the ReLU-ADC, the target reference voltage is used to adjust the quantization range of the preset SAR-ADC to perform the scaling function operation in the full analog non-multiplication-addition operation; In the ReLU-ADC, the lower limit of the quantization range of the SAR-ADC is set by using the DAC minimum output voltage to perform the ReLU activation function operation in the full analog non-multiplication-addition operation; In the ReLU-ADC, analog-to-digital conversion and quantization operations are performed on the bias multiplication and addition analog calculation results in a SAR-ADC manner.

6. A fully analog non-multiplication-addition computing device for analog in-memory computing, characterized in that: include: A training module, used for training a target full model to obtain weights and batch normalization parameters of the target full model, wherein the batch normalization parameters include bias information and a scaling factor; A multiplication-addition operation module, used to perform multiplication-addition operations and batch normalized bias calculations in the target full model based on a preset analog in-memory calculation array and an analog bias array, and in combination with the weights and the bias information, to obtain corresponding bias multiplication-addition simulation calculation results; A generating module, configured to generate a preset target reference voltage of the ReLU-ADC based on a preset global resistor ladder divider or the analog bias array and in combination with the scaling factor, and to set a DAC minimum output voltage of the ReLU-ADC; The non-multiplication-addition operation module is used to enable the ReLU-ADC to perform a full-analog non-multiplication-addition operation on the bias multiplication-addition analog calculation result based on the target reference voltage and the DAC minimum output voltage, wherein the full-analog non-multiplication-addition operation includes a scaling function operation, a ReLU activation function function operation and a quantization operation.

7. The device according to claim 6, characterized in that The multiplication and addition operation module comprises: A first construction unit is used to connect a plurality of preset analog in-memory computing units in parallel to construct the analog in-memory computing array, and input the output result of the previous layer and the weight value into the plurality of analog in-memory computing units to perform multiplication and addition operations in an analog in-memory computing manner to obtain a simulated multiplication and addition calculation result; A second construction unit, configured to construct the analog bias array by using the plurality of analog in-memory computing units, and input the bias information into the plurality of analog in-memory computing units to obtain a target analog bias signal; The analog addition unit is used to perform an analog addition operation on the analog multiplication and addition calculation result and the analog bias signal based on the analog in-memory calculation array and the analog bias array to obtain the biased multiplication and addition analog calculation result.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the full-analog non-multiplication-addition operation method for analog in-memory calculation as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the full-analog non-multiplication-addition operation method for analog in-memory calculation as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the full analog non-multiplication-addition operation method for analog in-memory calculation as described in any one of claims 1-5.

Images