Analog arithmetic unit and neuromorphic device

Abstract

An analog arithmetic unit according to an embodiment includes a first input terminal, a voltage-current conversion circuit, a current addition / subtraction circuit, a current-voltage conversion circuit, and a division circuit. The voltage-current conversion circuit performs exponential function conversion on an input voltage applied to the first input terminal and outputs the input voltage as a first current. The current adding circuit obtains the sum or difference of the first current and the reference current or the conversion current. A current-to-voltage conversion circuit converts the first current, the sum, and the difference into a voltage. The division circuit obtains the ratio of the first voltage and the second voltage obtained by the current-voltage conversion circuit.

Description

Technical Field

[0001] This disclosure relates to analog computing units and neuromorphic devices. This application claims priority based on US18 / 190,264, filed March 27, 2023, the contents of which are incorporated herein by reference. Background Technology

[0002] In recent years, edge computing, which distributes data processing, has attracted much attention. Edge computing is the opposite of cloud computing, which centrally processes data in the cloud. Edge computing has the advantages of not requiring large-scale servers like cloud computing, reducing network load, and easily enhancing security.

[0003] On the other hand, it is difficult to install large-scale computing units on edge terminals used for edge computing, which requires reducing the computing load on edge terminals.

[0004] The computational load of software-based neural networks is high. Software computations are typically performed using digital processing units (DSPs) on general-purpose devices such as CPUs and GPUs. High DSP load leads to longer processing times and increased power consumption. Therefore, proposals have included devices specifically designed for neural networks. For example, Patent Document 1 discloses a reservoir element that converts the computations performed by a reservoir in a neural network into analog processing using physical devices.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: International Publication No. 2021 / 192069 Summary of the Invention

[0008] (a) Technical problems to be solved

[0009] In neural networks, when the input is output to the next neuron, the result of the multiplication-accumulation operation is substituted into the activation function for a nonlinear transformation. If the activation function is processed by a digital arithmetic unit, the processing steps are numerous and the computation time is consumed. In addition, if the computational load is high, the power consumption of the device will increase.

[0010] This disclosure was made in view of the above circumstances, and provides an analog processor that converts the operation of an activation function into an analog processing using a physical device.

[0011] (II) Technical Solution

[0012] The analog arithmetic unit of this embodiment includes a first input terminal, a voltage-to-current conversion circuit, a current addition / subtraction circuit, a current-to-voltage conversion circuit, and a division circuit. The voltage-to-current conversion circuit performs an exponential function transformation on the input voltage applied to the first input terminal and outputs it as a first current. The current addition circuit obtains at least one of the sum and difference between the first current and a reference current or one or more converted currents, wherein one or more of the converted currents are obtained by performing an exponential function transformation on an input voltage applied to at least one input terminal different from the first input terminal. The current-to-voltage conversion circuit converts at least two of the first current, the sum, and the difference into voltages. The division circuit obtains the ratio of the first voltage obtained by the current-to-voltage conversion circuit to a second voltage.

[0013] (III) Beneficial Effects

[0014] The analog arithmetic unit described above can simulate the operation of the activation function through physical circuits, etc. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the first example of a neural network simulated by a neuromorphic device according to the first embodiment.

[0016] Figure 2 This is a schematic diagram of a second example of a neural network simulated by a neuromorphic device according to the first embodiment.

[0017] Figure 3 This is a schematic diagram of the analog arithmetic unit of the first embodiment.

[0018] Figure 4 This is a schematic diagram of the voltage amplifier circuit of the first embodiment.

[0019] Figure 5 This is a conceptual diagram of the computational processing when the Softmax function is operated entirely by a digital arithmetic unit.

[0020] Figure 6 This is a schematic diagram of another example of the analog arithmetic unit of the first embodiment.

[0021] Figure 7 This is a schematic diagram of the analog arithmetic unit according to the second embodiment.

[0022] Figure 8 This is a conceptual diagram of the computation process when the Softmax function is operated on using a simulation arithmetic unit.

[0023] Figure 9 This is a schematic diagram of another example of the analog arithmetic unit of the second embodiment.

[0024] Figure 10This is a schematic diagram of the analog arithmetic unit according to the third embodiment.

[0025] Figure 11 This is a schematic diagram of another example of the analog arithmetic unit in the third embodiment.

[0026] Figure 12 This is a schematic diagram of the analog arithmetic unit according to the fourth embodiment. Detailed Implementation

[0027] The present embodiment will now be described in detail with appropriate reference to the accompanying drawings. In the drawings used in the following description, for ease of understanding of the features of this disclosure, characteristic parts are sometimes shown enlarged, and the dimensional ratios of structural elements may sometimes differ from the actual dimensions. The materials, dimensions, etc., illustrated in the following description are merely examples, and this disclosure is not limited to these contents; appropriate modifications can be made to achieve the effects of this disclosure.

[0028] [First Implementation]

[0029] Figure 1 This is a schematic diagram of the neural network simulated by the neuromorphic device in the first embodiment. Figure 1 The neural network N1 shown is a conceptual diagram of the reservoir calculation. Figure 1 The neural network N1 shown has an input layer L in The reservoir layer R and the output layer L out Input layer L in and output layer L out Connected to the reservoir layer R.

[0030] Input layer L in Input signal S in Input pool layer R. Alternatively, there can be no input layer L. in That is, the input signal S can also be... in The input is directly fed into the reservoir layer R. Alternatively, the input signal S can be fed into the reservoir layer R. in After weighting, input the data into the reserve pool layer R.

[0031] The storage pool layer R stores data from the input layer L. in Input signal S in The input signal S is converted into other signals. The reservoir layer R has multiple nodes n. Within the reservoir layer R, the connection weights between nodes n are set, for example, by random numbers. Furthermore, the connection coefficients representing the connection weights between nodes n can be set, for example, in a way that maximizes the information content of the output signal and the desired signal. in It changes non-linearly.

[0032] Input signal Sin The nodes change over time by interacting with each other within the reservoir layer R. Node n is analogous to a neuron in a neural circuit, and the connections between nodes n are analogous to synapses. Multiple nodes n are randomly connected. For example, there is a case where a signal output from a node n at time t returns to the node n that output the signal at time t+1. Node n processes the information recursively based on the signals at time t and time t+1.

[0033] Regarding the output layer L out In other words, a signal is input from the reservoir layer R, and an output signal S based on that signal is output. out .

[0034] Output layer L out For example, it can have an activation function F and a comparator C. The activation function F performs a nonlinear transformation on the signal from the reservoir layer R. For example, the softmax function, sigmoid function, and hyperbolic tangent function are examples of activation functions F.

[0035] The Softmax function transforms multiple input values, normalizes them so that the sum of the multiple output values ​​is "1.0", and outputs the activation rate corresponding to each input.

[0036] The Softmax function performs the operation of equation (1) below. In equation (1), x... i It is the input to the Softmax function, y i It is the output of the Softmax function.

[0037] [Formula 1]

[0038]

[0039] The Sigmoid function is a function derived by modeling the properties of biological nerve cells.

[0040] The Sigmoid function performs the following operation as shown in equation (2). In equation (2), x is the input to the Sigmoid function, and f(x) is the output from the Sigmoid function.

[0041] [Formula 2]

[0042]

[0043] The hyperbolic tangent function is a function that converts an input value to a value in the range of -1.0 to 1.0 and outputs it.

[0044] The hyperbolic tangent function is operated on as shown in equation (3). In equation (3), x is the input to the hyperbolic tangent function, and f(x) is the output from the hyperbolic tangent function.

[0045] [Formula 3]

[0046]

[0047] Comparator C compares the output from the activation function F with the teacher data Dt. The teacher data Dt is, for example, the teacher labels in a multi-class classification problem. Comparator C may contain a processor.

[0048] Output layer L out The learning and reasoning processes are performed. Output layer L out During the learning process, the cross-entropy error between the output of the activation function F and the teacher data Dt is calculated, and the nodes n of the pool layer R and the output layer L are adjusted accordingly. out The connection weights w between them. Learning can be batch learning or online learning.

[0049] Output layer L out During inference processing, the input signal S will be used as the basis. in The inference result of the connection weight w is used as the output signal S out Output.

[0050] Here, reservoir calculation is shown as an example of neural network N1, but it is not limited to this example.

[0051] For example, Figure 2 This is the second example of a schematic diagram of the neural network N2 that is imitated by the neuromorphic device in the first embodiment. Figure 2 The neural network N2 shown is a forward propagation neural network with nodes arranged in layers.

[0052] Figure 2 The neural network N2 shown, for example, has an input layer L in Intermediate layer L m and output layer L out Input layer L in Intermediate layer L m and output layer L out Each layer has multiple nodes n. The intermediate layer L m It can also have two or more layers.

[0053] In the input layer L in Node n and intermediate layer L m Between nodes n, and in the intermediate layer L m node n and output layer L out Between nodes n, connection weights w are set. These connection weights w are adjusted based on the result obtained by comparing the output from the activation function F with the teacher data Dt using a comparator C. The activation function F compares the output from the output layer L... out The output undergoes a nonlinear transformation. The connection weights w are determined during the learning process.

[0054] Figure 3 This is a schematic diagram of the analog arithmetic unit 100 according to the first embodiment. The analog arithmetic unit 100 implements the activation function processing using the circuitry of a physical device. The analog arithmetic unit 100, for example, mimics the processing of the Softmax function in the activation function, and can replace... Figure 1 as well as Figure 2 The computational processing unit that performs the Softmax function operation.

[0055] The analog arithmetic unit 100 includes multiple input terminals 10, a voltage-to-current conversion circuit 20, a current adder circuit 30, current-to-voltage conversion circuits 41 and 42, and a division circuit 50. One of the multiple input terminals 10 is the first input terminal 10A. Any one of the multiple input terminals 10 can be used as the first input terminal 10A. The current adder circuit 30 is an example of a current addition and subtraction arithmetic circuit.

[0056] An input voltage is applied to input terminal 10. Here, the input voltage is the difference between the input potential and the ground potential. Figure 1 In the case of neural network N1, the input voltage corresponds to the individual signals obtained by multiplying the output from the reservoir layer R by the connection weights w. Figure 2 In the case of neural network N2, the input voltage and the voltage from the output layer L out The output signal corresponds to this. Figure 3 The example shown has four input terminals 10, but the number of input terminals 10 is not limited.

[0057] Input terminal 10 is connected, for example, to voltage amplifier circuit 11. Figure 4 This is an example of a circuit diagram for voltage amplifier circuit 11. Figure 4 The voltage amplifier circuit 11 shown makes the input voltage V in Multiply, as the output voltage V out Output. Input voltage V in With output voltage V out Satisfy V out =2V in The relationship.

[0058] The output voltage V from voltage amplifier circuit 11 out A portion of the voltage is divided and applied to the voltage-to-current conversion circuit 20. The voltage amplifier circuit 11 amplifies the input voltage V. in This multiplication allows the voltage-to-current conversion circuit 20 to apply the input voltage V. in The same voltage.

[0059] The voltage-to-current conversion circuit 20 includes, for example, an exponential conversion element. The exponential conversion element outputs a current that varies exponentially with respect to the input voltage. There can be one or more voltage-to-current conversion circuits 20. For example, the voltage-to-current conversion circuits 20 have the same number as the input terminals 10. For example, multiple voltage-to-current conversion circuits 20 are connected to each of the input terminals 10 respectively. The voltage-to-current conversion circuit 20 performs an exponential conversion on the input voltage applied to each of the multiple input terminals and outputs it as a current. For example, the input voltage applied to the first input terminal 10A is performed an exponential conversion and output as a first current I1 from the voltage-to-current conversion circuit 20.

[0060] Exponential conversion elements are, for example, diodes. Diodes are, for example, pn junction diodes using a diffusion layer. Diodes are, for example, formed on a semiconductor substrate such as Si.

[0061] Equation (4) below represents the current-voltage characteristics of the diode. In equation (4), I... s It is the leakage current, V is the voltage applied to the diode, and I is the current output from the diode. T It is represented by kT / e. T is the temperature, k is the Boltzmann constant, e is the electron charge, and m is the emission coefficient. The emission coefficient is 1 or 2; in the case of diodes formed on Si semiconductors, m=1 is more common. At room temperature (T=300K), V T =26mV. The voltage V applied to the diode is much greater than V0. T , you can use I=I S (exp(V / mV)) T () can be approximated.

[0062] [Formula 4]

[0063]

[0064] As shown in equation (4), a diode is a nonlinear converter that outputs a current that varies exponentially with respect to the input voltage. The nonlinear conversion in this diode is analogous to the conversion of the input signal x in the Softmax function. i Convert to e xi The corresponding processing.

[0065] The current replication circuit 21 is connected to the voltage-to-current conversion circuit 20. The current replication circuit 21 replicates the current output from the voltage-to-current conversion circuit 20. The current replication circuit 21 outputs a current of equal magnitude to the current output from the voltage-to-current conversion circuit 20 to a line connected in parallel with the line to which the voltage-to-current conversion circuit 20 is connected. The current replication circuit 21 may include, for example, a bipolar transistor or a field-effect transistor.

[0066] The current adding circuit 30 obtains the sum of the various currents converted by the voltage-to-current conversion circuit 20. For example, it obtains the sum of a first current I1 and one or more converted currents (I2, I3, I4), where the first current I1 is the input voltage V1 applied to the first input terminal 10A. in Obtained by performing an exponential function transformation, one or more conversion currents (I2, I3, I4) are the input voltages (V2) applied to each of the other input terminals 10 that are different from the first input terminal 10A. in V3 in V4 in The current is obtained by performing an exponential function transformation. The current adding circuit 30 is connected, for example, to each of the multiple current replication circuits 21. The currents replicated by each of the multiple current replication circuits 21 are collected (merged) through the current adding circuit 30 and thus added together. For example, the first wiring through which the first current I1 flows is connected to the second wiring through which the converted currents I2, I3, and I4 flow, respectively, within the current adding circuit 30, thereby merging the first current I1 with each of the converted currents I2, I3, and I4. The current adding circuit 30 has the function of adding the currents by merging multiple currents.

[0067] The current-to-voltage conversion circuit 41 converts the various currents obtained by the voltage-to-current conversion circuit 20 into voltages. For example, one of the current-to-voltage conversion circuits 41 converts a first current I1 into a voltage, while the other current-to-voltage conversion circuits 41 convert various converted currents I2, I3, and I4 into voltages. The current-to-voltage conversion circuit 42 converts the sum of the various currents obtained by the voltage-to-current conversion circuit 20 into a voltage. This voltage is generated between the terminals of each current-to-voltage conversion circuit 41 and each voltage-to-current conversion circuit 20.

[0068] There are, for example, multiple current-to-voltage conversion circuits 41. The number of current-to-voltage conversion circuits 41 is, for example, the same as the number of input terminals 10. Each current-to-voltage conversion circuit 41 is, for example, connected to any one of the current replication circuits 21. Current-to-voltage conversion circuit 42 is, for example, connected to the current adding circuit 30.

[0069] Current-to-voltage conversion circuits 41 and 42 include, for example, impedance elements. These impedance elements are preferably variable impedance elements. Current-to-voltage conversion circuits 41 and 42 are, for example, shunt resistors. A shunt resistor is a resistor installed in a circuit for the purpose of current detection, which linearly converts current into voltage. Here, an example of using a variable resistor with control terminals is shown as current-to-voltage conversion circuits 41 and 42.

[0070] The division circuit 50 obtains the ratio of a first voltage obtained by the current-to-voltage conversion circuit to a second voltage. The first voltage is, for example, the voltage obtained by converting a first current I1 or each of the converted currents I2, I3, and I4. The second voltage is, for example, the voltage obtained by converting the sum of the first current I1 and the converted currents I2, I3, and I4. The division circuit 50 calculates, for example, the ratio of the voltage obtained by converting each current to the voltage obtained by converting the sum of the currents. The voltage obtained by the current-to-voltage conversion circuit 41 corresponds to the voltage obtained by converting each current. The voltage obtained by the current-to-voltage conversion circuit 42 corresponds to the voltage obtained by converting the sum of the currents.

[0071] exist Figure 3 In the process, the division circuit 50 uses the voltage obtained by the current-to-voltage conversion circuit 42 to standardize the various voltages obtained by the current-to-voltage conversion circuit 41. The voltage V obtained by the current-to-voltage conversion circuit 42... sum For example, to satisfy V sum The adjustment is made by changing the resistance (impedance) of the variable resistors in the current-to-voltage conversion circuits 41 and 42. All variable resistors have the same resistance (impedance). Vcc is the power supply voltage. R1 and R2 are the resistance values ​​of the resistors included in the division circuit 50. The division circuit 50, for example, has a reference voltage setting circuit including resistors R1 and R2. The reference voltage "1" is set by setting the resistance values ​​of resistors R1 and R2. The reference voltage of the division circuit 50 can be arbitrarily set. The voltage V is adjusted using the voltage V... sum To standardize it to "1", the sum of the various voltages obtained by the current-voltage conversion circuit 41 is "1".

[0072] The individual voltages obtained by the current-voltage conversion circuit 41 are standardized by using the total voltage obtained by the current-voltage conversion circuit 42. Thus, the analog arithmetic unit 100 can directly output the division result obtained by dividing the voltages of each current-voltage conversion circuit 41 by the total voltage. Furthermore, by standardizing the total voltage, the effects of temperature changes can be reduced. As shown in equation (4), the current-voltage characteristics of a diode are affected by temperature. By standardizing each voltage using the total voltage, the effects of temperature on each diode can be reduced.

[0073] The division circuit 50 is not limited to Figure 3 The circuit shown. For example, it could also be a circuit comprising: a memory that stores each of the voltages converted by the current-to-voltage conversion circuit 41 and the total voltages converted by the current-to-voltage conversion circuit 42; and an arithmetic unit that divides each voltage by the total voltage.

[0074] Next, the operation of the analog arithmetic unit 100 of the first embodiment will be described.

[0075] First, apply input voltage V1 to each input terminal 10. in V2 in V3 in V4 in Input voltage V1 in V2 in V3 in V4 in These correspond to the inputs to the activation function F, respectively.

[0076] The voltage input from input terminal 10 is amplified by various voltage amplifier circuits 11. Input voltage V1 in V2 in V3 in V4 in For example, the voltage is amplified by a factor of 2 by voltage amplifier circuit 11. Input voltage V1 in V2 in V3 in V4 in The amplification ratio is not limited to 2 times, and can be set according to the voltage division ratio of the voltage-to-current conversion circuit 20 and the current replication circuit 21. The following explanation uses the case where the amplification ratio in the voltage amplifier circuit 11 is 2 times as an example.

[0077] Next, the voltage 2V1, amplified by voltage amplifier circuit 11, in 2V2 in 2V3 in 2V4 in The voltage is applied to the voltage-to-current conversion circuit 20. The bipolar transistor in the voltage-to-current conversion circuit 20 is connected in series with the current replication circuit 21. The base and collector of this bipolar transistor are short-circuited, exhibiting the same voltage-to-current characteristics as the voltage-to-current conversion circuit 20. Therefore, a voltage based on the voltage division ratio of the voltage-to-current conversion circuit 20 and the current replication circuit 21 is applied to both the voltage-to-current conversion circuit 20 and the current replication circuit 21. For example, when the amplification ratio of the voltage amplifier circuit 11 is 2x and the voltage division ratio of the voltage-to-current conversion circuit 20 is 50%, the voltage is applied to the input voltage V1. in V2 in V3 in V4 in The same voltage is applied to the voltage-to-current conversion circuit 20.

[0078] Each voltage-to-current conversion circuit 20 converts the input voltage V1 in V2 in V3 in V4 inEach voltage is transformed using an exponential function and output as currents I1, I2, I3, and I4, respectively. Input voltage V1 in V2 in V3 in V4 in It is the difference between the input potential and the ground potential.

[0079] For example, as shown in equation (4), the voltage-to-current conversion circuit 20 converts the input voltage V1 in V2 in V3 in V4 in Perform an exponential transformation and output currents I1, I2, I3, and I4. This exponential transformation is similar to the transformation of the input signal x in the Softmax function. i Convert to e xi The corresponding processing.

[0080] In addition, the current replication circuit 21 outputs currents I1, I2, I3, I4 of the same magnitude as the currents I1, I2, I3, I4 output from the voltage-current conversion circuit 20 to the current-voltage conversion circuit 41.

[0081] In addition, currents I1, I2, I3, and I4 of the same magnitude as those output from the voltage-to-current conversion circuit 20 are applied to the current adding circuit 30. In the current adding circuit 30, currents I1, I2, I3, and I4 merge to form a combined current I. sum .

[0082] The current-to-voltage conversion circuit 41 converts currents I1, I2, I3, and I4 into voltage V1. out V2 out V3 out V4 out Additionally, the current-to-voltage conversion circuit 42 will combine the current I. sum Converted to voltage V sum Here, voltage is generated between the two terminals of each current-to-voltage conversion circuit. Voltage V sum For example, V sum =R2 / (R1+R2)×Vcc.

[0083] Division circuit 50 calculates voltage V sum With voltage V1 out V2 out V3 out V4 out The ratio. Using voltage V sum Standardized voltages V1 out V2 out V3 out V4 outThese voltages V1 are output from the analog arithmetic unit. out V2 out V3 out V4 out With the output signal y in the Softmax function i Corresponding. The output signal is input to... Figure 1 as well as Figure 2 The comparator C in the code.

[0084] The analog arithmetic unit 100 of the first embodiment can reduce the computational load consumed by arithmetic processing. The analog arithmetic unit 100 performs signal conversion by converting analog physical quantities such as current and voltage. When executing the activation function F using a digital arithmetic unit, it is necessary to perform arithmetic processing by substituting the input signal into the activation function. The analog arithmetic unit 100 of the first embodiment does not require this operation, thus reducing the computational load of the arithmetic unit.

[0085] In addition, the analog arithmetic unit 100 of the first embodiment can shorten the time spent on arithmetic processing.

[0086] Figure 5 This is a conceptual diagram of the computational processing when all operations of the activation function F are performed by a digital arithmetic unit (software). Figure 5 The activation function F shown is a softmax function. The software performs the following operation: It processes the input signal x... i The first operation C1 of the exponential transformation is to obtain the signal e after the exponential transformation. xi The second operation C2 of the summation, and the individual signals e obtained by exponential transformation. xi The third operation, C3, is the division by the sum obtained in the second operation. Both the first operation C1 and the third operation C3 involve n operations. That is to say, the digital arithmetic unit needs to perform 2n+1 operations sequentially.

[0087] In contrast, the analog arithmetic unit 100 performs the processing corresponding to the first operation C1 via the voltage-to-current conversion circuit 20. Furthermore, the analog arithmetic unit 100 performs the processing corresponding to the second operation C2 via the current addition circuit 30. Additionally, the analog arithmetic unit 100 performs the processing corresponding to the third operation C3 via the current-to-voltage conversion circuit 41, the current-to-voltage conversion circuit 42, and the division circuit 50. These processes are performed in parallel within the analog arithmetic unit 100.

[0088] In the case of a digital arithmetic unit, the third operation cannot be performed before the first operation C1 and the second operation C2 are completed; these operations must be performed sequentially. In contrast, the analog arithmetic unit 100 can perform these processes in parallel by replacing them with conversions of physical quantities. That is, the analog arithmetic unit 100 can shorten the computation time. In addition, by integrating each operation step and sequence into the circuit, the analog arithmetic unit 100 has a low computational load and can be driven with low power consumption.

[0089] This concludes the detailed explanation of the first embodiment based on an example; however, the simulation processor of the first embodiment is not limited to this example.

[0090] Figure 6 This is a schematic diagram of another example of the analog arithmetic unit of the first embodiment. Figure 6 In the analog arithmetic unit 100A shown, the current replication circuit 21 also functions as the voltage-to-current conversion circuit 20, and does not have a separate exponential function element (such as a diode) as the voltage-to-current conversion circuit 20. In this respect, it is similar to... Figure 3 The analog arithmetic unit 100 shown is different. Since there is no exponential function element connected in series, the same reference numerals are used for the same structures as in the analog arithmetic unit 100A, and the description is omitted.

[0091] The current replication circuit 21 includes, for example, bipolar transistors and field-effect transistors. These transistors, like diodes, exponentially transform the input voltage and output it as current. In this case, the voltage amplifier circuit 11 amplifies the input voltage by a factor of 1 and outputs it. That is, the voltage amplifier circuit 11 functions as a voltage follower circuit. In the analog arithmetic unit 100, in order to distribute voltages equal to the input voltage to both the voltage-to-current conversion circuit 20 and the current replication circuit 21, the voltage amplifier circuit 11 amplifies the input voltage by a factor of 2. However, in the analog arithmetic unit 100A, the input voltage is directly input to the current replication circuit 21.

[0092] Like analog arithmetic unit 100, analog arithmetic unit 100A can perform signal conversion by converting analog physical quantities such as current and voltage. Analog arithmetic unit 100A achieves the same effect as analog arithmetic unit 100, and has the advantages of fewer components and the ability to operate at low voltage.

[0093] [Second Implementation]

[0094] Figure 7 This is a schematic diagram of the analog arithmetic unit 101 according to the second embodiment. The analog arithmetic unit 101 implements the activation function processing using the circuitry of a physical device. For example, the analog arithmetic unit 101 mimics the processing of the Softmax function in the activation function.

[0095] The analog arithmetic unit 101 includes multiple input terminals 10, a voltage-to-current conversion circuit 20, a current adder circuit 31, a current-to-voltage conversion circuit 43, a division circuit 50, a switch 60, and an analog-to-digital converter 70. One of the multiple input terminals 10 is the first input terminal 10A. Any one of the multiple input terminals 10 can be used as the first input terminal 10A. The current adder circuit 30 is an example of a current addition and subtraction arithmetic circuit. In the analog arithmetic unit 101, the same reference numerals are used for structures identical to those in the analog arithmetic unit 100, and descriptions are omitted.

[0096] The current adding circuit 31 calculates the sum of the various currents converted by the voltage-to-current conversion circuit 20. The current adding circuit 31 is connected, for example, to each of the current replicating circuits 21. For instance, the first wiring through which the first current I1 flows is connected to the second wiring through which the converted currents I2, I3, and I4 flow, respectively, within the current adding circuit 31, thereby merging the first current I1 with each of the converted currents I2, I3, and I4. The currents output from each of the current replicating circuits 21 are merged through the current adding circuit 31.

[0097] The current-to-voltage conversion circuit 43 is connected to the current adding circuit 31. The current-to-voltage conversion circuit 43 converts the various currents obtained by the voltage-to-current conversion circuit 20 into voltages. Furthermore, the current-to-voltage conversion circuit 43 converts the sum of the various currents obtained by the voltage-to-current conversion circuit 20 into a voltage. The current applied to the current-to-voltage conversion circuit 43 is changed by turning the switch 60 on and off. The current-to-voltage conversion circuit 43 includes, for example, an impedance element. The impedance element is preferably a variable impedance element, for example. A field-effect transistor can be used as the impedance element, but it is not limited to this.

[0098] Switch 60 may use known switches. Switch 60 may be, for example, a transistor, an Ovonic Threshold Switch (OTS) that utilizes a phase change in the crystal layer, a Metal-Insulator Transition (MIT) switch that utilizes a change in the band structure, a Zener diode or an avalanche diode that utilizes a breakdown voltage, or a device whose conductivity varies with atomic position.

[0099] Switch 60 may be located, for example, between input terminal 10 and voltage-to-current conversion circuit 20. Switch 60 may also be located between current replication circuit 21 and current addition circuit 31.

[0100] The analog-to-digital converter 70 converts the voltage output from the current-to-voltage conversion circuit 43 into a digital value. Alternatively, the analog-to-digital converter 70 may not be used.

[0101] The division circuit 50 obtains the ratio of a first voltage obtained by the current-to-voltage conversion circuit to a second voltage. The first voltage is, for example, the voltage obtained by converting a first current I1 or each of the converted currents I2, I3, and I4. The second voltage is, for example, the voltage obtained by converting the sum of the first current I1 and the converted currents I2, I3, and I4. The division circuit 50 calculates, for example, the ratio of the voltage obtained by converting each current to the voltage obtained by converting the sum of the currents. The division circuit 50 includes, for example, a memory and an arithmetic logic unit (ALU). The memory stores, for example, the voltages obtained by the current-to-voltage conversion circuit 43. The ALU divides the voltage obtained by converting each current by the voltage obtained by converting the sum of the currents.

[0102] Next, the operation of the analog arithmetic unit 101 in the second embodiment will be described.

[0103] First, turn on all switches 60. Then, apply input voltage V1 to each input terminal 10. in V2 in V3 in V4 in Each voltage-to-current conversion circuit 20 converts the input voltage V1 in V2 in V3 in V4 in The exponential function transformation is performed respectively, and the results are output as currents I1, I2, I3, and I4 respectively. This process is the same as that of the analog arithmetic unit 100 in the first embodiment.

[0104] Next, currents I1, I2, I3, and I4 of the same magnitude as those output from the voltage-to-current conversion circuit 20 are applied to the current adding circuit 31. In the current adding circuit 31, currents I1, I2, I3, and I4 merge to form a combined current I. sum .

[0105] The current-to-voltage conversion circuit 43 will combine the current I sum Converted to voltage V sum Voltage V sum This is the voltage generated between the two terminals of the current-to-voltage conversion circuit 43. Voltage V sum The voltage V is converted to digital by the analog-to-digital converter 70 and stored in the division circuit 50. sum The calculation of e in the Softmax function xi The sum of (i=1,2,…,n) corresponds to this. That is, the voltage V stored in the division circuit 50... sum It corresponds to the denominator of the Softmax function.

[0106] Next, one of the switches 60 is turned on, while the others are turned off. In this case, only the current (e.g., the first current I1) converted by the voltage-to-current conversion circuit 20 that turns on the switches 60 is applied to the current adding circuit 31 and the current-to-voltage conversion circuit 43. The current-to-voltage conversion circuit 43 converts the applied first current I1 into a voltage V1. The voltage V1 is the voltage generated between the two terminals of the current-to-voltage conversion circuit 43. The voltage V1 is converted into a digital value by the analog-to-digital converter 70 and stored in the division circuit 50.

[0107] Then, the conducting switch 60 is continuously switched. By switching the conducting switch 60, the voltage V for each case where i=1,2,…,n is calculated. i Each voltage V i (i=1,2,…,n) are stored in the division circuit 50. Here, the voltage V is calculated. i This corresponds to the numerator of the Softmax function. Voltage V i These are the voltages generated between the two terminals of the current-to-voltage conversion circuit 43.

[0108] The division circuit 50 divides the voltage V i (i=1,2,…,n) divided by voltage V sum The result of this division is related to the output signal y in the Softmax function. i Corresponding. The output signal is input to. Figure 1 as well as Figure 2 In comparator C.

[0109] The analog arithmetic unit 101 of the second embodiment can achieve the same effect as the analog arithmetic unit 100 of the first embodiment.

[0110] In addition, the analog arithmetic unit 101 of the second embodiment can shorten the time spent on arithmetic processing.

[0111] Figure 8 This is a conceptual diagram of the processing of the analog arithmetic unit 101. The analog arithmetic unit 101 performs processing corresponding to the first operation C1 and the second operation C2 through a voltage-to-current conversion circuit 20, a current addition circuit 31, and a current-to-voltage conversion circuit 43. The first operation C1 and the second operation C2 are switched by turning the switch 60 on and off. The analog arithmetic unit 101 performs the third operation C3 through a division circuit 50. The third operation C3 can be performed as soon as at least one of the second operation C2 and the first operation C1 has been completed. Therefore, the analog arithmetic unit 101 can perform the first operation C1 and the third operation C3 in parallel, which can shorten the calculation time compared to the case where calculations are performed using a device arithmetic unit (software).

[0112] Furthermore, the analog arithmetic unit 101 of the second embodiment can perform all of the first calculation and the second calculation by using the switch 60 to time-division switch the current input to the current-to-voltage conversion circuit 43. Therefore, the analog arithmetic unit 101 only requires one current-to-voltage conversion circuit 43. As a result, the analog arithmetic unit 101 of the second embodiment can reduce the component area.

[0113] in addition, Figure 9 This is a schematic diagram of the analog arithmetic unit 101A of the first modified example. The analog arithmetic unit 101A has multiple units, each consisting of multiple input terminals 10, a voltage-to-current conversion circuit 20, a current adder circuit 31, a current-to-voltage conversion circuit 43, a switch 60, and an analog-to-digital converter 70. The number of units is not limited to two, but can also be three or more.

[0114] The analog arithmetic unit 101A of the first modification can perform the various processes of the first operation C1 in parallel through multiple units. Therefore, the analog arithmetic unit 101A of the first modification can shorten the time consumed by the operation.

[0115] Furthermore, in the analog arithmetic units 101 and 101A of the second embodiment, variations of the analog arithmetic unit as in the first embodiment can also be applied. For example, with... Figure 6 Similarly, in the example shown, the current replication circuit 21 also functions as the voltage-to-current conversion circuit 20, and it is not necessary to set up an exponential function element (such as a diode) as the voltage-to-current conversion circuit 20.

[0116] [Third Implementation]

[0117] Figure 10 This is a schematic diagram of the analog arithmetic unit 102 according to the third embodiment. The analog arithmetic unit 102 implements the processing of the activation function using the circuitry of a physical device. For example, the analog arithmetic unit 102 mimics the processing of the Sigmoid function in the activation function.

[0118] The analog arithmetic unit 102 includes a first input terminal 10A, a second input terminal 10B, a voltage-to-current conversion circuit 20, a current addition circuit 30, current-to-voltage conversion circuits 41 and 42, and a division circuit 50. In the analog arithmetic unit 102, the same reference numerals are used to label the same structures as in the analog arithmetic unit 100, and descriptions are omitted.

[0119] Next, the operation of the analog arithmetic unit 102 in the third embodiment will be described.

[0120] Apply input voltage V1 to the first input terminal 10A inA reference voltage VC is applied to the second input terminal 10B. The reference voltage VC is the voltage at which the current flowing to the exponential converter is negligible, for example, the built-in potential of a diode.

[0121] Input voltage V1 in The voltage is converted into voltage V1 by voltage amplifier circuit 11, voltage-to-current conversion circuit 20, and current-to-voltage conversion circuit 41. out Voltage V1 out This corresponds to exp(x) of the Sigmoid function. Furthermore, the reference voltage VC is converted into voltage VC by voltage amplifier circuit 11, voltage-to-current conversion circuit 20, and current-to-voltage conversion circuit 41. out Voltage VC out It corresponds to exp(0) of the Sigmoid function.

[0122] That is, by using voltage V1 out and voltage VC out It can replace the operation of the Sigmoid function shown in equation (2) with the processing in the physical device.

[0123] The analog arithmetic unit 102 of the third embodiment reduces the computational load of the arithmetic processing. The analog arithmetic unit 102 does not perform the operations of a digital arithmetic unit; instead, it performs signal conversion through the conversion of analog physical quantities such as current and voltage. When executing the activation function F using a digital arithmetic unit, each calculation step and sequence must be performed sequentially in the arithmetic processing of substituting the input signal into the activation function. In the third embodiment, this operation of the analog arithmetic unit 102 is integrated into the analog signal conversion within the circuit, thus reducing the computational load of the analog arithmetic unit.

[0124] This concludes the detailed explanation of the third embodiment based on an example; however, the simulation processor of the third embodiment is not limited to this example.

[0125] For example, in the analog arithmetic unit 102 of the third embodiment, it is also related to... Figure 6 Similarly, in the example shown, the current replication circuit 21 also functions as the voltage-to-current conversion circuit 20, so an additional exponential function element (such as a diode) is not required for the voltage-to-current conversion circuit 20. In this case, the amplification factor of the voltage amplifier circuit is not 2 times, but 1 times is sufficient.

[0126] in addition, Figure 11 This is a schematic diagram of another example of the analog arithmetic unit in the third embodiment. Figure 11The analog arithmetic unit 102A shown includes a first input terminal 10A, a voltage-to-current conversion circuit 20, a current adder circuit 32, current-to-voltage conversion circuits 41 and 42, a divider circuit 51, and DC sources 81 and 82. By setting the impedance of resistor R2 to a reference voltage such that the voltage generated across resistor R2 is used, the current output from DC source 82 becomes the corresponding reference current I. c The reference current I of DC source 82 c The data is copied to DC source 81 via a current mirror circuit, etc. In analog arithmetic unit 102A, the same reference numerals are used for structures identical to those in analog arithmetic unit 102, and descriptions are omitted.

[0127] The current replication circuit 21 also functions as the voltage-to-current conversion circuit 20. The current addition circuit 32 is an example of a current addition / subtraction circuit. The current addition circuit 32 obtains the first current I1 converted by the voltage-to-current conversion circuit 20 and the reference current I... c The sum. The current adding circuit 32 is connected, for example, to the current replicating circuit 21 and the current-to-voltage conversion circuit 42. The first current I1 and the reference current I... c The flow directions are the same. The first current I1 and the reference current I c The currents are combined (converged) through the current adding circuit 32, thus adding them together. For example, the first wiring through which the first current I1 flows and the reference current I... c The second wiring is connected within the current adding circuit 32, thereby connecting the first current I1 with the reference current I. c To converge.

[0128] Reference current I c Output from DC source 81. Figure 10 In the analog arithmetic unit 102 shown, a reference current is generated by converting the reference voltage VC. Figure 10 The current I2), but Figure 11 In the example shown, the reference current I is determined by using the voltage between the terminals of resistor R2 as the reference potential. c This directly generates a DC source 81. In this case, since there is no need to input an external reference voltage, the number of circuit components can be reduced.

[0129] The current-to-voltage conversion circuit 41 converts the first current I1 obtained by the voltage-to-current conversion circuit 20 into a voltage. The current-to-voltage conversion circuit 42 compares the first current I1 with the reference current I... c The sum (total current I) sum The current is converted into voltage. These voltages are generated between the terminals of the current-to-voltage conversion circuits 41 and 42.

[0130] The division circuit 51 obtains the ratio of the first voltage to the second voltage obtained by the current-to-voltage conversion circuits 41 and 42. The first voltage is, for example, the voltage V1 obtained by converting the first current I1 into a voltage. out The second voltage is, for example, the voltage between the first current I1 and the reference current I. c The sum (total current I) sum The voltage V1 obtained by voltage conversion sum The division circuit 51 includes a DC source 82. DC source 82 is a copy of DC source 81, and outputs the same reference current I. c The division circuit 50, for example, calculates the voltage V1 obtained by converting the first current I1. out With the first current I1 and the reference current I c The sum (total current I) sum The voltage V1 obtained by conversion sum than.

[0131] Next, the operation of the analog arithmetic unit 102 will be explained.

[0132] Apply input voltage V1 to the first input terminal 10A in DC source 81 outputs reference current I. c .

[0133] Input voltage V1 in The voltage is converted into voltage V1 by voltage-to-current conversion circuit 20 and current-to-voltage conversion circuit 41. out Voltage V1 out This corresponds to exp(x) of the Sigmoid function. Additionally, the first current I1, replicated by the current replication circuit 21, is combined with the reference current I through the current addition circuit 32. c The first current I1 merges with the reference current I. c The sum (total current I) sum The voltage is converted into voltage V1 by the current-to-voltage conversion circuit 42. sum Voltage V1 sum It corresponds to the Sigmoid function exp(x) + exp(0) = exp(x) + 1.

[0134] That is, by using voltage V1 out and voltage V1 sum It can replace the operation of the Sigmoid function shown in equation (2) with the processing in the physical device.

[0135] [Fourth Implementation]

[0136] Figure 12This is a schematic diagram of the analog arithmetic unit 103 according to the fourth embodiment. The analog arithmetic unit 103 implements the processing of the activation function using the circuitry of a physical device. For example, the analog arithmetic unit 103 can implement the processing of the hyperbolic tangent function in the activation function.

[0137] The analog arithmetic unit 103 includes a first input terminal 10A, a voltage-to-current conversion circuit 20, a current addition circuit 32, a current subtraction circuit 33, current-to-voltage conversion circuits 41 and 42, a division circuit 51, a DC source 81, 82, and 83, a sign removal circuit 91, and a sign recovery circuit 92. In the analog arithmetic unit 103, structures identical to those in the analog arithmetic unit 102A are labeled with the same reference numerals, and descriptions are omitted. The current addition circuit 32 and the current subtraction circuit 33 are examples of current addition and subtraction arithmetic circuits, respectively.

[0138] The reference current I of DC source 82 c The reference potential is uniquely determined by using the voltage between the terminals of resistor R2 as the reference potential. The currents from DC sources 81 and 83 are used to replicate the reference current I through a current replication circuit, etc. c The DC source 83 is connected in the direction of the flowing current to the wiring that connects the current-to-voltage conversion circuit 41 and the voltage-to-current conversion circuit 20.

[0139] The current subtraction circuit 33 obtains the first current I1 converted by the voltage-to-current conversion circuit 20 and the reference current I. c The difference. The current subtraction circuit 33 is connected, for example, to the DC source 83, the current replication circuit 21, and the current-to-voltage conversion circuit 41.

[0140] First current I1, reference current I c and the current I flowing through the current-to-voltage conversion circuit 41 dif The current is collected through current subtraction circuit 33.

[0141] The sign removal circuit 91 is connected to the first input terminal 10A to calculate the input voltage V1 applied to the first input terminal 10A. in The absolute value of the input voltage V1 is obtained by calculating the absolute value of the input voltage V1. in The absolute value of the input voltage V1 applied to the first input terminal 10A. in Remove positive and negative (sign) information.

[0142] The sign recovery circuit 92 adds a sign to the division result of the division circuit 50, restoring the sign removed by the sign removal circuit 91. The sign recovery circuit 92 is connected, for example, to both ends of the current-to-voltage conversion circuit 42. The sign recovery circuit 92 adds the same sign as the sign removed by the sign removal circuit 91 to the output of the analog arithmetic unit 103. By adding the sign information to the output of the analog arithmetic unit 103 using the sign recovery circuit 92, the analog arithmetic unit 103 can output a value in the range of -1 to +1.

[0143] Next, the operation of the analog arithmetic unit 13A will be explained.

[0144] Apply input voltage V1 to the first input terminal 10A in Input voltage V1 in The sign removal circuit 91 removes the positive and negative information, and the result is applied as the absolute value. DC sources 81 and 83 output reference current I, respectively. c Input voltage V1 in The voltage is amplified by 2 times by voltage amplifier circuit 11, becoming 2V1. in .

[0145] Amplified input voltage V1 in The voltage-to-current conversion circuit 20 converts the voltage into a first current I1. The first current I1 is then copied by the current replication circuit 21 and flows to the current-to-voltage conversion circuit 41 and the current-to-voltage conversion circuit 42, respectively.

[0146] In the current adding circuit 32, the first current I1 and the reference current I c The first current I1 and the reference current I converge. c The flow directions are the same. That is, they both become currents flowing out of the current adding circuit 32. In the current adding circuit 32, the first current I1 and the reference current I are obtained. c The sum (total current I) sum Total current I sum The voltage is converted into voltage V1 by the current-to-voltage conversion circuit 42. sum Voltage V1 sum It corresponds to the denominator term exp(2x)+1 of the hyperbolic tangent function.

[0147] In the current subtraction circuit 33, the first current I1 and the reference current I c The first current I1 merges with the reference current I. c The flow direction is opposite. That is, the first current I1 becomes the current flowing out of the current subtraction circuit 33, and in contrast, the reference current I... c This becomes the current flowing into the current subtraction circuit. Here, according to Kirchhoff's laws, I... c +I difThe relationship =I1 holds. Therefore, I dif =I1-I c The current flowing through the current-to-voltage conversion circuit 41 (differential current I) dif ) becomes the difference between the first current I1 and the reference current I c The obtained current. In the current subtraction circuit 33, the first current I1 and the reference current I are obtained. c The difference (differential current I) dif Voltage V1 dif It corresponds to the numerator exp(2x)-1 of the hyperbolic tangent function.

[0148] The division circuit 51 obtains voltage V1 sum With voltage V1 dif The ratio, where voltage V1 sum Based on the first current I1 and the reference current I c The sum of the voltages, V1 dif Based on the first current I1 and the reference current I c difference.

[0149] That is, by using voltage V1 dif and voltage V1 sum It can replace the operation of the hyperbolic tangent function shown in equation (3) with the processing in the physical device.

[0150] This concludes the detailed explanation of the fourth embodiment based on an example; however, the simulation processor of the fourth embodiment is not limited to this example.

[0151] For example, in the analog arithmetic unit 103 of the fourth embodiment, it may also be combined with... Figure 3 The example shown similarly separates the functions through the current replication circuit 21 and the voltage-to-current conversion circuit 20.

[0152] For example, in the analog arithmetic unit 103 of the fourth embodiment, it may also be combined with... Figure 10 Similarly, the reference current is generated by converting the reference voltage VC as shown in the example.

[0153] The analog arithmetic unit has been described in detail above using some embodiments as examples. However, the structure of the analog arithmetic unit is not limited to these embodiments and can be modified and altered in various ways. For example, the characteristic structures of the first to fourth embodiments can be applied to other embodiments.

[0154] Explanation of reference numerals in the attached figures:

[0155] N1, N2: Neural network; L in R: Input layer; L: Reservoir layer; m : Intermediate layer; L out: Output layer; F: Activation function; C: Comparator; n: Node; 10: Input terminal; 11: Voltage amplifier circuit; 20: Voltage-to-current conversion circuit; 21: Current replication circuit; 30, 31, 32: Current addition circuit; 33: Current subtraction circuit; 41, 42, 43: Current-to-voltage conversion circuit; 50: Division circuit; 60: Switch; 70: Analog-to-digital converter; 81, 82, 83: DC source; 91: Sign removal circuit; 92: Sign recovery circuit.

Claims

1. An analog arithmetic unit, comprising: First input terminal; A voltage-to-current conversion circuit performs an exponential function transformation on the input voltage applied to the first input terminal and outputs it as the first current. A current addition / subtraction circuit obtains at least one of the sum and difference between the first current and a reference current or one or more conversion currents, wherein... One or more of the aforementioned conversion currents are obtained by exponentially transforming the input voltage applied to at least one input terminal different from the first input terminal; A current-to-voltage conversion circuit that converts at least two of the first current, the sum, and the difference into voltage; and The division circuit obtains the ratio of the first voltage to the second voltage converted by the current-voltage conversion circuit.

2. The analog arithmetic unit according to claim 1, characterized in that, The current addition and subtraction circuit calculates the sum of the first current and the converted current. The first voltage is obtained by converting the first current into a voltage. The second voltage is obtained by voltage conversion of the sum of the first current and the converted current.

3. The analog arithmetic unit according to claim 1, characterized in that, The current addition and subtraction circuit calculates the sum of the first current and the reference current. The first voltage is obtained by converting the first current into a voltage. The second voltage is obtained by voltage conversion of the sum of the first current and the reference current.

4. The analog arithmetic unit according to claim 1, characterized in that, The current addition and subtraction circuit calculates the difference between the first current and the reference current, and the sum of the first current and the reference current, respectively. The first voltage is obtained by voltage conversion of the difference between the first current and the reference current. The second voltage is obtained by voltage conversion of the sum of the first current and the reference current.

5. The analog arithmetic unit according to claim 1, characterized in that, The voltage-to-current conversion circuit includes diodes.

6. The analog arithmetic unit according to claim 1, characterized in that, It also includes a switch that controls the current flowing to the current-to-voltage conversion circuit.

7. The analog arithmetic unit according to claim 1, characterized in that, It also has a second input terminal. The reference voltage of the voltage-to-current conversion circuit is applied to the second input terminal. The reference current is obtained by converting the reference voltage into current.

8. The analog arithmetic unit according to claim 1, characterized in that, It also has a DC power supply. The DC source outputs the reference current.

9. The analog arithmetic unit according to claim 1, characterized in that, The division circuit has a reference voltage setting circuit.

10. The analog arithmetic unit according to claim 1, characterized in that, The current addition / subtraction circuit has at least one first wiring for the flow of the first current, and a second wiring for the flow of the reference current or the conversion current. The first wiring is connected to the second wiring. The first current is combined with the reference current or the conversion current to obtain the sum or the difference.

11. The analog arithmetic unit according to claim 1, characterized in that, The current-to-voltage conversion circuit includes impedance elements.

12. The analog arithmetic unit according to claim 1, characterized in that, The current-to-voltage conversion circuit includes a variable impedance element.

13. The analog arithmetic unit according to claim 1, characterized in that, It also features a sign removal circuit and a sign recovery circuit. The sign removal circuit is connected to the first input terminal, calculates the absolute value of the input voltage applied to the first input terminal, and removes the sign of the input voltage. The sign recovery circuit adds the sign to the division result of the division circuit, thereby recovering the sign that was removed by the sign removal circuit.

14. A neuromorphic device comprising the analog arithmetic unit of claim 1.

Images