Adjustable activation neuron circuit based on phase change memory and multi-layer inference acceleration device

Abstract

This invention discloses an adjustable activation neuron circuit and a multilayer inference acceleration device based on phase-change memory (PCM), relating to the field of micro-nano electronics technology. It includes: a neuron input terminal for receiving data to be activated; a neuron output terminal for outputting data after nonlinear activation processing; an activation parameter adjustment circuit connected to an activation function circuit, which includes: a first transmission gate, a second transmission gate, a third transmission gate, a fourth transmission gate, a PCM, and an operational amplifier. The inverting input terminal of the operational amplifier receives the data signal to be activated, the non-inverting input terminal is grounded, and the output terminal serves as the neuron output terminal. The activation parameter adjustment circuit is used to adjust the resistance state of the PCM through pulse signals, changing the nonlinear relationship between the output voltage and the input current. Based on the physical characteristics of PCM, this invention develops a neuron circuit with continuously adjustable activation functions, improving the data processing capability of neural networks.

Description

Technical Field

[0001] This invention relates to the field of micro-nano electronics technology, and more specifically, to an adjustable activation neuron circuit based on phase-change memory and a multilayer inference acceleration device. Background Technology

[0002] With the development of artificial intelligence technology, deep neural networks have demonstrated excellent performance in tasks such as image recognition and natural language processing. However, their high computational cost and power consumption limit their deployment on edge devices. Therefore, neuromorphic architectures based on computing-in-memory (CIM) have gradually become a research hotspot, attempting to accelerate neural networks at the hardware layer by integrating non-volatile memories (such as RRAM, MRAM, PCM, etc.).

[0003] In such non-von Neumann structures, the cross-array effectively implements synaptic multiply-accumulate operations, while the activation neuron module is responsible for introducing nonlinear characteristics and is an important source of neural network inference capabilities. Currently, activation functions mainly adopt forms such as ReLU, sigmoid, and tanh, whose hardware implementation usually relies on digital logic or high-precision analog-to-digital converters (ADCs), leading to increased power consumption, area, and latency. To address these challenges, researchers have begun to explore the hardware implementation of analog activation functions. However, existing analog schemes rely on approximation operations, such as CMOS-based Tanh functions and MRAM-based Sigmoid functions, but these schemes rely on piecewise approximation, which has shortcomings in terms of continuity and accuracy. Another study proposed a scheme to directly construct the Tanh activation function based on phase-change memory (PCM), which avoids piecewise processing, but introduces differential approximation circuits, increasing the complexity and stability of the inter-layer connection circuit system of the neural network, which is not conducive to the deployment of large-scale neural network arrays. Therefore, there is an urgent need to propose a neuron circuit design that is compact, has a high-speed response, and has a continuously adjustable activation function, which can make full use of the intrinsic characteristics of the device and has the ability to dynamically adjust and reuse. Summary of the Invention

[0004] To address the shortcomings and improvement needs of existing technologies, this invention provides an adjustable activation neuron circuit based on phase-change memory. The purpose is to develop a compact, high-speed, and continuously adjustable activation function neuron circuit by utilizing the physical characteristics of phase-change memory, while also possessing dynamic adjustment and reuse capabilities.

[0005] To achieve the above objectives, according to one aspect of the present invention, an adjustable activation neuron circuit based on a phase-change memory is provided, comprising:

[0006] The neuron's input terminal is used to receive data to be activated.

[0007] The neuron's output terminal is used to output the data after nonlinear activation processing.

[0008] The system includes an activation parameter adjustment circuit and an activation function circuit, wherein the activation parameter adjustment circuit is connected to the activation function circuit, and the activation function circuit includes: a first transmission gate, a second transmission gate, a third transmission gate, a fourth transmission gate, a phase-change memory, and an operational amplifier.

[0009] The inverting input of the operational amplifier receives the data signal to be activated from the input of the neuron; the non-inverting input of the operational amplifier is grounded; and the output serves as the output of the neuron.

[0010] The first end of the first transmission gate is connected to the input end of the neuron, the second end of the first transmission gate is connected to the second end of the third transmission gate, and the first end of the second transmission gate is connected to the first end of the fourth transmission gate. Each transmission gate is switched by a control signal to achieve circuit mode switching. The first end of the phase-change memory is connected to the point where the first and third transmission gates are connected, and the second end of the phase-change memory is connected to the point where the second and fourth transmission gates are connected. The inverting input end of the operational amplifier is connected to the first end of the first transmission gate, and the output end of the operational amplifier is connected to the second end of the second transmission gate. The phase-change memory, the first transmission gate, and the second transmission gate form a feedback path for converting current signals to voltage signals (IV conversion).

[0011] An activation parameter adjustment circuit is used to adjust the resistance state of the phase-change memory through a pulse signal, thereby changing the nonlinear relationship between the output voltage at the neuron's output terminal and the neuron's input current.

[0012] The present invention also discloses a multilayer inference acceleration device based on the above-mentioned phase-change memory and an adjustable activation neuron circuit, comprising:

[0013] An activation parameter adjustment circuit is used to output a pulse signal;

[0014] Multiple activation function circuits are connected to the activation parameter adjustment circuit, with each activation function circuit connected in parallel and each activation function circuit used to process one channel of data to be activated.

[0015] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0016] (1) Compact structure: It does not rely on complex comparators or analog segmented approximation circuits. The complete activation operation can be achieved with only one operational amplifier and one PCM device, which is suitable for array deployment;

[0017] (2) Low power consumption and high speed: Activation calculation occurs in the analog domain, eliminating the need for digital-to-analog converters (ADCs) or high-precision digital circuits, simplifying inter-layer connections in the CIM architecture, and reducing overall power consumption and latency;

[0018] (3) Supports online reconstruction: The transmission gate dynamic switching architecture and the activation function parameters are adjustable to adapt to the activation curve requirements of different datasets or network structures, and have adaptive capabilities.

[0019] (4) Neuromorphic computing friendly: This structure is easy to integrate with the CIM structure, and is suitable for building large-scale neuromorphic computing chips to achieve optimal on-chip energy efficiency of neural networks. Attached Figure Description

[0020] Figure 1 A circuit diagram of an adjustable activation neuron circuit based on a phase-change memory according to an embodiment of the present invention;

[0021] Figure 2 This is a logic state diagram of the control signals in an adjustable activation neuron circuit based on a phase-change memory according to an embodiment of the present invention.

[0022] Figure 3 This describes the subthreshold IV characteristics of a phase-change memory according to an embodiment of the present invention.

[0023] Figure 4 This is a nonlinear mapping curve of the output voltage to the input current of the phase-change memory according to an embodiment of the present invention;

[0024] Figure 5 This is a mathematical diagram illustrating the embedding method and implementation of the adjustable activation neuron circuit of the phase-change memory in a neural network according to an embodiment of the present invention.

[0025] Figure 6 This is a block diagram of a multilayer inference acceleration device based on a phase-change memory-based adjustable activation neuron circuit according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0027] In this invention, the terms "first," "second," etc. (if present) in the invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0028] The purpose of this invention is to provide an adjustable activation neuron circuit based on phase-change memory (PCM). By directly utilizing the inherent subthreshold electrical characteristics of PCM and combining it with the negative feedback mechanism of an operational amplifier, a nonlinear activation function is directly constructed in the analog domain. The activation function constructed by the adjustable activation neuron circuit not only possesses excellent nonlinear response characteristics, but also achieves dynamic adjustment of the activation function by introducing multiple sets of transmission gate control mechanisms to switch the activation function adjustment mode and adjusting the resistance state of the PCM, thus realizing the design of an adjustable activation function.

[0029] Figure 1 This is a circuit diagram of an adjustable activation neuron circuit based on a phase-change memory according to an embodiment of the present invention. Figure 1 As shown, the adjustable activation neuron circuit includes: a neuron input terminal IN, a neuron output terminal OUT, an operational amplifier, a phase-change memory (PCM), an activation parameter adjustment circuit, and four transmission gate switches: the first transmission gate TG1, the second transmission gate TG2, the third transmission gate TG3, and the fourth transmission gate TG4. Specifically: the inverting input terminal of the operational amplifier is connected to the neuron input terminal IN to receive the data to be activated, such as an analog current signal from the preceding neural network. This analog current signal contains information about the data to be activated. The non-inverting input terminal of the operational amplifier is grounded, and the output terminal of the operational amplifier is connected to the neuron output terminal OUT. In other words, the output terminal of the operational amplifier serves as the neuron output terminal OUT, outputting the data after nonlinear activation processing. The phase-change memory PCM is connected between the output terminal and the inverting input terminal of the operational amplifier via transmission gate switches TG1-TG4, forming a negative feedback path for converting the current signal into a voltage signal, i.e., IV conversion. Specifically, the activation function circuit consists of a first transmission gate TG1, a second transmission gate TG2, a third transmission gate TG3, a fourth transmission gate TG4, a phase-change memory (PCM), and an operational amplifier. The inverting input of the operational amplifier receives the data signal to be activated from the neuron's input terminal IN and is connected to the first terminal of the first transmission gate. The non-inverting input is grounded, and the output is connected to the second terminal of the second transmission gate TG2, serving as the neuron's output terminal OUT. The first terminal of the first transmission gate TG1 is connected to the neuron's input terminal IN, the second terminal of the first transmission gate TG1 is connected to the second terminal of the third transmission gate TG3, and the first terminal of the second transmission gate TG2 is connected to the first terminal of the fourth transmission gate TG4. Each transmission gate is turned off by a control signal Din, enabling circuit mode switching—that is, dynamic switching between "activation operation mode" and "parameter adjustment mode" for the adjustable activation neuron circuit. The first terminal of the phase-change memory (PCM) is connected to the points where the first and third transmission gates TG1 and TG3 are connected, and the second terminal of the PCM is connected to the points where the second and fourth transmission gates are connected.

[0030] Furthermore, the two electrodes of the phase change memory (PCM), namely the first and second terminals, are also connected to the two control ports of the activation parameter adjustment circuit through the third transmission gate TG3 and the fourth transmission gate TG4.

[0031] Figure 2 The diagram below illustrates the logic state of the control signal Din in an adjustable activation neuron circuit based on a phase-change memory (PCM) according to an embodiment of the present invention. When the control signal Din is high, TG1 and TG2 are turned on, and TG3 and TG4 are turned off. The PCM is connected to the operational amplifier and forms a feedback loop with TG1 and TG2. The adjustable activation neuron circuit is in activation calculation mode. The input current received at the neuron input terminal IN is mapped to a nonlinear output voltage through the PCM and output from the neuron output terminal OUT. When Din is low, TG1 and TG2 are turned off, and TG3 and TG4 are turned on. The PCM is disconnected from the operational amplifier and connected to the activation parameter adjustment circuit. The activation parameter adjustment circuit outputs a pulse signal to the PCM, such as a SET / RESET signal. By programming the amplitude, pulse width, and repetition count of the pulse signal, the resistance state of the PCM is changed, thereby adjusting the parameters of the activation function curve.

[0032] Figure 3 To illustrate the subthreshold IV characteristics of a phase-change memory according to an embodiment of the present invention, as shown in the figure, in the amorphous region of the phase-change memory (PCM), its subthreshold IV characteristics exhibit nonlinear behavior, which can be expressed in the following form:

[0033]

[0034] Where q is the electron charge; τ0 is the electron escape time constant; N T The concentration of trapped state carriers is z; the trap spacing is E. a The activation energy of the device material; k B U is the Boltzmann constant; T is the temperature; S is the cross-sectional area of ​​the amorphous region; δ is the thickness of the amorphous region. To simplify calculations, this expression can be equivalently simplified to the following Sinh function form:

[0035] I = Bsinh(AV) (2)

[0036] Figure 3 This is a curve fitting diagram of the subthreshold IV of the phase change memory.

[0037] Figure 4 This is a nonlinear mapping curve of the output voltage to the input current of a phase-change memory according to an embodiment of the present invention. Figure 4 Combining Figure 3It is explained that within the amorphous region of the phase-change memory (PCM), the subthreshold IV characteristic of the PCM exhibits a Sinh function form, which can be simplified to I = Bsinh(AV). Because the high input impedance of the operational amplifier in the transimpedance amplifier (TIA) circuit structure used in this invention ensures that the current at its inverting input is zero (meaning close to zero), the input current will flow entirely through the PCM. Since the PCM is connected in the operational amplifier feedback loop, the relationship between the input current and output voltage can be equivalently transformed as follows:

[0038] V=A′Arsinh(B′I) (3)

[0039] Where V is the output voltage, I is numerically equal to the negative of the input current, and A' and B' are the reciprocals of A and B, respectively.

[0040] By programming the phase-change memory (PCM) with SET / RESET pulses, the resistance value of the PCM can be adjusted in multiple levels, such as... Figure 4 As shown in states 1, 2, and 3 (medium resistance states), the functional relationship between output voltage and input current exhibits Arsinh activation curves of different shapes, demonstrating the tunability of the activation function. In high resistance states, such as state 3, the output curve is steeper, with a stronger nonlinear response, suitable for representing a "harder" activation function. In low resistance states, such as state 1, the output curve is flatter and approximately linear, suitable for smoother activation requirements. State 2 falls between high and low resistance states, corresponding to a moderate nonlinear response. Different activation functions can be applied to the activation requirements of different tasks, thus increasing the adaptability of this scheme. This result verifies that the tunable activation neuron circuit based on the physical characteristics of phase-change memory proposed in this invention achieves an adjustable Arsinh activation function in the analog domain through a simple neuron structure, exhibiting good scalability and system compatibility.

[0041] Figure 5 This diagram illustrates the embedding method and mathematical function of an adjustable activation neuron circuit based on a phase-change memory in a neural network, according to an embodiment of the present invention. Figure 5As shown in Figure (a), the adjustable activation neuron circuit based on phase-change memory (PCM) is integrated into a crossbar array structure in a neural network. It mainly consists of three parts: the crossbar array, the activation neuron circuit, and the activation parameter adjustment circuit. The synaptic array is formed by x parallel word lines and n parallel bit lines perpendicularly intersecting each other. A PCM is set at each intersection point of a word line and a bit line (represented by "G" in the figure). The conductance of the PCM is used to store weights Wij, where i is the word line number and j is the bit line number. When an external analog voltage signal V1 to Vm is input to the word line, matrix multiplication is performed through the PCM array, and the current is calculated according to Kirchhoff's laws and output by the bit line. The bit lines, as the output terminals of the array, are connected one-to-one to the inverting input terminals of the operational amplifiers of the lower-level activatable neuron circuits to activate the weighted analog current signal. The core components of each activatable neuron circuit consist of an operational amplifier, a transmission gate group, and a PCM. For specific components, refer to [reference needed]. Figure 1 The description in the text will not be repeated here. The activation parameter adjustment circuit generates pulse signals, such as SET / RESET signals, and adjusts the resistance state of the phase-change memory by programming the amplitude, pulse width, and number of repetitions of the pulse signals.

[0042] Furthermore, Figure 5 Figure (b) in the diagram illustrates the neural network mathematical function performed by structure (a), namely, realizing the linear process of matrix multiplication and accumulation and the nonlinear process of activation. Through the tunable neuron circuit disclosed in this invention, a natural fusion of computation and activation in analog neural networks is achieved. Compared to traditional mixed-signal solutions, this significantly reduces the AD / DA data conversion requirements and simplifies the overall system architecture based on the inherent electrical characteristics of phase-change memory. In particular, the tunability of the activation function in this invention allows for programming and adaptation when deploying different types of tasks or datasets, exhibiting high adaptability and low power consumption.

[0043] Figure 6 This is a block diagram of a multilayer inference acceleration device based on a phase-change memory-based tunable activation neuron circuit according to an embodiment of the present invention. Figure 6 As shown, the multi-layer inference acceleration device is a partial structure in a neural network. For example... Figure 6 The multi-layer inference acceleration device shown includes an activation parameter adjustment circuit for outputting pulse signals; and multiple activation function circuits connected to the activation parameter adjustment circuit, wherein each activation function circuit is connected in parallel and each activation function circuit is used to process one channel of data to be activated.

[0044] Specifically, when a neural network comprises multiple weighted cross arrays, each cross array performs an analog matrix-vector multiplication (MVM) operation once to generate an output current. To achieve nonlinear transformations between layers, the adjustable activation function neuron circuit based on phase-change memory (PCM) is placed between adjacent cross arrays. Specifically: the row current output of the previous layer's cross array is directly connected to the activation function circuit, with each row current corresponding to an activation function circuit. The activation function circuit performs Arsinh-type nonlinear activation operations based on the physical characteristics of PCM; the output voltage can directly drive the input of the next layer's cross array without the need for additional voltage / current conversion modules or A / D / D / A conversion interfaces; the entire link is calculated and mapped within the analog domain, maintaining data flow consistency.

[0045] Furthermore, the TIA structure is a key unit for achieving IV decoupling in the cross-array. This invention cleverly integrates it with nonlinear activation operations, greatly simplifying the circuit architecture and improving the system's energy efficiency. Compared to traditional digital activation functions and constrained analog activation function schemes, the structure of this invention is more suitable for the implementation of highly integrated neural network chips.

[0046] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A circuit for adjustable activation of neurons based on phase-change memory, characterized in that, include: The neuron's input terminal is used to receive data to be activated; The neuron's output terminal is used to output the data after nonlinear activation processing. The system includes an activation parameter adjustment circuit and an activation function circuit, wherein the activation parameter adjustment circuit is connected to the activation function circuit, and the activation function circuit includes: a first transmission gate, a second transmission gate, a third transmission gate, a fourth transmission gate, a phase-change memory, and an operational amplifier. The inverting input of the operational amplifier receives the data to be activated from the input of the neuron, the non-inverting input of the operational amplifier is grounded, and the output is used as the output of the neuron. The first end of the first transmission gate is connected to the input end of the neuron, the second end of the first transmission gate is connected to the second end of the third transmission gate, and the first end of the second transmission gate is connected to the first end of the fourth transmission gate. Each transmission gate is controlled to turn off by a control signal to achieve circuit mode switching. The circuit modes include: nonlinear activation mode and parameter adjustment mode. The first end of the phase-change memory is connected to the point where the first and third transmission gates are connected, the second end of the phase-change memory is connected to the point where the second and fourth transmission gates are connected, the inverting input end of the operational amplifier is connected to the first end of the first transmission gate, and the output end of the operational amplifier is connected to the second end of the second transmission gate. The phase-change memory, the first transmission gate, and the second transmission gate constitute a feedback path. The activation parameter adjustment circuit is used to send pulse signals to adjust the resistance state of the phase change memory, thereby changing the nonlinear relationship between the output signal at the neuron output end and the input signal at the neuron input end.

2. The adjustable activation neuron circuit based on phase-change memory according to claim 1, characterized in that, When the control signal is high, the first transmission gate and the second transmission gate are turned on, the phase-change memory is connected to the operational amplifier, and the adjustable activation neuron circuit operates in nonlinear activation mode.

3. The adjustable activation neuron circuit based on phase-change memory according to claim 1, characterized in that, When the control signal is low, the third and fourth transmission gates are turned on, the activation parameter adjustment circuit is connected to the phase change memory, and the adjustable activation neuron circuit works in parameter adjustment mode.

4. The adjustable activation neuron circuit based on phase-change memory according to claim 3, characterized in that, When the adjustable activation neuron circuit operates in parameter adjustment mode, the activation parameter adjustment circuit applies a pulse signal to the phase change memory to adjust the crystallization degree of the phase change memory and obtain the nonlinear curve of the activation function.

5. The neuron circuit according to claim 1, characterized in that, The operational amplifier has high impedance, and the current at the inverting input of the operational amplifier is zero.

6. The neuron circuit according to claim 1, characterized in that, The phase change materials of the phase change memory include: Ge2Sb2Te5, GeSbTe, Sb2Te3, and GeTe.

7. The neuron circuit according to claim 4, characterized in that, The activation function includes: inverse hyperbolic sine function.

8. A multilayer inference acceleration device based on a phase-change memory-based tunable activation neuron circuit, characterized in that, include: An activation parameter adjustment circuit according to any one of claims 1-7 is used to output a pulse signal; The activation function circuit described in any one of claims 1-7 is connected to the activation parameter adjustment circuit, wherein each activation function circuit is connected in parallel and each activation function circuit is used to process one channel of data to be activated.

Images