Neuromorphic device and neuromorphic system including the same

Abstract

A neuromorphic device and a neuromorphic system including the same are provided. The neuromorphic device includes a neuron block, a spike transmission circuit, and a spike reception circuit. The neuron block includes a plurality of neurons connected by a plurality of synapses, and is configured to generate and operate spikes. The spike transmission circuit generates a non-binary transmission signal based on a plurality of transmission spike signals output from the neuron block, and transmits the non-binary transmission signal to a transmission channel, wherein the non-binary transmission signal includes information about transmission spikes included in the plurality of transmission spike signals. The spike reception circuit receives a non-binary reception signal from the transmission channel, and generates a plurality of reception spike signals including reception spikes based on the non-binary reception signal, and provides the plurality of reception spike signals to the neuron block, wherein the non-binary reception signal includes information about the reception spikes.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority from Korean Patent Application No. 10-2019-0152184 filed on November 25, 2019, in the Korean Intellectual Property Office (KIPO), the entire contents of which are incorporated herein by reference. Technical Field

[0003] Example embodiments relate generally to semiconductor integrated circuits, and more particularly to neuromorphic devices and neuromorphic systems including the same. Background Art

[0004] The human brain comprises hundreds of billions of neurons connected in a complex neuromorphic network. Neuromorphic devices emulate the human nervous system or brain. Unlike the architecture of a typical computer central processing unit, where the arithmetic unit, instruction processor, and memory unit exist independently, neuromorphic devices can process information in parallel. Neuromorphic devices can be defined by two-dimensional or three-dimensional connections of multiple neurons. Each neuron can be configured using circuits corresponding to the axon, dendrites, and cell body in the same manner as the constituent elements of a biological neuron. As the number of neurons forming a nervous system increases, the efficiency of information exchange between neurons decreases. Summary of the Invention

[0005] Some example embodiments may provide a neuromorphic device and a neuromorphic system including the neuromorphic device, which are capable of more efficiently implementing a spiking neural network.

[0006] According to an example embodiment, a neuromorphic device includes a neuron block, a spike transmission circuit, and / or a spike reception circuit. The neuron block includes a plurality of neurons connected by a plurality of synapses, and is configured to generate and operate spikes. The spike transmission circuit generates a non-binary transmission signal based on a plurality of transmission spike signals output from the neuron block, and transmits the non-binary transmission signal to a transmission channel, wherein the non-binary transmission signal includes information about transmission spikes included in the plurality of transmission spike signals. The spike reception circuit receives a non-binary reception signal from the transmission channel, and generates a plurality of reception spike signals including reception spikes based on the non-binary reception signal, to provide the plurality of reception spike signals to the neuron block, wherein the non-binary reception signal includes information about the reception spikes.

[0007] According to an example embodiment, a neuromorphic system includes a plurality of neuromorphic devices and a transmission channel configured to electrically connect the plurality of neuromorphic devices. The plurality of neuromorphic devices each include a neuron block, a spike transmitting circuit, and a spike receiving circuit.

[0008] According to an exemplary embodiment, a neuromorphic device includes: a neuron block including a plurality of neurons connected via a plurality of synapses, configured to generate and operate spikes; a spike transmission circuit configured to generate a pulse amplitude modulated (PAM) transmission signal based on a plurality of transmission spike signals output from the neuron block, and configured to transmit the PAM transmission signal to a transmission channel; and a spike reception circuit configured to receive the PAM reception signal from the transmission channel. The spike transmission circuit includes: a spike converter configured to generate transmission spike data including timing information of the transmission spikes based on the plurality of transmission spike signals output from the neuron block; a packetizer configured to generate a transmission data packet based on the transmission spike data; and a modulator configured to modulate the transmission data packet to generate the PAM transmission signal. The spike reception circuit includes: a demodulator configured to demodulate the PMA reception signal to generate a reception data packet; a depacketizer configured to generate reception spike data including timing information of the reception spike based on the reception data packet; and a spike generator configured to generate a plurality of reception spike signals based on the reception spike data.

[0009] A neuromorphic device and a neuromorphic system according to example embodiments may effectively implement a spiking neural network by packetizing spike signals and modulating data packets to transmit the spike signals as non-binary signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Example embodiments of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0011] Figure 1 is a block diagram illustrating a neuromorphic device according to example embodiments.

[0012] Figure 2 is a diagram illustrating a spiking neural network according to example embodiments.

[0013] Figure 3 is an example neuron in a spiking neural network according to an example embodiment.

[0014] Figure 4 is a diagram illustrating spike timing dependent plasticity (STDP) of neurons.

[0015] Figure 5 is a diagram used to describe the spike signal output from a neuron.

[0016] Figure 6 and Figure 7 is a diagram for describing a method of implementing a spiking neural network according to example embodiments.

[0017] Figure 8 is a block diagram illustrating a spiking transceiver circuit included in a neuromorphic device according to an example embodiment.

[0018] Figures 9A to 9G is a diagram illustrating an example format of a data packet used in a neuromorphic device according to example embodiments.

[0019] Figure 10 is a diagram for describing error correction code (ECC) levels in a neuromorphic device according to example embodiments.

[0020] Figure 11 is a diagram illustrating an example of a pulse amplitude modulation (PAM) signal in a neuromorphic device according to example embodiments.

[0021] Figure 12 is a block diagram illustrating an example embodiment of a modulator included in a neuromorphic device according to an example embodiment.

[0022] Figure 13 It is shown from Figure 12 Timing diagram of an example of a PAM signal output by the modulator.

[0023] Figure 14 is a diagram illustrating an example eye pattern of a PAM signal.

[0024] Figure 15 is a block diagram illustrating an example embodiment of a demodulator included in a neuromorphic device according to an example embodiment.

[0025] 16A to 16D is a diagram illustrating an example embodiment of a spiking neural network according to example embodiments.

[0026] Figures 17 to 20 is a diagram illustrating a neuromorphic system according to example embodiments.

[0027] Figure 21 and Figure 22 is a diagram for describing an example of a deep learning neural network structure driven by a neuromorphic system according to example embodiments.

[0028] Figure 23 is a block diagram illustrating a computing system according to an example embodiment. DETAILED DESCRIPTION

[0029] Various example embodiments will be described more fully hereinafter with reference to the accompanying drawings, which illustrate some example embodiments. In the accompanying drawings, like reference numerals represent like elements throughout. Repetitive descriptions may be omitted.

[0030] Figure 1 is a block diagram illustrating a neuromorphic device according to example embodiments.

[0031] Reference Figure 1 , the neuromorphic device NDEV includes a spike transmission circuit TX and / or a spike reception circuit RX. The combination of the spike transmission circuit TX and the spike reception circuit RX may be referred to as a spike transceiver circuit TRX.

[0032] The neuron block NBLK includes a plurality of neurons NR connected by a plurality of synapses for performing spike generation and computation. The neuron block NBLK may correspond to a neuromorphic core or a neuromorphic chip including a plurality of neuromorphic cores. Figures 2 to 5 The neuron block NBLK is further described.

[0033] The spike transmission circuit TX generates a non-binary transmission signal TNB based on the plurality of transmission spike signals TSPK output from the neuron block NBLK and transmits the non-binary transmission signal TNB to the transmission channel CHNN. The non-binary transmission signal TNB includes information about transmission spikes included in the plurality of transmission spike signals TSPK.

[0034] The spike receiving circuit RX receives the non-binary receiving signal RNB from the transmission channel CHNN and generates a plurality of receiving spike signals RSPK including receiving spikes based on the non-binary receiving signal RNB to provide the plurality of receiving spike signals RSPK to the neuron block NBLK. The non-binary receiving signal RNB includes information about the receiving spikes.

[0035] According to example embodiments, the non-binary transmit signal TNB and the non-binary receive signal RNB may be pulse-based signals. For example, the pulse-based signals may include pulse width modulation (PWM) signals, pulse amplitude modulation (PAM) signals, pulse position modulation (PPM) signals, etc. In particular, as will be described below with reference to Figures 11 to 15 As described, the non-binary signal transmitted between the plurality of neuromorphic devices may be a PAM signal.

[0036] The neuromorphic device NDEV and the neuromorphic system including the neuromorphic device NDEV according to example embodiments may effectively implement a spiking neural network by modulating a spike signal to transmit the spike signal as a non-binary signal.

[0037] Figure 2is a diagram illustrating a spiking neural network according to example embodiments, Figure 3 is an example neuron in a spiking neural network according to an example embodiment, Figure 4 is a diagram illustrating spike timing dependent plasticity (STDP) of neurons.

[0038] Reference Figure 2 , the spiking neural network may include neurons NR11 to NR1N of a first layer LV1 and neurons NR21 to NR2M of a second layer LV2, such that the neurons NR11 to NR1N of the first layer LV1 and the neurons NR21 to NR2M of the second layer LV2 may be connected via a synaptic array SYNARR (i.e., a network of synaptic connections (e.g., feedforward connections)). For simplicity, in Figure 2 Only two layers of neurons are shown in FIG, but there may be fewer or more layers of neurons in a spiking neural network. It should be noted that some neurons can be connected to other neurons in the same layer via lateral connections. In addition, some neurons can be connected back to neurons in the previous layer via feedback connections.

[0039] like Figure 2 As shown, each neuron in the first layer LV1 can receive input signals X1 to XN, and the input signals X1 to XN can be generated by the neurons in the previous layer ( Figure 2 The current applied to each neuron of the first layer LV1 can accumulate on the neuronal membrane of each neuron to charge the membrane potential. When the membrane potential reaches its threshold, the neuron can excite and generate an output spike to be transmitted to the neurons of the next layer (for example, the neurons of the second layer LV2). In some modeling methods, the neurons can continuously transmit signals to the neurons of the next layer. The signal is typically a function of the membrane potential. This behavior can be simulated or emulated in hardware and / or software including analog and digital implementations.

[0040] When a spike is transmitted from a neuron in one layer to a neuron in another layer, the information transmitted is represented only by the frequency and number of spikes, or the timing of the spikes, not the amplitude. The importance of a spike can be determined by the weights applied to the connections between neurons, as described below.

[0041] like Figure 2As shown, the transmission of spikes from neurons in one layer to neurons in another layer can be achieved through a network of synaptic arrays (or simply referred to as "synapses") SYNARR. With respect to the synaptic SYNARR, the neurons of the first layer LV1 can be considered as presynaptic neurons, and the neurons of the second layer LV2 can be considered as postsynaptic neurons. The synaptic SYNARR can receive output signals (e.g., spikes) from neurons in the first layer LV1 and scale these signals according to adjustable synaptic weights W1~WP, where P is the total number of synaptic connections between neurons in the first layer LV1 and the second layer LV2. The scaled signals can be combined as input signals for each neuron in the second layer LV2. Each neuron in the second layer LV2 can generate output spikes Y1~YM based on the corresponding combined input signal. Another synaptic network (not shown) can be used to transmit output spikes Y1~YM to neurons in another layer.

[0042] Spiking neural networks can be emulated by processing circuits such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices (PLDs), discrete gate or transistor logic, discrete hardware components, software modules executed by a processor, or any combination thereof. Spiking neural networks can be used in a wide variety of applications such as image and pattern recognition, machine learning, motor control, and more, as will be discussed below. Figure 21 and Figure 22 Each neuron in a spiking neural network can be implemented as a neuron circuit. The neuron membrane, which is charged to a threshold value for initiating an output spike, can be implemented as, for example, a capacitor that integrates the current flowing through it, and a smaller memristor element can replace the capacitor.

[0043] The functionality of a neuron processor emulating a spiking neural network can depend on the weights of synaptic connections, which control the strength of connections between neurons. The synaptic weights can be stored in non-volatile memory to preserve the functionality of the neuron processor after power is removed.

[0044] Figure 3 An example of an operation performed by one node (ie, one neuron ND included in a spiking neural network) is shown.

[0045] When N inputs a1~a1 are provided to the node ND n When the node ND can input n a1~a n Respectively with the corresponding n weights w1~w nIn the multiplication, n values ​​obtained by the multiplication may be summed, an offset 'b' may be added to the summed value, and an output value may be generated by applying the value to which the offset 'b' is added to a specific function 'σ'.

[0046] like Figure 3 As shown, neurons ND can receive multiple input signals, which can be signals external to the nervous system, signals generated by other neurons in the same nervous system, or both. The input signals can be current, conductance, voltage, real value and / or complex value. The input signals can include numerical values ​​with fixed-point or floating-point representation. These input signals can be transmitted to neurons via synaptic connections, which scale the signals according to adjustable synaptic weights. The neurons can combine the scaled input signals and use the combined scaled inputs to generate output signals.

[0047] In a spiking neural network, neurons ND may be implemented using processing circuits, for example, as analog circuits, digital circuits, or combined circuits including analog and digital components.

[0048] During the process of training a spiking neural network, synaptic weights may be initialized with random values ​​(e.g., Figure 2 The weights W1~WP or Figure 3 The weights w1~w n ), and increases or decreases the synaptic weight according to a learning rule. Those skilled in the art will appreciate that examples of learning rules include, but are not limited to, spike timing dependent plasticity (STDP) learning rule, Hebb rule, Oja rule, Bienenstock-Copper-Munro (BCM) rule, and the like.

[0049] STDP is a learning process that regulates the strength of synaptic connections between neurons. The connection strength is regulated based on the relative timing of the output of a particular neuron and the input spikes (e.g., action potentials) received. Under the STDP process, if the input spike of a neuron tends, on average, to occur just before the output spike of that neuron, long-term potentiation (LTP) will occur. Thus, the particular input becomes stronger. On the other hand, if the input spike tends, on average, to occur immediately after the output spike, long-term depression (LTD) will occur. Thus, the particular input becomes weaker, hence the name "spike timing dependent plasticity". Therefore, inputs that cause the postsynaptic neuron to become excited are more likely to contribute in the future, while inputs that do not cause postsynaptic spikes are less likely to contribute in the future.

[0050] The STDP learning rule can effectively adapt the synaptic weight of the synapse connecting the presynaptic neuron and the postsynaptic neuron based on the time difference between the spike time t_PRE of the presynaptic neuron and the spike time t_POST of the postsynaptic neuron (e.g., t=t_POST-t_PRE). The typical expression of STDP is: if the time difference is positive (the presynaptic neuron fires before the postsynaptic neuron), then increase the synaptic weight (e.g., strengthen the synapse); if the time difference is negative (the postsynaptic neuron fires before the presynaptic neuron), then decrease the synaptic weight (e.g., inhibit the synapse).

[0051] In the STDP process, exponential decay is often used to implement the change of synaptic weights over time.

[0052] Figure 4 An example graph shows how synaptic weight changes according to STDP vary with the relative timing of presynaptic and postsynaptic spikes. If the presynaptic neuron fires before the postsynaptic neuron, the corresponding synaptic weight can increase, as shown in portion GP1 of the graph. This weight increase can be referred to as the LTP of the synapse. As can be seen from portion GP1 of the graph, the amount of LTP can decrease roughly exponentially as the difference between the presynaptic spike time and the postsynaptic spike time increases. As shown in portion GP2 of the graph, the opposite order of firing can reduce the synaptic weight, thereby causing LTD of the synapse. In order to account for the correlation of causal inputs from the layer, the intersection point PX (y=0) of the x-axis can be configured to coincide with the maximum time lag.

[0053] Figure 5 is a diagram used to describe the spike signal output from a neuron.

[0054] Reference Figure 5 The first presynaptic neuron NR_PRE1, the second presynaptic neuron NR_PRE2, the first postsynaptic neuron NR_POST1, and the second postsynaptic neuron NR_POST2 can generate spike signals SSPK1 to SSPK4, respectively. When the spike signal SSPK1 including spikes SPK1 and SPK2 and the spike signal SSPK2 including spikes SPK3 and SPK4 are input to the first postsynaptic neuron NR_POST1, the first postsynaptic neuron NR_POST1 can output a spike signal SSPK3 including a spike SPK5. Conversely, the second postsynaptic neuron NR_POST2, which receives only the spike signal SSPK2, can output a spike signal SSPK4 that does not include any spikes.

[0055] like Figure 5As shown, the first presynaptic neuron NR_PRE1 can generate spikes SPK1 and SPK2 at time points t1 and t5, respectively, the second presynaptic neuron NR_PRE2 can generate spikes SPK3 and SPK4 at time points t2 and t4, respectively, and the first postsynaptic neuron NR_POST1 can generate a spike SPK5 at time point t6. These time points of generating spikes can be included as timing information in the above-mentioned information on sending spikes and receiving spikes.

[0056] Figure 6 and Figure 7 is a diagram for describing a method of implementing a spiking neural network according to example embodiments.

[0057] Reference Figure 6 and Figure 7 , the first neuromorphic device NDEV1 and the second neuromorphic device NDEV2 may be electrically connected through a transmission channel CHNN ( S100 ).

[0058] Using the first spike transceiver circuit TRX1 of the first neuromorphic device NDEV1, a non-binary transmit signal TNB including information about the transmit spikes is generated based on a plurality of transmit spike signals TSPK1-TSPKN including transmit spikes output from transmit neurons TNR of the first neuron block NBLK1 of the first neuromorphic device NDEV1, and the non-binary transmit signal TNB is provided to the transmission channel CHNN (S200).

[0059] A second spike transceiver circuit TRX2 of the second neuromorphic device NDEV2 generates a plurality of reception spike signals RSPK1-RSPKM including reception spikes based on a non-binary reception signal RNB received through a transmission channel CHNN, and provides the plurality of reception spike signals RSPK1-RSPKM to a reception neuron RNR of a second neuron block NBLK2 of the second neuromorphic device NDEV2 (S300).

[0060] In this way, spiking neural networks can be implemented by combining multiple neuromorphic devices. Spiking neural networks can be implemented more efficiently by converting spiking signals and non-binary signals between multiple neuromorphic devices.

[0061] Figure 8 is a block diagram illustrating a spiking transceiver circuit included in a neuromorphic device according to an example embodiment.

[0062] Reference Figure 8 , the spike transceiver circuit 500 may include a spike transmitting circuit TX and / or a spike receiving circuit RX.

[0063] The spike transmission circuit TX may include a spike converter 11 , a packetizer 12 , a modulator 100 and / or an ECC encoder 13 .

[0064] The spike converter 11 may generate transmit spike data TDT including timing information for a transmit spike TSPK, with reference to a reference time provided by a timer. The packetizer 12 may generate a transmit data packet TPKT based on the transmit spike data TDT. The modulator 100 may modulate the transmit data packet TPKT to generate a non-binary transmit signal TNB. The ECC encoder 13 may generate parity bits by performing ECC encoding on at least one of the header or payload of the transmit data packet TPKT, and the packetizer 12 may add the parity bits to the transmit data packet TPKT.

[0065] The spike receiving circuit RX may include a demodulator 200 , a depacketizer 22 , a spike generator 21 and / or an ECC decoder 23 .

[0066] The demodulator 200 can demodulate the non-binary receive signal RNB to generate a receive data packet RPKT. The depacketizer 22 can generate receive spike data RDT, including timing information of receive spikes, based on the receive data packet RPKT. The spike generator 21 can generate a plurality of receive spike signals RSPK based on the receive spike data RDT, with reference to a reference time provided by a timer. The ECC decoder 23 can correct errors in the receive data packet RPKT based on the parity bits included in the receive data packet RPKT.

[0067] In some example embodiments, the ECC encoder 13 and the ECC decoder 23 may be implemented to share at least a portion of components. In some example embodiments, the ECC encoder 13 and the ECC decoder 23 may be selectively enabled, or may be omitted. A timer may be included in a neuromorphic device including the spike transceiver circuit 500 to provide a reference time for synchronizing the overall operation of the neuromorphic device.

[0068] In this way, the spike transmission circuit TX can generate a transmission data packet TPKT based on the information about the transmission spikes and modulate the transmission data packet TPKT into a non-binary transmission signal TNB, while the spike reception circuit RX can demodulate the non-binary reception signal RNB into a reception data packet RPKT. A spike neural network can be effectively implemented by packaging the spike signals and modulating the data packets into non-binary signals.

[0069] Figures 9A to 9G is a diagram illustrating an example format of a data packet storable in a memory for use in a neuromorphic device according to example embodiments.

[0070] Reference Figures 9A to 9G The data packet may include a header HDR and a payload. The data packet may be the aforementioned transmit data packet output from the spike transmit circuit and the aforementioned receive data packet input to the spike receive circuit. The header HDR may include information such as the identifier PSRC of the transmit neuron block that generated the transmit spikes SPK1 to SPK3, the identifier PDST of the receive neuron block or spike receive circuit that received the data packet, the number NS of transmit spikes SPK1 to SPK3 included in the transmit data packet, and the like. Although not shown in the drawings, the header HDR may also include information such as whether the ECC function is enabled and the ECC encoding scheme.

[0071] In some example embodiments, Figure 9A As shown, the payload may include identifiers DST1 - DST3 of receiving neurons for receiving the transmission spikes SPK1 - SPK3 , and timing information (eg, time stamps TSTM1 - TSTM3 ) of the transmission spikes SPK1 - SPK3 .

[0072] In some example embodiments, Figure 9B As shown, the payload may include: identifiers DST1-DST3 of receiving neurons for receiving the transmission spikes SPK1-SPK3, timing information of the transmission spikes SPK1-SPK3 (eg, timestamps TSTM1-TSTM3), and weight values ​​W1-W3.

[0073] In some example embodiments, Figure 9C As shown, the spike transmission circuit can generate a header parity bit HPRT by performing ECC encoding on the header HDR and add the header parity bit HPRT to the transmitted data packet. Furthermore, the spike transmission circuit can generate a spike parity bit SPRT by performing ECC encoding on the payload and add the spike parity bit SPRT to the transmitted data packet. The spike reception circuit can correct errors in the received data packet based on the parity bits HPRT and SPRT included in the received data packet.

[0074] According to an example embodiment, Figure 9D As shown, the spike transmission circuit can perform ECC encoding only on the header HDR to add the header parity bit HPRT to the transmission data packet, and can omit ECC encoding on the payload.

[0075] In some example embodiments, Figure 9EAs shown, the spike transmission circuit can generate a first parity bit PRT1 by performing ECC encoding on the header HDR and the identifiers DST1 to DST3 of the receiving neurons, and add the first parity bit PRT1 to the transmission data packet. In addition, the spike transmission circuit can generate a second parity bit PRT2 by performing ECC encoding on the timestamps TSTM1 to TSTM3 and the weight values ​​W1 to W3, and add the second parity bit PRT2 to the transmission data packet. In some embodiments, the ECC level of the first parity bit PRT1 can be higher than the ECC level of the second parity bit PRT2. Figure 10 Describes the ECC level.

[0076] Figure 9F and Figure 9G Some example embodiments of multi-level coding corresponding to a transmission scheme using a PAM4 signal are shown.

[0077] like Figure 9F As shown, the four signal levels of the PAM4 signal may correspond to values ​​of "11," "10," "01," and "10" of 2-bit data. The values ​​"11" and "10" represent some example embodiments in which the most significant bit (MSB) is 1, and the values ​​"01" and "00" represent some example embodiments in which the MSB is 0.

[0078] like Figure 9G As shown, in the PAM4 transmission scheme, the header HDR and the identifier DST of the receiving neuron can be set at the MSB of the data packet, and the timestamp TSTM and the weight value W can be set at the least significant bit (LSB) of the data packet. The spike transmission circuit can generate a first parity bit PRT1 by performing ECC encoding on the header HDR and the identifier DST, and add the first parity bit to the MSB of the data packet. In addition, the spike transmission circuit can generate a second parity bit PRT2 by performing ECC encoding on the timestamp TSTM and the weight value W, and add the second parity bit to the LSB of the data packet. In some example embodiments, the ECC level of the first parity bit PRT1 can be set to be higher than the ECC level of the second parity bit PRT2.

[0079] In this way, the code rate can be increased and / or the ECC overhead can be reduced by applying different ECC levels according to the importance of the transmitted information.

[0080] Figure 10 is a diagram for describing error correction code (ECC) levels in a neuromorphic device according to example embodiments.

[0081] exist Figure 10In , SEC means single error correction, DED means double error detection, and DEC means double error correction. Figure 10 The number of parity bits corresponding to the Hamming code or the extended Hamming code is shown. Figure 10 The exemplary embodiments in are non-limiting examples, and exemplary embodiments are not limited thereto. For example, if included in a Bose-Chaudhuri-Hocquenghem (BCH) code, a Reed-Solomon code, etc., the number of parity bits may be determined differently.

[0082] You can do this by Figure 10 The appropriate ECC level is selected from the multiple ECC levels shown to adjust the degree of error correction and the performance of the system. As the number of neurons included in the spiking neural network increases, the demand for higher capacity and / or higher speed data transmission may increase. The operating speed of neuromorphic devices and neuromorphic systems can be increased by transmitting information about spikes using a PAM scheme, which will be referred to below. Figures 11 to 15 Describe this.

[0083] Figure 11 is a diagram illustrating an example of a pulse amplitude modulation (PAM) signal in a neuromorphic device according to example embodiments.

[0084] Reference Figure 11 , a PAM4 signal with four signal levels can be generated in a PAM transmitter. The lowest voltage level V0 of the PAM4 signal can be mapped to the 2-bit data "00". The highest voltage level V3 of the PAM4 signal can be mapped to the 2-bit data "10". The lowest voltage level V0 and the highest voltage level V3 of the PAM4 signal correspond to the high swing level. The intermediate voltage levels V1 and V2 of the PAM4 signal can be mapped to the 2-bit data "01" and "11" respectively. The intermediate voltage levels V1 and V2 corresponding to the 2-bit data "01" and "11" can both correspond to the low swing level. The mapping between the above-mentioned voltage levels V0, V1, V2 and V3 and the data can be a mapping based on Gray code. However, Figure 11 The illustrated mapping between voltage levels and data bits is an example, and it will be appreciated that the mapping may be varied variously.

[0085] In the PAM4 transmitter described below, a driver for generating a low-swing signal and a driver for generating a high-swing signal may be separately provided. Furthermore, it will be appreciated that in a pulse amplitude modulation scheme, three or more drivers capable of driving multiple voltage swings in addition to the low and high swings may be provided.

[0086] Figure 12is a block diagram illustrating an example embodiment of a modulator included in a neuromorphic device according to an example embodiment.

[0087] Reference Figure 12 , the modulator may be implemented as a PAM transmitter 100. The PAM transmitter 100 may include a serializer 110, a PAM encoder 120, a first voltage regulator 130, a second voltage regulator 140, a low swing (LS) driver 150, a high swing (HS) driver 160, and / or a constant current load switch 170.

[0088] The serializer 110 converts the data Data_P to be transmitted into serial data Data_S in the form of a continuous data stream. In an example embodiment, the data Data_P may be provided from a processing circuit (e.g., a processor or a baseband unit such as various data processing blocks). The serializer 110 may output the serial data Data_S in the form of continuous binary bits.

[0089] The PAM encoder 120 encodes the data Data_S in 2-bit units. For example, the PAM encoder 120 divides the continuous data Data_S into 2-bit units. The PAM encoder 120 generates a drive signal D00, D01, D11, or D10 corresponding to the logic value of each 2-bit unit. The PAM encoder 120 transmits the generated drive signal to the low-swing driver 150 or the high-swing driver 160. For example, in some example embodiments where the logic value of the 2-bit data is "00" or "10," the PAM encoder 120 transmits the drive signal "D00" or "D10" to the high-swing driver 160. In some example embodiments where the logic value of the 2-bit data is "01" or "11," the PAM encoder 120 transmits the drive signal "D01" or "D11" to the low-swing driver 150.

[0090] In addition, in some example embodiments, when the PAM encoder 120 transmits the driving signal “D01” or “D11” to the low-swing driver 150, the PAM encoder 120 may generate a first enable signal LS_EN for activating the low-swing driver 150. In some example embodiments, when the PAM encoder 120 transmits the driving signal “D00” or “D10” to the high-swing driver 160, the PAM encoder 120 may generate a second enable signal HS_EN for activating the high-swing driver 160.

[0091] When the high-swing driver 160 is activated and the low-swing driver 150 is deactivated, the PAM encoder 120 may turn on the constant current load switch 170. In some example embodiments, when the PAM encoder 120 transmits a drive signal "D00" or "D10" to the high-swing driver 160, the PAM encoder 120 may deactivate the low-swing driver 150 and simultaneously turn on the constant current load switch 170. If the constant current load switch 170 is turned on, a current path is formed between the voltage regulators 130 and 140. In some example embodiments, even if the current path through the low-swing driver 150 is blocked, the current path formed through the constant current load switch 170 allows the voltage regulators 130 and 140 to maintain stable operation. To control the constant current load switch 170, for example, the PAM encoder 120 may use the second enable signal HS_EN. However, it should be understood that in some example embodiments, the first enable signal LS_EN may be used to turn the constant current load switch 170 on or off.

[0092] The first voltage regulator 130 provides the first low swing voltage VLS_H to the low swing driver 150. The first voltage regulator 130 may be configured to provide a first low swing voltage VLS_H according to a reference voltage Vref ( Figure 12 The first voltage regulator 130 outputs the first low-swing voltage VLS_H corresponding to the optimal high signal level (not shown). The first voltage regulator 130 continuously generates the first low-swing voltage VLS_H and provides the first low-swing voltage VLS_H to the low-swing driver 150. That is, even if the low-swing driver 150 does not generate a signal, the first voltage regulator 130 can continue to remain enabled.

[0093] The second voltage regulator 140 provides the second low-swing voltage VLS_L to the low-swing driver 150. The second voltage regulator 140 can be configured to output the second low-swing voltage VLS_L corresponding to the optimal low signal level based on the reference voltage Vref. The second voltage regulator 140 continuously generates the second low-swing voltage VLS_L and provides the second low-swing voltage VLS_L to the low-swing driver 150. That is, even if the low-swing driver 150 does not generate a signal, the second voltage regulator 140 can remain in an enabled state.

[0094] The low-swing driver 150 generates a low-swing signal corresponding to the driving signal "D01" or "D11" (a case where the logic value of 2-bit data is "01" or "11"). The low-swing driver 150 generates a low-swing signal having a low-swing level (e.g., Figure 11The low swing driver 150 generates the low swing signal in response to the first enable signal LS_EN from the PAM encoder 120. The low swing driver 150 may transmit the generated low swing signal to the output terminals TXP and TXN.

[0095] The high-swing driver 160 generates a high-swing signal corresponding to the driving signal "D00" or "D10" (a case where the logic value of 2-bit data is "00" or "10"). The high-swing driver 160 generates a high-swing signal having a high swing level (e.g., Figure 11 The high swing driver 160 generates the high swing signal in response to the second enable signal HS_EN from the PAM encoder 120. The high swing driver 160 may transmit the generated high swing signal to the output terminals TXP and TXN.

[0096] The low-swing signal and the high-swing signal are transmitted to the output terminals TXP and TXN exclusively. For example, at the time when the low-swing signal is output, the high-swing signal does not exist. That is, at the time when the low-swing driver 150 outputs the low-swing signal, the high-swing driver 160 can be deactivated, and the output terminals of the high-swing driver 160 can maintain a high-impedance state, and vice versa. At the time when the high-swing driver 160 outputs the high-swing signal, the low-swing driver 150 can be deactivated, and the output terminals of the low-swing driver 150 can maintain a high-impedance state.

[0097] The constant current load switch 170 provides a current path between the first voltage regulator 130 and the second voltage regulator 140 under the control of the PAM encoder 120. When the low-swing driver 150 is activated, the low-swing driver 150 provides a current path between the first voltage regulator 130 and the second voltage regulator 140. However, when the low-swing driver 150 is deactivated, the current path formed within the low-swing driver 150 is blocked.

[0098] Figure 13 It is shown from Figure 12 Timing diagram of an example of a PAM signal output by the modulator.

[0099] Reference Figure 13 , at the transmission power supply voltage VDDTX and the level of “0V”, the PAM transmitter 100 continuously performs high swing and low swing operations to generate a PAM transmission signal.

[0100] The low-swing operation is performed between a first low-swing voltage VLS_H provided by the first voltage regulator 130 and a second low-swing voltage VLS_L provided by the second voltage regulator 140. If the high-swing driver 160 is deactivated and the low-swing driver 150 is activated, the PAM transmit signal at the output terminals TXP and TXN swings between the levels of the first low-swing voltage VLS_H and the second low-swing voltage VLS_L. In some example embodiments, during a period ΔT1 when the low-swing enable signal LS_EN is activated, the low-swing driver 150 may generate the PAM transmit signal using the first low-swing voltage VLS_H and the second low-swing voltage VLS_L. Therefore, the PAM transmit signal swings between the first low-swing voltage VLS_H and the second low-swing voltage VLS_L at the time when the low-swing enable signal LS_EN is activated.

[0101] In contrast, during the period ΔT2 in which the high-swing enable signal HS_EN is activated, the low-swing driver 150 is deactivated. In some example embodiments, the high-swing driver 160 may generate a PAM transmit signal using the transmit power supply voltages VDDTX and VSS. Therefore, the PAM transmit signal swings between a level greater than the first low-swing voltage VLS_H and a level less than the second low-swing voltage VLS_L.

[0102] Figure 14 is a diagram illustrating an example eye pattern of a PAM signal.

[0103] Reference Figure 14 , through the effective voltage swings of the low-swing driver 150 and the high-swing driver 160 , the eye diagram of the PAM transmission signal provides characteristics of an eye diagram in a typical PAM4 signal scheme.

[0104] like Figure 14 As shown, the output transmission signal of the PAM4 signal scheme can support four levels and can transmit 2 bits of data per unit interval (UI).

[0105] Figure 15 is a block diagram illustrating some example embodiments of a demodulator included in a neuromorphic device according to example embodiments.

[0106] Reference Figure 15 The PAM receiver 200 may include a first switched capacitor summer (SCS1) 210, a second switched capacitor summer (SCS2) 220, a first comparator 230, a second comparator 240 and a third comparator 250, a multiplexer 260 and / or a termination circuit 270.

[0107] The PAM input signals RXINP and RXINN transmitted through the transmission line are transmitted to input pads of the PAM receiver 200. The PAM input signals RXINP and RXINN are converted into voltage signals having a preset level by the termination circuit 270 and then transmitted to the first switched capacitor summer 210 and the second switched capacitor summer 220.

[0108] The first switched capacitor summer 210 generates a first receive signal RXP and a second receive signal RXN from the PAM input signals RXINP and RXINN. The second switched capacitor summer 220 has the same structure as the first switched capacitor summer 210, except that an offset "a" is applied to the second switched capacitor summer 220. The first receive signal RXP and the second receive signal RXN generated in the first switched capacitor summer 210 are respectively transmitted to the corresponding inputs of the second comparator 240 and the third comparator 250 to determine the least significant bit (LSB), and are also transmitted to the corresponding inputs of the first comparator 230 to determine the most significant bit (MSB).

[0109] Second switched capacitor summer 220 receives PAM input signals RXINP and RXINN to generate offset received signals RXP+ and RXN+. Second switched capacitor summer 220 generates third received signal RXP+ and fourth received signal RXN+ obtained by applying an offset "α" to PAM input signals RXINP and RXINN. To apply the offset "α," second switched capacitor summer 220 has a structure for capacitor switching and boosts a voltage corresponding to the offset "α." Second switched capacitor summer 220 may use clock signal CK for capacitor switching. Third received signal RXP+ and fourth received signal RXN+ generated by second switched capacitor summer 220 are provided to corresponding inputs of second comparator 240 and third comparator 250, respectively, to determine the LSB.

[0110] First comparator 230 determines the logic value of the MSB of 2-bit data transmitted within a unit interval UI. First comparator 230 determines the logic value of the MSB using the first receive signal RXP and the second receive signal RXN, which are not offset. The technique for determining the logic value of the MSB by first comparator 230 is the same or substantially the same as the technique for determining the logic value using the level of a differential signal in a conventional NRZ-based receiver. The first receive signal RXP and the second receive signal RXN may have a substantially symmetrical relationship about a center voltage. That is, the MSB of a 2-bit data unit transmitted during a unit interval can be determined using the levels of the first receive signal RXP and the second receive signal RXN. For example, if the voltage of the second receive signal RXN is greater than the MSB reference voltage VMSB, first comparator 230 may determine the MSB of the PAM received data of the 2-bit data unit as a logic "1."

[0111] The second comparator 240 determines the first least significant bit value of a 2-bit data unit transmitted within a unit interval UI by using the first receive signal RXP without an applied offset and the fourth receive signal RXN+ with an applied offset. The fourth receive signal RXN+ has a voltage level that is increased by an offset "α" compared to the second receive signal RXN. The second comparator 240 can compare the first receive signal RXP with the fourth receive signal RXN+ increased by an offset "α" and output an inverted value of the actually transmitted first least significant bit value as a comparison result. Therefore, an inverter can be connected to the output terminal of the second comparator 240 to restore the inverted value of the first least significant bit value. When the MSB is a logic "1," the first least significant bit value output by the second comparator 240 can be valid as the LSB output by the PAM receiver 200.

[0112] The third comparator 250 determines the logic value of the second least significant bit value of the 2-bit data unit transmitted within one unit interval UI by using the second received signal RXN without the applied offset and the third received signal RXP+ with the applied offset. Compared with the first received signal RXP, the third received signal RXP+ has a voltage level that is increased by the offset "α". The third comparator 250 can compare the second received signal RXN with the third received signal RXP+ increased by the offset "α" and can output the second least significant bit value as the comparison result. When the logic value of the MSB determined by the first comparator 230 is "0", the second least significant bit value output by the third comparator 250 can be valid as the LSB output by the PAM receiver 200.

[0113] The multiplexer 260 selects any one of the outputs of the second comparator 240 and the third comparator 250 according to the logic value of the MSB provided from the first comparator 230. In some example embodiments where the logic value of the MSB determined by the first comparator 230 is "0", the multiplexer 260 outputs the second least significant bit value determined by the third comparator 250 as the LSB. Conversely, in some example embodiments where the logic value of the MSB determined by the first comparator 230 is "1", the multiplexer 260 outputs the first least significant bit value determined by the second comparator 240 as the LSB.

[0114] The termination circuit 270 may be implemented using an impedance circuit to match the input impedance of the PAM receiver 200. The PAM input signals RXINP and RXINN are transmitted as voltage signals to the first switched capacitor summer 210 and the second switched capacitor summer 220 by the termination circuit 270.

[0115] A neuromorphic device and a neuromorphic system including a plurality of neuromorphic devices according to an exemplary embodiment may implement an effective spiking neural network by transmitting packetized spike signals in the form of PAM signals, as described with reference to Figures 11 to 15 described.

[0116] 16A to 16D is a diagram illustrating an example embodiment of a spiking neural network according to example embodiments.

[0117] Reference Figure 16A , the spike neural network SNN1 may include a plurality of neuromorphic devices NDEV1-NDEV6 commonly connected to a transmission channel CHNN. Each of the neuromorphic devices NDEV1-NDEV6 may include a spike transceiver circuit for transmitting spike information in the form of a non-binary signal as described above. Figure 16A In the spiking neural network SNN1, neuromorphic devices NDEV1~NDEV6 can exchange spike information with any other neuromorphic device.

[0118] Figure 16B The spiking neural network SNN2 has a one-dimensional systolic configuration, Figure 16C The spiking neural network SNN3 has a two-dimensional contraction configuration, while Figure 16D The spiking neural network SNN4 has a three-dimensional contraction configuration. Figure 16B 、 Figure 16C and Figure 16D In the collapsed configuration, in some neuromorphic devices, the spike transceiver circuit TRX may include only a spike transmitting circuit for transmitting spike information, and may omit a spike receiving circuit for receiving spike information.

[0119] Reference Figure 16B , multiple neuromorphic devices NDEV1 ˜ NDEV4 can be connected in a one-dimensional manner through a transmission channel CHNN, and spike information can be propagated, for example, in a rightward direction.

[0120] Reference Figure 16C , multiple neuromorphic devices NDEV11 ˜ NDEV44 may be connected in a two-dimensional manner through a transmission channel CHNN, and spike information may be propagated, for example, in a rightward direction and a bottomward direction.

[0121] Reference Figure 16D , multiple neuromorphic devices NDEV111-NDEV222 can be connected in a three-dimensional manner through horizontal transmission channels HCHNN and vertical transmission channels VCHNN. In each layer of layers LAY1 and LAY2, spike information can be propagated, for example, in the right direction and in the bottom direction. Between layers LAY1 and LAY2, spike information can be propagated, for example, in the downward direction (from LAY1 to LAY2).

[0122] In this way, various spiking neural networks can be implemented using multiple neuromorphic devices. At least one neuromorphic device can use a spiking transmit circuit in a spiking transceiver circuit TRX to transmit a non-binary transmit signal to a transmission channel, and at least one neuromorphic device can use a spiking receive circuit in the spiking transceiver circuit TRX to receive the non-binary transmit signal as a non-binary receive signal via the transmission channel.

[0123] Figures 17 to 20 is a diagram illustrating a neuromorphic system according to example embodiments.

[0124] Reference Figure 17 The neuromorphic system 600 may include multiple neuromorphic devices NDEV in the same semiconductor die SD, where the semiconductor die SD includes a semiconductor substrate SUB and a dielectric layer DLY located on (or above) the semiconductor substrate SUB. The first neuromorphic device may include a first neuromorphic core NRC1 and a first spike transceiver circuit TRX1, the second neuromorphic device may include a second neuromorphic core NRC2 and a second spike transceiver circuit TRX2, and the third neuromorphic device may include a third neuromorphic core NRC3 and a third spike transceiver circuit TRX3. Transmission channels CHNN for transmitting non-binary signals between the multiple neuromorphic devices may be included in metal lines formed in the dielectric layer DLY on (or above) the semiconductor substrate SUB.

[0125] Reference Figure 18The neuromorphic system 700 includes a base substrate 60 and a plurality of semiconductor dies 41, 42, 43, and 50 stacked on the base substrate 60. The stacked semiconductor dies 41, 42, and 43 can be packaged as a chip 40, and the semiconductor die 50 can be packaged as another separate chip. The packaged chips 40 and 50 can be mounted on the base substrate 60 using bumps BMP and can be electrically connected to each other through the bumps BMP, vertical contacts VC, and / or signal lines 62 of the base substrate 60.

[0126] The stacked semiconductor chip 40 includes a plurality of semiconductor dies 41 , 42 , and 43 stacked in a vertical direction. Figure 18 A non-limiting example of three stacked semiconductor dies 41, 42, and 43 is shown, and the number of stacked semiconductor dies in the stacked semiconductor chip 40 may vary. Semiconductor die 41 may include internal circuit INT1, transceiver circuit TRX1, and through-silicon via (TSV1); semiconductor die 42 may include internal circuit INT2, transceiver circuit TRX2, and through-silicon via (TSV2); and semiconductor die 43 may include internal circuit INT3, transceiver circuit TRX3, and through-silicon via (TSV3). Another semiconductor die 50 may include internal circuit INTC, transceiver circuit TRXC, and / or vertical contact VC.

[0127] Each data path PTH may include through-substrate vias TSV1, TSV2, and TSV3 formed in semiconductor dies 41, 42, and 43, respectively. Figure 18 It is shown that two through-substrate vias in adjacent semiconductor dies are directly connected. The through-substrate vias can also be electrically connected to each other through bumps, pads, metal lines, etc.

[0128] Each internal circuit INTi (i=1, 2, 3) and INTC may include various integrated circuits depending on the type of semiconductor chips 40 and 50. For example, the stacked semiconductor chip 40 may be a neuromorphic device including at least one neuron block as described above. Each transceiver circuit TRXi and TRXC may be a spike transceiver circuit configured to convert spike information into a non-binary signal and / or vice versa and transmit the non-binary signal between neuron blocks.

[0129] Thus, the first neuromorphic device may be included in a first semiconductor die, and the second neuromorphic device may be included in a second semiconductor die stacked in a vertical direction with the first semiconductor die. In some example embodiments, the transmission channel for transmitting the non-binary signal may include a through-substrate via formed in the first semiconductor die and the second semiconductor die. Figure 18As shown, the transmission channel may further include a signal line 62 formed at a base substrate 60 on which the packaged chips 40 and 50 are mounted, so that a non-binary signal may be transmitted through the signal line 62 between the packaged chips 40 and 50 .

[0130] Figure 18 A structure is shown in which semiconductor dies 41 , 42 , and 43 including neuromorphic devices are vertically stacked except for a logic semiconductor die 50 , and the logic semiconductor die 50 is electrically connected to the package chip 40 through an interposer or base substrate 60 . Figure 19 A structure in which a logic semiconductor die and a semiconductor die including a neuromorphic device are vertically stacked is shown.

[0131] Reference Figure 19 , the neuromorphic system 800 may include a base substrate BSUB and a plurality of semiconductor dies stacked on the base substrate BSUB. The lowest semiconductor die may be a logic semiconductor die, and the neuromorphic device may be formed in the semiconductor die above the logic semiconductor die. The semiconductor die may include transceiver circuits TRXi and internal circuits INTi (i=1, 2, 3, 4). The data path PTH may include through-substrate vias TSV1 to TSV4 and TSVC formed in the semiconductor die including the neuromorphic device and the logic semiconductor die, respectively.

[0132] The base substrate BSUB may be the same as or include an interposer. The base substrate BSUB may be a printed circuit board (PCB). External connection elements such as conductive bumps (BMPs) may be formed on the lower surface of the base substrate BSUB, and internal connection elements such as conductive bumps may be formed on the upper surface of the base substrate BSUB. A resin RSN may be used to encapsulate the stacked semiconductor dies.

[0133] Figure 20 A semiconductor package including multiple neuromorphic devices according to example embodiments is shown. Figure 20 , a semiconductor package 900 may include one or more stacked neuromorphic devices SND 910 and a central processing unit (CPU) 920. The stacked neuromorphic devices 910 and CPU 920 may be mounted on an interposer 930, and the interposer 930 may be mounted on a package substrate 940.

[0134] The stacked neuromorphic devices 910 may be packaged into a packaged chip, and the packaged chip may be mounted on an interposer 930 of a base substrate 940. The first neuromorphic device may be included in the first packaged chip, and the second neuromorphic device may be included in a second packaged chip different from the first packaged chip. In some example embodiments, the above-described transmission channel for transmitting non-binary signals may include a signal line formed in the interposer 930 or in the base substrate 930 on which the first and second packaged chips are mounted.

[0135] Figure 21 and Figure 22 is a diagram for describing an example of a deep learning neural network structure driven by a neuromorphic system according to example embodiments.

[0136] The neuromorphic system according to example embodiments may be used to drive neural network systems and / or machine learning systems such as an artificial neural network (ANN) system, a convolutional neural network (CNN) system, a deep neural network (DNN) system, a deep learning system, a deconvolutional neural network, a recurrent neural network (RNN) (optionally including long short-term memory (LSTM) units and / or gated recurrent units (GRU)), a stacked neural network (SNN), a state-space dynamic neural network (SSDNN), a deep belief network (DBN), a generative adversarial network (GAN), and / or a restricted Boltzmann machine (RBM).

[0137] Alternatively or additionally, such a machine learning system may include other forms of machine learning models (e.g., linear and / or logistic regression, statistical clustering, Bayesian classification, decision trees, dimensionality reduction (e.g., principal component analysis), and expert systems) and / or combinations thereof (including ensembles such as random forests).

[0138] For example, the neuromorphic system can execute and process various services and / or applications, such as image classification services, user authentication services, advanced driver assistance system (ADAS) services, voice assistant services, automatic speech recognition (ARS) services, etc.

[0139] Reference Figure 21 , a general neural network may include an input layer IL, a plurality of hidden layers HL1, HL2, ..., HLn and an output layer OL.

[0140] The input layer IL may include i input nodes x1, x2, ..., xi, where i is a natural number. Input data (e.g., vector input data) IDAT of length i may be input to the input nodes x1, x2, ..., xi, such that each element of the input data IDAT is input to a corresponding input node among the input nodes x1, x2, ..., xi.

[0141] The plurality of hidden layers HL1, HL2, ..., HLn may include n hidden layers, where n is a natural number, and may include a plurality of hidden nodes h 1 1.h 1 2. h 1 3. ..., h 1 m 、h 2 1.h 2 2. h 2 3. ..., h 2 m 、h n 1.h n 2. h n 3. ..., h n m For example, the hidden layer HL1 may include m hidden nodes h 1 1.h 1 2. h 1 3. ..., h 1 m , the hidden layer HL2 may include m hidden nodes h 2 1.h 2 2. h 2 3. ..., h 2 m , the hidden layer HLn may include m hidden nodes h n 1.h n 2. h n 3. ..., h n m , where m is a natural number.

[0142] The output layer OL may include j output nodes y1, y2, ..., y j , where j is a natural number. Output nodes y1, y2, ..., y j Each of the may correspond to a corresponding category in the categories to be classified. The output layer OL may output an output value ODAT (e.g., a category score or a simple score) associated with the input data IDAT for each category. The output layer OL may be referred to as a fully connected layer and may indicate, for example, the probability that the input data IDAT corresponds to a car.

[0143] Figure 21The structure of the neural network shown can be represented by information about branches (or connections) between nodes, which are illustrated as lines, and weight values ​​(not shown) assigned to each branch. Nodes within a layer may not be directly connected to each other, but nodes in different layers may be fully or partially connected to each other.

[0144] Each node (for example, node h 1 1) can receive the output of a previous node (e.g., node x1), can perform a computation, operation, or calculation on the received output, and can output the result of the computation, operation, or calculation to a subsequent node (e.g., node h 2 1). Each node can calculate the value to be output by applying a specific function (for example, a nonlinear function) to the input.

[0145] Typically, the structure of a neural network is pre-set, and weight values ​​for connections between nodes are appropriately set using data with known answers for the categories to which they belong. This data with known answers is called "training data," and the process of determining weight values ​​is called "training." The neural network "learns" during training. A set of independently trainable structures and weight values ​​is called a "model," and the process of predicting which category the input data belongs to using the model with the determined weight values ​​and then outputting the predicted value is called "testing."

[0146] Reference Figure 22 , a convolutional neural network can include multiple layers CONV1, RELU1, CONV2, RELU2, POOL1, CONV3, RELU3, CONV4, RELU4, POOL2, CONV5, RELU5, CONV6, RELU6, POOL3 and FC.

[0147] Unlike general neural networks, each layer of a convolutional neural network can have three dimensions: width, height, and depth. Therefore, the data input to each layer can be volume data with three dimensions: width, height, and depth. For example, if Figure 22 The size of the input image in is 32 width units (e.g., 32 pixels) and 32 height units (e.g., 32 pixels) and three color channels R, G, and B, and the size of the input data IDAT corresponding to the input image can be 32x32x3. Figure 22 The input data IDAT in can be called input volume data or input activation volume.

[0148] Convolutional layers CONV1, CONV2, CONV3, CONV4, CONV5, and CONV6 can each perform a convolution operation on input volume data. For example, in image processing, a convolution operation refers to an operation that processes image data based on a mask having weight values, and obtains an output value by multiplying the input value by the weight value and adding all the product values. The mask can be called a filter, a window, or a kernel.

[0149] The RELU layers RELU1, RELU2, RELU3, RELU4, RELU5, and RELU6 can all perform rectified linear unit operations on the input. The pooling layers POOL1, POOL2, and POOL3 can all perform downsampling operations on the input volume data along the spatial dimensions of width and height. Typically, a convolutional layer (e.g., CONV1) and a RELU layer (e.g., RELU1) can form a pair of CONV / RELU layers in a convolutional neural network. Paired CONV / RELU layers can be repeatedly arranged in a convolutional neural network, and the pooling layer can be periodically inserted into the convolutional neural network to reduce the image space size and extract image features.

[0150] The output layer or fully connected layer FC can output the result (e.g., class score) of the input volume data IDAT for each class. For example, when the convolution operation and downsampling operation are repeated, the input volume data IDAT corresponding to the two-dimensional image can be converted into a one-dimensional matrix or vector. For example, the fully connected layer FC can represent the probability that the input volume data IDAT corresponds to a car, a truck, an airplane, a ship, and a horse.

[0151] Figure 23 is a block diagram illustrating a computing system according to an example embodiment.

[0152] Reference Figure 23 The computing system 1200 includes an application processor 1210, a connection unit 1220, a volatile memory device (VM) 1230, a non-volatile memory device (NVM) 1240, a user interface 1250, and a neuromorphic system (NSYS) 1260, any or all of which may be implemented in a processing circuit. According to an example embodiment, the computing system 1200 may correspond to a mobile device such as a smartphone.

[0153] The application processor 1210 can execute applications such as a web browser, game applications, and a video player. The connection unit 1220 can perform wired or wireless communication with external devices. The volatile memory device 1230 can store data processed by the application processor 1210 or can be used as a working memory. The non-volatile memory device 1240 can store a boot image for booting the mobile system 1200. The user interface 1250 can include at least one input device such as a keypad or a touch screen, and at least one output device such as a speaker or a display device.

[0154] The neuromorphic system 1260 may include at least one neuron block and a spike transceiver circuit TRX configured to transmit and / or receive non-binary signals including spike information, as described above. Additionally, the application processor 1210 and / or the volatile memory device 1230 may also include a spike transceiver circuit TRX.

[0155] As described above, the neuromorphic device and the neuromorphic system according to example embodiments may effectively implement a spiking neural network by packetizing spike signals and modulating data packets to transmit the spike signals as non-binary signals.

[0156] The present invention can be applied to any electronic device and system. For example, the present invention can be applied to systems such as mobile phones, smart phones, personal digital assistants (PDAs), portable multimedia players (PMPs), digital cameras, camcorders, personal computers (PCs), server computers, workstations, laptop computers, digital TVs, set-top boxes, portable game consoles, navigation systems, wearable devices, Internet of Things (IoT) devices, Internet of Everything (IoE) devices, e-books, virtual reality (VR) devices, augmented reality (AR) devices, etc.

[0157] The foregoing is illustrative of example embodiments and should not be construed as limiting thereof. Although a few example embodiments have been described, those skilled in the art will readily appreciate that many modifications may be made to the example embodiments without departing substantially from the inventive concept.

Claims

1. A neuromorphic device comprising: a neuron block, the neuron block including a plurality of neurons connected by a plurality of synapses, for performing spike generation and computation; a spike transmission circuit configured to generate a non-binary transmission signal based on a plurality of transmission spike signals output from the neuron block and configured to transmit the non-binary transmission signal to a transmission channel, the non-binary transmission signal including information about transmission spikes included in the plurality of transmission spike signals; as well as a spike receiving circuit configured to receive a non-binary receive signal from the transmission channel and to generate a plurality of receive spike signals including receive spikes based on the non-binary receive signal to provide the plurality of receive spike signals to the neuron block, the non-binary receive signal including information about the receive spikes. wherein the spike transmission circuit is further configured to generate a transmission data packet based on the information about the transmission spike, and is configured to modulate the transmission data packet to generate the non-binary transmission signal, The transmit data packet includes a header and a payload, wherein the payload includes an identifier of a receiving neuron for receiving the transmit spike and timing information of the transmit spike, The non-binary transmission signal has four signal levels corresponding to 2-bit data. wherein the identifier of the receiving neuron for receiving the transmitted spike in the header and the payload is set to the most significant bit in the 2-bit data, The timing information of the transmission spike in the payload is set in the least significant bit of the 2-bit data.

2. The neuromorphic device of claim 1 , wherein: The non-binary transmit signal and the non-binary receive signal are pulse-based signals.

3. The neuromorphic device of claim 1 , wherein: The non-binary transmit signal and the non-binary receive signal are pulse amplitude modulated signals.

4. The neuromorphic device of claim 1 , wherein: The header includes an identifier of the neuron block that generated the transmit spike and the number of spikes included in the transmit data packet.

5. The neuromorphic device of claim 4, wherein: The spike transmission circuit is configured to generate header parity bits by performing error correction code encoding on the header, and add the header parity bits to the transmission data packet.

6. The neuromorphic device of claim 5, wherein: The spike transmission circuit is configured to generate spike parity bits by performing error correction code encoding on the payload and add the spike parity bits to the transmission data packet.

7. The neuromorphic device of claim 1 , wherein: The spike receiving circuit is configured to demodulate the non-binary receive signal to generate a receive data packet, and is configured to generate the plurality of receive spike signals based on the receive data packet including an identifier of a receive neuron for receiving the receive spike and timing information of the receive spike.

8. The neuromorphic device of claim 1 , wherein: The peak sending circuit includes: a spike converter configured to generate transmit spike data including timing information of the transmit spike; a packetizer configured to generate the transmission data packet based on the transmission spike data; and A modulator is configured to modulate the transmit data packet to generate the non-binary transmit signal.

9. The neuromorphic device of claim 8, wherein: The peak sending circuit also includes: an error correction code encoder configured to generate parity bits by performing error correction code encoding on at least one of the header or the payload of the transmission data packet, The packetizer adds the parity bit to the sending data packet.

10. The neuromorphic device of claim 1 , wherein: The peak receiving circuit includes: a demodulator configured to demodulate the non-binary received signal to generate a received data packet; a depacketizer configured to generate reception spike data including timing information of the reception spike based on the reception data packet; and A spike generator is configured to generate the plurality of receive spike signals based on the receive spike data.

11. The neuromorphic device of claim 10, wherein: The peak receiving circuit further includes: An error correction code decoder is configured to correct errors in the received data packet based on parity bits included in the received data packet.

12. The neuromorphic device of claim 1 , wherein: The neuron block, the spike transmitting circuit, and the spike receiving circuit are included in the same semiconductor die.

13. A neuromorphic system comprising: multiple neuromorphic devices; as well as a transmission channel configured to electrically connect the plurality of neuromorphic devices, Wherein, the plurality of neuromorphic devices each include: a neuron block, the neuron block including a plurality of neurons connected by a plurality of synapses, for performing spike generation and computation; a spike transmission circuit configured to generate a non-binary transmission signal based on a plurality of transmission spike signals output from the neuron block, and configured to transmit the non-binary transmission signal to a transmission channel, the non-binary transmission signal including information about transmission spikes included in the plurality of transmission spike signals; and a spike receiving circuit configured to receive a non-binary receive signal from the transmission channel and to generate a plurality of receive spike signals including receive spikes based on the non-binary receive signal to provide the plurality of receive spike signals to the neuron block, the non-binary receive signal including information about the receive spikes. wherein the spike transmission circuit is further configured to generate a transmission data packet based on information about the transmission spike, and is configured to modulate the transmission data packet to generate the non-binary transmission signal, The transmit data packet includes a header and a payload, wherein the payload includes an identifier of a receiving neuron for receiving the transmit spike and timing information of the transmit spike, The non-binary transmission signal has four signal levels corresponding to 2-bit data. wherein the identifier of the receiving neuron for receiving the transmitted spike in the header and the payload is set to the most significant bit in the 2-bit data, The timing information of the transmission spike in the payload is set in the least significant bit of the 2-bit data.

14. The neuromorphic system of claim 13, wherein: the plurality of neuromorphic devices include a first neuromorphic device and a second neuromorphic device, the first neuromorphic device configured to transmit the non-binary transmit signal to the transmission channel using the spike transmit circuit, and the second neuromorphic device configured to receive the non-binary transmit signal provided from the first neuromorphic device over the transmission channel as the non-binary receive signal, and Wherein, the plurality of neuromorphic devices form a spiking neural network.

15. The neuromorphic system of claim 14, wherein: The first neuromorphic device and the second neuromorphic device are included in a same semiconductor die, and the transmission channel includes a metal line formed over a semiconductor substrate in the semiconductor die.

16. The neuromorphic system of claim 14, wherein: The first neuromorphic device is included in a first semiconductor die, and the second neuromorphic device is included in a second semiconductor die stacked in a vertical direction with the first semiconductor die, and the transmission channel includes a through-substrate via located in the first semiconductor die and the second semiconductor die.

17. The neuromorphic system of claim 14, wherein: The first neuromorphic device is included in a first packaged chip, and the second neuromorphic device is included in a second packaged chip different from the first packaged chip, and the transmission channel includes a signal line located at a base substrate on which the first packaged chip and the second packaged chip are mounted.

18. The neuromorphic system of claim 13, wherein: The non-binary transmit signal and the non-binary receive signal are pulse amplitude modulated signals.

19. A neuromorphic device comprising: a neuron block, the neuron block including a plurality of neurons connected by a plurality of synapses, for performing spike generation and computation; a spike transmission circuit configured to generate a pulse amplitude modulated transmission signal based on a plurality of transmission spike signals output from the neuron block, and configured to transmit the pulse amplitude modulated transmission signal to a transmission channel; as well as a spike receiving circuit configured to receive a pulse amplitude modulated receiving signal from the transmission channel, Wherein, the peak sending circuit includes: a spike converter configured to generate transmit spike data including timing information of transmit spikes based on a plurality of transmit spike signals output from the neuron block; a packetizer configured to generate a transmit data packet based on the transmit spike data, wherein the transmit data packet includes a header and a payload, the payload including an identifier of a receiving neuron for receiving the transmit spike and timing information of the transmit spike; and a modulator configured to modulate the transmit data packet to generate the pulse amplitude modulated transmit signal, wherein the pulse amplitude modulated transmit signal has four signal levels corresponding to 2-bit data, wherein an identifier of a receiving neuron for receiving the transmit spike in the header and the payload is set at a most significant bit in the 2-bit data, wherein timing information of the transmit spike in the payload is set at a least significant bit in the 2-bit data, and Wherein, the peak receiving circuit includes: a demodulator configured to demodulate the pulse amplitude modulated received signal to generate a received data packet; a depacketizer configured to generate reception spike data including timing information of reception spikes based on the reception data packet; and A spike generator is configured to generate a plurality of reception spike signals based on the reception spike data.

Images