Nonlinear current attenuation in neural networks

Abstract

Non-linear current attenuation in neural networks The present description relates to a neural network (200) which comprises at least one first neuron (204) connected to a second neuron (201) via at least one synapse (205) and a non-linear current attenuator (202) configured to receive a first current (Iin200) from said at least one synapse (205) and to supply a second current (Iin200) to said second neuron (201). Figure for abstract: Fig. 2

Description

Title of the invention: Nonlinear current attenuation in neural networks Technical field

[0001] The present description relates generally to electronic systems and devices and their implementations. The present description relates more particularly to neural networks and their implementations. Prior art

[0002] Neural networks and, more specifically, artificial neural networks are a type of artificial intelligence and machine learning models based on the principles of neural organization found in biological neural networks constituting animal brains.

[0003] It would be desirable to be able to improve, at least in part, certain aspects of the implementation of neural networks. Summary of the invention

[0004] There is a need for more efficient neural networks.

[0005] There is a need for neural networks that use current attenuation between layers of neurons.

[0006] There is a need for neural networks that utilize current attenuation between a synapse and a spiking neuron.

[0007] There is a need for more efficient neural networks comprising a synapse that uses multi-level phase-shifting memory cells.

[0008] There is a need for neural networks that can compensate for drift and dispersion in the resistance of memory cells used to form a synapse.

[0009] There is a need for neural networks that can compensate for drift and dispersion in the resistance of phase-shifting memory cells used to form a synapse. One embodiment overcomes some or all of the drawbacks of known neural networks.

[0010] One embodiment provides a neural network that uses a non-linear current attenuator between two layers of neurons.

[0011] One embodiment provides a neural network that uses a non-linear current attenuator between a synapse and a spiking neuron.

[0012] One embodiment provides a neural network that includes at least a first neuron connected to a second neuron via at least one synapse and a nonlinear current attenuator configured to receive a first current from said at least one synapse and to supply a second current to said second neuron.

[0013] According to one embodiment, said at least one synapse comprises a multi-level memory cell.

[0014] According to one embodiment, said at least one synapse comprises a multi-level phase-modified memory cell.

[0015] According to one embodiment, said first current is a reading current of said at least one first neuron.

[0016] According to one embodiment, the neural network comprises at least two first neurons and at least two synapses, in which said first current is the sum of third currents provided by each of said at least two synapses.

[0017] According to one embodiment, said at least one first neuron is a spiking neuron.

[0018] According to one embodiment, said second neuron is a pulse neuron.

[0019] According to one embodiment, said non-linear current attenuator is non-linear. linear with respect to said first current.

[0020] According to one embodiment, said second current depends on the first current via an exponential function.

[0021] According to one embodiment, said second current is given by the following mathematical formula: [Math 1] ÎOUt3QQ = in which: - Ilin300 represents said first current; - SDFmax represents the maximum gain provided by said attenuator; - SDFmin represents the minimum gain provided by said attenuator; and - 2 is an experimentally defined factor. Brief description of the drawings

[0022] These characteristics and advantages, as well as others, will be explained in detail in the following description of particular embodiments given without limitation in relation to the attached figures among which:

[0023] [Fig.l] represents the principle of two layers of a fully connected neural network;

[0024] [Fig.2] represents an embodiment of a part of a neural network;

[0025] [Fig.3] represents a detailed embodiment of a part of a network of neurons;

[0026] [Fig.4] represents a graph illustrating the embodiment of [Fig.3]; and

[0027] [Fig.5] represents other graphs illustrating the embodiment of [Fig.3]. Description of the embodiments

[0028] The same elements have been designated by the same references in the different figures. In particular, the structural and / or functional elements common to the different embodiments may have the same references and may have identical structural, dimensional and material properties.

[0029] For the sake of clarity, only the steps and elements useful for understanding the described embodiments have been shown and are detailed.

[0030] Unless otherwise specified, when referring to two elements connected to each other, this means directly connected without intermediate elements other than conductors, and when referring to two elements connected (in English "coupled") to each other, this means that these two elements can be connected or be connected by means of one or more other elements.

[0031] In the following description, when reference is made to absolute position qualifiers, such as the terms "front", "back", "top", "bottom", "left", "right", etc., or relative position qualifiers, such as the terms "above", "below", "upper", "lower", etc., or to orientation qualifiers, such as the terms "horizontal", "vertical", etc., reference is made unless otherwise specified to the orientation of the figures.

[0032] Unless otherwise specified, the expressions "about", "approximately", "substantially", and "of the order of" mean to within 10%, preferably to within 5%.

[0033] The embodiments described above relate to implementations of neural networks and, more particularly, to implementations of spiking neural networks. A neural network is a machine learning model based on the principles of neural organization found in the biological neural networks constituting animal brains. Nowadays, neural networks are implemented by electronic circuits and devices. A neural network comprises several layers of neurons, each neuron being connected to the neurons of the other layers by connections called synapses which constitute the weighted memory.

[0034] In an electronic implementation of neural networks, each neuron is a transfer function cell transferring voltages and currents to other neurons via a synapse which is, in practice, one or more memory cells, preferably a memory cell. The use of the current attenuator between a synapse and a neuron is fundamental for the proper functioning of neural networks. In some cases, multi-level memory cells are used to constitute part of a synapse, such as multi-level phase-shift memory cells. These memory cells can be faced over time with a phenomenon of dispersion and resistance drift which can lead to implementation problems. The embodiments described above propose using a non-linear current attenuator between a synapse and a neuron to compensate for these resistance differences.

[0035] Furthermore, the embodiments described above are particularly suitable for inclusion in a chip configured to implement artificial intelligence functions.

[0036] [Fig.l] very schematically represents a neural network 100 comprising two layers of neurons totally connected to each other.

[0037] The neural network 100 comprises two layers 110 and 120 of neurons, in which the neurons are connected by synapses 130. More particularly, the layer 110 comprises several neurons 111 and the layer 120 comprises several neurons 121. Each neuron 111 is connected to each neuron 121 of the layer 120 via a synapse 130 and, consequently, each neuron 121 is connected to each neuron 111 of the layer 110 via a synapse 130.

[0038] Several types of neurons exist. Neurons 111 and 121 may be spiking neurons. Spiking neurons are artificial neurons that very closely mimic natural neural networks. They are adapted to receive analog data and to provide spiking data, i.e., spiking-shaped data. A detailed example of a spiking neuron will be described in connection with [Fig. 3].

[0039] A portion of a synapse 130 may be implemented using memory cells. More particularly, a portion of a synapse 130 may be implemented by a multi-level memory cell, i.e., a memory capable of storing more than two different values ​​of a data item. According to one embodiment, the synapse 130 may be implemented using a phase-modified memory cell (PCM cell) and, more particularly, a multi-level phase-modified memory cell.

[0040] A phase-change memory cell uses a phase transition of a material to store binary data. When the material is in a first phase, the value of the data is a first value, for example a one (1) and, when the material is in a second phase, the value of the data is a second value different from the first value, for example a zero (0). A read operation is based on the resistance value of said material.

[0041] A multi-level phase-change memory cell uses a material capable of changing its resistance value to more than two distinct values. More particularly, such a memory cell can store eight (8) distinct values. However, a common concern with a multi-level phase-change memory cell multiple levels is a drift and dispersion of resistance, which means that the value of its material drifts over time. The embodiment described below proposes to compensate for this drift.

[0042] [Fig.2] represents, very schematically and in the form of blocks, an embodiment of a part 200 of a neural network. Part 200 is concentrated on inputs of a neuron.

[0043] The portion 200 comprises a neuron 201 (SN). According to one embodiment, the neuron 201 is a spiking neuron. The neuron 201 may receive a current Iout200 from another layer of neurons in the neural network. As previously indicated, a detailed example of a spiking neuron will be described in relation to [Fig.3].

[0044] The portion 200 further comprises a non-linear current attenuator 202 (NL CURRENT ATTENUATOR) configured to provide a current Iout200 to said neuron 201. The attenuator 202 can receive a current lin200 and to apply a non-linear function to said current lin200 in order to provide a current Iout200. In other words, the current Iout200 is non-linear with respect to the current Iin200. A detailed example of such an attenuator will be described in relation to [Fig.3].

[0045] The portion 200 further comprises a layer 203 of neurons 204 (SN) and the associated synapse 205 (S) of the neural networks. According to one example, each neuron 204 is connected to the neuron 201 via a synapse 205. More particularly, each neuron 204 is connected to the current attenuator 202 via a synapse 205. According to one embodiment, like the neuron 201, each neuron 204 is a spiking neuron.

[0046] According to one embodiment, each synapse 205 is modeled by a memory cell and, more particularly, by a multi-level memory cell. According to a preferred example, a portion of each synapse 205 may be modeled by a phase-changing memory cell and, more particularly, by a multi-level phase-changing memory cell. Each synapse 205 is connected to a neuron 201 via the non-linear current attenuator 202. More particularly, all the outputs of the synapses 205 are connected to an input node of the attenuator 202. Thus, the current lin200 is the sum of each output current of the synapses 205. Furthermore, each output current of the synapses 205 is the result of voltage pulses converted into current by resistive elements of the synapses 205 and, more particularly, by the memory cells of the synapses 205.The inventors have discovered that the use of a non-linear current attenuator between the synapses 205 and a neuron 201 compensates for a phenomenon of dispersion and resistance drift which appears in the synapses 205.

[0047] [Fig. 3] is a circuit diagram of a part 300 of a neural network of the type of part 200 described in relation to [Fig. 2].

[0048] Like part 200 of [Fig.2], part 300 is focused on inputs of a neuron 301. Part 300 comprises: - a neuron 301; - a non-linear current attenuator 302; - a layer 303 of synapses 304 connected to neuron 301.

[0049] According to one embodiment, the neuron 301 is a postsynaptic analog neuron and, for example, a spiking neuron. The neuron 301 is configured to receive a current Iout300 and to provide an output voltage Vout300.

[0050] According to one example, the neuron 301 is made up of metal-oxide-semiconductor field effect transistors or MOSFET transistors, or MOS transistors. More particularly, the neuron 301 is made up of N-channel MOS transistors or N-type MOS transistors or NMOS transistors, or P-channel MOS transistors or P-type MOS transistors or PMOS transistors.

[0051] According to an example, the neuron 301 comprises a first current mirror circuit containing two NMOS transistors N301 and N302. A drain terminal of the transistor N301 is connected, preferably connected, to a node providing a current Iout300 and is also connected to gate terminals of the transistors N301 and N302. Source terminals of the transistors N301 and N302 are both connected, preferably connected, to a node providing a reference voltage GND300, such as ground.

[0052] According to an example, the neuron 301 comprises a second current mirror circuit containing two PMOS transistors P301 and P302. A drain terminal of the transistor P301 is connected, preferably connected, to a source terminal of the transistor N302. Source terminals of the transistors P301 and P302 are both connected, preferably connected, to a node providing a supply voltage Vdd300. A drain terminal of the transistor P302 is connected, preferably connected, to gate terminals of transistors P301 and P302 and provides a current Iinput300 which is an image of the current Iout300.

[0053] According to an example, the neuron 301 comprises a PMOS transistor P303 and an NMOS transistor N303. A source terminal of the transistor P303 is connected, preferably connected, to the drain terminal of the transistor P302 and therefore receives the current Iinput300. Drain terminals of the transistors P303 and N303 are connected, preferably connected, to each other. The source terminal of the transistor N303 receives the reference voltage GND300.

[0054] According to one example, the neuron 301 comprises an NMOS transistor N304 and a capacitor Cmem300. A drain terminal of the transistor N304 is connected, preferably connected, to the drain terminals of the transistors N303 and P303 and a terminal The source terminal of transistor N304 receives the reference voltage GND300. The gate terminal of transistor N304 receives a control voltage Vleak300. The capacitor Cmem300 is connected between the drain terminals of transistors N303 and P303 and the node providing the reference voltage GND300. Transistor N304 is used as the leakage element of neuron 301. The capacitor Cmem300 is a storage capacitor, also called a membrane capacitor.

[0055] According to an example, the neuron 301 comprises two (2) PMOS transistors P304 and P305 and three (3) NMOS transistors N305, N306 and N307. A source terminal of the transistor P304 receives the supply voltage Vdd300. Drain terminals of the transistors P304 and N305 are connected, preferably connected, to each other. A source terminal of the transistor N305 receives the reference voltage GND300. Gate terminals of the transistors P304 and N305 are connected, preferably connected, to the drain terminals of the transistors P303 and N303. A source terminal of the transistor P305 receives the supply voltage Vdd300. Drain terminals of the transistors P305 and N306 are connected, preferably connected, to each other. A source terminal of transistor N306 is connected, preferably connected, to the drain terminal of transistor N307. Gate terminals of transistor P305 and N306 are connected, preferably connected, to the drain terminals of transistors P304 and N305.A source terminal of transistor N307 is connected, preferably connected, to the reference voltage GND300. A gate terminal of transistor N307 receives a control voltage Vrefra300. The control voltage Vrefra300 is used to control the refractory period of neuron 301, i.e. the duration during which neuron 301 becomes insensitive to any input current Iinput300. .

[0056] According to one example, the neuron 301 comprises a capacitor Crefra300 connected, preferably connected, between the drain terminals of the transistors P305 and N306.

[0057] According to an example, the neuron 301 comprises two (2) PMOS transistors P306 and P307 and two (2) NMOS transistors N308 and N309. Source terminals of the transistors P306 and P307 are connected, preferably connected, to each other. Drain terminals of the transistors P306 and N308 are connected, preferably connected, to each other. A source terminal of the transistor N308 receives the reference voltage GND300. Gate terminals of the transistors P306 and N308 are connected, preferably connected, to the drain terminals of the transistors P305 and N306. Drain terminals of the transistors P307 and N309 are connected, preferably connected, to each other. A source terminal of the transistor N309 receives the reference voltage GND300. Gate terminals of transistors P307 and N309 are connected, preferably connected, to the drain terminals of transistors P306 and N308.

[0058] According to one example, the neuron 301 comprises three (3) PMOS transistors P308, P309 and P310 and three (3) NMOS transistors N310, N311 and N312. Source terminals of the Transistors P308, P309 and P310 are connected, preferably connected, to each other. Drain terminals of transistors P308 and N310 are connected, preferably connected, to each other. A source terminal of transistor N310 receives the reference voltage GND300. Gate terminals of transistors P308 and N310 are connected, preferably connected, to the drain terminals of transistors P306 and N308. Drain terminals of transistors P309 and N311 are connected, preferably connected, to each other. A source terminal of transistor N311 receives the reference voltage GND300. Gate terminals of transistors P309 and N311 are connected, preferably connected, to the drain terminals of transistors P308 and N310. Drain terminals of transistors P310 and N312 are connected, preferably wired, to each other. A source terminal of transistor N312 receives the reference voltage GND300.Gate terminals of transistors P310 and N312 are connected, preferably connected, to each other and provide the output voltage Vout300 of neurons 301.

[0059] According to one embodiment, the layer 303 comprises several synapses 304 placed in a matrix or a column. Only a part of each synapse 304 is shown in [Fig.3], this part comprises a memory cell Mem301 (PCM) and a selection NMOS transistor N313 or selector N313. According to one embodiment, the memory cell Mem301 is a multi-level memory cell. According to one embodiment, the memory cell Mem301 is a phase modification memory cell and, for example, a multi-level phase modification memory cell. As previously indicated, the memory cell Mem301 is faced with a resistance drift phenomenon.A first conduction terminal of transistor N313 is connected, preferably connected, to the output of memory cell Mem301 and a second conduction terminal of transistor N313 provides a read current of synapses 304, also called inference current of synapses 304. A gate terminal of transistor N313 receives a selection voltage.

[0060] All synapses 304 of layer 303 have their output connected, preferably connected, to each other. This means that all second conduction terminals of transistors N313 are connected, preferably connected, to each other. Layer 303 provides a current Iinfer300 to current attenuator 302. Current Iinfer300 is the sum of all read currents of synapses 304.

[0061] According to one embodiment, the nonlinear current attenuator 302 comprises a first current mirror circuit containing two NMOS transistors N314 and N315. A drain terminal of the transistor N314 is connected, preferably connected, to a node providing the current Iinfer300 and is also connected to the gate terminals of the transistors N314 and N315. Source terminals of the transistors N314 and N315 are both connected, preferably connected, to a node providing a reference voltage GND300, such as ground.

[0062] According to one example, the attenuator 302 comprises a second current mirror circuit containing two PMOS transistors P311 and P312. A drain terminal of the transistor P311 is connected, preferably connected, to a source terminal of the transistor N315. Source terminals of the transistors P311 and P312 are both connected, preferably connected, to a node providing a supply voltage Vdd300. A drain terminal of the transistor P312 is connected, preferably connected, to gate terminals P311 and P312 and provides a current Iin300 which is an image of the current Iinfer300 which is also called the input current Iin300.

[0063] According to an example, the attenuator 302 comprises two NMOS transistors N316 and N317. A first conduction terminal of the transistor N316 is connected, preferably connected, to the drain terminal of the transistor P312. A second conduction terminal of the transistor N316 is connected, preferably connected, to a source terminal of the transistor N317. A gate terminal of the transistor N317 receives the supply voltage Vdd300. A drain terminal of the transistor N317 provides an intermediate voltage Vint300.

[0064] According to one example, the attenuator 302 comprises eight (8) NMOS transistors N318, N319, N320, N321, N322, N323, N324 and N325 and four (4) PMOS transistors P313, P314, P315 and P316.

[0065] Gate terminals of transistors N318, N319, N320 and N321 are all connected, preferably connected, to each other, to the gate terminal of transistor N317 and to the source terminal of transistor N317. Transistors N318, N319, N320 and N321 are all connected, preferably connected, in series. In other words, a first conduction terminal of transistor N318 is connected, preferably connected, to a first conduction terminal of transistor N319. A second conduction terminal of transistor N319 is connected, preferably connected, to a first conduction terminal of transistor N320. A second conduction terminal of transistor N320 is connected, preferably connected, to a first conduction terminal of transistor N321.

[0066] Gate terminals of transistors N322, N323, N324 and N325 are all connected, preferably connected, to each other. Transistors N322, N323, N324 and N325 are all connected, preferably connected, in series. In other words, a first conduction terminal of transistor N322 is connected, preferably connected, to a first conduction terminal of transistor N323. A second conduction terminal of transistor N323 is connected, preferably connected, to a first conduction terminal of transistor N324. A second conduction terminal of transistor N324 is connected, preferably connected, to a first conduction terminal of transistor N325.

[0067] A first conduction terminal of transistor P313 is connected, preferably connected, to a second conduction terminal of transistor N321. A second conduction terminal of transistor P313 is connected, preferably connected, to a first conduction terminal of transistor P314. A first conduction terminal of transistor P315 is connected, preferably connected, to a second conduction terminal of transistor N325. A second conduction terminal of transistor P315 is connected, preferably connected, to a first conduction terminal of transistor P316. Gate terminals of transistors P313 and P315 are connected, preferably connected, to each other and to the first conduction terminal of transistor P315. Gate terminals of transistors P314 and P316 are connected, preferably connected, to each other and to the first conduction terminal of transistor P316.

[0068] According to one example, the attenuator 302 comprises an NMOS transistor N326. A first conduction terminal of the transistor N326 is connected, preferably connected, to the drain terminal of the transistor P312. A second conduction terminal of the transistor N326 is connected, preferably connected, to the drain terminal of the transistor N317 and also provides the intermediate voltage Vint300.

[0069] According to an example, the attenuator 302 comprises two NMOS transistors N326' and N327 and a PMOS transistor P317. A drain terminal of the transistor N326' is connected, preferably connected, to the second conduction terminals of the transistors N318 and N322. A source terminal of the transistor N326' receives the reference voltage GND300. Gate terminals of the transistors N327 and N328 are connected, preferably connected, to each other, the drain terminal of the transistor N327 and the source terminal of the transistor P317. A source terminal of the transistor N327 receives the reference voltage GND300.

[0070] According to an example, the attenuator 302 comprises two (2) NMOS transistors N328 and N329 and a PMOS transistor P318. A source terminal of the transistor P318 is connected, preferably connected, to the source terminal of the transistor P317. A gate terminal of the transistor P318 is connected, preferably connected, to the gate terminal of the transistor P317. A drain terminal of the transistor P318 is connected, preferably connected, to the drain terminal of the transistor N329. A source terminal of the transistor N329 is connected, preferably connected, to the drain terminal of the transistor N328. A source terminal of the transistor N329 receives the reference voltage GND300. Gate terminals of the transistors N328 and N329 are connected, preferably connected, to each other and to the drain terminal of the transistor N328.

[0071] According to one example, the attenuator 302 comprises two (2) NMOS transistors N330 and N331 and two (2) PMOS transistors P319 and P320. A source terminal of the transistor P319 is connected, preferably connected, to the source terminal of transistor P317. A gate terminal of transistor P319 is connected, preferably connected, to the gate terminal of transistor P317. A source terminal of transistor P320 is connected, preferably connected, to the source terminal of transistor P317. A gate terminal of transistor P320 is connected, preferably connected, to the gate terminal of transistor P317. A drain terminal of transistor N330 is connected, preferably connected, to the drain terminal of transistor P319. A source terminal of transistor N330 receives the reference voltage GND300. Gate terminals of transistors N330 and N331 are connected, preferably connected, to each other, to the drain terminal of transistor N330 and to the source terminal of transistor P318.

[0072] According to an example, the attenuator 302 comprises two (2) NMOS transistors N332 and N332 and a PMOS transistor P321. A source terminal of the transistor P321 is connected, preferably connected, to the source terminal of the transistor P317. A gate terminal of the transistor P321 is connected, preferably connected, to the gate terminal of the transistor P317. A drain terminal of the transistor P321 is connected, preferably connected, to the drain terminal of the transistor N333 and to the gate terminal of the transistor N333. A source terminal of the transistor N333 is connected, preferably connected, to the drain terminal of the transistor N332. A source terminal of the transistor N333 receives the reference voltage GND300. Gate terminals of transistors N332 and N333 are connected, preferably connected, to each other and to the source terminal of transistor N331.

[0073] A self-biasing current reference circuit of attenuator 302 includes transistors N326 to N333 and P317 to P321 and generates a subthreshold current that is carried by current mirror N327 to N326 that acts to limit the Iout300 of the current attenuator. The current bias carried by N326 is the bias current.

[0074] As previously indicated, the attenuator 302 is a non-linear current attenuator configured to provide an output current Iout300 whose evolution is non-linear with respect to the evolution of the input current Iin300. In the example shown in [Fig. 3], the output current Iout300 depends on the input current Iin300 via a one-phase exponential decay function. More particularly, the output current Iout300 is given by the following mathematical formula: [Math 2] T — _________Zt'n300_________ lOUlJUV - SDFmux-SDFimH^fi^ ' in which: - SDFmax represents the maximum gain provided by the attenuator 302, i.e. the maximum current reduction factor (SDF); - SDFmin represents the minimum gain provided by the attenuator 302, i.e. the minimum current reduction factor (SDF); and - X is an experimentally defined coefficient.

[0075] Furthermore, the function of the current reduction factor SDF of the attenuator 302 is given by the following mathematical formula: [Math 1] SDF ( / m300 ) = SDFmax - SDFmin^*1"*™

[0076] The current roll-off factor (SDF) or attenuator gain, defined as the ratio of current IIin300 to current Iout300, is not linear with respect to Iin300 for some exemplary operating conditions or some exemplary operating mode. It is not linear because the output current Iout300 is initially constant regardless of the increase in Iin300. This mode is triggered whenever the voltage Vint300 is sufficient to saturate transistor N326 even during the start of operation, i.e., when current Iin300 is zero. Transistors N316 to N317 are connected to transistor N326 as source followers. Thus, a change in voltage Vint300 will not only cause a change in the gate voltage of transistors N317 and N316, but will also change the drain voltage of N326.Transistor N326 will be in saturation whenever the drain voltage is greater than substantially four times the thermal potential. The common mode rejection ratio (CMRR) of attenuator 302 is high when the bias current carried by transistor N326 is constant, i.e., transistor N326 is in saturation, so that the output current Iout300 depends only on the differential input change of the gate terminals of transistors N316 and N317, which depends on the current Iin300. As the current Iin300 increases, the differential input change increases. Finally, the current Iout300 is initially constant because during the start of operation, the current Iin300, i.e., the input current change, is not sufficient to control / change the current Iout300.

[0077] On the other hand, when Vint300 is not sufficient to saturate transistor N326 and its bias current is not constant, i.e. it is rather in the high transconductance triode region, so that the common mode rejection ratio is reduced and the current Iout300 depends on the common mode voltage of transistor N326. Thus, any increase in Vint300 will generate an increase in Iout300 and thus Iout300 is not initially constant or saturated. Thus, Iout300 is not constant and increases when Iin300 increases, even for low values ​​of the latter, i.e. the current reduction factor (SDF) of the attenuator is linear for this example of operating mode.

[0078] [Fig.4] represents a graph illustrating the current reduction factor (SDF) of the attenuator 302 described in relation to [Fig.3] with respect to the input current In300 (lin(A)).

[0079] The gain of the attenuator 302 is defined by dividing the input current In300 by the output current Iout300.

[0080] The graph of [Fig.4] represents a series 401 of points illustrating a first configuration of the attenuator 302 and a series 402 of points illustrating a second configuration of the attenuator 302. Both configurations of the attenuator 302 have the same non-linear configuration mode, with only a different level of tunability. The non-linear operation is achieved whenever a voltage Vbias300 is sufficient to saturate the transistor N326 from the very beginning of the operation of the attenuator (for low values ​​of Iin300). The common mode rejection ratio of the attenuator 302 is high enough to make the current Iout300 constant and not change whenever the common mode voltage of the gate voltage of the transistors N317 and N316, which increases with In300, is not high enough.Thus, for low values ​​of Iin300, Iin300 increases when Iout300 is constant, which makes the gain / current reduction factor non-linear - this condition is described by curves 401 and 402 for Vbias300 equal to 0.1 V and for Vbias300 equal to 0.2 V. To tune it, we change Vbias300. When we increase Vbias300, the drain voltage of transistor N326 increases and makes it saturate earlier whenever the Vbias300 voltage is equal to 0.22 V (curve 401) instead of 0.15 V (curve 402).

[0081] The graph in [Fig.4] directly shows that the attenuator 302 is non-linear.

[0082] [Fig.5] comprises a series of graphs 510 illustrating the result of using the attenuator 302 described in relation to [Fig.3].

[0083] The series of graphs 510 includes: - a graph 511 illustrating the normalization of reading currents of the synapses 304 described in relation to [Fig.3], after the use of a linear current attenuator at an initial instant; - a graph 512 illustrating the normalization of read currents of the synapses 304 after the use of a linear current attenuator at another instant which follows the initial instant; - a graph 513 illustrating the normalization of reading currents of the synapses 304 after the use of the non-linear current attenuator 302 having the first configuration, at said other instant which follows the initial instant; - a graph 514 illustrating the normalization of read currents of the synapses 304 after the use of the non-linear current attenuator 302 having the second configuration, at said other instant which follows the initial instant; and - a graph 515 illustrating the comparison of the resistance drift phenomenon between the case illustrated by graphs 511 to 514.

[0084] The difference between graphs 511 and 512 shows the resistance drift phenomenon described previously. Graphs 513 to 515 show the impact of using a non-linear current attenuator and, more particularly, the impact of using the non-linear current attenuator 302 of [Fig. 3]. A non-linear current attenuator has the effect of reducing the resistance drift phenomenon.

[0085] Various embodiments and variations have been described. Those skilled in the art will understand that certain features of these various embodiments and variations could be combined, and other variations will occur to those skilled in the art.

[0086] Finally, the practical implementation of the embodiments and variants described is within the reach of those skilled in the art from the functional indications given above.

Claims

Claims

1. A neural network circuit (200; 300) comprising at least one first neuron (204; 304) connected to a second neuron (201; 301) via at least one synapse (205) and a non-linear current attenuator (202; 302) configured to receive a first current (Iin200; Iin300) from said at least one synapse (205) and to supply a second current (Iout200; Iout300) to said second neuron (201; 301).

2. The circuit of claim 1, wherein said at least one synapse (205) comprises a multi-level memory cell.

3. The circuit of claim 2, wherein said at least one synapse (205) comprises a multi-level phase-shift memory cell.

4. Circuit according to any one of claims 1 to 3, wherein said first current (Iin200; Iin300) is a read current of said at least one first neuron (204; 304).

5. A circuit according to any one of claims 1 to 4, comprising at least two first neurons (204; 304) and at least two synapses (205), wherein said first current (Iin200; Iin300) is the sum of third currents provided by each of said at least two synapses (205).

6. A circuit according to any one of claims 1 to 5, wherein said at least one first neuron (204; 304) is a spiking neuron.

7. A circuit according to any one of claims 1 to 6, wherein said second neuron (201; 301) is a spiking neuron.

8. A circuit according to any one of claims 1 to 7, wherein said non-linear current attenuator (202; 302) is non-linear with respect to said first current (Iin200; Iin300).

9. A circuit according to claim 8, wherein said second current (Iout300) depends on the first current (Iin300) via an exponential function.

10. A circuit according to claim 9, wherein said second current (Iout300) is given by the following mathematical formula: [Math 4] Zo«ï300 - sDFmax-SDFmin*^^ ' in which: - Ilin300 represents said first current (Iin200; Iin300); - SDFmax represents the maximum gain provided by said attenuator (302); - SDFmin represents the minimum gain provided by said attenuator (302); and - 2 is an experimentally defined factor.

Images