Neuromorphic device and neuromorphic system

Abstract

A neuromorphic device (300) for updating a weight between a first neuron (301) and a second neuron (303) that are interconnected. The neuromorphic device (300) is provided with a memdevice (302) having an electric characteristic that indicates a weight, a first weight update circuit (304a) and a second weight update circuit (304b). The first weight update circuit (304a) and the second weight update circuit (304b) change the electric characteristic of the memdevice (302) by a back propagation method using at least first data that is the output data of the first neuron (301) and second data that is the output data of the second neuron (303).

Description

[Technical field]

[0001] The present disclosure relates to neuromorphic devices and systems. [Background technology]

[0002] Neuromorphic systems are known that use devices capable of analog calculations rather than digital calculations. Neuromorphic systems are systems that realize neural networks using hardware. Neuromorphic systems are sometimes called neuromorphic systems.

[0003] Patent Documents 1 and 2 describe electronic devices used in neural networks. The electronic devices of Patent Documents 1 and 2 are devices that realize neural networks in hardware. The electronic devices of Patent Documents 1 and 2 include a circuit that holds weights, which are the connection strengths between neurons. The circuit has a plurality of transistors and capacitive elements. The potential held in the capacitive elements is analog data. The electronic devices of Patent Documents 1 and 2 write weights to the capacitive elements of the circuit and read out weights from the capacitive elements of the circuit.

[0004] Non-Patent Document 1 describes a cellular neural network using an oxide semiconductor. The cellular neural network of Non-Patent Document 1 is a network in which a neural network is realized by hardware. The cellular neural network of Non-Patent Document 1 includes a memory element that holds a weight, which is the connection strength between neurons. The memory element is a memory element that uses an oxide semiconductor. The memory element holds analog data. The cellular neural network of Non-Patent Document 1 writes a weight to the memory element and reads the weight from the memory element. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-80162 [Patent Document 2] Japanese Patent Publication No. 2020-109715 [Non-patent literature]

[0006] [Non-Patent Document 1] MUTSUMI KIMURA, YUKI KOGA, HIROKI NAKANISHI, TOKIYOSHI MATSUDA, TOMOYA KAMEDA, AND YASUHIKO NAKASHIMA, "In-Ga-Zn-O Thin-Film Devices As Synapse Elements in a Neural Network", 27 November 2017, IEEE Journal of the Electron Devices Society(Volume:6), Page(s):100-105 Summary of the Invention [Problem to be solved by the invention]

[0007] However, neither Patent Document 1, Patent Document 2, nor Non-Patent Document 1 discloses or suggests a configuration for learning the above-mentioned weights by backpropagation.

[0008] One aspect of the present disclosure aims to provide a neuromorphic device and a neuromorphic system having a novel configuration. [Means for solving the problem]

[0009] In order to solve the above problems, a neuromorphic device according to one embodiment of the present disclosure is a neuromorphic device that updates a weight, which is a connection strength between a first neuron and a second neuron that are interconnected. The neuromorphic device includes a mem device and a weight update circuit. The mem device has a variable electrical characteristic that indicates the weight. The weight update circuit includes a semiconductor element and changes the electrical characteristic to update the weight. The weight update circuit changes the electrical characteristic by a backpropagation method by driving the semiconductor element using at least first data and second data. The first data is output data of the first neuron. The second data is output data of the second neuron. Effect of the Invention

[0010] According to one aspect of the present disclosure, a neuromorphic device and a neuromorphic system having a novel configuration can be provided. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram for explaining an error backpropagation method. [Diagram 2] FIG. 11 is another explanatory diagram for explaining the error backpropagation method. [Diagram 3] FIG. 1 is a schematic diagram of a neuromorphic device according to an embodiment of the present disclosure. [Figure 4] A diagram showing the electrical characteristics of a mem device included in the above-mentioned neuromorphic device and a schematic configuration of the mem device. [Diagram 5] A figure showing the electrical characteristics of another mem device included in the above-mentioned neuromorphic device and a schematic configuration of the mem device. [Figure 6] FIG. 2 is an explanatory diagram for explaining the operation of the neuromorphic device. [Figure 7] FIG. 2 is a diagram illustrating an example of a hardware configuration of the neuromorphic device. [Figure 8]10 is an explanatory diagram for explaining the effect of the above-mentioned neuromorphic device. FIG. [Figure 9] 10 is an explanatory diagram for explaining the effect of the above-mentioned neuromorphic device. FIG. [Figure 10] 10 is an explanatory diagram for explaining the effect of the above-mentioned neuromorphic device. FIG. [Figure 11] FIG. 2 is a diagram illustrating another example of the hardware configuration of the neuromorphic device. [Figure 12] FIG. 13 is a diagram showing an application example of the above-mentioned neuromorphic device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <Background to this disclosure> Artificial intelligence is a technology that will be central to the society of the future. Neural networks are a representative technology that will realize artificial intelligence. Conventional neural networks are complex and large-scale software that runs on high-performance hardware. For this reason, conventional neural networks have issues such as the large size of the hardware and the increased power consumption that accompanies software execution.

[0013] In contrast, a neuromorphic system is a system that realizes a neural network using hardware. A neuromorphic system is a system that mimics the neural circuits of the living brain at the hardware level. Neuromorphic systems have the advantages of reduced hardware size and low power consumption.

[0014] However, neuromorphic systems have the following problems. Backpropagation, a neural network learning algorithm, is easy to implement in neuromorphic systems if it is software. On the other hand, implementing backpropagation in hardware in neuromorphic systems has not yet been realized. To realize the above advantages of neuromorphic systems, there are high hopes for implementing backpropagation in hardware in neuromorphic systems.

[0015] Therefore, the present inventors conducted extensive research to make it possible to implement the backpropagation algorithm in a neuromorphic system, and as a result, have come up with the present disclosure, which makes it easy to implement the backpropagation algorithm in hardware.

[0016] <Embodiment> Hereinafter, one embodiment of the present disclosure will be described in detail.

[0017] (Backpropagation) First, a brief explanation will be given of the backpropagation method. The backpropagation method is a known learning method used in multilayer neural networks. Therefore, only matters considered necessary for easy understanding of one embodiment of the present disclosure will be explained below.

[0018] FIG. 1 is an explanatory diagram for explaining the error backpropagation method. As shown in FIG. 1, a multi-layer neural network 100 is composed of an input layer 101, an intermediate layer 102, and an output layer 103. Each layer includes multiple neurons. The open circles in FIG. 1 each represent a neuron included in each layer. One open circle corresponds to one neuron.

[0019] The input layer 101 includes five neurons. The hidden layer 102 includes four neurons. The output layer 103 includes three neurons. The hidden layer 102 is one layer. However, the configuration of the multi-layer neural network 100 in FIG. 1 is merely an example. The present disclosure is not limited to the number of hidden layers 102 in FIG. 1 and the number of neurons included in each layer.

[0020] Each neuron included in the input layer 101 is connected to all neurons included in the hidden layer 102. Each neuron included in the hidden layer 102 is connected to all neurons included in the output layer 103.

[0021] The input data group 111 includes five pieces of input data input to each neuron included in the input layer 101. The output data group 112 includes three pieces of output data output from each neuron included in the output layer 103. The teacher data group 113 includes three pieces of teacher data corresponding to each neuron included in the output layer 103. Each piece of teacher data is a target value of the output data of each neuron included in the output layer 103.

[0022] The backpropagation method is a method for updating weights by comparing each output data included in the output data group 112 with each teacher data included in the teacher data group 113 corresponding to each output data (step indicated by reference numeral 121) and correcting weights, which are connection strengths between neurons included in different layers, based on the comparison results (step indicated by reference numeral 122). In this specification, the process of updating weights using the backpropagation method is referred to as "learning". Also, in this specification, the process of outputting the output data group 112 based on the input data group 111 and the updated weights in the multilayer neural network 100 that has performed the above-mentioned "learning" is referred to as "inference".

[0023] Equation (1) is solved by using the backpropagation algorithm to find the weights W ij The weight W when updating ij The correction amount ΔW ij Here, the weight W ij is the weight between each neuron included in the input layer 101 and each neuron included in the hidden layer 102 in FIG. 1. ij The subscripts i and j in ij This means that the weight between the i-th neuron included in the input layer 101 and the j-th neuron included in the hidden layer 102 is given.

number

[0024] In addition, the output data Z k The subscript k in the output data Z k is the output data output from the k-th neuron included in the output layer 103. k The subscript k of the bar is the training data Z k The bar indicates the target value of the output data of the k-th neuron included in the output layer 103. jk The subscripts j and k of jk is the weight between the j-th neuron included in the hidden layer 102 and the k-th neuron included in the output layer 103. j The subscript j in j is the output data output from the j-th neuron included in the hidden layer 102. i The subscript i in the i is the output data output from the i-th neuron included in the input layer 101.

[0025] Figure 2 is another explanatory diagram for explaining the error backpropagation method. As shown in Figure 2, the multi-layer neural network 200 is composed of a first layer 201, ···, an (m - 1)-th layer 202, an m-th layer 203, an (m + 1)-th layer 204, ···, and an n-th layer 205. The first layer 201 is an input layer. The n-th layer 205 is an output layer. The (m - 1)-th layer 202, the m-th layer 203, and the (m + 1)-th layer 204 are all intermediate layers. Note that both m and n are natural numbers, and m < n. Each layer constituting the multi-layer neural network 200 includes a plurality of neurons. Similar to Figure 1, the white circles in Figure 2 each represent a neuron included in each layer. Also, similar to Figure 1, one white circle corresponds to one neuron.

[0026] However, the configuration of the multi-layer neural network 200 in Figure 2 is merely an example. The present disclosure is not limited to the number of intermediate layers in Figure 2 and the number of neurons included in each layer.

[0027] Each neuron included in the first layer 201, which is the input layer, is connected to all the neurons included in a second layer (not shown). Each neuron included in the (m - 1)-th layer 202 is connected to all the neurons included in the m-th layer 203. Each neuron included in the m-th layer 203 is connected to all the neurons included in the (m + 1)-th layer 204. Each neuron included in an (n - 1)-th layer (not shown) is connected to all the neurons included in the n-th layer 205.

[0028] Equation (2) is an equation representing the input data u input to the j-th neuron included in the m-th layer 203. j m

Number

[0029] Equation (3) expresses the output data y j m This is an equation representing:

[0030]

number

number

[0031] Equation (5) expresses the output data y k m+1 This is an equation representing:

[0032]

number

[0033]

number

[0034]

number

number

[0035] Equation (9a) represents the case where m=n, that is, the error data for the j-th neuron included in the n-th layer 205, which is the output layer. On the other hand, equation (9b) represents the case where m≠n, that is, the error data for the j-th neuron included in each layer other than the output layer.

[0036]

number

[0037] (Neuromorphic Devices) 1. Configuration Fig. 3 is a schematic diagram of a neuromorphic device 300 according to an embodiment of the present disclosure. As shown in Fig. 3, the neuromorphic device 300 includes a mem device 302 having a variable electrical characteristic indicating a weight, which is a connection strength between a first neuron 301 and a second neuron 303 that are interconnected, and a first weight update circuit 304a and a second weight update circuit 304b that change the electrical characteristic of the mem device 302 to update the weight.

[0038] In the example of FIG. 3, the first neuron 301 is the i-th neuron included in the (m-1)-th layer 202 of FIG. 2. y i m-1 is the output data (first data) output from the first neuron 301. The second neuron 303 is the j-th neuron included in the m-th layer 203 in FIG. 2. y j m is the output data (second data) output from the second neuron 303.

[0039] Fig. 4 is a diagram showing electrical characteristics and a schematic configuration of a memristor, which is an example of the memristor device 302. The memristor has electrical characteristics indicated by reference numeral 401 in Fig. 4. The memristor has a schematic configuration indicated by reference numeral 402 in Fig. 4.

[0040] A memristor is a passive element that has electrical characteristics that allow it to memorize the electric charge that passes through it and change its resistance value, as shown by the hysteresis characteristic indicated by reference numeral 401 in FIG. 4. A memristor is a memory element. As a memory element, a memristor holds analog data. A memristor can be used as a non-volatile memory.

[0041] As shown in the schematic configuration of FIG. 4 and indicated by reference numeral 402, the memristor includes an Al layer 42 disposed on a base 41, a Ga-Sn-O (GTO) layer 43 disposed on the Al layer 42, and an Au layer 44 disposed on the GTO layer 43. The Al layer 42 is the negative electrode, and the Au layer 44 is the positive electrode. When a voltage V is applied between the Al layer 42 and the Au layer 44, a current I flows through the GTO layer 43. That is, charges pass through the GTO layer 43. The passage of the charges changes the resistance value of the GTO layer 43. Note that the memristor is a known device, and so a detailed description is omitted.

[0042] When the memdevice 302 is a memristor, the weight between the first neuron 301 and the second neuron 303 is represented using a variable resistance value as described above.

[0043] Fig. 5 is a diagram showing electrical characteristics and a schematic configuration of a memcapacitor, which is an example of the memdevice 302. The memcapacitor has electrical characteristics shown by reference numeral 501 in Fig. 5. The memcapacitor has a schematic configuration shown by reference numeral 502 in Fig. 5.

[0044] A memcapacitor is a passive element that has electrical characteristics that not only store electric charge in a capacitor but also change the capacitance of the capacitor depending on the history of the applied voltage, as shown by the hysteresis characteristic indicated by reference numeral 501 in Fig. 5. A memcapacitor is a memory element. As a memory element, a memcapacitor holds analog data. A memcapacitor can be used as a non-volatile memory.

[0045] As shown in the schematic configuration of memcapacitor 502 in Fig. 5, memcapacitor includes a Pt layer 52 disposed on a base 51, a bismuth lanthanum titanate (BLT) layer 53 disposed on the Pt layer 52, and an Au layer 54 disposed on the BLT layer 53. The Pt layer 52 is a negative electrode, and the Au layer 54 is a positive electrode. When a voltage V is applied between the Pt layer 52 and the Au layer 54, an electric charge Q is accumulated in the BLT layer 53. The application of this voltage V changes the capacitance value of the capacitor consisting of the Pt layer 52, the BLT layer 53, and the Au layer 54. Note that the memcapacitor is a known device, and a detailed description thereof will be omitted.

[0046] When the memo device 302 is a memocapacitor, the weight between the first neuron 301 and the second neuron 303 is expressed using the variable capacitance value described above.

[0047] The first weight update circuit 304a is disposed between the output side of the first neuron 301 and the input side of the mem device 302. The input side of the first weight update circuit 304a is connected to the output side of the first neuron 301. The output side of the first weight update circuit 304a is connected to the input side of the mem device 302.

[0048] The second weight update circuit 304b is disposed between the output side of the mem device 302 and the input side of the second neuron 303. The input side of the second weight update circuit 304b is connected to the output side of the mem device 302. The output side of the second weight update circuit 304b is connected to the input side of the second neuron 303.

[0049] However, when performing "learning" for updating weights expressed using the electrical characteristics of the mem device 302, the first weight updating circuit 304a is placed between the output side of the first neuron 301 and the input side of the mem device 302, as shown in Fig. 3. The input side of the first weight updating circuit 304a is connected to the output side of the first neuron 301, and the output side of the first weight updating circuit 304a is connected to the input side of the mem device 302.

[0050] 3, when performing the above-mentioned "learning", the second weight updating circuit 304b is placed between the output side of the mem device 302 and the input side of the second neuron 303. The input side of the second weight updating circuit 304b is connected to the output side of the mem device 302, and the output side of the second weight updating circuit 304b is connected to the input side of the second neuron 303.

[0051] As shown in FIG. 3, the weight W between the first neuron 301 and the second neuron 303 is ij m-1,m The correction amount ΔW ij m-1,m is expressed by equation (10). Note that equation (10) is the same as equation (8).

[0052]

number

[0053] 2. Hardware configuration example Hereinafter, the operation of the neuromorphic device will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is an explanatory diagram for explaining the operation of the neuromorphic device. Fig. 7 is a diagram showing an example of the hardware configuration of the neuromorphic device.

[0054] As shown in Figure 6, a neuromorphic device 600, which is an example of the neuromorphic device 300 of Figure 3, includes a mem device 602, a first weight update circuit 604a, and a second weight update circuit 604b. The mem device 602 corresponds to the mem device 302 of Figure 3. The first weight update circuit 604a corresponds to the first weight update circuit 304a of Figure 3. The second weight update circuit 604b corresponds to the second weight update circuit 304b of Figure 3.

[0055] As shown in FIG. 6, the neuromorphic device 600 calculates a weight W ij The n-th layer 612 is the output layer. The (n-1)-th layer 611 is the layer immediately before the n-th layer 612.

[0056] In addition, instead of the multi-layer neural network 610, the neuromorphic device 600 uses a weight W between the i-th neuron included in the n-1-th layer 611 and the j-th neuron included in the n-th layer 612 in a simple perceptron in which the n-1-th layer 611 is an input layer and the n-th layer 612 is an output layer. ij You may update the following.

[0057] As shown in Figure 6, by using equation (9a), the weight W ij The correction amount ΔW ij is expressed by equation (11). ΔW ij =-η(y j -y j bar)y j (1-y j )x i (11) where y j is the output data output from the j-th neuron included in the n-th layer 612. j The bar indicates the target value of the output data of the j-th neuron included in the n-th layer 612. y j The bar is included in the training data set 613. i is the output data output from the i-th neuron included in the (n−1)-th layer 611.

[0058] As shown in FIG. 6, the first weight update circuit 604a and the second weight update circuit 604b update the sub-equation “(y j -y j bar)y j (1-y j )xi " is a circuit that represents the above in hardware.

[0059] As shown in Fig. 7, the first weight updating circuit 604a in Fig. 6 is composed of an n-type MOS transistor (first n-type MOS transistor) 604a-1. The second weight updating circuit 604b in Fig. 6 is composed of an n-type MOS transistor (second n-type MOS transistor) 604b-1, a p-type MOS transistor 604b-2, and a CMOS transistor 604b-3.

[0060] As shown in FIG. 7, the n-type MOS transistor 604a-1 is connected between Vdd / 2, which is half the power supply voltage Vdd, and the input side of the MEMS device 602. The n-type MOS transistor 604a-1 outputs output data x i The n-type MOS transistor 604b-1 is connected between the output side of the memdevice 602 and the p-type MOS transistor 604b-2, and has a gate terminal to which the output data y j The p-type MOS transistor 604b-2 has a gate terminal to which the output data y denoted by reference numeral 603 is input. j The CMOS transistor 604b-3 has a gate terminal to which the teacher data y denoted by reference numeral 605 is input via a capacitor 606. j It has an input terminal to which bar is input, and an output terminal connected to a p-type MOS transistor 604b-2.

[0061] As shown in FIG. 7, in the sub-formula indicated by the symbol B in the formula (11) in FIG. 6, the sub-formula indicated by the symbol B-1 “(y j -y j The sub-expression "y j " is expressed by an n-type MOS transistor 604b-1. j )" is represented by a p-type MOS transistor 604b-2. i" is represented by an n-type MOS transistor 604a-1.

[0062] The reason why each of the above sub-formulas is expressed in terms of each of the above semiconductor devices will now be described in more detail.

[0063] The sub-formula "(y j -y j bar)": The sub-formula "(y j -y j The positive or negative value of "y" indicates the output data j and the training data y j It is determined by the magnitude relationship with the bar. Also, the sub-formula "(y j -y j The absolute value of "y j bar and output data y j is the absolute value of the difference between

[0064] In response to this, the input terminal of the CMOS transistor 604b-3 is connected to the capacitor 606 via "(y j -y j The voltage corresponding to "(y j -y j If the voltage corresponding to "(y bar)" is higher than the threshold voltage of the CMOS transistor 604b-3, the CMOS transistor 604b-3 outputs the ground voltage GND to the output terminal. j -y j When the voltage corresponding to "Vdd" is lower than the threshold voltage of the CMOS transistor 604b-3, the CMOS transistor 604b-3 outputs the power supply voltage Vdd to the output terminal. That is, the voltage output to the output terminal of the CMOS transistor 604b-3 switches between the ground voltage GND and the power supply voltage Vdd depending on whether the voltage applied to the input terminal is positive or negative.

[0065] This switching changes the magnitude relationship between Vdd / 2 on the input terminal side of MEM device 602 and the voltage on the output terminal side of MEM device 602. That is, if the voltage output to the output terminal is the ground voltage GND, the voltage on the output terminal side will be smaller than the voltage on the input terminal side of MEM device 602. On the other hand, if the voltage output to the output terminal is the power supply voltage Vdd, the voltage on the output terminal side will be greater than the voltage on the input terminal side of MEM device 602.

[0066] The direction of the current flowing through the memdevice 602 changes according to the change in the magnitude relationship. The resistance value of the memdevice 602 changes according to the direction of the current, based on the electrical characteristics of the memdevice 602. This current direction is expressed by the subexpression "(y j -y j The resistance is expressed as a positive or negative value of the subexpression "(y j -y j The time is expressed as an absolute value of "(bar)".

[0067] The sub-formula "y j ": The conductance of n-type MOS transistor 604b-1 increases as the voltage applied to the gate terminal increases. This increase in conductance increases the amount of current flowing through memdevice 602. Due to the electrical characteristics of memdevice 602, the resistance value of memdevice 602 changes depending on the amount of current. This resistance value is expressed by the subexpression "y j " is expressed as follows.

[0068] The sub-formula "(1-y j )": The conductance of p-type MOS transistor 604b-2 increases as the voltage applied to the gate terminal decreases. This increase in conductance increases the amount of current flowing through mem device 602. Due to the electrical characteristics of mem device 602, the resistance value of mem device 602 changes depending on the amount of current. This resistance value is expressed by the subexpression "(1-y j )".

[0069] The sub-formula "x" shown in symbol B-4 i ": The conductance of n-type MOS transistor 604a-1 increases as the voltage applied to the gate terminal increases. This increase in conductance increases the amount of current flowing through memdevice 602. Due to the electrical characteristics of memdevice 602, the resistance value of memdevice 602 changes depending on the amount of current. This resistance value is expressed by the subexpression "x i " is expressed as follows.

[0070] 3.Effects The effect of the neuromorphic device 600 in FIG. 7 will be described below with reference to FIG. 8, FIG. 9, and FIG. 10. FIG. 8 shows the preconditions established to simplify the formula (11) in FIG. 7 when verifying the effect of the neuromorphic device 600. As shown in FIG. 8, when the output data x i is data that represents "0" or "1". In addition, the training data y j The bar represents data that represents "0" or "1". As shown in Figure 8, the output data x i If is "0", the training data y j Whether the bar is "0" or "1", the correction amount Δw ij will result in "0".

[0071] In contrast, the output data x i If is "1", the training data y j If the bar is "0", the correction amount Δw from equation (11) ij HA-ηy j y j (1-y j ) On the other hand, the training data y j If the bar is "1", the correction amount Δw from equation (11) ij is -η(y j -1)y j (1-y j )

[0072] Figure 9 shows the output data y j and the correction amount Δw of equation (11) ij10 shows the results of a circuit simulation using HSPICE.

[0073] The graph in Figure 10 is the same as the graph in Figure 9, except for "x i =1 / y j bar=0", "x i =0 / y j bar=0", "x i =1 / y j bar=1" and "x i =0 / y j It can be seen that the results in Fig. 10 are almost identical to those in Fig. 9.

[0074] 4. Variations Fig. 11 shows a modified example of the neuromorphic device 600. The neuromorphic device 700 of Fig. 11 differs from the neuromorphic device 600 of Fig. 7 in that the neuromorphic device 700 of Fig. 11 further includes a switching element 1101, a switching element 1102, a switching element 1103, and a switching element 1104.

[0075] As shown in FIG. 11, when neuromorphic device 700 performs the above-mentioned "learning," switching element 1101 connects terminal 1101a to terminal 1101b, switching element 1102 connects terminal 1102a to terminal 1102b, switching element 1103 connects terminal 1103a to terminal 1103b, and switching element 1104 connects terminal 1104a to terminal 1104b.

[0076] On the other hand, when neuromorphic device 700 performs the above-mentioned "inference," switching element 1101 connects terminal 1101a to terminal 1101c, switching element 1102 connects terminal 1102a to terminal 1102c, switching element 1103 connects terminal 1103a to terminal 1103c, and switching element 1104 connects terminal 1104a to terminal 1104c.

[0077] When the neuromorphic device 700 performs the above-mentioned "learning" by switching the switching elements 1101 to 1104, the n-type MOS transistor 604a-1, the n-type MOS transistor 604b-1, the p-type MOS transistor 604b-2, and the CMOS transistor 604b-3 are driven. That is, the neuromorphic device 700 operates the first weight update circuit 604a and the second weight update circuit 604b in FIG. 6. On the other hand, when the neuromorphic device 700 performs the above-mentioned "inference", the n-type MOS transistor 604a-1, the n-type MOS transistor 604b-1, the p-type MOS transistor 604b-2, and the CMOS transistor 604b-3 are not driven. That is, the neuromorphic device 700 does not operate the first weight update circuit 604a and the second weight update circuit 604b in FIG. 6.

[0078] <Other embodiments> The present disclosure can be used in all neural networks and neuromorphic systems. For example, in a neural network that combines multiple convolutional layers and several final fully connected layers as shown in FIG. 12, the above "learning" is performed in the fully connected layers. The present disclosure can be used in the fully connected layers.

[0079] 7, the n-type MOS transistor 604a-1 is arranged on the input side of the mem device 602, and the n-type MOS transistor 604b-1, the p-type MOS transistor 604b-2, and the CMOS transistor 604b-3 are arranged on the output side of the mem device 602. However, the present disclosure is not limited to this arrangement. All of the n-type MOS transistor 604a-1, the n-type MOS transistor 604b-1, the p-type MOS transistor 604b-2, and the CMOS transistor 604b-3 may be arranged on the input side of the mem device 602, or conversely, all of them may be arranged on the output side of the mem device 602.

[0080] Furthermore, by arranging at least one of the n-type MOS transistor 604a-1, the n-type MOS transistor 604b-1, the p-type MOS transistor 604b-2, and the CMOS transistor 604b-3 rather than arranging all of them, the backpropagation algorithm can be easily implemented in hardware.

[0081] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present disclosure.

[0082] <Summary> A neuromorphic device according to aspect 1 of the present disclosure is a neuromorphic device that updates a weight, which is a connection strength between an interconnected first neuron and a second neuron, and includes a mem device having a variable electrical characteristic that indicates the weight, and a weight update circuit that includes a semiconductor element and changes the electrical characteristic to update the weight, and the weight update circuit changes the electrical characteristic by a backpropagation method by driving the semiconductor element using at least first data, which is output data of the first neuron, and second data, which is output data of the second neuron.

[0083] A neuromorphic device according to aspect 2 of the present disclosure is the same as in aspect 1 above, in which the weight update circuit has a first weight update circuit and a second weight update circuit, the first weight update circuit drives the semiconductor element using at least the first data, and the second weight update circuit drives the semiconductor element using at least the second data.

[0084] A neuromorphic device according to aspect 3 of the present disclosure is the same as aspect 1 or 2 above, in which the first neuron and the second neuron are respectively included in different layers constituting a neural network, and the weight update circuit changes the electrical characteristics by backpropagation by driving the semiconductor element using the first data, the second data, and teacher data of the neural network.

[0085] A neuromorphic device according to aspect 4 of the present disclosure is, in any one of aspects 1 to 3 above, wherein the second neuron is included in an output layer of a multilayer neural network, and the first neuron is included in a layer of the multilayer neural network one layer before the output layer, and the weight update circuit changes the electrical characteristics by a backpropagation method by driving the semiconductor element using teacher data corresponding to the first data, the second data, and the output data of the second neuron.

[0086] A neuromorphic device according to aspect 5 of the present disclosure is, in any one of aspects 1 to 3 above, such that the second neuron is included in an output layer of a simple perceptron and the first neuron is included in an input layer of the simple perceptron, and the weight update circuit changes the electrical characteristics by a backpropagation method by driving the semiconductor element using teacher data corresponding to the first data, the second data, and the output data of the second neuron.

[0087] A neuromorphic device according to aspect 6 of the present disclosure is, in any of aspects 3 to 5 above, wherein the mem device has an input terminal which is a terminal on the first neuron side and an output terminal which is a terminal on the second neuron side, and there are one or more semiconductor elements, and each of the semiconductor elements is either a first n-type MOS transistor having a gate terminal to which the first data is input, a second n-type MOS transistor having a gate terminal to which the second data is input, a p-type MOS transistor having a gate terminal to which the second data is input, or a CMOS transistor having an input terminal to which the teacher data is input, and each of the semiconductor elements is arranged on either the input terminal side or the output terminal side of the mem device.

[0088] A neuromorphic device according to aspect 7 of the present disclosure is the same as in aspect 6 above, in which the weight update circuit has a first weight update circuit and a second weight update circuit, the first weight update circuit including at least the first n-type MOS transistor, and the second weight update circuit including at least any of the second n-type MOS transistor, the p-type MOS transistor, and the CMOS transistor.

[0089] A neuromorphic device according to an eighth aspect of the present disclosure is the neuromorphic device of any one of the first to seventh aspects, wherein the memdevice is any one of a memristor, a memcapacitor, or a meminductor.

[0090] A neuromorphic system according to aspect 9 of the present disclosure comprises an interconnected first neuron and a second neuron, and a neuromorphic device according to any of aspects 1 to 8 above, which updates a weight which is the connection strength between the first neuron and the second neuron.

[0091] A neuromorphic system according to aspect 10 of the present disclosure is, in aspect 9 above, such that the neuromorphic system activates the weight update circuit when updating the weights, but does not activate the weight update circuit when using the weights to transmit information between the first neuron and the second neuron. [Explanation of symbols]

[0092] 100, 200, 610 Multilayer Neural Networks 101 Input Layer 102 Middle Class 103 Output Layer 300, 600, 700 Neuromorphic Devices 301 First Neuron 302, 602 Memdevice 303 Second Neuron 304a, 604a First weight update circuit (weight update circuit) 304b, 604b Second weight update circuit (weight update circuit) 604a-1 n-type MOS transistor (first n-type MOS transistor) 604b-1 n-type MOS transistor (second n-type MOS transistor) 604b-2 p-type MOS transistor 604b-3 CMOS transistor

Claims

1. A neuromorphic device for updating weights, which are connection strengths between interconnected first and second neurons, comprising: a memdevice having a variable electrical characteristic indicative of said weight; a weight update circuit including a semiconductor device and configured to change the electrical characteristics to update the weights; Equipped with the weight update circuit drives the semiconductor element using at least first data, which is output data of the first neuron, and second data, which is output data of the second neuron, thereby changing the electrical characteristic by a backpropagation method; the memdevice has an input terminal which is a terminal on the first neuron side and an output terminal which is a terminal on the second neuron side, The semiconductor element may be one or more, and each of the semiconductor elements may include: A first n-type MOS transistor; A second n-type MOS transistor; A p-type MOS transistor; CMOS transistors and is one of A neuromorphic device, wherein each of the semiconductor elements is arranged on either the input terminal side or the output terminal side of the memdevice.

2. the weight update circuit includes a first weight update circuit and a second weight update circuit; the first weight update circuit drives the semiconductor device using at least the first data; The neuromorphic device according to claim 1 , wherein the second weight update circuit drives the semiconductor element using at least the second data.

3. the first neuron and the second neuron are included in different layers constituting a neural network, The neuromorphic device according to claim 1 or 2, wherein the weight update circuit changes the electrical characteristics by a backpropagation algorithm by driving the semiconductor element using the first data, the second data, and training data of the neural network.

4. the second neuron is included in an output layer of a multi-layer neural network; the first neuron is included in a layer of the multilayer neural network that is one layer before the output layer; The neuromorphic device according to any one of claims 1 to 3, wherein the weight update circuit changes the electrical characteristics by a backpropagation method by driving the semiconductor element using teacher data corresponding to the first data, the second data, and the output data of the second neuron.

5. The second neuron is included in an output layer of a simple perceptron; the first neuron is included in an input layer of the simple perceptron; The neuromorphic device according to any one of claims 1 to 3, wherein the weight update circuit changes the electrical characteristics by a backpropagation method by driving the semiconductor element using teacher data corresponding to the first data, the second data, and the output data of the second neuron.

6. The first data is input to a gate terminal of the first n-type MOS transistor, The second data is input to a gate terminal of the second n-type MOS transistor, The second data is input to a gate terminal of the p-type MOS transistor, The neuromorphic device according to any one of claims 3 to 5, wherein the teacher data is input to an input terminal of the CMOS transistor.

7. the weight update circuit includes a first weight update circuit and a second weight update circuit; the first weight update circuit includes at least the first n-type MOS transistor; The neuromorphic device according to claim 6 , wherein the second weight update circuit includes at least any one of the second n-type MOS transistor, the p-type MOS transistor, and the CMOS transistor.

8. The neuromorphic device according to any one of claims 1 to 7, wherein the memdevice is one of a memristor, a memcapacitor, or a meminductor.

9. a first neuron and a second neuron which are interconnected; The neuromorphic device according to any one of claims 1 to 8, which updates a weight, which is a connection strength between the first neuron and the second neuron; A neuromorphic system comprising:

10. The neuromorphic system comprises: activating the weight update circuit when updating the weights; The neuromorphic system of claim 9 , wherein the weight update circuit is not operated when the weight is used to transmit information between the first neuron and the second neuron.

Images