Silicon brain

Abstract

The modern era is an era of artificial intelligence (AI). Semiconductor technology that supports artificial intelligence today has special specifications and high performance, but consumes too much electrical power. As utilization of artificial intelligence continues to expand, not only would production of electric power eventually become insufficient, but artificial intelligence utilization itself may cause climate change. In view of such a situation, the present invention proposes artificial-intelligence-dedicated semiconductor technology with which power consumption can be greatly reduced. [Problem] Conventional semiconductor chips for artificial intelligence are processors that are suitable for processing many multiply-add operations in parallel. The power consumption of high-speed and large-scale-parallelized processors only increases. [Solution] In the present application, such multiply-add operations are performed by a dedicated built-in circuit instead of a processor, making it possible to suppress large-scale power consumption.

Description

Silicon Brain

[0001] The present invention relates to a technology for integrating a neural network on a silicon chip (IC chip).

[0002] Conventional semiconductor computing systems utilize a memory device (memory) and a processor (a processor such as a CPU) working together. A memory device (semiconductor memory) consists of a collection (array, cell array, memory cell array, or memory element array) of memory elements called memory cells (memory elements, bit cells, or simply cells or elements). Each element consists of at least a source, drain, and gate (or control gate). The source and drain can each be connected to a bit line. The gate is connected to a word line. This connection is typically made through a contact (terminal), such as a word line contact (terminal) or a bit line contact (terminal). When such a collection of elements is distributed on a two-dimensional plane, each memory element is accessed via word lines (WL) and bit lines (BL) arranged perpendicularly in the X and Y directions of the two-dimensional plane. For example, the address of a memory element located at the intersection of the Ath word line and the Bth bit line is (A, B). This is called the address of the memory element. Here, A is specifically referred to as the address on the X axis (X address). B is specifically called the address on the Y axis (Y address).

[0003] For a long time, the mainstream of semiconductor memory technology development was to integrate as many memory elements as possible on the surface of a silicon wafer using semiconductor manufacturing processes in accordance with Moore's Law (see Non-Patent Document 1). However, in recent years (since 2015), it has become difficult to increase the integration density of memory elements on a two-dimensional plane, and the method of placing memory elements in three-dimensional space has become mainstream, even at the mass-produced level. In this case, addresses can be represented as (A, B, C), where C is the address on the Z axis (Z address), which is perpendicular to the XY plane.

[0004] However, whether two-dimensional or three-dimensional, the current method of recording information in semiconductor memory devices is based on memory elements, and when each memory element (cell) has two values, 0 and 1, it is said that one cell has a memory capacity (amount of information that can be stored) of 1 bit. If there are two such memory elements, the memory capacity is said to be 2 bits. In this case, there are four possible combinations of 0 and 1: (00), (01), (10), and (11). In this case, the number can be calculated by raising to the power of 2. If a cell array is made up of N memory elements, the memory capacity of the cell array is N bits. In this case, the number can be calculated by raising to the power of 2 N.

[0005] Therefore, the amount of information (number of bits) of a conventional semiconductor device is expressed as the number of cases in logarithms with a base value of 2. Even when so-called multi-value technology is used, the base value of the logarithm simply becomes 4 or 8, and such logarithms can always be converted to logarithms with a base value of 2, so even when multi-value technology is used, information is still described in bits.

[0006] In contrast, the human brain is not made up of memory elements. If there were something that corresponds to a memory element, it would be the cell body that makes up part of the nerve cell, but this cell body does not store information such as 0 or 1.

[0007] As shown simply in Figure 1, a nerve cell (neuron) generally consists of three parts: a cell body, multiple (e.g., dozens) dendrites, and an axon. The cell body can receive external input from these multiple dendrites. The axon generally extends longer than the dendrites, and its tip further branches into dozens or even hundreds of branches. The ends of these branched axons are called axon terminals.

[0008] As shown simply in Figure 2, the axon terminal approaches one of the dendrites of another cell body and forms a junction called a synapsis.

[0009] There are two cell bodies, A and B. Cell body A receives multiple external inputs x(n) via multiple dendrites (n), where n is an integer between 1 and N. Cell body A assigns a weight W(n) to each input x(n). The signal summed according to these weights is called SUM. SUM is transferred to one of the axon terminals via the axon. When SUM exceeds a certain threshold (threshold of exitation), the neuron generates an action potential, driving synapsis and transmitting neurotransmitters from cell body A to cell body B.

[0010] This threshold changes as signals are transmitted repeatedly. In other words, by repeatedly learning from experience, synaptic connections become stronger, break, or are replaced. The strengthening of synaptic connections can be explained by a lowering of the threshold. The disconnection of synapses can be explained by a rising of the threshold. The replacement of synapses can be explained by a change in the distribution of synaptic thresholds.

[0011] This is modeled as shown in Figure 3. When neurotransmitters are transmitted, the output y is set to 1 (y=1), and when not, it is set to y=0. This model is called a perceptron, and is widely used in deep learning and machine learning.

[0012] There are two main ways to implement a perceptron on a computer.

[0013] In the traditional method, the input x(n), synaptic weight w(n), SUM, threshold of excitation, and output y are all represented as bits, i.e., a computer program.

[0014] The problem with this method is that it places a heavy load on computers. There is a greater need than ever before to improve calculation speeds and reduce power consumption. Deep learning and machine learning require the instantaneous processing of huge amounts of data, and if calculations that place a heavy load were to flood the world, the power consumption of data centers would increase explosively, making it difficult to operate them practically. Furthermore, there are growing concerns that this could accelerate global warming (see Non-Patent Document 2).

[0015] The main cause of the limit on computation speed is excessive data communication between the processor and main memory. Although processors can still be made faster, the communication speed of the data bus between the processor and main memory has reached a ceiling. This is called the von Neumann bottleneck (or memory bus problem).

[0016] The main reason for this increase in power consumption is that the main memory devices currently in use are volatile memory devices called dynamic random access memory (DRAM), which means that the power consumed by refreshing recorded data is becoming non-negligible.

[0017] A recent trend is to replicate the perceptron directly on a semiconductor chip in order to avoid the von Neumann bottleneck and simultaneously reduce power consumption. However, the neural network of the human brain is generally designed to generate synapses between two unspecified pairs of neurons. While current semiconductor technology makes it possible to place perceptrons at precisely specified addresses on a two-dimensional plane or in three-dimensional space, it is not easy to replicate synapses between arbitrary pairs of neurons or to freely reassign them according to learning.

[0018] Generally speaking, a neural network refers to a network of neural circuits. Originally, it referred to a network with countless synapses, as shown in Figure 2, and exists in living organisms. A model that includes the neurons and synapses in Figure 1 is the perceptron in Figure 3.

[0019] If we extract the part of the perceptron in Figure 3 enclosed by the dotted line, it looks like Figure 4. However, let's say the number of dendrites, N, is 4, the input is x(p), and the weight is w(p). Here, p is an integer between 1 and N and refers to an individual dendride. The sum of the products of the input x(p) and the synaptic weight w(p) from 1 to N is SUM in the perceptron in Figure 3, and t(1,1) in Figure 4.

[0020] The parts not enclosed by dotted lines in Figure 3, i.e., the parts related to action potentials, are missing in Figure 4. Nevertheless, in Figure 5, t(2,1) and t(3,1) are added to the second column, and t(1,2) is added to the third column. The arrow from the first column to the first row of the second column means that the input x(p) is multiplied by the weight w(p,1,1) and then added in the first row of the second column. This means calculating the scalar product (product-sum operation) of the row vector with x(p) as an element and the column vector with w(p,1,1) as an element. The result of this calculation is t(1,1).

[0021] The weight has three arguments because we added 1 to represent the first column to the second column, and 1 to represent the sum of the first row of the second column. In other words, the weight from x(p) to t(1,1) is w(p,1,1).

[0022] In Figure 5, the input from the second column to the first row of the third column is further calculated. That is, the arrow from the second column to the first row of the third column means that the input t(q,1) is multiplied by the weight w(q,1,2) and then added together in the first row of the second column. This means that the scalar product (product-sum operation) of the row vector with t(q,1) as an element and the column vector with w(q,1,2) as an element is calculated. The result of this calculation is t(1,2).

[0023] In Figure 6, t(2,2) and t(3,2) are added to the third column, and y is added to the fourth column. The arrow from the third column to the first row of the fourth column (i.e., y) means that the input t(r,2) is multiplied by the weight w(r,1,3) and then added together. This means calculating the scalar product (product-sum operation) of the row vector with t(r,2) as an element and the column vector with w(r,1,3) as an element. The result of this calculation is y.

[0024] In Figure 7, we also take into consideration the arrow from x(p) in the first column to the second row in the second column. Although not shown in the figure, the weight w(p,2,1) is assigned to the newly added arrow.

[0025] In Figure 8, we also take into consideration the arrow from x(p) in the first column to the third row in the second column. Although not shown in the figure, the weight w(p,3,1) is assigned to the newly added arrow.

[0026] In Figure 9, we also consider an arrow from t(q,1) in the second column to row 2 in the third column. Although not shown in the figure, the newly added arrow is assigned a weight w(q,2,2).

[0027] In Figure 10, we also take into account the arrow from t(q,1) in the second column to row 3 in the third column. Although not shown in the figure, the newly added arrow is assigned a weight w(q,3,2). As can be seen, the column number (k) increases as we move to the right in the figure, and at the same time, the layer depth of learning deepens. In other words, the column number (k) is related to the layer depth of learning.

[0028] In neural networks commonly used in the field of artificial intelligence, the first column in Figure 10 is called the input layer. The final column in Figure 10 (for example, column 4) is called the output layer. The multiple layers between the input and output layers (for example, columns 2 and 3) are called hidden layers. When there are multiple hidden layers (number of layers, or number of hidden layers), the system is called deep learning. Deep learning is believed to achieve higher performance as the number of hidden layers increases. An epoch is the period from the input layer to the calculation of output y. The weights are readjusted and output y is recalculated from the same input until the desired value is reached. Various feedback methods are used to bring output y closer to the desired value. Learning is completed when the desired output y is finally obtained after repeated epochs. Therefore, the learning time is calculated by multiplying the time required to process one epoch by the number of epochs required to complete learning. The time required to process one epoch is determined by the chip's ability to process countless thread operations (multiply-and-accumulate operations) in parallel.

[0029] The left diagram in Figure 11 resembles Figure 4. The arrow from the kth column to the jth row of the (k+1)th column represents the (k+1)th column, jth row, obtained by multiplying the input t(p,k) by the weight w(p,j,k+1). This means calculating the scalar product (thread operation or multiply-and-accumulate operation) of the row vector with t(p,k) as an element and the column vector with w(p,j,k+1) as an element. The result of this calculation is t(j,k+1). This calculation result can be provided externally to the AI ​​chip and stored in main memory or storage. In other words, the column number k represents the layer number of the hidden layer. However, when k = 0, it is the input layer. Also, when k = M, it is the output layer. The number j represents the output element number (output element number) of the (k+1)th layer. The number i represents the input element number (input element number) of the kth layer.

[0030] The thread processing unit that calculates t(j,k+1) is denoted T(j,k). It can be considered as the element in the jth row and kth column of the thread matrix. That is, T(j,k) is the device (or circuit) that calculates the thread in the jth row of the (k+1)th layer. In the right diagram of Figure 11, the thread processing unit T(j,k) is placed at a position on the semiconductor chip determined by j for a given k. In other words, j and the position (or placement) on the semiconductor chip are related. In the example on the right of Figure 11, j searches for a position from the top left to the right on the chip, and when it reaches the edge, it moves down one row and searches again from left to right. Generally, the number of output rows j in the (k+1)th column is an integer between 1 and L(k+1). However, since the number of elements (rows) L in each column (each layer) varies for each k, it is an integer function L(k) with k as an argument. That is, L(k+1) threads t(j, k+1) are input to the thread calculation device T(j', k+1) in the (k+1)th layer, and thread t(j', k+2) can be calculated as the output to row j', column (k+2). Here, j' is an integer between 1 and L(k+2). Also, k is an integer between 1 and M. In this case, the number of elements of the weight w in FIG. 11 is L(k+1)L(k+2), which is obtained by multiplying the number of rows in the input layer by the number of rows in the output layer. That is, it is the product of L(k+1) and L(k+2). In the example of FIG. 10, L(0)L(1)L(2)=36 and M=3. However, L(M)=1.

[0031] Let's return to Figure 10. The hidden layer is made up of the overlapping of multiple thread operations. The total number of hidden layers is given by M-1. However, in Figure 10, M = 3, i.e., two columns, is used as an example. Therefore, when calculating T(j, k+1), L(k) is the number of elements input to the thread processing unit. Therefore, the number of elements of the weight w when transferring from the kth column to the k+1th column in the hidden layer is L(k)L(k+1), that is, the product of L(k) and L(k+1). Since the integer k is repeated from 0 to M, the total number of elements of the weight w required to calculate the output y is L(0)L(1)L(2)...L(M-1). This is the product of multiplication from L(0) to L(M). However, since the number of outputs y is 1, we set L(M) = 1. Storing all weight elements in main memory, etc., would require a significantly larger bit capacity for complex deep learning.

[0032] To calculate the output y, first, L(0) inputs are used to calculate L(1) threads, t(1,1), t(2,1)… t(L(1),1), (thread operation). Next, these are used as inputs to calculate L(2) threads, t(1,2), t(2,2)… t(L(2),2), (thread operation). This process is repeated from k = 1 to M to calculate the output y. Calculating the output y is essentially the same as calculating the threads. Thus, the number of thread operations required to calculate the output y using deep learning is L(1) + L(2) + … + L(M-1) + L(M), where L(M) = 1. In the example in Figure 10 (L(1) = 3, L(2) = 3, L(3) = 1, M = 3), a total of seven threads are required: 3 + 3 + 1.

[0033] Next, we will discuss the Graphic Processor Unit (GPU), which is often used in image analysis.

[0034] Generally, a GPU consists of multiple cores. For example, it consists of 1000 512-bit cores. Thus, the number of cores is much greater than that of a typical CPU (around 10).

[0035] GPUs are processor chips originally specialized for image processing, and because they have such a large number of cores, they are better suited than CPUs (central processing units) for parallel processing of multiple rotation matrices. The more complex the 3D images being processed, the more rotation matrices need to be processed in parallel. Calculating a 3D rotation matrix involves multiplying a 3x3 rotation matrix by a 3D column vector (pre-rotation column vector). The result is a 3D column vector (post-rotation column vector).

[0036] The first component of the rotated column vector is the scalar product (thread operation, or multiply-and-accumulate operation) of the first row of the rotation matrix and the pre-rotation column vector. In other words, it can be calculated using the thread T(a, b) mentioned above. As an example, if N=3, the following calculation result is obtained: T(a, b) = t(1, b) × w(1, a, b+1) + t(2, b) × w(2, a, b+1) + t(3, b) × w(3, a, b+1). This calculation result is provided off-chip and can be saved in main memory or storage as needed.

[0037] The second component of the rotated column vector is the scalar product (thread operation, or multiply-and-accumulate operation) of the second row of the rotation matrix and the pre-rotation column vector. In other words, it can be calculated using the thread T(c,d) mentioned above. As an example, if N=3, the following calculation result is obtained: T(c,d) = t(1,d) × w(1,c,d+1) + t(2,d) × w(2,c,d+1) + t(3,d) × w(3,c,d+1). This calculation result can be provided off-chip and stored in main memory or storage.

[0038] The third component of the rotated column vector is the scalar product (thread operation, or multiply-and-accumulate operation) of the third row of the rotation matrix and the pre-rotation column vector. In other words, it can be calculated using the thread T(e, f) mentioned above. As an example, if N=3, the following calculation result is obtained: T(e, f) = t(1, f) × w(1, e, f+1) + t(2, f) × w(2, e, f+1) + t(3, f) × w(3, e, f+1). This calculation result is provided outside the chip and can be stored in main memory or storage.

[0039] As such, three threads (multiplication and accumulation) are required to calculate one 3D rotation matrix. In other words, both deep learning and image analysis of complex 3D shapes (which requires countless 3D rotation matrices) can be performed by parallel processing of countless threads (multiplication and accumulation). In other words, the computational processing required for image analysis and deep learning is virtually identical. This similarity in the computational aspects of image analysis and deep learning is why GPUs are widely used in deep learning. For example, computations related to one thread can be assigned to a single core. Alternatively, computations related to two or three threads can be combined and assigned to a single core. In any case, GPUs, with their vastly greater number of cores than CPUs, are more suitable processors for deep learning than CPUs.

[0040] However, GPUs consume very high amounts of power, ranging from 500W to around 1kW. Increasing power consumption is a problem for AI servers that utilize countless GPUs. One solution is to dramatically increase calculation speed. For example, even if the power consumption per chip increases threefold, if the chip's processing speed increases 100 times, power consumption can be reduced to 3%. In other words, a 97% reduction in power consumption is expected. However, a 100-fold increase in processing speed means that 100 times the AI ​​can be used for the same cost. In other words, humans have inherent desires, so they end up using 100 times the AI, dramatically increasing power consumption by three times. Therefore, this method will ultimately not be able to reduce the power consumption of AI servers.

[0041] Another method is to reduce the amount of information handled. GPUs handle not only pixel coordinates but also subtle color tones and brightness, so they handle fixed-length data of 8, 16, and 32 bits, as well as floating-point numbers. However, if you are only performing threaded operations specialized for deep learning, it is sufficient to handle only fixed-length 8-bit data. This means that the amount of information handled to perform the same threaded operations can be significantly reduced. The Neural Network Processing Unit (NPU) was developed based on this concept. The power consumption of an NPU is about one-tenth that of a GPU. In other words, reducing the amount of information handled in threaded operations is an effective way to reduce power consumption.

[0042] The present invention has been made in view of the above circumstances, and provides a method for reducing power consumption in an information processing system using a neural network.

[0043] The present invention employs the following means to solve the above problems. The solution proposed by the present invention is characterized in that a thread processing device mounted on a semiconductor chip is a component, the thread processing device returns an output in response to an input, the input is made up of a plurality of input elements, the input elements are provided from outside the thread processing device, the thread processing device has weight cells, the weight cells are integrated on the semiconductor chip, and the output is related to the input and the state of the weight cells,

[0044] The thread processing unit further comprises a plurality of cells and a sense amplifier, wherein the cells have first, second, third and fourth terminals, the first terminal connected to a bit line, the second terminal connected to a control word line, the third terminal connected to a selected word line, and the fourth terminal connected to a sum data line, the sum data line having first and second select gates and a summing capacitor between the first and second select gates, the second select gate being sandwiched between the summing capacitor and the sense amplifier, the cells comprise a select transistor, a select capacitor and the weight cell, the weight cell having a first source, a first drain and a first gate, the first drain connected to the first terminal and the first gate connected to the second terminal, the select transistor having a second source, a second drain and a second gate, the second gate connected to the third terminal, The second source is connected to the fourth terminal; the selection capacitor has two terminals; one terminal of the selection capacitor is connected to the fourth terminal; the other terminal of the selection capacitor is connected to the second drain; and the first source and the second drain are connected.

[0045] According to the present invention, by simulating part of the neural network functions within a silicon chip (IC chip), it is possible to reduce the power consumed by artificial intelligence. The best mode for carrying out the invention will now be described in detail.

[0046] First, let's look at how power is consumed in thread operations (how multiply-accumulate operations work).

[0047] Next, the mechanism of the sum-of-products operation (or thread operation) will be described with reference to FIG.

[0048] First, as an example, prepare three pieces of data (A, B, C). Data A is the first input to the adder. Data B is the first input to the multiplicator. Data C is the second input to the multiplicator. The output of the multiplier, i.e., the product of data B and data C, becomes the second input to the adder. The output of the adder is again substituted for data A. This is repeated the number of times (N) that is the number of input elements.

[0049] FIG. 13 is a diagram showing how data is exchanged between an AI chip such as a GPU or NPU and a main memory such as a DRAM.

[0050] Every time a processor such as an AI chip accesses main memory, power is consumed through the memory bus. In Figure 12, an assignment to data A is an overwrite in main memory. Data A is read from main memory and used as the first input to the adder. Data B is read and retrieved from main memory. Data C is read and retrieved from main memory.

[0051] The power consumption for one overwrite to the main memory is PO. The power consumption for each data read from the main memory is PR. In this case, the power consumption for the calculation of the scalar product of a row vector and a column vector with N elements (thread operation or multiply-and-accumulate operation) is N x (PO + PR + PR + PR). (First embodiment)

[0052] What would happen if the input C to the multiplier could be processed on-chip, that is, within the AI ​​chip without using main memory? As shown in Figure 14, the number of times data is retrieved from main memory per multiplication would be reduced from three to two. Therefore, the power consumption for the scalar product (thread operation, or multiply-and-accumulate operation) of a row vector and a column vector with N elements would be N × (PO + PR + PR).

[0053] The weight w(i, j, k) in neural network calculations corresponds to the data C. As there are three integer arguments (i, j, k), it is a three-dimensional matrix. In deep learning, k becomes large, and as the input and output of each thread calculation become larger, i and j also become larger. This means that a large number of elements of data C must be handled. For this reason, the system corresponding to Figure 12 (a conventional example) uses HBM (High-Band Width Memory), which is a vertical stack of multiple DRAMs connected via TSV, as the main memory.

[0054] Therefore, in order to handle data C on-chip, the AI ​​chip and HBM must be integrated into a single chip, which is quite difficult.

[0055] This application attempts a different approach. As shown in FIG. 15 , data B consisting of N external elements is acquired from outside the chip (or AI chip) as a first input. N weight elements are acquired from on-chip as a second input. These threads are calculated, and the results are output as data A. Data A can be provided externally to the chip (or AI chip). The number of output elements is set to 1, since all that is required is to calculate a scalar product. In this case, the main memory accesses are PR when acquiring data B and PS when outputting data A. Assuming that PS and PO are equal, the power consumption for the scalar product (thread operation or multiply-and-accumulate operation) of a row vector and a column vector with N elements is N × PR + PO. In this way, power consumption can be reduced without relying on the integrated main memory. This represents an example of the effect of the thread operation device (Present MAD) of this application.

[0056] In other words, by using this application, it is possible to reduce power consumption by 2N × PR + (N-1) × PO each time the calculation of the scalar product (product-sum operation) of a row vector and a column vector with N elements is performed as a thread operation.

[0057] Therefore, if the same thread operation is repeated K times to execute one epoch, power consumption per epoch can be reduced by K × (2N × PR + (N-1) × PO).

[0058] Therefore, if the number of epochs required to complete one learning is Q, it is possible to save power consumption of Q×K×(2N×PR+(N−1)×PO) to complete one learning (Second Embodiment).

[0059] Fig. 16 is an example of an equivalent circuit diagram for explaining the first embodiment. By using this circuit, it is possible to enjoy the benefits of Fig. 15. That is, Fig. 16 is an example of an equivalent circuit of a thread processing device.

[0060] That is, this circuit makes it possible to calculate t(j, k+1) (thread calculation or product-sum calculation) from t(1, k), t(2, k), ... t(N-1, k), and t(N, k) in FIG. 11, taking into account weights w(1, j, k+1), w(2, j, k+1), ... w(N, j, k+1). That is, this equivalent circuit is for calculating thread T(j, k) in FIG. 11. When calculating that thread, it is possible to significantly reduce power consumption, as described in FIG. 15.

[0061] First, input t(i, k) can be input through N appropriately selected bit lines BL(i), where i is an integer between 0 and N-1, and N is an integer greater than or equal to 1. The total number of bit lines in Figure 16 is NN, where NN is generally an integer greater than or equal to N. Each bit line is connected to a bit line select gate SBL(i). Therefore, N inputs can be conveniently selected from the NN input candidates. The selected N bit lines correspond to the dendride through which the Soma Body receives stimuli (input) from the outside (other neurons).

[0062] One cell (silicon brain cell, SB cell) corresponds to each bit line. Therefore, the equivalent circuit of this embodiment is composed of NN SB cells, the same number as the number of bit lines. In other words, in this embodiment, one thread is modeled with NN SB cells. Each SB cell consists of a weight cell (for example, a type of non-volatile memory cell), a selection transistor, and a selection capacitor.

[0063] The weight cell that constitutes part of the SB cell is, for example, a nonvolatile memory cell, and has a charge storage region such as a charge trap layer or a floating gate that can store charge even without a power source.

[0064] The nonvolatile memory cell (i.e., an example of a weight cell) comprises a source, a drain, a control gate, and a channel portion in addition to the charge storage region. The drain of the nonvolatile memory is connected to a corresponding bit line, and the control gate is connected to a control word line (CWL).

[0065] The select transistor that constitutes part of the SB cell is a type of transistor that consists of a source, a drain, and a select gate.

[0066] The source of the select transistor is connected to an add data line (ADL), the drain of the select transistor is connected to a source of the nonvolatile memory, and the gate of the select transistor is connected to a select word line (SWL).

[0067] A select capacitor, which is part of an SB cell, is associated with the select transistor and has two terminals, one connected to the drain of the select transistor and the other connected to the source of the select transistor.

[0068] Thus, the N SB cells are connected in parallel to the summing line.

[0069] The summing line has a left selection gate (SGL) and a right selection gate (SGR), and the source side terminals of the NN SB cells are arranged between the left selection gate and the right selection gate.

[0070] Between the left select gate and the right select gate, there is further provided an adder capacitor (ADC) which serves as an adder (ADDER) according to the present invention. The location of the adder capacitor can be anywhere between SGL and SGR, and it can also be placed between two bit lines.

[0071] The summing capacitor has two terminals, one of which is connected to the summing line, and the other terminal (arrow) can be connected to any circuit, for example, to the source line.

[0072] A sense amplifier (S / A) is installed outside the right select gate (the side without the source terminal of the SB cell), and the right select gate is between the sense amplifier and the summing capacitor.

[0073] In general, the capacitance of the summing capacitor must be greater than the capacitance of each of the selection capacitors. For example, the capacitance of the summing capacitor must be greater than NN times the capacitance of the selection capacitor. (Third embodiment)

[0074] It is desirable that the nonvolatile memory constituting the SB cell is specially adjusted to have a large S-factor.

[0075] Figure 17 shows an example of the voltage-current characteristics of two types of nonvolatile memory cells (NV). Gate voltage is the voltage applied to the control gate. Drain current is the current that flows through the drain in response to the applied gate voltage.

[0076] The left corresponds to a typical nonvolatile memory cell with a sufficiently small S-factor, while the right corresponds to a nonvolatile memory cell (i.e., a weight cell) related to this application, with a sufficiently large S-factor. In both cases, writing (P) and erasing (E) are possible repeatedly. The mechanism is basically the same, and Fowler-Nordheim tunneling (FNT) is used as an example. First, as an example, consider a nonvolatile memory cell that includes a channel region (or substrate) and a charge storage region (such as a floating gate or charge trap layer). When FNT occurs, charged particles (such as electrons or holes) move between the channel region and the charge storage region.

[0077] In other words, FNTs cause charge transfer between the channel and charge storage regions within a nonvolatile memory cell (or weight cell). For example, electrons in the channel move to the charge storage region. Conversely, electrons in the charge storage region move to the channel. Alternatively, holes in the channel move to the charge storage region. Conversely, holes in the charge storage region move to the channel. One of the key features of this application is that the state change using FNTs (in this example, the change between the written state and the erased state) is a change in the number of charged particles (or charge) stored in the charge storage region. Therefore, changing the charge stored in the charge storage region within a nonvolatile memory cell (or weight cell) requires only the movement of electrons or holes within the weight cell, and does not necessarily require large currents to flow through the bit line or word line. Therefore, this application is characterized by extremely low power consumption for changing the state of the weight cell.

[0078] For example, in the current-voltage characteristics of a typical nonvolatile memory cell (left), programming (P) increases the threshold voltage, shifting the characteristic curve to the right. Conversely, erasing (E) decreases the threshold voltage, shifting the characteristic curve to the left. This threshold voltage change occurs due to changes in the amount of charge accumulated in the charge storage region. In the written state (0), the S-factor is small, so the current rises sharply at the threshold voltage Vt0. In the erased state (1), the S-factor is small, so the current rises sharply at the threshold voltage Vt1. Because of this steep current rise, there is a region (encircled by the dotted line) between Vt1 and Vt0 where no current flows in the written state (0) but does in the erased state (1). The wider this region is, the better.

[0079] In the current-voltage characteristics of a weighted cell (a nonvolatile memory cell (right) as an example) related to this application, when writing (P) is performed, the threshold voltage increases, and the characteristic curve shifts to the right. Conversely, when erasing (E) is performed, the threshold voltage decreases, and the characteristic curve shifts to the left. This change in threshold voltage occurs due to the change in the amount of charge accumulated in the charge storage region. In the written state (0), the S factor is large, so the current increases gradually at the threshold voltage Vt0. In the erased state (1), the S factor is large, so the current increases gradually at the threshold voltage Vt1. Because the current increase is so gradual, even in the written state (0), a certain amount of current flows between Vt1 and Vt0. This characteristic differs from that of ordinary nonvolatile memory and is suitable for recording state changes in an analog manner.

[0080] In this embodiment, such analog nonvolatile memory cells (ANV) are used as weight cells that constitute part of the SB cells. In other words, it is desirable that all of the NN nonvolatile memory cells (weight cells) in FIG. 16 are ANVs.

[0081] There are several ways to increase the S-factor of the weight cell or analog nonvolatile memory cell (ANV), but care must be taken to prevent the S-factor of the select transistor from increasing. For example, it is desirable to take measures such as shortening the gate length of the ANV, making the diffusion layer of the ANV deeper than usual, or reducing the amount of on-injection into the channel of the ANV. (Fourth Embodiment)

[0082] FIG. 18 is a diagram illustrating how the circuit of FIG. 16 operates to obtain thread T(j, k).

[0083] First, all nonvolatile memory cells in the entire thread are erased at once (Thread Erase).

[0084] Next, programming is performed on only the nonvolatile memory cells associated with the bit lines selected in the thread. In this selective programming, it is desirable to gradually increase the threshold voltage Vt by dividing it into multiple short pulses so that the increase in Vt can be adjusted.

[0085] Next, if necessary, erase only the nonvolatile memory cells related to the selected bit lines in the thread. In this selective erase (Selective Program), it is desirable to gradually decrease the threshold voltage Vt by dividing it into multiple short pulses so that the decrease in Vt can be adjusted.

[0086] However, if the precision of the selective writing is sufficient, the selective erasure can be omitted.

[0087] Next, the first measurement (Measure 1) is performed. What is measured here is not the threshold voltage Vt of the nonvolatile memory cell. What is measured in this application is the charge stored in the summing capacitor (ADC). If the situation is properly arranged, the charge on the summing capacitor can be measured by the sense amplifier (S / A) as the potential of the ADL. In this first measurement (Measure 1), the amount of charge before the adder is activated is measured. As an example, let's call this value Q1.

[0088] Next, the adder is operated. The operation steps are divided into four stages (Ope1, Ope2, Ope3, Ope4).

[0089] After the adder is activated, a second measurement (Measure 2) is immediately taken to obtain Q2.

[0090] The value of T(j, k) is obtained from the value measured the second time (Q2) and the value measured the first time (Q1). However, the value of T(j, k) is the value of a function with Q2 and Q1 as arguments.

[0091] The function can be selected from various options depending on convenience, such as the difference between Q2 and Q1, the square of this difference, the ratio of the logarithm of Q2 to the logarithm of Q1, etc. In this embodiment, the difference obtained by subtracting Q1 from Q2 is used as an example.

[0092] The value of T(j, k) thus obtained is stored in the main memory as data A in Fig. 15. It is then read out as data B as needed.

[0093] The data corresponding to data C in Fig. 15 is stored in the weight cell. An example is the threshold voltage value (Vt) written to the analog nonvolatile memory cell (ANV) in Fig. 16.

[0094] Even an NPU (fixed length 8 bits), which consumes less power than a GPU, still consumes the necessary power to write, erase, and read data to 8 bits in main memory to handle one element of the weight w(i, j, k). This consumes power. A GPU allocates more bits to each weight element, resulting in even higher power consumption. As mentioned above, the number of weight elements is L(1)L(2) ...L(M). In other words, the deeper the hidden layer (M-1) is, and the larger the number of inputs and outputs in each column, L(k), the higher the power consumption.

[0095] In contrast, in this application, only one bit of the analog nonvolatile memory (ANV) of the SB cell is allocated to handle one element of the weight w(i, j, k). As shown in the right diagram of Figure 17, by continuously changing the Vt of the ANV, the drain current, which is the output for a given gate voltage, changes continuously. This is the same as a continuously variable resistance. Therefore, one bit of the ANV can correspond to analog data, which means that it can also accommodate fixed-length data longer than 8 bits. This is another advantage of this application.

[0096] FIG. 19 is a table for explaining an example of a method for executing the thread erase of FIG.

[0097] A reference voltage Vref (for example, 0 V) ​​is applied to the control word line CWL, and an on-voltage Von is applied to the selected word line SWL. Von is a voltage that switches on the select transistor. Furthermore, an erase voltage Vers (for example, 20 V) is applied to the addition data line ADL, and Von is applied to the left select gate SGL. Furthermore, Von is applied to all bit line select gates SBL (*), where * is an integer from 0 to NN-1. In this way, all bit lines BL are selected and an erase voltage Vers (for example, 20 V) is applied.

[0098] FIG. 20 is a table for explaining an example of a method for executing the selective program of FIG.

[0099] An intelligent write voltage Vint.pgm is applied to the control word line CWL, and an on-voltage Von is applied to the selected word line SWL. Furthermore, a reference voltage Vref (for example, 0V) is applied to the addition data line ADL, and Von is applied to the left select gate SGL. Furthermore, Von is applied to all bit line select gates SBL (**), where ** is the number of the selected bit line (selected bit line), an integer between 0 and NN-1. In this way, a reference voltage Vref (for example, 0V) is selectively applied to the selected bit line BL.

[0100] However, the intelligent write voltage Vint.pgm is a voltage applied to the control gate for writing data to the nonvolatile memory cell in small increments.

[0101] An example is shown in Figure 21. In this example, a potential (Vpulse) of the same magnitude is repeatedly applied. The write (P) on the right side of Figure 17 is performed little by little by adjusting the pulse potential, the time for applying the pulse, i.e., the pulse period, the standby period between pulses, the number of times the pulse is applied, etc.

[0102] Another example is shown in Figure 22. In this example, pulses are applied while gradually increasing the pulse potential. By adjusting the pulse potential increase width (pulse shift), pulse time (pulse period), standby period between pulses (standby period), number of pulse applications, etc., the write (P) on the right side of Figure 17 is performed little by little.

[0103] There are various other methods for applying pulses, but in any case, the purpose is to gradually adjust the change in Vt caused by writing as shown in the right diagram of Figure 17. Figure 40 shows yet another example, in which the potential of each pulse is lowered as the change in Vt approaches the target value.

[0104] FIG. 23 is a table for explaining an example of a method for performing the selective erase of FIG.

[0105] A reference voltage Vref (for example, 0V) is applied to the control word line CWL, and an off-voltage Voff is applied to the selected word line SWL. Furthermore, the addition data line ADL is set to a floating state, and Voff is applied to both the left select gate SGL and the right select gate SGR. The ADL is set to a floating state to prevent it from unnecessarily affecting selective erasure. Furthermore, Von is applied to the bit line select gate SBL (**) of the selected bit line, where ** is the number of the selected bit line (selected bit line), an integer between 0 and NN-1. In this way, the intelligent erase voltage Vint.ers is selectively applied to the selected bit line BL.

[0106] Here, the intelligent erase voltage Vint.ers is a voltage applied to the control gate for gradually erasing the nonvolatile memory cell. Examples of Vint.ers are similar to those shown in Figures 21, 22, and 40, so detailed description will be omitted.

[0107] FIG. 24 is a table illustrating an example of a method for performing an initial measurement (measure 1).

[0108] 16, when charge accumulates in the summing capacitor (ADC), the potential of the summing data line ADL changes. While SGL is off and SGR is on, the sense amplifier (S / A) can measure this potential change.

[0109] The control word line CWL is set to a floating state to prevent CWL from having an undesired effect on the measurement. Furthermore, an off-voltage Voff is applied to the selected word line SWL, and Voff is initially applied to both the left select gate and the right select gate SGR. Furthermore, Voff is applied to all bit line select gates. In other words, all bit lines are unselected. To ensure that any bit line does not have an undesired effect on the measurement, the bit lines are set to a floating state. Thus, the summing data line ADL is set to a floating state. Next, the voltage applied to SGR is changed from Voff to Von, which opens SGR and allows the sense amplifier S / A to measure the initial potential of ADL. This potential is related to the charge Q1 stored in the summing capacitor at this point.

[0110] 25 is a table showing an example of a method for executing a thread operation, in which the thread operation is divided into four steps (Ope-1, Ope-2, Ope-3, Ope-4).

[0111] First, in Operation 1, Vread is applied to the control word line CWL, and Voff is applied to the selected word line SWL. Here, Vread is the read voltage applied to the control gate to read the Vt of the analog nonvolatile memory cell (ANV). Generally, Vread should be within the voltage region where the current-voltage characteristics change smoothly, as shown in the right diagram of Figure 17. If the S factor is sufficiently large, Vread should be higher than Vt0 in the diagram. It should also be lower than the voltage at which the current rise in the erased state saturates. The Vread shown in the right diagram of Figure 17 satisfies the above conditions.

[0112] Furthermore, the addition data line ADL is set to a floating state, Voff is applied to the left selection gate SGL and the right selection gate SGR, and Voff is also applied to the selected word line SWL.

[0113] Furthermore, all bit line select gates SBL are set to Voff. Next, input voltages Vin(0), Vin(1), ..., Vin(N-1) are applied to the selected bit lines. Here, Vin corresponds to the inputs t(1,k), t(2,k), ..., t(N,k) in the left diagram of Figure 11. Here, N is generally an integer less than or equal to NN. Thus, since Vin is an input signal, it does not need to be fixed to a constant value.

[0114] To select N bit lines, the voltage applied to the corresponding N bit line select gates SBL is changed from Voff to Von. Thus, Vin is applied to the drain of the analog nonvolatile memory cell (ANV) of the SB cell selected by the selected bit (selected SB cell) through each selected bit line. Since the ANV is a weight cell, which is equivalent to a variable resistor in this example, while Vread is applied to the CWL, charge flows into the selection capacitor in the selected SB cell according to Vin and the Vt of the ANV. While Voff is applied to the selected word line SWL, the selection transistor in the selected SB cell is turned off, so this charge is stored in the selection capacitor for a certain period of time.

[0115] Next, in Operation 2, the voltage applied to the CWL is changed from Vread to Voff, and the voltage applied to the selected SBL is changed from Von to Voff. To avoid any unnecessary effects on the ADL, the selected bit line is set to a floating state. This is a maintenance state that maintains the charge on the selected capacitor.

[0116] Subsequently, in Operation 3, the voltage applied to the selected word line SWL is changed from Voff to Von, causing the charge in the selected capacitor to move to the summing capacitor (ADC) through the summing data line ADL.

[0117] Subsequently, in Operation Operand 4, first, the voltage applied to the selected word line SWL is changed from Von to Voff.

[0118] Next, in Measure-2, the voltage applied to the right select gate SGR is changed from Voff to Von. Here, a sense amplifier is located outside (to the right of) SGR. This allows the sense amplifier to sense the potential of the sum data line ADL. Since the potential of the sum data line ADL is determined by the charge accumulated in the summing capacitor, the sense amplifier can sense the charge Q2 of the summing capacitor. Figure 27 lists the above procedure as an example.

[0119] Incidentally, the difference in charge between the summing capacitors of Measure 1 and Measure 2 is determined by the sum of the charges accumulated in the N selected capacitors between Ope-1 and Ope-4, so the sense amplifier senses the sum of the charges of the N selected capacitors.

[0120] Furthermore, the charge on each selected capacitor is related to the Vin of each input and the Vt of each ANV. In other words, the charge on each selected capacitor is determined by taking into account the voltage drop across the corresponding Vin and the corresponding ANV (a variable resistor with Vt as an argument). Therefore, the sense amplifier senses the sum of Vin and ANV Vt within the selected SB cell.

[0121] In this way, it is considered that at least the charge dQ of each selected capacitor is determined by a function f of the corresponding Vin and the Vt of the corresponding ANV. As an example, it can be written as dQ = f(Vin, Vt). However, f does not need to be a known function.

[0122] Nevertheless, the charge dQ of each selected capacitor can be considered to be proportional to the corresponding Vin. Furthermore, the proportionality coefficient can be considered to depend on the Vt of the corresponding ANV. Therefore, for example, it can be written as dQ = Vin × w(Vt). Here, this w(Vt) corresponds to the weight w in Figure 11. This is indeed in the form of a product. However, although the ANV stores Vt, it does not store the weight (w) itself. In other words, the weight cell (for example, the ANV) stores something related to the weight.

[0123] As an example, the charge dQ on the selection capacitor is summed up by the summing capacitor to obtain Q. This is sensed by the sense amplifier as a potential difference V. This V becomes the calculation result of thread T(j,k). In other words, the output sensed by the sense amplifier is related to the state of each weight cell and the input.

[0124] However, in AI neural network calculations, this thread T(j,k) is not used as is as part of the input to the next layer. In fact, it is transformed using an activation function before being used as input to the next layer. Activation functions vary depending on the problem being solved using AI. In other words, it is convenient to define them using software. In other words, the CPU (Central Processing Unit) used in conjunction with the AI ​​chip dedicated to thread calculations described above transforms the thread T(j,k) using the activation function before returning it to the AI ​​chip.

[0125] An example will be described using Fig. 28. Fig. 28 shows an example of a method for handling thread t(j, k) calculated between an AI chip, a CPU, and a main memory.

[0126] For example, thread t(j,k) calculated by an AI chip is sent to the CPU. The CPU stores the received thread in main memory such as DRAM. The CPU reads this thread when needed and converts it to Act(j,k) using an activation function. This Act(j,k) is passed to the AI ​​chip as the next input. In other words, t(*,k) in Figure 11 is Act(*,k), where * is an integer between 1 and N.

[0127] In order to increase the S-factor, the analog nonvolatile memory cell ANV shortens the gate length, dilutes the impurity concentration, and deepens the diffusion layer. Therefore, it is manufactured differently from normal nonvolatile memory, which prefers a small S-factor. This makes it more susceptible to fluctuations (sigma) in the current characteristics than normal. There are various causes of fluctuations, but the most prominent is random impurity fluctuations.

[0128] Such fluctuations appear as fluctuations in Vt and the S factor. Therefore, the effect of this fluctuation will be mixed into the charge of the ADC, which is the result of the thread operation. In this case, the formula for the charge of the selected capacitor mentioned above can be expanded to dQ = Vin × w(Vt, sigma).

[0129] So, is this a problem? The bottom line is no.

[0130] This is because the fluctuations mentioned above are automatically taken into account when updating the weights (synaps weights, w) stored in main memory each time learning is repeated.

[0131] Let's say the weight when fluctuations are ignored is w0(Vt). In contrast, the weight actually stored in main memory after going through the mechanism in Figure 28 is w(Vt, sigma). In other words, it is different from w0(Vt). We can think of it as a weight that has already been updated taking fluctuations into account and stored in main memory. This is because the amount of charge read by the sense amplifier automatically includes all fluctuations. When updating the weights through repeated learning, the weights that include fluctuations are automatically updated in main memory, so their influence can be ignored. This is one of the features of the present application.

[0132] In other words, the weight w can be seen as a model in which the SB cell in Figure 29 is replaced with a capacitor. The SB cell has two terminals, one of which is connected to the bit line BL(*) in Figure 16, and the other to the addition data line ADL, where * is an integer from 0 to N-1.

[0133] 29 shows the state when Vin is applied between the ADL and the bit line. When Vread is applied to ANV and the selection transistor is off, a transient current flows between the two terminals until the charge on the selection capacitor is saturated.

[0134] This transient current causes charge to accumulate in the selected capacitor over time. This is shown in Figure 30. When Vin is applied at time t0, the absolute value of the charge increases over time from t0. In the case of a low Vt, the charge saturates at time t1 and the transient current disappears. In the case of a high Vt, the charge saturates at time t2 and the transient current disappears. Generally, t2 is greater than t1.

[0135] An ANV with a low Vt can be considered a low resistance. Conversely, an ANV with a high Vt can be considered a high resistance. That is, at low Vt, the current is high due to the low resistance, so charge accumulates quickly. Therefore, the slope is high. On the other hand, at high Vt, the current is low due to the high resistance, so it takes longer for charge to accumulate. Therefore, the slope is low.

[0136] The sense time is ts - t0. In other words, Von is applied to the corresponding bit line select gate SBL only between t0 and ts. In other words, Vin is applied only between t0 and ts to allow a transient current to flow. However, ts is assumed to be shorter than t1.

[0137] When the application of Vin is stopped at time ts and the transient current disappears, the charge stored in the selected capacitor at that time differs between when Vt is low and when Vt is high. This is reflected as a different value of weight w when added on the addition data line ADL. The added result is stored in main memory. Incidentally, in this application, there is no need to store each component of weight w in main memory. This makes it possible to dramatically reduce power consumption and the required bit capacity of the main memory.

[0138] Here, ts must always be smaller than or equal to t1. Therefore, there exists a minimum Vt corresponding to the weight w to be stored. In other words, the minimum Vt is a function of ts. However, ts can be controlled by software.

[0139] Returning to Figure 18, a wait state may be required between two adjacent steps. Figure 31 is a table illustrating one example of how such a wait state can be implemented.

[0140] First, the selected word line CWL is set to a floating state, Voff is applied to the selected word line SWL, the addition data line is set to a floating state, Voff is applied to the left select gate and the right select gate, and all bit lines are deselected.

[0141] The wait state shown in Fig. 31 can be inserted between two adjacent columns in Fig. 27 as needed. Such a wait state may be necessary when the response from the CPU is relatively long. (Fifth embodiment)

[0142] FIG. 32 is a diagram showing an example of the flow of one epoch.

[0143] The integers used for incrementing are j and k. However, k has a special meaning in determining the structure of the neural network. That is, when k = 0, it represents the input layer, and when k = M, it represents the output layer. The hidden layers range from k = 1 to k = M-1. That is, the number of hidden layers is M-1. When k = M, the output y is calculated. That is, the output y is the charge added in the Mth layer. Also, j is the element number of the output element in the (k+1)th column obtained by thread calculation, and is an integer between 1 and L(k+1).

[0144] Here, i is the element number of the input element x(i) input in the kth layer when performing thread calculation, and is an integer between 1 and L(k). However, if the number of elements in the input layer (k=0) is N, then L(0) = N. Input elements can be input from outside the thread calculation device.

[0145] At Start, both increments shall start from 1.

[0146] Next, the first thread operation is executed to obtain the output t(1, 2) (Get t(1, 2)). Using the activation function, this is converted into the activation output element Act(1, 2) and stored in main memory.

[0147] Next, the next thread operation is executed to obtain the output t(2, 2) (Get t(2, 2)). Using the activation function, this is converted into the activation output element Act(2, 2) and stored in main memory.

[0148] This process is repeated L(2) times. The activation output element Act(j,2) is provided externally from the semiconductor chip. In some cases, the activation output element Act(j,2) is stored in main memory, where j is an integer between 1 and L(2). The L(2) activation output elements Act(1,2), Act(2,2), … Act(L(2),2) are combined together and called the activation output of the second layer. (The first layer is the input layer.) The activation output of the (k+1)th layer is obtained by incrementing k. The activation output of the (k+1)th layer is the combination of the L(k+1) activation output elements Act(1,k+1), Act(2,k+1), … Act(L(k+1),k+1).

[0149] Next, check whether k is smaller than M. If it is smaller than M, assign (overwrite) the activated output element to the input element and update (increment) k. In other words, increase it by 1, and repeat this process until k = M. Then, calculate the Y value when k = M. However, when k = M, it is possible to use or not use an activation function.

[0150] In this way, activation output elements Act(j, k+1) are calculated for all possible j and k and stored in main memory as needed. Y values ​​are also calculated (Get Y) and stored in main memory as needed. This completes one learning session. In other words, with the calculation of Y values ​​(Get Y), one epoch is completed and the process ends (End).

[0151] 33 is a diagram for explaining an example of a method for repeating epochs. Here, the number of epochs (number of learning times) is represented by g (epoch number).

[0152] First, at the start, set g = 1.

[0153] Y(g) is calculated using the method in Figure 32, where Y(g) is the Y value described above. The Y value calculated the first time is Y(1), the Y value calculated the second time is Y(2), and so on, and the Y value calculated the gth time is Y(g).

[0154] Next, the difference (or residual) between the latest Y(g) and the immediately previous Y(g-1) is compared with a predetermined value. If the absolute value of this residual is deemed sufficiently small compared to the predetermined value, Y(g) is deemed to have converged. If it has not converged and g is not smaller than the upper limit gupp, the calculation is deemed to have failed (failed) and the calculation is terminated (End). Alternatively, if g is smaller than the upper limit gupp, g is incremented (increased by 1), Y is calculated again, and this is set as Y(g+1).

[0155] On the other hand, if Y(g) has converged, Y = Y(g), Y is output, and learning ends (End). The final output Y can be saved to the main memory outside the AI ​​chip, if necessary. At this time, the most recent g is the epoch number. The epoch number is useful for expressing the learning efficiency of an algorithm. In other words, a large epoch number indicates low learning efficiency, and a small epoch number indicates high learning efficiency.

[0156] The learning speed is expressed as the product of the operating speed of the AI ​​chip, which is expressed as the clock frequency, bandwidth, etc., and the number of epochs.

[0157] As described above, the total number of elements of the weight w required to calculate the output Y is L(0), L(1), ... L(M-1). (Sixth embodiment)

[0158] Instead of the ANV in FIG. 16 , a variable resistor, a memristor, or the like can be used. The ANV, variable resistor, memristor, and the like in this application can be collectively referred to as a synaptic weight cell. That is, the synaptic weight can be stored in the weight cell as some kind of state change. FIG. 34 shows an example of an equivalent circuit diagram for a general weight cell. Weight cells are available in two-terminal and three-terminal configurations. The three-terminal type has a control word line CWL, as shown in FIG. 16 . The two-terminal type can omit the control word line CWL, as shown in FIG. 34 . In the two-terminal type, the state of the weight cell can be changed by applying a predetermined electrical, electromagnetic, or optical stimulus between the two terminals.

[0159] The weight cell related to this application is, for example, a three-terminal, transistor-type electronic component (electronic device, or simply, device) having a charge storage region, where the charge storage region is, for example, a floating gate or a charge trap layer.

[0160] An example of a weight cell relevant to this application is a memristor, which is a two-terminal device whose state changes depending on the charge or magnetic flux transmitted through it.

[0161] The weight cell of interest in this application is, for example, a conductive bridge RAM (CBRAM), which is a two-terminal device in which the resistance (or conductivity) is controlled by passing an appropriate current through it, causing ions or the like to move.

[0162] The weight cell relevant to this application is, for example, a metal switch, which is a two-terminal device that controls resistance (or conductivity) by moving metal ions or the like when an appropriate current is passed through it. It is also a variant of CBRAM.

[0163] The weight cell of interest in this application is, for example, a phase change memory (PC Memory), which is a two-terminal device whose resistance (or conductivity) is controlled by passing an appropriate current through it, causing a phase change.

[0164] The weight cell of interest in this application is, for example, a resistive memory, which is a two-terminal device that changes its resistance (or conductivity) by passing an appropriate current through it.

[0165] The weight cell related to this application is, for example, a spintronic device. This is a device that controls the resistance (or conductivity) by applying an appropriate electric field and passing an appropriate current nearby to apply this electric field, thereby causing a change in resistance. Examples include spin transfer torque devices, spin wave devices, magnetic domain walls, etc. The weight cell has two terminals in addition to the part that applies the electric field.

[0166] The weight cells of interest in this application are, for example, optical devices, which are two-terminal photoelectric devices that use the photoelectric effect to control the flow of electrons and thus the synaptic weight.

[0167] FIG. 34 is a diagram showing an example of an equivalent circuit according to the present invention when a two-terminal weighted cell is used.

[0168] Since the weight cell has two terminals, it is possible to omit the control word line CWL. Examples of two-terminal cells include memristors, CBRAM, metal switches, phase-change memories, resistive memories, spin devices, and optical devices.

[0169] When the weight cell according to the present invention is a three-terminal cell, the equivalent circuit according to the present invention is, for example, as shown in FIG.

[0170] When the three-terminal weight cell has a charge storage region (such as a floating gate or a charge trap layer), FIG. 35 is equivalent to FIG.

[0171] In general, the characteristics of a three-terminal weight cell having a charge storage region (such as a floating gate or a charge trap layer) are analog type with a large S factor, as described above on the right side of FIG.

[0172] When the S factor of a three-terminal weighted cell with a charge storage region (such as a floating gate or charge trap layer) is small, it is possible to control more than two states by programming (P) and erasing (E), as shown in Figure 36. In this figure, as an example, a two-bit per cell (2 bits / cell) with four states is used. In this case, the four states are (11), (10), (01), and (00), as shown in the second row of Figure 37. This is also known as MLC.

[0173] The third row in Figure 37 shows the case of 8 states (3-bit parcel, 3 bits / cell). The states are (111), (110), (101), (100), (011), (010), (001), and (000). This is also known as TLC.

[0174] The fourth row in Figure 37 shows the case of 16 states (4-bit parcel, 4 bits / cell). The states are (1111), (1110), (1101), (1100), (1011), (1010), (1001), (1000), (0111), (0110), (0101), (0100), (0011), (0010), (0001), (0000). This is also known as QLC.

[0175] Although not specifically shown, 32 states (5-bit per cell) are also possible. This is also known as PLC. Since it is self-evident, we will omit the details, but the number of states per cell can be increased arbitrarily. As mentioned above, the increase does not have to be in increments of two. Since it is self-evident, we will omit the details, but it is possible to increase the number of discretized states per cell arbitrarily.

[0176] The name of one line in Figure 37 is 1 bit parcel (1 bit / cell) on the left side of Figure 17. This is also known as SLC.

[0177] As mentioned above, the weight cells of the present application can have an arbitrarily large number of discretized states.

[0178] FIG. 38 is a diagram showing an example having dummy cells.

[0179] The dummy cell can be placed adjacent to the weight cell to ensure stable operation of the weight cell. The dummy cell may or may not be the same element as the weight cell. In the example of FIG. 38, the weight cell is an ANV, and the dummy cell is a cell having a charge storage region. However, the threshold voltage of this dummy cell may or may not be variable. The dummy cell may or may not be connected to a word line. The positions of the dummy cell and the weight cell may be interchanged. The number of dummy cells is not limited to one. The number of dummy cells is arbitrary.

[0180] FIG. 39 is a diagram showing an example having dummy cells.

[0181] A dummy cell can be placed adjacent to a weight cell to ensure stable operation of the weight cell. The dummy cell may or may not be the same element as the weight cell. The dummy cell may or may not change state like the weight cell. The dummy cell may or may not be connected to a word line. The weight cell may or may not be connected to a word line. The positions of the weight cell and the dummy cell may be interchanged. The number of dummy cells is not limited to one. The number of dummy cells is arbitrary.

[0182] Finally, the weight cell of the present application can form part of the SB cell of the present application. The SB cell can be integrated on the semiconductor chip (such as an AI chip) of the present application. Thus, the weight cell can be integrated on the semiconductor chip. As described above, the weight cell can store some state related to the weight (synaptic weight) or a change in that state as a weight element, rather than storing the weight itself. As described above, the weight element can be expressed in the form of a three-dimensional matrix w(i, j, k). As the number of hidden layers increases, i.e., when dealing with more complex problems using deep learning, the number of elements in w(i, j, k) becomes very large. In other words, accessing main memory every time a weight is updated requires very large power consumption. Therefore, integrating weight cells directly on an AI chip, as in the present application, contributes to significantly reducing power consumption for utilizing deep learning (artificial intelligence).

[0183] In the present application, there is no need to pass a large current through the bit line or word line to change the state of the weight cell. In other words, the state change of the weight cell occurs due to the limited movement of charged particles (electrons, holes, etc.) inside the weight cell. Therefore, there is an advantage that the power consumption required to rewrite the state of the weight cell is extremely low. Furthermore, what is sensed to read the state of the weight cell is the charge accumulated in the selection capacitor and the summing capacitor ADC. Therefore, there is also no need to pass a large current through the bit line or word line to sense the state change of the weight cell. Thus, there is an advantage that the power consumption to sense the weight (synaptic weight) is extremely low.

[0184] The thread processing device according to the present application is an integrated circuit or an embedded circuit having multiple SB cells as components, i.e., a circuit mounted or integrated on an AI chip, or a part thereof. The thread processing device according to the present application can be integrated on a semiconductor chip. The semiconductor chip is an example of an AI chip, and is preferably a chip independent of the main memory. As shown in FIG. 11, multiple thread processing devices can be arranged on the semiconductor chip. The semiconductor chip is characterized by functioning in cooperation with another processor that controls the operation of the multiple thread processing devices.

[0185] Goodon E. Moore, “Cramming more components onto integrated circuits”, Electronics, volume 38, Number 8, April 19, 1965. Masanet, E.; Shhehabi, A.; Lei, N.; Smith, S.; Koomey, J. Recalibrating global data center energy-use estimates. Science 2020, vol. 3667, 984―986.

[0186] The features of the present invention have been described above.

[0187] The technical scope of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

[0188] This will make it possible to provide a semiconductor chip that significantly reduces the power consumption of deep learning.

[0189] 1 is a diagram illustrating an example of a neuron. 2 is a diagram illustrating an example of a synapse. 3 is a diagram illustrating an example of the concept of a perceptron. 4 is a diagram illustrating an example of a thread operation. 5 is a diagram illustrating an example of a combination of thread operations. 6 is a diagram illustrating an example of a combination of thread operations. 7 is a diagram illustrating an example of a combination of thread operations. 8 is a diagram illustrating an example of a combination of thread operations. 9 is a diagram illustrating an example of a thread. 10 is a diagram illustrating a conventional example of a thread operation method. 11 is a diagram illustrating an example of a von Neumann bottleneck. 12 is a diagram illustrating an example of a method of multiply-accumulate operation when data C is on-chip. 13 is a diagram illustrating an example of a thread operation related to the present application. 14 is a diagram illustrating an example of a circuit for calculating threads related to the present application. 15 is a diagram illustrating an example of electrical characteristics of a non-volatile memory cell constituting an SB cell related to the present application. 16 is a diagram illustrating an example of a sequence for calculating threads related to the present application. 17 is a diagram illustrating an example of thread erase related to the present application. 18 is a diagram illustrating an example of a selective program related to the present application. 19 is a diagram illustrating an example of a selective program related to the present application. A diagram explaining an example of a selective program method related to the present application. A diagram explaining an example of a selective erase method related to the present application. A diagram explaining an example of a measurement method related to the present application. A diagram explaining an example of a product-sum operation related to the present application. A diagram explaining an example of a measurement method related to the present application. A diagram explaining an example of a product-sum operation method related to the present application. A diagram explaining an example of an interaction between a thread and an activation function. A diagram explaining an example of an SB cell related to the present application. A diagram explaining an example of the relationship between the charge accumulated in a capacitor and time. A diagram explaining an example of an idle state related to the present application. A diagram explaining an example of one learning cycle (one epoch) related to the present application. A diagram explaining an example of neural network calculation related to the present application. A diagram explaining an example of an equivalent circuit diagram related to the present application. A diagram explaining an example of an equivalent circuit diagram related to the present application. A diagram explaining an example of the characteristics of a weight cell related to the present application.1 is a diagram illustrating an example of the characteristics of a weight cell according to the present application; FIG. 2 is a diagram illustrating an example having a dummy cell; FIG. 3 is a diagram illustrating an example having a dummy cell; FIG. 4 is a diagram illustrating an example of a selective program method according to the present application.

Claims

1. A semiconductor device comprising a thread processing unit mounted on a semiconductor chip, the thread processing unit returning an output in response to an input, the input consisting of a plurality of input elements, the input elements being provided from outside the thread processing unit, the thread processing unit having weight cells, the weight cells being integrated on the semiconductor chip, and the output being related to the input and the state of the weight cells.

2. The semiconductor device according to claim 1, characterized in that the thread processing device has a plurality of cells and a sense amplifier, the cells have first, second, third and fourth terminals, the first terminal connected to a bit line, the second terminal connected to a control word line, the third terminal connected to a selected word line, and the fourth terminal connected to an addition data line, the addition data line having first and second select gates, and an addition capacitor between the first and second select gates, and the second select gate is sandwiched between the addition capacitor and the sense amplifier.

3. The semiconductor device according to claim 2, characterized in that the cell has a selection transistor, a selection capacitor, and the weight cell, the weight cell has a first source, a first drain, and a first gate, the first drain connected to the first terminal, the first gate connected to the second terminal, the selection transistor has a second source, a second drain, and a second gate, the second gate connected to the third terminal, and the second source connected to the fourth terminal, the selection capacitor has two terminals, one terminal of the selection capacitor connected to the fourth terminal, the other terminal of the selection capacitor connected to the second drain, and the first source and the second drain are connected.

4. The semiconductor device according to claim 1, characterized in that the weight cell has a charge storage region and a channel portion, and charged particles are exchanged between the channel portion and the charge storage region, and the state of the weight cell is changed by adjusting the amount of charge stored in the charge storage region.

5. The semiconductor device according to claim 2, wherein the thread calculation device calculates the output in accordance with a plurality of calculation steps, the plurality of calculation steps having a first step, the first step applying an erase voltage to the first terminal, applying a reference voltage to the second terminal, applying an on-voltage to the third terminal, applying an erase voltage to the fourth terminal, and applying an on-voltage to the first select gate.

6. The semiconductor device according to claim 5, wherein the plurality of operation steps include a second step, wherein the second step applies a reference voltage to the first terminal, applies an intelligent write voltage to the second terminal, applies an on-voltage to the third terminal, applies a reference voltage to the fourth terminal, and applies an on-voltage to the first select gate.

7. The semiconductor device according to claim 5, wherein the plurality of operation steps include a third step, which comprises applying an intelligent erase voltage to the first terminal, applying a reference voltage to the second terminal, applying an off voltage to the third terminal, setting the fourth terminal in a floating state, applying an off voltage to the first select gate, and applying an off voltage to the second select gate.

8. The semiconductor device according to claim 5, wherein the plurality of operation steps include a fourth step, which sets the first terminal in a floating state, sets the second terminal in a floating state, applies an off-voltage to the third terminal, sets the fourth terminal in a floating state, applies an off-voltage to the first select gate, and applies an on-voltage to the second select gate.

9. The semiconductor device according to claim 5, wherein the plurality of operation steps include a fifth step, which includes applying an input voltage to the first terminal, applying a read voltage to the second terminal, applying an off voltage to the third terminal, setting the fourth terminal in a floating state, applying an off voltage to the first select gate, and applying an off voltage to the second select gate.

10. The semiconductor device according to claim 5, wherein the plurality of operation steps include a sixth step, which comprises: placing the first terminal in a floating state; applying an off-voltage to the second terminal; applying an off-voltage to the third terminal; placing the fourth terminal in a floating state; applying an off-voltage to the first select gate; and applying an off-voltage to the second select gate.

11. The semiconductor device according to claim 5, wherein the plurality of operation steps include a seventh step, which comprises: placing the first terminal in a floating state; applying an off-voltage to the second terminal; applying an on-voltage to the third terminal; placing the fourth terminal in a floating state; applying an off-voltage to the first select gate; and applying an off-voltage to the second select gate.

12. The semiconductor device according to claim 5, wherein the plurality of operation steps include an eighth step, which comprises: placing the first terminal in a floating state; applying an off-voltage to the second terminal; applying an off-voltage to the third terminal; placing the fourth terminal in a floating state; applying an off-voltage to the first select gate; and applying an off-voltage to the second select gate.

13. The semiconductor device according to claim 5, wherein the plurality of operation steps include a ninth step, which comprises: setting the first terminal in a floating state; applying an off-voltage to the second terminal; applying an off-voltage to the third terminal; setting the fourth terminal in a floating state; applying an off-voltage to the first select gate; and applying an on-voltage to the second select gate.

14. A method for controlling a thread processing device mounted on a semiconductor chip, the thread processing device returning an output in response to an input, the input being composed of a plurality of input elements, the input elements being provided from outside the thread processing device, the thread processing device having weight cells, the weight cells being integrated on the semiconductor chip, the output being related to the input and a state of the weight cells, the output having first and second arguments, using an activation function according to the first argument to convert the output into an activated output element, comparing the first argument with a first integer, and if the first argument is smaller than the first integer, incrementing the first argument, the first integer being determined according to the second argument, comparing the second argument with a second integer, and if the second argument is smaller than the second integer, substituting the activated output element for the input element, and incrementing the second argument, and if the second argument is equal to or greater than the second integer, The output is a function Y.

15. The software according to claim 14, characterized in that: the function Y has a third argument; when the third argument is a third integer, the function Y takes a first value; the third argument is incremented; when the third argument is a fourth integer, the function Y takes a second value; the difference between the fourth integer and the third integer is 1; the residual between the first value and the second value is compared with a predetermined value; when the absolute value of the residual is deemed to be sufficiently small compared to the predetermined value, the function Y is deemed to have converged; and the third argument is deemed to be the number of epochs.

16. The semiconductor device according to claim 1, characterized in that the weight cell has two terminals, and the state of the weight cell is changed by applying an electrical, electromagnetic, or optical stimulus between the two terminals.

17. The semiconductor device according to claim 1, wherein the weight cells are capable of arbitrarily increasing the number of discretized states.

18. The semiconductor device according to claim 1, wherein the weight cells are adjacent to an arbitrary number of dummy cells.

19. The semiconductor device according to claim 3, wherein the capacitance of said addition capacitor is greater than the capacitance of said selection capacitor.

Images