Silicon brain

Abstract

To provide a semiconductor device in which a neural net is integrated in a silicon IC chip capable of saving power consumed by artificial intelligence.SOLUTION: The on-chip thread computation device (PresentMAD) acquires, as a first input, a datum B consisting of N external elements from outside the AI chip, acquires, as a second input, a datum C consisting of N weight elements from on-chip, computes these threads, and provides them as a datum A to outside the chip. The number of elements of the output is 1 because it is only necessary to calculate the scalar product. At this time, the access to the main memory is the power consumption PR when acquiring the data B from the main memory and the power consumption PS when outputting the data A of the calculation result. When it is assumed that PS is equal to power consumption PO of one overwriting to the main memory, the power consumption related to the scalar product of the row vector and the column vector of the number of elements N is N * PR + PO, and the power consumption can be reduced.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present invention relates to a technique for integrating a neural network on a silicon chip (IC chip). [Background technology]

[0002] Conventional semiconductor computing methods involve the cooperation of a memory device (memory) and a processing unit (a processor such as a CPU). A memory device (semiconductor memory) consists of a collection (array, cell array, memory cell array, or memory element array) of memory elements called memory cells (memory elements, bit cells, or simply cells or elements). Each element consists of at least a source, a drain, and a gate (or control gate). The source and drain can each be connected to a bit line. The gate is each connected to a word line. This connection is typically made through a contact (terminal), such as a word line contact (terminal) or a bit line contact (terminal). When a collection of such elements is distributed on a two-dimensional plane, each memory element is accessed via word lines (WL) and bit lines (BL) arranged in the mutually perpendicular X and Y directions of the two-dimensional plane. For example, the address of a memory element located at the intersection of the Ath word line and the Bth bit line is (A, B). This is called the address of the memory element. Here, A is specifically referred to as the address on the X axis (X address). B is specifically called the address on the Y axis (Y address).

[0003] For a long time, the mainstream of semiconductor memory technology development was to integrate as many memory elements as possible on the surface of a silicon wafer using semiconductor manufacturing processes in accordance with Moore's Law (see Non-Patent Document 1). However, in recent years (since 2015), it has become difficult to increase the integration density of memory elements on a two-dimensional plane, and the method of placing memory elements in three-dimensional space has become mainstream, even at the mass-produced level. In this case, addresses can be represented as (A, B, C), where C is the address on the Z axis (Z address), which is perpendicular to the XY plane.

[0004] However, whether it is two-dimensional or three-dimensional, the current method of recording information in semiconductor memory devices is based on memory elements, and when each memory element (cell) has two values, 0 and 1, it is said that one cell has a memory capacity (amount of information that can be stored) of 1 bit. If there are two such memory elements, the memory capacity is said to be 2 bits. In this case, there are four possible combinations of 0 and 1: (00), (01), (10), and (11). In this case, the number can be calculated by raising to the power of 2. If a cell array is made up of N memory elements, the memory capacity of the cell array is N bits. In this case, the number can be calculated by raising to the power of 2N.

[0005] Therefore, the amount of information (number of bits) in conventional semiconductor devices is expressed as the number of cases in logarithms with a base of 2. Even when so-called multi-value technology is used, the base of the logarithm simply becomes 4 or 8, and such logarithms can always be converted to logarithms with a base of 2, so even when multi-value technology is used, information is still expressed in bits.

[0006] In contrast, the human brain is not made up of memory elements. If there were something that corresponds to a memory element, it would be the cell body that makes up part of the nerve cell, but this cell body does not store information such as 0 or 1.

[0007] As shown simply in Figure 1, a nerve cell (neuron) generally consists of three parts: a cell body, multiple (e.g., dozens) dendrites, and an axon. The cell body can receive external input from these multiple dendrites. The axon generally extends longer than the dendrites, and its tip further branches into tens to hundreds of branches. The tip of these branched axons is called the axon terminal.

[0008] As shown simply in Figure 2, the axon terminal approaches one of the dendrites of another cell body and forms a junction called a synapsis.

[0009] There are now two cell bodies, cell A and cell B. Cell body A receives multiple external inputs x(n) through multiple dendrites (n), where n is an integer between 1 and N. Cell body A assigns a weight W(n) to each input x(n). The signal summed according to these weights is called SUM. SUM is transferred to one of the axon terminals via the axon. When SUM exceeds a certain threshold (threshold of exitation), the neuron generates an action potential, driving synapsis and transmitting neurotransmitters from cell body A to cell body B.

[0010] This threshold changes as signals are transmitted repeatedly. In other words, by repeatedly learning from experience, synaptic connections become stronger, break, or are replaced. The strengthening of synaptic connections can be explained by a lowering of the threshold. The disconnection of synapses can be explained by a rising of the threshold. The replacement of synapses can be explained by a change in the distribution of synaptic thresholds.

[0011] This is modeled as shown in Figure 3. When neurotransmitters are transmitted, the output y is set to 1 (y=1), and when not, it is set to y=0. This model is called a perceptron, and is widely used in deep learning and machine learning.

[0012] There are two main ways to implement a perceptron on a computer.

[0013] In the traditional method, the input x(n), synaptic weight w(n), SUM, threshold of excitation, and output y are all represented as bits, i.e., a computer program.

[0014] The problem with this method is that it places a heavy load on computers. There is a greater need than ever before to improve calculation speeds and reduce power consumption. Deep learning and machine learning require the instantaneous processing of huge amounts of data, and if calculations that place a heavy load were to flood the world, the power consumption of data centers would increase explosively, making it difficult to operate them practically. Furthermore, there are growing concerns that this could accelerate global warming (see Non-Patent Document 2).

[0015] The main cause of the limit on computing speed is excessive data communication between the processor and main memory. Although processors can still be made faster, the communication speed of the data bus between the processor and main memory has reached a ceiling. This is called the von Neumann bottleneck (or memory bus problem).

[0016] The main reason for this increase in power consumption is that the main memory devices currently in use are volatile memory devices called dynamic random access memory (DRAM), which means that the power consumed by refreshing recorded data is becoming non-negligible.

[0017] A recent trend is to replicate the perceptron directly on a semiconductor chip in order to avoid the von Neumann bottleneck and simultaneously reduce power consumption. However, the neural networks of the human brain are generally designed to generate synapses between two unspecified pairs of neurons. While current semiconductor technology makes it possible to place perceptrons at precisely specified addresses on a two-dimensional plane or in three-dimensional space, it is not easy to replicate synapses between arbitrary pairs of neurons or to freely reassign them according to learning.

[0018] Generally speaking, a neural network refers to a network of neural circuits. Originally, it refers to a network with countless synapses, as shown in Figure 2, and it actually exists in living organisms. A model that includes the neurons and synapses in Figure 1 is the perceptron in Figure 3.

[0019] If we extract the part of the perceptron in Figure 3 enclosed by the dotted line, it will look like Figure 4. However, let's say the number of dendrites N is 4, the input is x(p), and the weight is w(p). Here, p is an integer between 1 and N, and refers to an individual dendride. The sum of the products of the input x(p) and the synaptic weight w(p) from 1 to N is SUM in the perceptron in Figure 3, and t(1,1) in Figure 4.

[0020] The parts not enclosed by dotted lines in Figure 3, i.e., the parts related to action potentials, are missing in Figure 4. Nevertheless, in Figure 5, t(2,1) and t(3,1) are added to the second column, and t(1,2) is added to the third column. The arrow from the first column to the first row of the second column means that the input x(p) is multiplied by the weight w(p,1,1) and then added together in the first row of the second column. This means calculating the scalar product (sum-of-products operation) of the row vector with x(p) as an element and the column vector with w(p,1,1) as an element. The result of this calculation is t(1,1).

[0021] The reason there are three weight arguments is because we added 1 to represent the first column to the second column, and 1 to represent the sum of the first row of the second column. In other words, the weight from x(p) to t(1,1) is w(p,1,1).

[0022] In Figure 5, the input from the second column to the first row of the third column is further calculated. That is, the arrow from the second column to the first row of the third column means that the input t(q,1) is multiplied by the weight w(q,1,2) and then added together in the first row of the second column. This means that the scalar product (product-sum operation) of the row vector with t(q,1) as an element and the column vector with w(q,1,2) as an element is calculated. The result of this calculation is t(1,2).

[0023] In Figure 6, t(2,2) and t(3,2) are added to the third column, and y is added to the fourth column. The arrow from the third column to the first row of the fourth column (i.e., y) means that the input t(r,2) is multiplied by the weight w(r,1,3) and then added together. This means calculating the scalar product (sum of products) of the row vector with t(r,2) as an element and the column vector with w(r,1,3) as an element. The result of this calculation is y.

[0024] In Figure 7, we also take into consideration the arrow from x(p) in the first column to the second row in the second column. Although not shown in the figure, the newly added arrow is assigned a weight w(p,2,1).

[0025] In Figure 8, we also take into consideration an arrow from x(p) in the first column to the third row in the second column. Although not shown in the figure, the newly added arrow is assigned a weight w(p,3,1).

[0026] In Figure 9, we also consider an arrow from t(q,1) in the second column to row 2 in the third column. Although not shown in the figure, the newly added arrow is assigned a weight w(q,2,2).

[0027] In Figure 10, we also take into account the arrow from t(q,1) in the second column to row 3 in the third column. Although not shown in the figure, the newly added arrow is assigned a weight w(q,3,2). As can be seen, the column number (k) increases as we move to the right in the figure, and at the same time, the layer depth of learning deepens. In other words, the column number (k) is related to the layer depth of learning.

[0028] In neural networks, which are commonly used in the field of artificial intelligence, the first column in Figure 10 is called the input layer. The final column in Figure 10 (for example, column 4) is called the output layer. The multiple layers between the input and output layers (for example, columns 2 and 3) are called hidden layers. When there are multiple hidden layers (number of layers, or number of hidden layers), the system is called deep learning. Deep learning is thought to achieve higher performance as the number of hidden layers increases. An epoch is the period from the input layer to the calculation of the output y. The weights are readjusted and the output y is recalculated from the same input until the desired value is reached. Various feedback methods are used to bring the output y closer to the desired value. Learning is completed after repeated epochs, when the desired output y is finally obtained. Therefore, the learning time is calculated by multiplying the time required to process one epoch by the number of epochs required to complete learning. The time required to process one epoch is determined by the chip's ability to process countless thread operations (multiply-and-accumulate operations) in parallel.

[0029] The left diagram in Figure 11 is similar to Figure 4. The arrow from the kth column to the jth row of the (k+1)th column represents the input t(p,k) multiplied by the weight w(p,j,k+1) to obtain the jth row of the (k+1)th column. This represents the calculation of the scalar product (thread operation or multiply-and-accumulate operation) of the row vector with t(p,k) as an element and the column vector with w(p,j,k+1) as an element. The result of this calculation is t(j,k+1). This calculation result can be provided externally to the AI ​​chip and stored in main memory or storage. In other words, the column number k represents the layer number of the hidden layer. However, when k = 0, it is the input layer. Also, when k = M, it is the output layer. The number j represents the output element number (output element number) of the (k+1)th layer. The number i represents the input element number (input element number) of the kth layer.

[0030] If the thread processing unit that calculates t(j, k+1) is T(j, k), it can be regarded as the element in the jth row and kth column of the thread matrix. In other words, T(j, k) is the device (or circuit) that calculates the thread in the jth row of the (k+1)th layer. In the right diagram of Figure 11, the thread processing unit T(j, k) is placed at the position on the semiconductor chip determined by j for a given k. In other words, j and the position (or placement) on the semiconductor chip are related. In the example on the right of Figure 11, j searches for a position from the top left to the right on the chip, and when it reaches the edge, it moves down one row and searches again from left to right. Generally, the number of rows j of the output in the (k+1)th column is an integer between 1 and L(k+1). However, since the number of elements (rows) L in each column (each layer) is different for each k, it is an integer function L(k) with k as an argument. That is, L(k+1) threads t(j, k+1) are input to the thread calculation device T(j', k+1) in the (k+1)th layer, and thread t(j', k+2) can be calculated as the output to row j', column (k+2). Here, j' is an integer between 1 and L(k+2). Also, k is an integer between 1 and M. In this case, the number of elements of the weight w in FIG. 11 is L(k+1)L(k+2), which is obtained by multiplying the number of rows in the input layer by the number of rows in the output layer. That is, it is the product of L(k+1) and L(k+2). In the example of FIG. 10, L(0)L(1)L(2)=36 and M=3. However, L(M)=1.

[0031] Let's return to Figure 10. The hidden layer is made up of multiple overlapping thread operations. The total number of hidden layers is given by M-1. However, in Figure 10, M = 3, i.e., two columns, is used as an example. Therefore, when calculating T(j, k+1), L(k) is the number of elements input to the thread processing unit. Therefore, the number of elements of the weight w when transferring from the kth column to the k+1th column in the hidden layer is L(k)L(k+1), that is, the product of L(k) and L(k+1). Since the integer k is repeated from 0 to M, the total number of elements of the weight w required to calculate the output y is L(0)L(1)L(2)...L(M-1). This is the product of multiplication from L(0) to L(M). However, since the number of outputs y is 1, we set L(M) = 1. Storing all weight elements in main memory, etc., would require a significantly larger bit capacity for complex deep learning.

[0032] To calculate the output y, first, L(0) inputs are used to calculate (thread operation) L(1) threads, t(1,1), t(2,1)… t(L(1),1). Next, these are used as inputs to calculate (thread operation) L(2) threads, t(1,2), t(2,2)… t(L(2),2). This process is repeated from k=1 to M to calculate the output y. Calculating the output y is essentially the same as calculating the threads. Thus, the number of thread operations required to calculate the output y using deep learning is L(1) + L(2) + … + L(M-1) + L(M), where L(M)=1. In the example in Figure 10 (L(1) = 3, L(2) = 3, L(3) = 1, M=3), a total of 7 threads are required (3 + 3 + 1).

[0033] Next, we will discuss the Graphic Processor Unit (GPU), which is often used in image analysis.

[0034] Generally, a GPU consists of multiple cores. For example, it consists of 1000 cores with 512 bits each. Thus, the number of cores is much greater than that of a typical CPU (around 10).

[0035] GPUs are processor chips originally specialized for image processing, and because they have such a large number of cores, they are better suited than CPUs (central processing units) for parallel processing of multiple rotation matrices. The more complex the 3D images being processed, the more rotation matrices need to be processed in parallel. Calculating a 3D rotation matrix involves multiplying a 3x3 rotation matrix by a 3D column vector (pre-rotation column vector). The result is a 3D column vector (post-rotation column vector).

[0036] The first component of the post-rotation column vector is the scalar product (thread operation, or multiply-and-accumulate operation) of the first row of the rotation matrix and the pre-rotation column vector. In other words, it can be calculated using the thread T (a, b) mentioned above. As an example, if N=3, the following calculation result is obtained: T (a, b) = t (1, b) × w (1, a, b+1) + t (2, b) × w (2, a, b+1) + t (3, b) × w (3, a, b+1). This calculation result is provided outside the chip and can be saved in main memory or storage as needed.

[0037] The second component of the rotated column vector is the scalar product (thread operation, or multiply-and-accumulate operation) of the second row of the rotation matrix and the pre-rotation column vector. In other words, it can be calculated using the thread T(c,d) mentioned above. As an example, if N=3, the following calculation result is obtained: T(c,d) = t(1,d) × w(1,c,d+1) + t(2,d) × w(2,c,d+1) + t(3,d) × w(3,c,d+1). This calculation result is provided outside the chip and can be stored in main memory or storage.

[0038] The third component of the post-rotation column vector is the scalar product (thread operation, or multiply-and-accumulate operation) of the third row of the rotation matrix and the pre-rotation column vector. In other words, it can be calculated using the thread T(e, f) mentioned above. As an example, if N=3, the following calculation result is obtained: T(e, f) = t(1,f) × w(1,e,f+1) + t(2,f) × w(2,e,f+1) + t(3,f) × w(3,e,f+1). This calculation result is provided outside the chip and can be stored in main memory or storage.

[0039] As such, three threads (sum of products) are required to calculate one 3D rotation matrix. In other words, both deep learning and image analysis of complex 3D shapes (countless 3D rotation matrices) can be performed by parallel processing of countless threads (sum of products). In other words, the computational processing required for image analysis and deep learning is almost identical. Due to this similarity in the computations between image analysis and deep learning, GPUs are widely used in deep learning. For example, calculations related to one thread can be delegated to one core. Alternatively, calculations related to two or three threads can be combined and delegated to one core. In either case, GPUs are processors better suited to deep learning than CPUs because they have an overwhelmingly greater number of cores than CPUs.

[0040] However, GPUs consume very high amounts of power, ranging from 500W to around 1kW. Increasing power consumption is a problem for AI servers that utilize countless GPUs. One solution is to dramatically increase computational speed. For example, even if the power consumption per chip increases threefold, if the chip's processing speed increases 100 times, power consumption can be reduced to 3%. In other words, a 97% reduction in power consumption is expected. However, a 100-fold increase in processing speed means that 100 times the AI ​​can be used for the same cost. In other words, humans have inherent desires, and in the end, we end up using 100 times the AI, dramatically increasing power consumption by three times. Therefore, this method will ultimately not be able to reduce the power consumption of AI servers.

[0041] Another method is to reduce the amount of information handled. GPUs handle subtle color tones and brightness in addition to pixel coordinates, so they handle fixed-length data of 8, 16, and 32 bits, as well as floating-point numbers, all together. However, if you are only performing threaded operations specialized for deep learning, it is sufficient to handle only fixed-length 8-bit data. This means that the amount of information handled to perform the same threaded operations can be significantly reduced. The Neural Network Processing Unit (NPU) was developed based on this concept. The power consumption of an NPU is about one-tenth that of a GPU. In other words, reducing the amount of information handled in threaded operations is an effective way to reduce power consumption. DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0042] The present invention has been made in view of the above circumstances, and provides a method for reducing power consumption in an information processing system using a neural network. [Means for solving the problem]

[0043] In order to solve the above problems, the present invention employs the following means. The solution proposed by the present invention is A thread processing unit mounted on a semiconductor chip is used as a component, The thread processing unit returns an output in response to an input, the input comprises a plurality of input elements; The input element is provided from outside the thread processing device, The thread processing unit has a weight cell, The weight cells are integrated on the semiconductor chip; the output is related to the input and the state of the weight cell; It is characterized by the fact that

[0044] Furthermore, The thread processing unit includes a plurality of cells and a sense amplifier. the cell has first, second, third, and fourth terminals; the first terminal is connected to a bit line; the second terminal is connected to a control word line; the third terminal is connected to a selected word line; the fourth terminal is connected to an addition data line; the summing data line has first and second select gates; a summing capacitor between the first and second select gates; the second select gate is sandwiched between the summing capacitor and the sense amplifier; The cell includes a selection transistor, a selection capacitor, and the weight cell; the weight cell has a first source, a first drain, and a first gate; the first drain is connected to the first terminal; the first gate is connected to the second terminal; the select transistor has a second source, a second drain, and a second gate; the second gate is connected to the third terminal; the second source is connected to the fourth terminal; The selection capacitor has two terminals; one terminal of the selection capacitor is connected to the fourth terminal; the other terminal of the selection capacitor is connected to the second drain; The first source and the second drain are connected. It is characterized by: [Effects of the Invention]

[0045] According to the present invention, by simulating part of the neural network functions within a silicon chip (IC chip), it is possible to reduce the power consumed by artificial intelligence. The best mode for carrying out the invention will now be described in detail. BEST MODE FOR CARRYING OUT THE INVENTION

[0046] First, let's look at how threaded operations consume power. (Mechanism of multiply-and-accumulate operation)

[0047] Next, the mechanism of the sum-of-products operation (or thread operation) will be explained with reference to FIG.

[0048] First, as an example, prepare three pieces of data (A, B, C). Data A is the first input to the adder. Data B is the first input to the multiplicator. Data C is the second input to the multiplicator. The output of the multiplier, i.e., the product of data B and data C, becomes the second input to the adder. The output of the adder is again assigned to data A. This is repeated the number of times (N) that is the number of input elements.

[0049] FIG. 13 is a diagram showing how data is exchanged between an AI chip such as a GPU or NPU and a main memory such as a DRAM.

[0050] Every time a processor such as an AI chip accesses main memory, power is consumed through the memory bus. In Figure 12, an assignment to data A is an overwrite in main memory. Data A is read from main memory and used as the first input to the adder. Data B is read from main memory and retrieved. Data C is read from main memory and retrieved.

[0051] Let PO be the power consumed for one overwrite to main memory. Let PR be the power consumed each time data is retrieved from main memory. In this case, the power consumed for calculating the scalar product of a row vector and a column vector with N elements (thread operation, or multiply-and-accumulate operation) is N x (PO + PR + PR + PR). (First embodiment)

[0052] What would happen if the input C to the multiplier could be done on-chip, that is, inside the AI ​​chip without using main memory? As shown in Figure 14, the number of times data is retrieved from main memory per multiplication would be reduced from three to two. Therefore, the power consumption for the scalar product (thread operation, or multiply-and-accumulate operation) of a row vector and a column vector with N elements would be N × (PO + PR + PR).

[0053] The weights w(i, j, k) in neural network calculations correspond to the data C. As there are three integer arguments (i, j, k), it is a three-dimensional matrix. In deep learning, k becomes large, and as the input and output of each thread calculation become larger, i and j also become larger. In other words, a large number of elements of data C must be handled. For this reason, the system corresponding to Figure 12 (a conventional example) uses HBM (High-Band Width Memory), which is a vertical stack of multiple DRAMs connected via TSV, as the main memory.

[0054] Therefore, in order to handle data C on-chip, the AI ​​chip and HBM must be integrated into a single chip, which is quite difficult.

[0055] This application attempts a different approach. As shown in FIG. 15, data B consisting of N external elements is acquired from outside the chip (or AI chip) as a first input. N weight elements are acquired from on-chip as a second input. These threads are calculated, and the result is output as data A. Data A can be provided externally to the chip (or AI chip). The number of output elements is set to 1, since all that is required is to calculate a scalar product. In this case, the main memory accesses are PR when acquiring data B and PS when outputting data A. Assuming that PS and PO are equal, the power consumption for the scalar product (thread operation or multiply-and-accumulate operation) of a row vector and a column vector with N elements is N × PR + PO. In this way, it is possible to reduce power consumption without relying on the integrated main memory. This represents an example of the effect of the thread operation device (Present MAD) related to this application.

[0056] In other words, by using this application, it is possible to reduce power consumption by 2N×PR + (N-1)×PO each time the calculation of the scalar product (product-sum operation) of a row vector and a column vector with N elements is performed as a thread operation.

[0057] Therefore, if the same thread operation is repeated K times to execute one epoch, power consumption per epoch can be saved by K × (2N × PR + (N-1) × PO)).

[0058] Therefore, if the number of epochs required to complete one learning is Q, then it is possible to save Q×K×(2N×PR+(N-1)×PO)) of power consumption to complete one learning. (Second embodiment)

[0059] Fig. 16 is an example of an equivalent circuit diagram for explaining the first embodiment. By using this circuit, it is possible to enjoy the benefits of Fig. 15. That is, Fig. 16 is an example of an equivalent circuit of a thread processing device.

[0060] That is, this circuit makes it possible to calculate t(j, k+1) (thread calculation or product-sum calculation) from t(1, k), t(2, k), ... t(N-1, k), and t(N, k) in Figure 11, taking into account the weights w(1, j, k+1), w(2, j, k+1), ... w(N, j, k+1). That is, this equivalent circuit is for calculating thread T(j, k) in Figure 11. When calculating that thread, it is possible to significantly reduce power consumption, as explained in Figure 15.

[0061] First, input t(i, k) can be input through N appropriately selected bit lines BL(i), where i is an integer between 0 and N-1, and N is an integer greater than or equal to 1. The total number of bit lines in Figure 16 is NN, where NN is generally an integer greater than or equal to N. Each bit line is connected to a bit line selection gate SBL(i). Therefore, N inputs can be conveniently selected from the NN input candidates. The selected N bit lines correspond to the dendride through which the Soma Body receives stimuli (input) from the outside (other neurons).

[0062] One cell (silicon brain cell, SB cell) corresponds to each bit line. Therefore, the equivalent circuit of this embodiment is composed of NN SB cells, the same number as the number of bit lines. That is, in this embodiment, one thread is modeled with NN SB cells. Each SB cell consists of a weight cell (for example, a type of non-volatile memory cell), a selection transistor, and a selection capacitor.

[0063] The weight cell that constitutes part of the SB cell is, for example, a nonvolatile memory cell, and has a charge storage region such as a charge trap layer or a floating gate that can store charge even without a power source.

[0064] The nonvolatile memory cell (i.e., an example of a weight cell) comprises a source, a drain, a control gate, and a channel portion in addition to the charge storage region. The drain of the nonvolatile memory is connected to a corresponding bit line, and the control gate is connected to a control word line (CWL).

[0065] The select transistor that constitutes part of the SB cell is a type of transistor that consists of a source, a drain, and a select gate.

[0066] The source of the select transistor is connected to an add data line (ADL), the drain of the select transistor is connected to a source of the nonvolatile memory, and the gate of the select transistor is connected to a select word line (SWL).

[0067] A select capacitor, which is part of an SB cell, is associated with the select transistor and has two terminals, one connected to the drain of the select transistor and the other connected to the source of the select transistor.

[0068] Thus, the N SB cells are connected in parallel to the summing line.

[0069] The summing line has a left selection gate (SGL) and a right selection gate (SGR), and the source side terminals of the NN SB cells are arranged between the left selection gate and the right selection gate.

[0070] Between the left select gate and the right select gate, there is further provided an adder capacitor (ADC) which serves as an adder (ADDER) according to the present invention. The location of the adder capacitor can be anywhere between SGL and SGR, and it can also be placed between two bit lines.

[0071] The summing capacitor has two terminals, one of which is connected to the summing line, and the other terminal (arrow) can be connected to any circuit, for example, the source line.

[0072] A sense amplifier (S / A) is installed outside the right select gate (the side without the source terminal of the SB cell), and the right select gate is between the sense amplifier and the summing capacitor.

[0073] In general, the capacitance of the summing capacitor must be greater than the capacitance of each of the selection capacitors. For example, the capacitance of the summing capacitor must be greater than NN times the capacitance of the selection capacitor. (Third embodiment)

[0074] It is desirable that the nonvolatile memory constituting the SB cell is specially adjusted to have a large S-factor.

[0075] Figure 17 shows an example of the voltage-current characteristics of two types of nonvolatile memory cells (NV). Gate voltage is the voltage applied to the control gate. Drain current is the current that flows through the drain in response to the applied gate voltage.

[0076] The left corresponds to a typical nonvolatile memory cell with a sufficiently small S-factor, while the right corresponds to the nonvolatile memory cell (i.e., a weight cell) of this application, with a sufficiently large S-factor. In both cases, writing (P) and erasing (E) are possible repeatedly. The mechanism is essentially the same, and Fowler-Nordheim tunneling (FNT) is used as an example. First, consider a nonvolatile memory cell that includes a channel (or substrate) and a charge storage region (such as a floating gate or charge trap layer). When FNT occurs, charged particles (such as electrons or holes) move between the channel and the charge storage region.

[0077] In other words, FNTs cause charge transfer between the channel and charge storage regions within a nonvolatile memory cell (or weight cell). For example, electrons in the channel move to the charge storage region. Conversely, electrons in the charge storage region move to the channel. Alternatively, holes in the channel move to the charge storage region. Conversely, holes in the charge storage region move to the channel. One of the main features of this application is that the state change using FNTs (in this example, the change between the written state and the erased state) is a change in the number of charged particles (or charge) accumulated in the charge storage region. Therefore, to change the charge accumulated in the charge storage region within a nonvolatile memory cell (or weight cell), it is sufficient to simply move electrons or holes within the weight cell, and it is not necessarily required to pass large currents through the bit line or word line. Therefore, this application is characterized by extremely low power consumption for changing the state of the weight cell.

[0078] For example, in the current-voltage characteristics of a typical nonvolatile memory cell (left), when writing (P), the threshold voltage rises and the characteristic curve shifts to the right. Conversely, when erasing (E), the threshold voltage falls and the characteristic curve shifts to the left. This change in threshold voltage occurs due to changes in the amount of charge accumulated in the charge storage region mentioned above. In the written state (0), the S factor is small, so the current rises sharply at the threshold voltage Vt0. In the erased state (1), the S factor is small, so the current rises sharply at the threshold voltage Vt1. Because the current rises so sharply, there is a region (encircled by the dotted line) between Vt1 and Vt0 where no current flows in the written state (0) but does in the erased state (1). The wider this region is, the better.

[0079] In the current-voltage characteristics of a weighted cell (a nonvolatile memory cell (right) as an example) related to this application, when writing (P) is performed, the threshold voltage rises and the characteristic curve shifts to the right. Conversely, when erasing (E) is performed, the threshold voltage falls and the characteristic curve shifts to the left. This change in threshold voltage occurs due to the change in the amount of charge accumulated in the charge storage region mentioned above. In the written state (0), the S factor is large, so the current rises gradually at the threshold voltage Vt0. In the erased state (1), the S factor is large, so the current rises gradually at the threshold voltage Vt1. Because the current rises gradually like this, even in the written state (0), a certain amount of current flows between Vt1 and Vt0. This characteristic differs from ordinary nonvolatile memory and is suitable for recording state changes in an analog manner.

[0080] In this embodiment, such analog nonvolatile memory cells (ANV) are used as weight cells that constitute part of the SB cells. In other words, it is desirable that all of the NN nonvolatile memory cells (weight cells) in FIG. 16 are ANVs.

[0081] There are several ways to increase the S-factor of the weight cell or analog nonvolatile memory cell (ANV), but care must be taken not to increase the S-factor of the select transistor. For example, it is desirable to take measures such as shortening the gate length of the ANV, making the diffusion layer of the ANV deeper than usual, or reducing the amount of on-injection into the ANV channel. (Fourth embodiment)

[0082] FIG. 18 is a diagram illustrating how the circuit of FIG. 16 operates to obtain thread T(j, k).

[0083] First, all nonvolatile memory cells in the entire thread are erased at once (Thread Erase).

[0084] Next, programming is performed on only the nonvolatile memory cells associated with the bit lines selected in the thread. In this selective programming, it is desirable to gradually increase the threshold voltage Vt by dividing it into multiple short pulses so that the increase in Vt can be adjusted.

[0085] Next, if necessary, erase only the nonvolatile memory cells related to the selected bit lines in the thread. In this selective erase (Selective Program), it is desirable to gradually decrease the threshold voltage Vt by dividing it into multiple short pulses so that the decrease in Vt can be adjusted.

[0086] However, if the precision of the selective writing is sufficient, the selective erasure can be omitted.

[0087] Next, the first measurement (Measure 1) is performed. What is measured here is not the threshold voltage Vt of the nonvolatile memory cell. What is measured in this application is the charge stored in the summing capacitor (ADC). If the situation is properly arranged, the charge on the summing capacitor can be measured by the sense amplifier (S / A) as the potential of the ADL. In this first measurement (Measure 1), the amount of charge before the adder is activated is measured. As an example, let's call this value Q1.

[0088] Next, the adder is operated. The operation steps are divided into four stages (Ope1, Ope2, Ope3, Ope4).

[0089] After activating the adder, a second measurement (Measure 2) is immediately taken to obtain Q2.

[0090] The value of T(j, k) is obtained from the value measured the second time (Q2) and the value measured the first time (Q1). However, the value of T(j, k) is the value of a function with Q2 and Q1 as arguments.

[0091] The function can be selected from various options depending on convenience, such as the difference between Q2 and Q1, the square of that difference, the ratio of the logarithm of Q2 to the logarithm of Q1, etc. In this embodiment, the difference obtained by subtracting Q1 from Q2 is used as an example.

[0092] The value of T(j, k) obtained in this way is stored in the main memory as data A in Fig. 15. It is then read out as data B as needed.

[0093] The data corresponding to the data C in FIG. 15 is stored in the weight cell. An example is the threshold voltage value (Vt) written to the analog nonvolatile memory cell (ANV) in FIG.

[0094] Even an NPU (fixed length 8 bits) that consumes less power than a GPU still requires writing, erasing, and reading from 8 bits in main memory to handle one element of the weight w(i, j, k). This consumes the necessary power. A GPU allocates more bits to each weight element, so power consumption is even higher. Also, as mentioned above, the number of weight elements is L(1)L(2) ...L(M). In other words, the deeper the number of hidden layers (M-1) is, and the larger the number of inputs and outputs in each column, L(k), the higher the power consumption.

[0095] In contrast, in this application, only one bit of the analog nonvolatile memory (ANV) of the SB cell is allocated to handle one element of the weight w(i, j, k). As shown in the right diagram of Figure 17, by continuously changing the Vt of the ANV, the drain current, which is the output for a given gate voltage, changes continuously. This is the same as a continuously variable resistance value. Therefore, one bit of the ANV can correspond to analog data, which means that it can also handle fixed-length data longer than 8 bits. This is another advantage of this application.

[0096] FIG. 19 is a table for explaining an example of a method for executing the thread erase of FIG.

[0097] A reference voltage Vref (for example, 0V) is applied to the control word line CWL, and an on-voltage Von is applied to the selected word line SWL. Here, Von is a voltage that switches on the select transistor. Furthermore, an erase voltage Vers (for example, 20V) is applied to the addition data line ADL, and Von is applied to the left select gate SGL. Furthermore, Von is applied to all bit line select gates SBL(*), where * is an integer from 0 to NN-1. In this way, all bit lines BL are selected and an erase voltage Vers (for example, 20V) is applied.

[0098] FIG. 20 is a table for explaining an example of a method for executing the selective program of FIG.

[0099] An intelligent write voltage Vint.pgm is applied to the control word line CWL, and an on-voltage Von is applied to the selected word line SWL. Furthermore, a reference voltage Vref (for example, 0V) is applied to the addition data line ADL, and Von is applied to the left select gate SGL. Furthermore, Von is applied to all bit line select gates SBL (**), where ** is the number of the selected bit line (selected bit line), an integer between 0 and NN-1. In this way, a reference voltage Vref (for example, 0V) is selectively applied to the selected bit line BL.

[0100] However, the intelligent write voltage Vint.pgm is a voltage applied to the control gate for writing data to the nonvolatile memory cell in small increments.

[0101] An example is shown in Figure 21. In this example, a potential (Vpulse) of the same magnitude is repeatedly applied. By adjusting the pulse potential, the time for applying the pulse, i.e., the pulse period, the standby period between pulses, the number of times the pulse is applied, etc., the write (P) on the right side of Figure 17 is carried out little by little.

[0102] Another example is shown in Figure 22. In this example, pulses are applied while gradually increasing the pulse potential. By adjusting the pulse potential increase width (pulse shift), pulse time (pulse period), standby period between pulses, number of pulse applications, etc., the write (P) on the right side of Figure 17 is performed little by little.

[0103] There are various other methods for applying pulses, but in any case, the purpose is to gradually adjust the change in Vt caused by writing as shown in the right diagram of Figure 17. Figure 40 shows yet another example, in which the potential of each pulse is lowered as the change in Vt approaches the target value.

[0104] FIG. 23 is a table for explaining an example of a method for performing the selective erase of FIG.

[0105] A reference voltage Vref (for example, 0V) is applied to the control word line CWL, and an off-voltage Voff is applied to the selected word line SWL. Furthermore, the addition data line ADL is set to a floating state, and Voff is applied to both the left select gate SGL and the right select gate SGR. The ADL is set to a floating state to prevent it from unnecessarily affecting selective erasure. Furthermore, Von is applied to the bit line select gate SBL (**) of the selected bit line. ** is the integer number of the selected bit line (selected bit line) between 0 and NN-1. In this way, the intelligent erase voltage Vint.ers is selectively applied to the selected bit line BL.

[0106] The intelligent erase voltage Vint.ers is a voltage applied to the control gate for gradually erasing the nonvolatile memory cell. Examples of Vint.ers are similar to those shown in Figures 21, 22, and 40, so detailed explanations will be omitted.

[0107] FIG. 24 is a table illustrating an example of a method for performing an initial measurement (measure 1).

[0108] 16, when charge accumulates in the summing capacitor (ADC), the potential of the summing data line ADL changes. While SGL is off and SGR is on, the sense amplifier (S / A) can measure this potential change.

[0109] The control word line CWL is set to a floating state. This is to prevent CWL from having an undesired effect on the measurement. Furthermore, an off voltage Voff is applied to the selected word line SWL, and Voff is initially applied to both the left select gate and the right select gate SGR. Furthermore, Voff is applied to all bit line select gates. In other words, all bit lines are unselected. To ensure that any bit line does not have an undesired effect on the measurement, the bit lines are set to a floating state. Thus, the sum data line ADL is set to a floating state. Next, the voltage applied to SGR is changed from Voff to Von, which opens SGR and allows the sense amplifier S / A to measure the initial potential of ADL. This potential is related to the charge Q1 stored in the summing capacitor at this point.

[0110] 25 is a table showing an example of a method for executing a thread operation. In this example, the thread operation is divided into four steps (Ope-1, Ope-2, Ope-3, Ope-4).

[0111] First, in Operation 1, Vread is applied to the control word line CWL, and Voff is applied to the selected word line SWL. Here, Vread is the read voltage applied to the control gate to read the Vt of the analog nonvolatile memory cell (ANV). Generally, Vread should be within the voltage region where the current-voltage characteristics change smoothly, as shown in the right diagram of Figure 17. If the S factor is sufficiently large, Vread should be higher than Vt0 in the diagram. It should also be lower than the voltage at which the current rise in the erased state saturates. The Vread shown in the right diagram of Figure 17 satisfies the above conditions.

[0112] Furthermore, the addition data line ADL is set to a floating state, Voff is applied to the left selection gate SGL and the right selection gate SGR, and Voff is also applied to the selected word line SWL.

[0113] Furthermore, all bit line select gates SBL are set to Voff. Next, input voltages Vin(0), Vin(1), ..., Vin(N-1) are applied to the selected bit lines. Here, Vin corresponds to the inputs t(1,k), t(2,k), ..., t(N,k) in the left diagram of Figure 11. Here, N is generally an integer less than or equal to NN. Thus, since Vin is an input signal, it does not need to be fixed at a constant value.

[0114] To select N bit lines, the voltage applied to the corresponding N bit line select gates SBL is changed from Voff to Von. Thus, Vin is applied to the drain of the analog nonvolatile memory cell (ANV) of the SB cell selected by the selected bit (selected SB cell) through each selected bit line. Since the ANV is a weight cell, which is equivalent to a variable resistor in this example, while Vread is applied to the CWL, charge flows into the selection capacitor in the selected SB cell according to the Vt of Vin and ANV. While Voff is applied to the selected word line SWL, the selection transistor in the selected SB cell is turned off, so this charge is stored in the selection capacitor for a certain period of time.

[0115] Next, in Operation 2, the voltage applied to the CWL is changed from Vread to Voff, and the voltage applied to the selected SBL is changed from Von to Voff. To avoid any unnecessary effects on the ADL, the selected bit line is set to a floating state. This is a maintenance state that maintains the charge on the selected capacitor.

[0116] Subsequently, in Operation 3, the voltage applied to the selected word line SWL is changed from Voff to Von, causing the charge in the selected capacitor to move to the addition capacitor (ADC) through the addition data line ADL.

[0117] Subsequently, in Operation Operand 4, first, the voltage applied to the selected word line SWL is changed from Von to Voff.

[0118] Next, in Measure-2, the voltage applied to the right select gate SGR is changed from Voff to Von. Here, a sense amplifier is located outside (to the right of) SGR. This allows the sense amplifier to sense the potential of the sum data line ADL. Since the potential of the sum data line ADL is determined by the charge stored in the summing capacitor, the sense amplifier can sense the charge Q2 of the summing capacitor. Figure 27 lists the above-described procedure as an example.

[0119] Incidentally, the difference in charge between the summing capacitors of Measure1 and Measure2 is determined by the sum of the charges accumulated in the N selected capacitors between Ope-1 and Ope-4, so the sense amplifier senses the sum of the charges of the N selected capacitors.

[0120] Furthermore, the charge on each selected capacitor is related to the Vin of each input and the Vt of each ANV. In other words, the charge on each selected capacitor is determined by taking into account the voltage drop due to the corresponding Vin and the corresponding ANV (a variable resistor with Vt as an argument). Therefore, the sense amplifier senses the sum of Vin and ANV Vt in the selected SB cell.

[0121] In this way, at least, the charge dQ of each selected capacitor is considered to be determined by a function f of the corresponding Vin and the Vt of the corresponding ANV. As an example, it can be written as dQ=f(Vin, Vt). However, f does not need to be a known function.

[0122] Nevertheless, the charge dQ of each selected capacitor is considered to be proportional to the corresponding Vin. Furthermore, the proportionality coefficient is considered to depend on the Vt of the corresponding ANV. Therefore, for example, it can be written as dQ = Vin × w(Vt). Here, this w(Vt) corresponds to the weight w in Figure 11. This is indeed in the form of a product. However, although the ANV stores Vt, it does not store the weight (w) itself. In other words, the weight cell (for example, the ANV) stores something related to the weight.

[0123] As an example, the charge dQ on the selection capacitor is summed up by the summing capacitor to become Q. This is sensed by the sense amplifier as a potential difference V. This V becomes the calculation result of thread T(j,k). In other words, the output sensed by the sense amplifier is related to the state of each weight cell and the input.

[0124] However, in AI neural network calculations, this thread T(j,k) is not used as is as part of the input to the next layer. In fact, it is transformed using an activation function before being used as input to the next layer. Activation functions vary depending on the type of problem being solved using AI. In other words, it is convenient to define them using software. In other words, the CPU (Central Processing Unit) used in conjunction with the AI ​​chip dedicated to the calculation of the above-mentioned threads transforms the thread T(j,k) using the activation function before returning it to the AI ​​chip.

[0125] An example will be described using Figure 28. Figure 28 shows an example of a method for handling thread t(j, k) calculated between an AI chip, a CPU, and a main memory.

[0126] For example, thread t(j,k) calculated from the AI ​​chip is sent to the CPU. The CPU stores the received thread in main memory such as DRAM. The CPU reads this thread when needed and converts it to Act(j,k) using an activation function. This Act(j,k) is passed to the AI ​​chip as the next input. In other words, t(*,k) in Figure 11 is Act(*,k), where * is an integer between 1 and N.

[0127] Analog nonvolatile memory cells (ANV) use a shorter gate length, a lower impurity concentration, and a deeper diffusion layer to increase the S-factor. This means that they are fabricated differently from regular nonvolatile memory, which prefers a small S-factor. This makes it easier for fluctuations (sigma) to be introduced into the current characteristics than usual. There are various causes of fluctuations, but the most prominent is random impurity fluctuations.

[0128] Such fluctuations appear as fluctuations in Vt and the S factor. Therefore, the effect of this fluctuation will be mixed into the charge of the ADC, which is the result of the thread operation. In this case, the formula for the charge of the selected capacitor mentioned above can be expanded to dQ = Vin × w(Vt, sigma).

[0129] So, is this a problem? The bottom line is no.

[0130] This is because the above-mentioned fluctuations are automatically taken into account when updating the weights (synaps weights, w) stored in main memory each time learning is repeated.

[0131] Let's say the weight when fluctuations are ignored is w0(Vt). In contrast, the weight actually stored in main memory after going through the mechanism in Figure 28 is w(Vt, sigma). In other words, it is different from w0(Vt). We can think of it as a weight that has already been updated to take fluctuations into account and stored in main memory. This is because the amount of charge read by the sense amplifier automatically includes all fluctuations. When updating the weights through repeated learning, the weights that include fluctuations are automatically updated in main memory, so their influence can be ignored. This is one of the features of this application.

[0132] In other words, the weight w can be seen as a model in which the SB cell in Figure 29 is replaced with a capacitor. The SB cell has two terminals, one of which is connected to the bit line BL(*) in Figure 16, and the other is connected to the addition data line ADL, where * is an integer from 0 to N-1.

[0133] The state in which Vin is applied between the ADL and the bit line is shown in Figure 29. Here, when Vread is applied to ANV and the selection transistor is off, a transient current flows between the two terminals until the charge on the selection capacitor is saturated.

[0134] This transient current causes charge to accumulate in the selected capacitor over time. This is shown in Figure 30. When Vin is applied at time t0, the absolute value of the charge increases over time from t0. In the case of a low Vt, the charge saturates at time t1 and the transient current disappears. In the case of a high Vt, the charge saturates at time t2 and the transient current disappears. Generally, t2 is greater than t1.

[0135] An ANV with a low Vt can be considered a low resistance. Conversely, an ANV with a high Vt can be considered a high resistance. That is, at low Vt (Low Vt), the current is high due to the low resistance, so charge accumulates quickly. Therefore, the slope is high. On the other hand, at high Vt (High Vt), the current is low due to the high resistance, so it takes longer for charge to accumulate. Therefore, the slope is low.

[0136] The sense time is ts - t0. In other words, Von is applied to the corresponding bit line select gate SBL only between t0 and ts. In other words, Vin is applied only between t0 and ts to allow a transient current to flow. However, ts is assumed to be shorter than t1.

[0137] When the application of Vin is stopped at time ts and the transient current disappears, the charge stored in the selected capacitor at that time differs between when Vt is low and when Vt is high. This is reflected as a different value of weight w when added on the addition data line ADL. The added result is stored in main memory. Incidentally, in this application, there is no need to store each component of weight w in main memory. This makes it possible to dramatically reduce power consumption and the required bit capacity of the main memory.

[0138] Here, ts must always be smaller than or equal to t1. Therefore, there exists a minimum Vt corresponding to the weight w to be stored. In other words, the minimum Vt is a function of ts. However, ts can be controlled by software.

[0139] Returning to Figure 18, it may be necessary to have a wait state between two adjacent steps. Figure 31 is a table illustrating one example of how such a wait state can be implemented.

[0140] First, the selected word line CWL is set to a floating state, Voff is applied to the selected word line SWL, the addition data line is set to a floating state, Voff is applied to the left select gate and the right select gate, and all bit lines are deselected.

[0141] The wait states in Figure 31 can be inserted as needed between two adjacent columns in Figure 27. Such wait states may be necessary when the response from the CPU is relatively long. (Fifth embodiment)

[0142] FIG. 32 is a diagram showing an example of the flow of one epoch.

[0143] The integers used for incrementing are j and k. However, k has a special meaning in determining the structure of the neural network. That is, when k=0, it represents the input layer, and when k=M, it represents the output layer. The hidden layers range from k=1 to k=M-1. That is, the number of hidden layers is M-1. When k=M, the output y is calculated. That is, the output y is the charge added in the Mth layer. Also, j is the element number of the output element in the (k+1)th column obtained by thread calculation, and is an integer between 1 and L(k+1).

[0144] Here, i is the element number of the input element x(i) input in the kth layer when performing thread calculation, and is an integer between 1 and L(k). However, if the number of elements in the input layer (k=0) is N, then L(0)=N. Input elements can be input from outside the thread calculation device.

[0145] At Start, both increments shall start from 1.

[0146] Next, the first thread operation is executed and the output t(1, 2) is obtained (Get t(1, 2)). Using the activation function, this is converted into the activation output element Act(1, 2) and stored in main memory.

[0147] Next, the next thread operation is executed to obtain the output t(2, 2) (Get t(2, 2)). Using the activation function, this is converted into the activation output element Act(2, 2) and stored in main memory.

[0148] This process is repeated L(2) times. The activation output element Act(j,2) is provided externally from the semiconductor chip. In some cases, the activation output element Act(j,2) is stored in main memory, where j is an integer between 1 and L(2). The L(2) activation output elements Act(1,2), Act(2,2), … Act(L(2),2) are combined together to form the activation output of the second layer. (The first layer is the input layer.) The activation output of the (k+1)th layer is obtained by incrementing k. The activation output of the (k+1)th layer is the combination of the L(k+1) activation output elements Act(1,k+1), Act(2,k+1), … Act(L(k+1),k+1).

[0149] Next, check whether k is smaller than M. If it is smaller than M, assign (overwrite) the activated output element to the input element and update (increment) k. In other words, increase it by 1, and repeat this process until k = M. Then, calculate the Y value when k = M. However, when k = M, it is possible to use or not use an activation function.

[0150] In this way, activation output elements Act(j, k+1) are calculated for all possible j and k and stored in main memory as needed. Y values ​​are also calculated (Get Y) and stored in main memory as needed. This completes one learning session. In other words, with the calculation of Y values ​​(Get Y), one epoch is completed and the process ends (End).

[0151] 33 is a diagram for explaining an example of a method for repeating epochs. Here, the number of epochs (number of learning times) is represented by g (epoch number).

[0152] First, at the start, g = 1.

[0153] Y(g) is calculated using the method in Figure 32, etc., where Y(g) is the Y value itself. The Y value calculated the first time is Y(1), the Y value calculated the second time is Y(2), and so on, and the Y value calculated the gth time is Y(g).

[0154] Next, the difference (or residual) between the latest Y(g) and the immediately previous Y(g-1) is compared with a predetermined value. If the absolute value of this residual is deemed to be sufficiently small compared to the predetermined value, Y(g) is deemed to have converged. If it has not converged and g is not smaller than the upper limit gupp, the calculation is deemed to have failed (failed), and the calculation is terminated (End). Alternatively, if g is smaller than the upper limit gupp, g is incremented (increased by 1), Y is calculated again, and this is set as Y(g+1).

[0155] On the other hand, if Y(g) has converged, Y = Y(g), Y is output, and learning ends (End). The final output Y can be saved to the main memory outside the AI ​​chip, if necessary. At this time, the most recent g is the epoch number. The epoch number is useful for expressing the learning efficiency of an algorithm. In other words, a large epoch number indicates low learning efficiency, and a small epoch number indicates high learning efficiency.

[0156] The learning speed is expressed as the product of the operating speed of the AI ​​chip, which is expressed as the clock frequency or bandwidth, and the number of epochs.

[0157] As described above, the total number of elements of the weight w required to calculate the output Y is L(0), L(1), . . . L(M-1). (Sixth embodiment)

[0158] Instead of the ANV in FIG. 16, a variable resistor, a memristor, or the like can be used. The ANV, variable resistor, memristor, and the like related to this application can be collectively referred to as a synaptic weight cell. That is, the synaptic weight can be stored in the weight cell as some kind of state change. FIG. 34 shows an example of an equivalent circuit diagram for a general weight cell. There are two-terminal and three-terminal weight cells. In the case of a three-terminal weight cell, a control word line CWL is particularly required, as shown in FIG. 16. In the case of a two-terminal weight cell, the control word line CWL can be omitted, as shown in FIG. 34. In the case of a two-terminal weight cell, the state of the weight cell can be changed by applying a predetermined electrical, electromagnetic, or optical stimulus between the two terminals.

[0159] The weight cell related to this application is, for example, a three-terminal, transistor-type electronic component (electronic device, or simply, device) having a charge storage region, where the charge storage region is, for example, a floating gate or a charge trap layer.

[0160] The weight cell relevant to this application is, for example, a memristor, which is a two-terminal device whose state changes depending on the transmitted charge or magnetic flux.

[0161] The weight cell of interest in this application is, for example, a conductive bridge random access memory (CBRAM), which is a two-terminal device that controls resistance (or conductivity) by passing an appropriate current through it, thereby moving ions or the like.

[0162] The weight cell of interest in this application is, for example, a metal switch, a two-terminal device that controls resistance (or conductivity) by moving metal ions or the like when an appropriate current is passed through it. It is also a variant of CBRAM.

[0163] The weight cell of interest in this application is, for example, a phase change memory (PC Memory), which is a two-terminal device that controls the resistance (or conductivity) by causing a phase change when an appropriate current is passed through it.

[0164] The weight cell of interest in this application is, for example, a resistive memory, which is a two-terminal device that controls the resistance (or conductivity) by causing a change in resistance when an appropriate current is passed through it.

[0165] The weight cell related to this application is, for example, a spintronic device. This is a device that controls the resistance (or conductivity) by applying an appropriate electric field and passing an appropriate current nearby to apply this electric field, thereby causing a change in resistance. Examples include spin transfer torque devices, spin wave devices, magnetic domain walls, etc. It has two terminals in addition to the part that applies the electric field.

[0166] The weight cells of interest in this application are, for example, optical devices, which are two-terminal photoelectric devices that use the photoelectric effect to control the flow of electrons and thus the synaptic weight.

[0167] FIG. 34 is a diagram showing an example of an equivalent circuit according to the present invention when a two-terminal weighted cell is used.

[0168] Since the weight cell has two terminals, it is possible to omit the control word line CWL. Examples of two-terminal cells include memristors, CBRAM, metal switches, phase-change memories, resistive memory, spin devices, and optical devices.

[0169] When the weight cell according to the present invention is a three-terminal cell, the equivalent circuit according to the present invention is, for example, as shown in FIG.

[0170] When the three-terminal weight cell has a charge storage region (such as a floating gate or a charge trap layer), FIG. 35 is equivalent to FIG.

[0171] In general, the characteristics of a three-terminal weight cell having a charge storage region (such as a floating gate or a charge trap layer) are analog type with a large S factor, as described above on the right side of Figure 17.

[0172] When the S-factor of a three-terminal weight cell with a charge storage region (such as a floating gate or charge trap layer) is small, it is possible to control more than two states by writing (P) and erasing (E), as shown in Figure 36. In this figure, as an example, a two-bit cell (2 bits / cell) with four states is used. In this case, the four states are (11), (10), (01), and (00), as shown in the second row of Figure 37. This is also known as MLC.

[0173] The third row in Figure 37 shows the case of 8 states (3-bit parcel, 3 bits / cell). The states are (111), (110), (101), (100), (011), (010), (001), (000). This is also known as TLC.

[0174] The fourth row in Figure 37 shows the case of 16 states (4-bit parcel, 4 bits / cell). The states are (1111), (1110), (1101), (1100), (1011), (1010), (1001), (1000), (0111), (0110), (0101), (0100), (0011), (0010), (0001), (0000). This is also known as QLC.

[0175] Although not specifically shown, 32 states (5-bit per cell) are also possible. This is also known as PLC. As it is self-evident, we will omit the details, but the number of states per cell can be increased arbitrarily. As mentioned above, the increase does not necessarily have to be in increments of two. As it is self-evident, we will omit the details, but it is possible to increase the number of discretized states per cell arbitrarily.

[0176] The name of one line in Figure 37 is 1 bit parcel (1 bit / cell) on the left side of Figure 17. This is also known as SLC.

[0177] As mentioned above, the weight cells of the present application can have an arbitrarily large number of discretized states.

[0178] FIG. 38 is a diagram showing an example having dummy cells.

[0179] The dummy cell can be placed adjacent to the weight cell to ensure stable operation of the weight cell. The dummy cell may or may not be the same element as the weight cell. In the example of Figure 38, the weight cell is an ANV, and the dummy cell is a cell having a charge storage region. However, the threshold voltage of this dummy cell may or may not be variable. The dummy cell may or may not be connected to a word line. The positions of the dummy cell and the weight cell may be interchanged. The number of dummy cells is not limited to one. The number of dummy cells is arbitrary.

[0180] FIG. 39 is a diagram showing an example having dummy cells.

[0181] A dummy cell can be placed adjacent to a weight cell to ensure stable operation of the weight cell. The dummy cell may or may not be the same element as the weight cell. The dummy cell may or may not change state like the weight cell. The dummy cell may or may not be connected to a word line. The weight cell may or may not be connected to a word line. The positions of the weight cell and the dummy cell may be interchanged. The number of dummy cells is not limited to one. The number of dummy cells is arbitrary.

[0182] Finally, the weight cell of the present application can form part of the SB cell of the present application. The SB cell can be integrated on the semiconductor chip (such as an AI chip) of the present application. Thus, the weight cell can be integrated on the semiconductor chip. As described above, the weight cell can store some state related to the weight (synaptic weight) or a change in that state as a weight element, rather than storing the weight itself. As described above, the weight element can be expressed in the form of a three-dimensional matrix w(i, j, k). As the number of hidden layers increases, i.e., when dealing with more complex problems using deep learning, the number of elements in w(i, j, k) becomes very large. In other words, accessing main memory every time a weight is updated requires very large power consumption. Therefore, integrating weight cells directly on an AI chip, as in the present application, contributes to significantly reducing power consumption for utilizing deep learning (artificial intelligence).

[0183] In the present application, there is no need to pass a large current through the bit line or word line to change the state of the weight cell. In other words, the change in state of the weight cell occurs due to the limited movement of charged particles (electrons, holes, etc.) inside the weight cell. Therefore, there is an advantage that the power consumption required to rewrite the state of the weight cell is extremely low. Furthermore, what is sensed to read the state of the weight cell is the charge accumulated in the selection capacitor and the summing capacitor ADC. Therefore, there is also no need to pass a large current through the bit line or word line to sense the change in state of the weight cell. Thus, there is an advantage that the power consumption to sense the weight (synaptic weight) is extremely low.

[0184] The thread processing device according to the present application is an integrated circuit or an embedded circuit having multiple SB cells as components, i.e., a circuit mounted or integrated on an AI chip, or a part thereof. The thread processing device according to the present application can be integrated on a semiconductor chip. The semiconductor chip is an example of an AI chip, and is preferably a chip independent of the main memory. As shown in Figure 11, multiple thread processing devices can be arranged on the semiconductor chip. The semiconductor chip is characterized by functioning in cooperation with another processor that controls the operation of the multiple thread processing devices.

[0185] [Non-Patent Document 1] Goodon E. Moore, “Cramming more components onto integrated circuits”, Electronics, volume 38, Number 8, April 19, 1965. [Non-patent document 2] Masanet, E.; Shhehabi, A.; Lei, N.; Smith, S.; Koomey, J. Recalibrating global data center energy-use estimates. Science 2020, vol. 3667, 984―986.

[0186] The features of the present invention have been described above.

[0187] The technical scope of the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

[0188] [Industrial Applicability]

[0189] This will make it possible to provide a semiconductor chip that significantly reduces the power consumption of deep learning. [Brief explanation of the drawings]

[0190] [Figure 1] A diagram illustrating an example of a nerve cell. [Figure 2] 1 is a diagram illustrating an example of synapsis. [Figure 3] An example diagram illustrating the concept of a perceptron. [Figure 4] 1 is a diagram illustrating an example of a thread operation. [Figure 5] 10 is a diagram for explaining an example of a combination of thread operations. [Figure 6] 10 is a diagram for explaining an example of a combination of thread operations. [Figure 7] 10 is a diagram for explaining an example of a combination of thread operations. [Figure 8] 10 is a diagram for explaining an example of a combination of thread operations. [Figure 9] 10 is a diagram for explaining an example of a combination of thread operations. [Figure 10] 10 is a diagram for explaining an example of a combination of thread operations. [Figure 11] 1 is a diagram illustrating an example of a thread. [Figure 12] 1 is a diagram illustrating a conventional example of a thread operation method. [Figure 13] 1 is a diagram illustrating an example of a von Neumann bottleneck. [Figure 14] 10 is a diagram for explaining an example of a method for performing a product-sum operation when data C is on-chip. [Figure 15] 1 is a diagram illustrating an example of thread operations related to the present application. [Figure 16] 1 is a diagram illustrating an example of a circuit for calculating threads according to the present application. [Figure 17] 1 is a diagram illustrating an example of electrical characteristics of a nonvolatile memory cell constituting an SB cell according to the present application. [Figure 18] 1 is a diagram illustrating an example of a sequence for calculating threads according to the present application. [Figure 19]1 is a diagram illustrating an example of thread erase related to the present application. [Figure 20] 1 is a diagram illustrating an example of a selective programming method according to the present application. [Figure 21] 1 is a diagram illustrating an example of a selective programming method according to the present application. [Figure 22] 1 is a diagram illustrating an example of a selective programming method according to the present application. [Figure 23] 1 is a diagram illustrating an example of a selective erase method according to the present application. [Figure 24] 1 is a diagram illustrating an example of a measurement method according to the present application. [Figure 25] 1 is a diagram for explaining an example of a product-sum operation according to the present application. [Figure 26] 1 is a diagram illustrating an example of a measurement method according to the present application. [Figure 27] 1 is a diagram for explaining an example of a multiply-and-accumulate operation method according to the present application. [Figure 28] 10 is a diagram illustrating an example of the interaction between threads and activation functions. [Figure 29] 1 is a diagram illustrating an example of an SB cell according to the present application. [Figure 30] 10 is a diagram illustrating an example of the relationship between the charge accumulated in a capacitor and time. [Figure 31] 1 is a diagram illustrating an example of an idle state according to the present application. [Figure 32] 1 is a diagram illustrating an example of one cycle (one epoch) of learning related to the present application. [Figure 33] 1 is a diagram illustrating an example of neural network calculation related to the present application. [Figure 34] 1 is a diagram illustrating an example of an equivalent circuit diagram according to the present application. [Figure 35] 1 is a diagram illustrating an example of an equivalent circuit diagram according to the present application. [Figure 36] 1 is a diagram illustrating an example of the characteristics of a weight cell according to the present application. [Figure 37]1 is a diagram illustrating an example of the characteristics of a weight cell according to the present application. [Figure 38] 10 is a diagram illustrating an example having a dummy cell. [Figure 39] 10 is a diagram illustrating an example having a dummy cell. [Figure 40] 1 is a diagram illustrating an example of a selective programming method according to the present application.

Claims

1. A thread processing unit mounted on a semiconductor chip is used as a component, The thread processing unit returns an output in response to an input, the input comprises a plurality of input elements; The input element is provided from outside the thread processing device, The thread processing unit has a weight cell, The weight cells are integrated on the semiconductor chip; the output is related to the input and the state of the weight cell; A semiconductor device characterized by:

2. The thread processing unit includes a plurality of cells and a sense amplifier. the cell has first, second, third, and fourth terminals; the first terminal is connected to a bit line; the second terminal is connected to a control word line; the third terminal is connected to a selected word line; the fourth terminal is connected to an addition data line; the summing data line has first and second select gates; a summing capacitor between the first and second select gates; the second select gate is sandwiched between the summing capacitor and the sense amplifier; 2. The semiconductor device according to claim 1.

3. The cell includes a selection transistor, a selection capacitor, and the weight cell; the weight cell has a first source, a first drain, and a first gate; the first drain is connected to the first terminal; the first gate is connected to the second terminal; the select transistor has a second source, a second drain, and a second gate; the second gate is connected to the third terminal; the second source is connected to the fourth terminal; The selection capacitor has two terminals; one terminal of the selection capacitor is connected to the fourth terminal; the other terminal of the selection capacitor is connected to the second drain; The first source and the second drain are connected.

3. The semiconductor device according to claim 2.

4. The weight cell has a charge storage region and a channel portion, Charged particles are exchanged between the channel portion and the charge accumulation region; By adjusting the amount of charge stored in the charge storage region, Changing the state of a weight cell, 2. The semiconductor device according to claim 1.

5. The thread processing device calculates the output according to a plurality of processing steps; The plurality of calculation steps includes a first step, The first step comprises: applying an erase voltage to the first terminal; A reference voltage is applied to the second terminal; Applying an on-voltage to the third terminal; applying an erase voltage to the fourth terminal; applying an on-voltage to the first select gate; 3. The semiconductor device according to claim 2.

6. The plurality of calculation steps includes a second step, The second step comprises: A reference voltage is applied to the first terminal; applying an intelligent write voltage to the second terminal; Applying an on-voltage to the third terminal; A reference voltage is applied to the fourth terminal; applying an on-voltage to the first select gate; 6. The semiconductor device according to claim 5.

7. The plurality of calculation steps includes a third step, The third step comprises: applying an intelligent erase voltage to the first terminal; A reference voltage is applied to the second terminal; applying an off voltage to the third terminal; The fourth terminal is left floating; applying an off voltage to the first select gate; applying an off voltage to the second select gate; 6. The semiconductor device according to claim 5.

8. The plurality of calculation steps includes a fourth step, The fourth step includes: The first terminal is in a floating state, The second terminal is in a floating state, applying an off voltage to the third terminal; The fourth terminal is left floating; applying an off voltage to the first select gate; applying an on-voltage to the second select gate; 6. The semiconductor device according to claim 5.

9. The plurality of calculation steps includes a fifth step, The fifth step comprises: applying an input voltage to the first terminal; Applying a read voltage to the second terminal; applying an off voltage to the third terminal; The fourth terminal is left floating; applying an off voltage to the first select gate; applying an off voltage to the second select gate; 6. The semiconductor device according to claim 5.

10. The plurality of calculation steps includes a sixth step, The sixth step comprises: The first terminal is left floating; applying an off voltage to the second terminal; applying an off voltage to the third terminal; The fourth terminal is left floating; applying an off voltage to the first select gate; applying an off voltage to the second select gate; 6. The semiconductor device according to claim 5.

11. The plurality of calculation steps includes a seventh step, The seventh step comprises: The first terminal is left floating; applying an off voltage to the second terminal; Applying an on-voltage to the third terminal; The fourth terminal is left floating; applying an off voltage to the first select gate; applying an off voltage to the second select gate; 6. The semiconductor device according to claim 5.

12. The plurality of calculation steps includes an eighth step, The eighth step includes: The first terminal is left floating; applying an off voltage to the second terminal; applying an off voltage to the third terminal; The fourth terminal is left floating; applying an off voltage to the first select gate; applying an off voltage to the second select gate; 6. The semiconductor device according to claim 5.

13. The plurality of calculation steps includes a ninth step, The ninth step comprises: The first terminal is in a floating state, applying an off voltage to the second terminal; applying an off voltage to the third terminal; The fourth terminal is left floating; applying an off voltage to the first select gate; applying an on-voltage to the second select gate; 6. The semiconductor device according to claim 5.

14. Controlling the thread processing unit mounted on the semiconductor chip, The thread processing unit returns an output in response to an input, the input comprises a plurality of input elements; The input element is provided from outside the thread processing device, The thread processing unit has a weight cell, The weight cells are integrated on the semiconductor chip; the output is related to the input and the state of the weight cell; The output has first and second arguments, converting the output into an activated output element using an activation function according to the first argument; comparing the first argument to a first integer; if the first argument is less than the first integer, then incrementing the first argument; the first integer is dependent on the second argument; comparing the second argument to a second integer; When the second argument is less than the second integer, assigning the activated output element to the input element; incrementing the second argument, When the second argument is greater than or equal to the second integer, The output is a function Y. Software characterized by:

15. The function Y has a third argument, when the third argument is a third integer, the function Y takes a first value; incrementing the third argument, when the third argument is a fourth integer, the function Y takes a second value; the difference between the fourth integer and the third integer is 1; comparing a residual between the first value and the second value with a predetermined value; When the absolute value of the residual is deemed to be sufficiently small compared to the predetermined value, the function Y is deemed to have converged; The third argument is considered to be the epoch number.

15. The software of claim 14.

16. The weight cell has two terminals; By applying an electrical, electromagnetic, or optical stimulus between the two terminals, changing the state of the weight cells; 2. The semiconductor device according to claim 1.

17. The weight cells are: The number of discretized states can be increased arbitrarily.

2. The semiconductor device according to claim 1.

18. The weight cells are: Adjacent to any number of dummy cells 2. The semiconductor device according to claim 1.

19. The capacitance of the summing capacitor is greater than the capacitance of the selection capacitor.

4. The semiconductor device according to claim 3.

Images