Memory mapped neural network software and hardware system for ai applications

Abstract

A method for implementing a neuron computation model includes having a neuron receive a plurality of input values. For each input value, a comparison is made to a threshold tau value. For each successful or unsuccessful comparison, the neuron can add a value to a number of matches or mismatches and based on that number of matches or mismatches generate a neuron output. In some embodiments, a plurality of neurons form at least a portion of an artificial neural network (ANN) that is supported on a memory device having array of memory cells configured in rows and columns, where each neuron is represented by one column. To improve performance, a companion ASIC chip can be connected to the memory device to provide computational support.

Description

ROOG-OOIOOWOMemory Mapped Neural Network Software and Hardware System for Al Applications

[0001] Related Application

[0002] The present disclosure claims the priority benefit of U.S. Provisional Patent Application No. 63 / 763,151, filed on February 25, 2025, which is hereby incorporated by reference in its entirety.

[0003] Technical Field

[0004] The present disclosure generally relates to use of methods and systems for implementing a neuron computation model able to run on a memory array supported computational substrate.

[0005] Background

[0006] Artificial Intelligence (AI) is becoming an essential part of life and is increasingly integrated into various aspects of our daily routines, work, and leisure activities. Some major industries are already adopting AI in their service delivery systems such as healthcare, finance, transportation, manufacturing, and customer service. Often, AI systems can utilize machine learning systems known as a neural network (NN), neural net, or artificial neural network (ANN), which is a computational model inspired by the structure and functions of biological neural networks.

[0007] Unfortunately, current hardware and software systems that support ANN based AI can be extremely complex, requiring substantial energy intensive data processing. For example, many conventional ANN systems require learning approaches that involve having a neuron calculating a product for multiple inputs, adding the result to a programmable bias, sending that result through a non-linear activation, and providing the result as a neuron output. That neuron output is then passed to other neurons as input, with similar processing occurring. In practice, thousands of neurons arranged in multiple input, hidden, and output layers may need to perform computationally and energy intensive matrix calculations for large numbers of neurons.

[0008] What is needed are hardware and software systems that support computationally simpler neuron computation and neural network processing approaches.

[0009] Summary

[0010] In one embodiment, a method for implementing a neuron computation model includes the steps of having a neuron receive a plurality of input values; for each input value,ROOG-OOIOOWOmaking a comparison to a threshold tau value; for each successful comparison, having the neuron add a value such as 1 to a number of matches; or for each unsuccessful comparison, having the neuron add a value such as 1 to a number of mismatches; and based on the number of matches or mismatches, generating a neuron output.

[0011] In one embodiment, the neuron output is set to between 0 and 1.

[0012] In one embodiment, the comparison can be at least one of > or <

[0013] In one embodiment, the comparison can be at least one of > or <, and a bias value is used to determine whether > or < is selected for comparison.

[0014] In one embodiment, the threshold tau value is set during training.

[0015] In one embodiment, the bias value is set during training.

[0016] In one embodiment, the neuron output is at least in part determined by the number of matches or mismatches and a ReLU function.

[0017] In one embodiment, the neuron output is at least in part determined by a number of matches or mismatches and a sigmoid function.

[0018] In one embodiment, the neuron output is further adjusted using a programmed output strength.

[0019] In one embodiment, the neuron output is further adjusted using a programmed output strength that is set during training.

[0020] In one embodiment, the neuron output is further adjusted by multiplication with a programmed output strength.

[0021] In another embodiment, an AI system can include an artificial neural network (ANN) having at least one a neuron able to receive a plurality of input values, wherein for each input value to the neuron, a comparison can be made to a threshold tau value. For each successful comparison, the at least one neuron adds a selected value to a number of matches, or for each unsuccessful comparison, the neuron adds a value to a number of mismatches, and based on the number of matches or mismatches generates a neuron output to one or more other neurons in the ANN.

[0022] In one embodiment, the ANN is a hybrid neural network including a conventional neural network with at least one conventional neuron connected to a neural network having neurons that, for each successful comparison, add a selected value to a number of matches, or for each unsuccessful comparison, has the neuron add a value to a number of mismatches, and basedROOG-OOIOOWOon the number of matches or mismatches generate a neuron output.

[0023] In one embodiment, the neuron output is set to between 0 and 1 inclusive.

[0024] In one embodiment, the comparison can be at least one of > or <

[0025] In one embodiment, the comparison can be at least one of > or <, and a bias value of 1 or 0 is respectively used to determine whether > or < is selected for comparison.

[0026] In one embodiment, the threshold tau value is set during training.

[0027] In one embodiment, the bias value is set during training.

[0028] In one embodiment, the ANN is supported on a memory device having an array of memory cells configured in rows and columns and where more than one row is selected in a given column and the sense amplifier detects and outputs a value representing how many of the selected row’s cells are conductive.

[0029] In another embodiment, a memory device configured for operation in multiple modes includes an array of memory cells configured in rows and columns. A plurality of sense amplifiers can be connected to the array of memory cells and the array of memory cells has a first operational data storage and read mode, wherein circuitry performs a write operation of data to a memory cell by selecting one row from a multiple of rows and data is written and read from the memory cell using at least one sense amplifier attached to one of the columns. The array of memory cells has a second operational multi-layer perceptron (MLP) mode where more than one row is selected in a given column, and the sense amplifier detects and outputs a value representing how many of the selected row’s cells are conductive.

[0030] In one embodiment, the memory device has a mode selector to switch between the first and second operational modes.

[0031] In one embodiment, a single neuron is represented through one column of the array of memory cells.

[0032] In one embodiment, the sense amplifier is connected to the column to provide a sense amplifier output value corresponding to the output value for that neuron.

[0033] In one embodiment, the memory device supports a neural network comprising at least one neuron represented through a column of the array of memory cells.

[0034] In one embodiment, multiple operations can be performed in MLP mode where each sequential operation is related to determining the values of neurons represented through columns of the array of memory cells and arranged in one layer.ROOG-OOIOOWO

[0035] In one embodiment, values placed on selected rows are related to the values sense amplifiers read from the previous operations.

[0036] In one embodiment, a neural network includes at least one neuron represented through a column of the array of memory cells and input to a first hidden layer neuron is based on inputs given for a neural network.

[0037] In another embodiment, a method for implementing a neural network includes the steps of providing a memory device having array of memory cells configured in rows and columns, where one neuron can reside on a single column. Incoming values can be determined for each row by input values for each of the neurons selected and operations performed by the memory device based on the incoming values of the row and the threshold value programmed on a plurality of cells associated with the neuron, where the conduction of a cell represents a failure in the operation being checked. A value from a sense amplifier can be outputted in accordance with how many of the cells were determined to be conductive.

[0038] In one embodiment, there is no conduction path for rows that have cells on the column that are not a part of the neuron input.

[0039] In one embodiment, the intersection of rows and columns consists of a series of serially connected cells.

[0040] In one embodiment, the operation being performed is < or >.

[0041] In one embodiment, a sense amplifier is attached to a column where there are multiple cells being checked, the sense amplifier output acting like an NOR gate.

[0042] In one embodiment, the operation being checked for the neuron is an opposite of an operation for a given input to a neuron.

[0043] In one embodiment, each operation being performed has one cell associated with it.

[0044] In one embodiment, only one operation is performed for each input.

[0045] In one embodiment, the same input can be inputted into multiple rows whereby multiple conditions can be performed on the same input.

[0046] In one embodiment, two inputs are generated for each neuron input, one to check for each < or > condition.

[0047] In one embodiment, for < operation, where the neuron checks to see if the input of the neuron is < the threshold, the cell’s input would be the value of the input, and the threshold would be an actual threshold.ROOG-OOIOOWO

[0048] In one embodiment, for > operation, where the neuron checks to see if the input value is > the threshold of the cell, the second cell can be selected which can have (1-input value) as the input to the cell and the threshold programmed into the cell would be (1 -Threshold condition).

[0049] In one embodiment, each cell from the serially connected cells not being checked is left in a conductive state, without respect to the input value of its row.

[0050] In one embodiment, conductivity level of each cell is controlled by controlling the gate of a series device in series with the string.

[0051] In another embodiment, an AI system includes a plurality of neurons forming at least a portion of an artificial neural network (ANN) having at least one a neuron able to receive a plurality of input values, wherein for each input value to the neuron, a comparison can be made to a threshold tau value and for each successful comparison, the neuron adds a selected value to a number of matches, and based on the number of matches generates a neuron output to one or more other neurons in the ANN. A memory device having array of memory cells is configured in rows and columns and can be arranged so each column represents a single neuron and a companion ASIC chip connected to the memory device to provide computational support.

[0052] In one embodiment, a memory device supports multi-layer perceptron (MLP) operation, and the companion ASIC chip performs related math computations.

[0053] In one embodiment, the companion ASIC chip can provide initial inputs to the memory Al chip based on either inputs for training or inference. The memory Al chip can then respond with the output of neurons based on the inputs provided to the companion ASIC chip.

[0054] In one embodiment, the companion ASIC chip can parse neuron outputs and provide values to the memory Al chip for next layer calculations.

[0055] In one embodiment, the companion ASIC chip can provide the inputs out of order to the memory Al chip.

[0056] In one embodiment, the companion ASIC chip can replace the output of one of the neurons with a redundant neuron output.

[0057] In one embodiment, the companion ASIC chip sets the memory Al chip in the memory mode and can read one parameter from the memory Al chip.

[0058] In one embodiment, the companion ASIC chip completes multiplication using the read parameter.ROOG-OOIOOWO

[0059] In one embodiment, the inputs to the memory AI chip are generated from the resulting values.

[0060] In one embodiment, the companion ASIC chip can perform the multiply operation and send resulting values to the memory Al chip for MLP processing.

[0061] In another embodiment, a method for implementing a neural network system includes the steps of providing a memory Al chip including a neural network on a memory device having array of memory cells configured in rows and columns, where each column can represent a neuron. For each input value to the respective column neuron, a comparison can be made to a threshold tau value. Computational math support to the memory Al chip can be provided with a companion ASIC.

[0062] In one embodiment, inputs by the companion ASIC chip are provided to the memory Al chip, and outputs from the memory Al chip can be sent back to the companion ASIC chip.

[0063] In one embodiment, the companion ASIC chip is allowed to take inputs from a neural network system as input to the memory Al chip to provide correct voltages on the associated rows.

[0064] In one embodiment, the companion ASIC chip takes the inputs from the memory Al chip that represents a current hidden layer and rearranges inputs to provide correct values to the memory Al chip as inputs to the next hidden layer.

[0065] In one embodiment, output of some of the layers can be kept by the companion ASIC chip to be input to layers that are not the next hidden layer.

[0066] In one embodiment, the output of the last layer can be utilized by the companion ASIC chip to implement multiply / accumulate (MAC) operation for the output layer.

[0067] In one embodiment, large language models (LLMs) are supported that use a companion ASIC chip for attention and have the memory Al perform feed forward networks (FFNs).

[0068] In one embodiment, vision and vision language models are supported that use the companion ASIC chip for attention type activities and has the memory Al perform feed forward networks (FFNs).

[0069] In one embodiment, world models and video are supported that use the companion ASIC chip for attention type activities and has the memory Al perform feed forward networksROOG-OOIOOWO(FFNs).

[0070] In one embodiment, an autonomous driving model is supported that uses the companion ASIC chip for attention type activities and has the memory Al chip perform feed forward networks (FFNs).

[0071] In another embodiment, a plurality of neurons forms at least a portion of an artificial neural network (ANN) has at least one a neuron able to receive a plurality of input values. A memory device having array of memory cells can be configured in rows and columns, where each column represents a single neuron. A companion ASIC chip can be connected to the memory device to provide computational support.

[0072] In one embodiment, the companion ASIC chip provides an address of the cells needed for a next hidden layer.

[0073] In one embodiment, the companion ASIC chip provides a same address for the next hidden layer to create a recurrent neural network where output of one layer is provided to the same existing layer that was previously evaluated with different inputs.

[0074] In one embodiment, the companion ASIC combines multiple outputs read from different neurons to increase neuron size.

[0075] In one embodiment, the companion ASIC chip can be incorporated into the same silicon as the memory device.

[0076] In one embodiment, the companion ASIC chip can be incorporated into the same package as the memory device.ROOG-OOIOOWO

[0077] Brief Description of the Drawings

[0078] Figure 1A depicts a computational neuron accepting multiple inputs, determining the state of inputs relative to learned thresholds, and determining neuron output.

[0079] Figure 1B depicts a computational neuron accepting multiple inputs, determining the state of inputs relative to learned thresholds in a simpler manner as compared to the neuron illustrated with respect to Figure 1A, and determining neuron output.

[0080] Figure 2 depicts steps in a method for operating a computational neuron.

[0081] Figure 3A depicts a computational neural network operable using the novel computational neuron described herein.

[0082] Figure 3B depicts a hybrid convolutional neural network operable using conventional neural network input stages and the novel computational neuron network described herein.

[0083] Figure 4A depicts a memory mapped neural network hardware and software system switchable between a memory and a neural network mode.

[0084] Figure 4B depicts memory mapped neural network hardware and software system based on a NAND array.

[0085] Figure 4C depicts a memory mapped neural network hardware and software system.

[0086] Figure 4D depicts a memory mapped neural network hardware and software system supported by segmented chips.

[0087] Figure 5 depicts steps in operation of a memory device supporting an embodiment of computational neurons forming a neural network.

[0088] Figure 6 depicts a computer system capable of supporting or acting as a component of a system such as described herein.

[0089] In the Figures, reference signs can be omitted as is consistent with accepted engineering practice; however, a skilled person will understand that the illustrated components are understood in the context of the Figures as a whole, of the accompanying writings about such Figures, and of the embodiments of the claimed inventions.ROOG-OOIOOWO

[0090] Detailed Description of the Drawings

[0091] Figure 1A depicts a computational neuron 100A accepting multiple inputs, modifying the inputs determining the state of inputs relative to learned thresholds, and determining neuron output. As illustrated, the computational neuron 100A is a logical function that has a direct relationship between the neuron and the inputs that come from other sources, including previous layers of neurons. In one embodiment the computational neuron 100A looks at each of the current inputs, compares those inputs to a threshold (tau) value corresponding to sensitivity, and if the output is higher or lower than a programmed level, it considers that input as a logical 1. If all the input neurons have values that are higher or lower than those of their specific threshold sensitivity, then the output function would generate a 1. The value coming from that function is programmable. In other words, one can say that the output of this specific neuron is a 1, but it has value is at 0.5, for example. That value can be increased depending on the reinforcement of that computational neuron 100A. As an example, if the output of this computational neuron 100A specifies detection of a car as an example, it could start with a 0.5 and over time, as more and more cars are identified correctly, the value can be increased.

[0092] There are several advantages to use of such a computational neuron 100A. Each neuron can have a 1: 1 relationship with other neurons that feed it. In the traditional neural networks, If the number of neurons per layer is increased, the total value of the summation of neuron values times weights become very large. If one of the cells was to change, its output and weight would have to change significantly to make any appreciable difference in that total sum. To remedy that situation, extremely high accuracy processors are needed for each neuron calculation, resulting in system designs that use a floating point 32 bit or 16 bit operation which requires each neuron processor to be 32b or 16b processor. For computational neuron 100A, the relationship between the neuron and its input neuron is individualistic. The neuron allows each of its input neurons to have a different threshold sensitivity. In other words, a neuron with an output of 0.5 assigned, can set off a neuron in the next layer if the threshold sensitivity of the new neuron is set to 0.4. The same neuron with the output value of 0.5, will not fire a new neuron right to the one it set off if the new neuron’s threshold sensitivity is set to 0.6. So regardless of the number of inputs, the value of each neuron can be individually set and with very simple math calculations not requiring sophisticated processing.

[0093] In more detail, the computational neuron 100A accepts output of each incomingROOG-OOIOOWOneuron and compares it to a threshold value tau (τ) that can be programmed or learned. In some embodiments the incoming neuron’s output value can be checked to determine whether it is higher or lower than the threshold value. The threshold value can be determined through the training. A bias value, or b can determine the “higher or lower” function. If b=0, it can check whether the incoming value is lower than the τ and if b=1, it can check whether the incoming value is higher than the τ. There can be one b for each input, with these values for each input of the neuron being determined during training.

[0094] The computational neuron 100A then counts how many of the conditions have been met, either higher or lower cases, generating the value for number of matches. The value of the number of matches is then used by a RELU function to generate the output value for the computational neuron 100A. Assuming there are n inputs to a given neuron, in one embodiment:

[0095] - If all conditions are met, the output value will be 1

[0096] - There will be a number m which is less than n that if number of matches is less than n-m, the output of the neuron will be 0

[0097] - The number of matches that fall in between n-m and n will have values corresponding to the number of matches. As an example, if there were 20 inputs to a neuron, so n=20, and one can assume that one can display threshold sensitivity to the output for 10 matches below ALL, so RELU Sensitivity m=10. Conversely, in some embodiments one can count the non-matches and multiply that by 0.1 and subtract from 1. In such an embodiment, if 4 does not match the conditions, one would get 1- count * 0.1= 0.6 as an alternative method of calculating the value of neuron output.

[0098] - If the number of matches is 0-10 the output of the neuron value will be 0.

[0099] - If the number of matches is 11, the output will be 0.1.

[0100] - If the number of matches is 12, the output will be 0.2.

[0101] - If the number of matches is 18, the output will be 0.8.

[0102] - If the number of matches is 20, the output will be 1.

[0103] - In a generalized scheme output= [(number of matches -n) + m] X (1 / m)

[0104] - Where m is the number of the conditions of unmatch where the output RELU is impacted. So, if m=10, any conditions not matched higher than 10, the output will be set to 0.

[0105] There is further flexibility that an output strength can be programmed outside the value determined by the neuron. That value could be 1, and in that case, the output of the neuronROOG-OOIOOWOcan be what the algorithm above determined. However, if one sets the value of output strength to 0.7, as an example and the neuron output was determined by algorithm above to be 0.8, then the output of the neuron will be multiplication of 0.7 and 0.8, resulting in 0.56.

[0106] In some embodiments, input thresholds or neuron output can be modified to support ranges other than 0 to 1 and steps of 0.1. For example, 3 neurons can be considered as 3 decimal values of a neuron instead of 0 to 1. So, the 3 neurons combined act as a value that can be a lot greater. As another example, if a regression sine model is needed with an output of 2 digits, the output can be 2 neurons, where the first neuron would be the first decimal one-digit and the second would be the least significant digit of the output, Advantageously, using this approach an error of 1 digit in the least significant digit is 10 times less error than one digit error in the first digit. As such, when error is calculated, different multipliers can be put on each error of the neuron. Furthermore, if an output is 19, a slight mistake or error would cause 19 to become 20. This is well understood in decimal manipulation, but to teach a neural network may require a lot more training than is needed. To fix this, different outputs can be mapped to new values such that the output digits don’t change massively with a slight error. For example, the order of digits for odd values can be redefined as the most significant digit. Making00, 01,..., 09, 19, 18, 17,..11,10, 20, 21,..,29, 39, 38,..,31,30, 40, 31... This coding change reverses the order from 9 to 0 when the first digit is odd. This scheme allows for slight change in output from a neural network training not to cause major shift to the output. This can be extended to more digits. As an example, 199 output, with one digit mistake or error, would require the output to go to 200. All 3 neurons need to change. This scheme makes sure only one-digit changes by one value. Schemes like this can be applied to binary values as well. Very well-known codes such as gray coding of the values can make the learning of the system simpler since a bit of change would not cause all the digits to have to be changed. As an example, if the value to be predicted is 0111111, and with a small change, all the output neurons have to go to 100000, that is a lot to ask the neural network to learn. Use of codes such as gray coding, or coding explained above for decimal may make the job of the neural networks simpler.

[0107] Figure 1B depicts an alternative embodiment of a computational neuron 100B. Computational neuron 100B can accept multiple inputs and modify the output in a simpler manner as compared to the computational neuron 100A illustrated with respect to Figure 1A. In this embodiment, a bias value is not set, and the programmable threshold value tau (τ) is onlyROOG-OOIOOWOchecked for being higher or lower.

[0108] Figure 2 depicts steps in a method 200 for operating a computational neuron such as described with respect to Figures 1A and IB. In this embodiment, in a first step 210 a neuron receives a plurality of input values. In a second step 220, for each input value, a comparison is made to a threshold tau value. In a third step 230, for each successful comparison, the neuron adds a 1 to a number of matches. In a fourth step 240, based on the number of matches, a neuron output is generated. Optionally, in some embodiments a fifth step 250 can be used to adjust the neuron output using a programmed output strength that is not determined by the neuron. In the third step 230, it can be modified that for each Unsuccessful comparison, the neuron adds a 1 to a number of mismatches.

[0109] Figure 3A depicts a portion of a computational neural network 300A operable using the novel computational neuron described herein. As illustrated, inputs from a conventional flattened layer 310A or other suitable source are provided to input neurons 312A of the computational neural network 300A. These are passed to fully or sparsely connected neuron layers, including a hidden layer 314A and output layer 318A. In some embodiments, all or at least some of these neuron layers can be a computational neuron such as described with respect to Figures 1 A and IBFigure 3B depicts a convolutional neural network 300B operable using the novel computational neuron described herein. Similar to that described with respect to Figure 3A, the neural network 300B receives inputs from a suitable source. These inputs are provided to input neurons 322B of the computational neural network 300B. These are passed to fully or sparsely connected neuron layers, including hidden layers 310B, 312B, 314B, 316B, and output layer 318B. In some embodiments, all or at least some of these neuron layers can be a computational neuron such as described with respect to Figures 1 A and IB to provide a hybrid convolutional neural network.

[0110] In one embodiment, for example, an input matrix 320 of 1000 neurons could be split into 50 neurons of each having 20 inputs of the 1000 inputs. The output of those 50 neurons would then go into 2 neurons of each having 25 inputs for a total of 50 total inputs.Advantageously, these two rows of neurons 322C can replace 1000 input neurons of the current network. However, since the inputs are a subset of the total inputs, as an example, inputs 0-20 go to the first neuron, 21-40 go to the second neuron and so forth, during the training, the learning algorithm needs to be modified to take into account which inputs are going into each neuron. ForROOG-OOIOOWOeach layer of many neurons, the limit of neuron inputs split, and the total neurons divided by that limit and provide at least M (total neurons / limit) neurons. Each output can then be fed into another neuron in the next layer with M neurons / limit, being the number of neurons in that hidden layer. This continues until enough layers are created to account for the total number of neurons that needed to be processed. In some embodiments, M can be more than the minimum (total neurons / limit) with the extra neurons being redundant in terms of having different inputs per each neuron.

[0111] In other embodiments, the hybrid computational neural network 300B can have inputs with thresholds between 0 and 1 and outputs having values between 0 and 1. While such networks can be trained on a variety of tasks, there needs to be changes made to both input and output of conventional neural networks to make it compatible with the new hybrid computational neural network 300B. As an example, if the inputs coming in are numbers between $0 to $1000,000 for house values, the hybrid network can have to have the data coming in massaged to be between 0 and 1; and the output can be between $0 to$ 10,000 for price per square foot.Alternatively, there may be outputs that are in probabilistic form. To remedy this situation while minimizing needed neural network changes, the hybrid network can have either or both of the input layer and output layer support a conventional neuron model. If the output of those neurons is set to a sigmoid function, by default, the outputs can be between 0 and 1 and can be fed into the new ANN layers. For the output layer, inputs of the output layer can be fed from the new hybrid, and the output can be done the same as traditional NN.

[0112] Figure 4A depicts a memory mapped neural network hardware and software system 400A switchable between a memory and a neural network mode. In this embodiment, a NAND memory is capable of operating in either a NAND memory mode or operating in an Al mode supporting neural network emulation. In memory mode, system 400A acts just like a NAND memory capable of reading, erasing, and programming. In Al mode, multiple word lines can get selected, and the sensing is used to determine how many meet the conditions set in by modification of a < or > mode with bias set for each cell. In operation, the memory NAND device, through either command or a mode selection, can be placed in either mode. In the memory mode, the controller would write the values associated with the threshold and bias levels to each cell supporting a computational neuron such as discussed in this disclosure. The memory would then check to see if the data were written properly as one would do for any write operationROOG-OOIOOWOto the memory. In the Al mode, the same array would behave differently. Instead of one row getting selected, all associated rows for each neuron would get selected and the operations related to evaluation of the neuron value would proceed. The outputs of each computational neuron can then be sent to either an internal or an external controller to generate the proper voltages that are sent to the array to evaluate a next layer of neurons.

[0113] Figure 4B depicts structure of a memory mapped neural network hardware and software system 400B based on a NAND array. The NAND array consists of blocks of cells or string of cells in this case 410-416. Each neuron is formed by elements on a bitline (BL) or a column. The content of a neuron is based on the same Wordline (WL) number in each block. Shaded boxes are a part of the same neuron. The inputs to the shaded neuron are 420₁ and 420₂. The thresholds for each one of the τ is programmed into the floating gate of the cell. In this embodiment, the input which compares whether X > τ is the wordline 420. 420₁ can be compared with the value of the top shaded box 410₁ with a threshold which would be τ₀. The second input to the same neuron is 420₂. 420₂ is compared with the value of the top shaded box 410₂ threshold which would be τ₁. The neuron can have many inputs, m, and can span across the first cells in m blocks or NAND strings. If shaded boxes apply to first hidden layer, the same applies to shaded boxes of 414₁ and 414₂. Those shaded boxes can part of another neuron in hidden layer, for example.

[0114] The memory mapped neural network hardware and software system 400B can support various compare functions. Normally, the job of the sense amp 430B is to determine whether or how much current is conducted through a given cell. So, in this example, assuming cell 410₁ was being sensed, all other cells 410₂ thru 416₂ would be disabled. All cells 411₁ 416₁ would go to high value as to turn on and allow current to pass through the string. If cell 410₁ would conduct, then sense amp would determine that and would say the cell is erased. If the cell 410₁ was multi-level cell, then the sense amp would try different values on the wordline to determine the level of conductivity of the cell and provide a 00,01,10 or 11. In effect, cells that have 16 levels of conductivity programmed into them represent 4 cells. So, while they are physically one cell, they are 4 cells logically. In some embodiments, a compare of the wordline value to the threshold that the cell is programmed to is supported.

[0115] A compare < function can be emulated as follows on the system 400B. Assume all the first rows of every string have been turned on. Let’s say there is a neuron represented byROOG-OOIOOWOhaving 2 inputs and one can make sure the input values of the neuron XO and XI are both lower than the threshold or sensitivity of the computational neuron learned during training. Thresholds, or sensitivity levels can be programmed for the first NAND cells of the two consecutive strings. If one wants A and B condition, this is equivalent to (A-+ B-)- The sense Amp and the column combination act like a NOR gate; So, if one can invert A and B conditions, then the sense amp 430B acts like an AND gate function. If A=(xO< TO) AND B=(xl< TI), then if one inverts A and B, in essence one can get (x0> TO) NOR (xl> TI). If the values of the WL’s which are XO and XI are both lower than the corresponding thresholds, the two cells would stay off, and the sense Amp determines that all the conditions are met. Lower vs. higher is determined because the NOR function is done by A. So, if A and B were lower, the two inputs are opposite of the A and B function which are greater than. For the above AND of (X< T) function, it is equivalent to (x0> TO) NOR (xl> TI). The NOR function is supported by the WLs and BLs. If WL value goes to 1, then the NOR output, which is the same as Sense Amp output is pulled low, in effect, the WL interacting with the threshold of the cell acts as (X > τ) function making the WL₀- τ₀ to be the same as x₀-τ₀.

[0116] A compare > function can be emulated as follows on the system 400B. Given the NOR function of the bits and bitline, one can define a “less than” for the cell or (x0< TO). The actual value of X on a cell that has threshold of TO, has a function of (x0> TO). The opposite or inverted function of this needs to be determined by defining the state of the threshold of the cell to be the opposite value of what it should be. So, if one can assume the cell threshold values are between 0 and 1, the threshold of 0.3 would be placed as opposite which is (1-0.3) which is 0.7. So, if there is a check for X>0.3, one is actually check for threshold being >0.7. Furthermore, the wordline should also be inverted as well. This way if X is 0.4, the wordline would be (1-0.4) or 0.6. In this case, if one wanted to check (0.4>0.3), but were looking for inverted version to go to NOR. The (0.4>0.3) function needs to provide 0, which is the inversion of above. In this case, 0.6 >0.7 would result in 0. Since the WL acts as WL > threshold or (x> T), the opposite function is being fed into the NOR gate, which solves our problem.

[0117] A simultaneous compare > and < function can be emulated as follows on the system 400B. Given bias for each input decides whether to check if the input X is higher, if bias =1, or l, or Lower, if bias=0. Once the training occurs, a controller would program the values of thresholds for each neuron input as well as bias value for that neuron input. Regardless ofROOG-OOIOOWOwhether the condition of higher or lower, the success criteria for deciding whether the condition was met is to make sure there is no conduction in the NAND string. In this embodiment, 2 cells in the serially connected string of cells can be assigned to one neuron. One of the cells is to check whether the cell is < if bias is set to 0 and the other cell of the pair is to check whether the cell > threshold if bias =1. In this embodiment the two cells 410₁ and 414₁ and 410₂ and 414₂ are assigned as pairs. Shaded boxes check for the function < and >. If the bias is determined to be lower or 0 for the neuron input, one of the shaded box conditions is to be checked. If the bias is determined to be for higher or 1 for the neuron input, the other shaded box cell condition should determine the condition for that cell. In either case, the other shaded box cell of the pair will stay erased as the string conducts and the right cell would determine whether the string would conduct depending on the value of the wordline and threshold. The WL values placed on WL3 and WL1, 420₁ and 424₁ are determined by the value of X coming into the block 420. The Value of 420₁ will be the value corresponding to the input value X. Assume the output of the prior neuron was 0.4v. For the sake of this example, assume that 0.4V is the value that one would put on the wordline; So, 420₁ will see 0.4V. The Value of 424₁ will be the analog opposite of 0.4V. Using the earlier example WL voltage, (1-0.4) or 0.6V placed on it. That is if bias is 1 which is for checking > function. The cell Threshold associated with this example would be (1-0.3) or 0.7.

[0118] As will be understood, the memory mapped neural network hardware and software system 400B based on a NAND array can be arranged to provide a novel RELU implementation. Commonly, a ReLU (Rectified Linear Unit) is an activation function in neural networks that outputs the input value directly if it's positive, and outputs zero if it's negative. As an example, a multilevel cell (MLC) can replace the sense amplifier 430B. The MLC can have multiple registers showing the level of conduciveness of the selected cell. The conductivity or lack thereof of each neuron input depends on whether that neuron was checking for higher or lower condition. Further, the current for each string is limited to a fixed number by controlling the gate of VGS or VDS. By providing a current mirror mode, an analog voltage can be set on those gates that regardless of the gate overdrive of the cell, if conducting, will ensure the cell only draws a fixed amount. In effect, this scheme checks to see how many of the cells are conducting and provides an MLC equivalent as to how many of the cells are actually meeting the condition of either > or < as assigned to them. It provides a RELU type of output, so it is a multistate output NOR Gate, instead of providing a 0 or 1, and it provides a gradient output that shows how manyROOG-OOIOOWO1’s there are. 1 means 0 conduction and 15 / 16 means 1 cell conducting and so on.

[0119] In case the number of inputs to a neuron is limited to n, this example RELU system can count X number of inputs at a time and allow for sense amps to count how many do not match their criteria of either < or >. The RELU accumulator can then send the total for the X number to an external controller or an internal controller to count the total number after X number of inputs to a neuron are counted Y times. In other words, let’s say that each neuron can have 16 inputs, and the n number mentioned in the definition of neuron as the total number of inputs that impact the value of the output of the neuron is 16, then the system selects 164-neuron, and the sense amp counts how many meet the condition of higher or lower. The sense amp then sends the neuron, binary that shows how many passed to a controller. The system then enables the next 16 rows which are 16 other inputs of the neuron, and the sense amp then determines how many of them do not match the higher or lower condition. The next 4 digits, representing how many match or don’t match will then be sent to the controller and it adds it to the total match number. This process will continue to get a total number of mismatches for the entire neuron inputs, or part thereof.

[0120] In some embodiments, a companion chip or a controller on the current chip can add the total number of matches of each count and add them to the total of previous counts of 4bits and provide a much higher RELU value to the companion chip or the internal chip. The companion chip can then decide what levels of voltage should be applied to the next neuron layer based on the count. In some embodiments the companion chip can be an application specific integrated circuit (ASIC)

[0121] As will be understood, the memory mapped neural network hardware and software system 400B based on a NAND array can be arranged to support an Al network. As an example, a first hidden layer can be defined for all the cells in the first location of the string of NAND. The sense amp 430 determines the relationship between all the inputs for neurons which were on a BL or column 1 and provided an output sent to all the neurons in a second hidden layer. A second column, including the first cells in the NAND string still belongs to a first hidden layer. For a second neuron, the output of a column is determined by the sense amplifier of that column. The generated output that can be used as the second input for second hidden layer. An input B can be then sent to all or some of the cells in second hidden layer.

[0122] In one embodiment, a memory mapped neural network hardware and softwareROOG-OOIOOWOsystem is able to support computational neuron operation such as described herein. In one example, every 8 columns are organized to have one sense amp, with 32K resulting in around 4K outputs, which can then feed into around 4K inputs to the next hidden layer. The array architecture needs to adapt the number of outputs from a previous stage to match the number of inputs of the next hidden layer. The total hidden layers become 255 times the number of columns that are multiplexed for each sense amp., in this example, 8. That would mean that the total number of hidden layers would be 2k hidden layers. If one multiplies each one of these by 4 to account for number of banks, one can get the total number of parameters. In this case, there is 1Tb of data available for storing parameters. In some embodiments, the structure can be designed in such a way that the number of sense Amplifiers is equivalent to the number of LUNS, which is # of block X 4 in this case. If there is less than the total inputs available, the array can be organized in such a way to block off some of the columns for either a new network or an extension to the network neuron count.

[0123] In some embodiments, other banks can be arranged to increase the number of 4-bit, and the output of the neurons would go to the input of all the LUNS which is 4X times more. In this example, if there is one set of sense amps for all banks, and there is a 4 bit register for RELU calculations, the output of these registers would then turn into an analog voltage that would go to each cell that is 4-bit storage, and it can decide whether the output of the last stage of network is > or < the threshold set for the cell.

[0124] In some embodiments, the architecture of the neuron in memory is such that one column is the computational neuron as discussed herein. The computational neuron can be a portion of the column or BL. In some embodiments 10 WL’s can be selected at a time and a sequential read of the total number of matched vs. unmatched can be made. The controller can then sum up the total in the next step. This situation of serializing the evaluation of the portion of the BL would be efficient in terms of silicon use but may be slower since a number of serial operations need to be performed. Alternatively, one can allow one neuron to be mapped into multiple BLs. Alternatively, one can make the portion of neuron that is on a different BL a “don’t care” with respect to evaluation, with the input from a given earlier neuron not being allowed to impact how a current neuron behaves. In such a case, all the bitlines can be read in parallel and the values can be sent to a companion chip which can add the number of mismatches and come up with what the value of the neuron would be if n were much larger than 10.ROOG-OOIOOWO

[0125] Generally, “don’t care” functionality includes having an ANN that supports a number of layers, including hidden layers and the connection for each neuron is fully connected. In one embodiment, connectivity of each neuron is by the structure defined by the memory mapped neural network hardware and no parameters need to be shuffled around. In some embodiments a string can be divided into the upper half or the lower half, both related to the same cell. In other words, each neuron input would take on two cells. One cell is programmed to check for the condition of < and one other cell in the string is used for checking whether > function is called for. Both these cells are associated with the same input in a given neuron. Thresholds can be programmed into the threshold of NAND cells. Whether bias is 0 or Iwhich determines < or > function can be decided at a controller that is programming the NAND array. If < function is selected, the upper part of the string that is allocated for the < or bias=0 and the actual threshold is programmed in the cell. The mirroring cell that would have checked for the > function or bias=l would be left as erased so the string would conduct. If the bias = 1 and checking for > function, the controller would then program (1 -threshold) into the cell that is in the group of cells that is checking for > function. In the case of checking > function, the associated < function for the cell can be left as erased which conducts without respect to what voltage is put on the wordline.

[0126] In those cases where the “don’t care” condition is supported, cells associated with > or < would be programmed to the highest threshold. This threshold is higher or equal to the highest value voltage of the wordline and keeps the cell off no matter what. In some embodiments, one can program both thresholds to a level high enough that one of them stays off.

[0127] Advantageously, the memory mapped neural network hardware and software system permits a programmable output strength for each neuron to be processed, with each neuron having an array that is for Al calculations and another array for the value or the strength of the neuron output. One array in the memory can be allocated for storing the values that need to be processed, and another array can have Al comparison circuitry. When the outputs from a data array are read and output from the Al comparison side are read, they will both be shifted to the companion controller or chip. The data array can store the strength for each neuron, and an Al comparison circuit output can have an output of RELU showing how many of the conditions have been met. In some embodiments, the companion chip can take these two as inputs and provides an output, in this example a multiplication for each neuron to provide an output thatROOG-OOIOOWOwould be fed to the next neuron layer.

[0128] As an example, to get the output of RELU to show 0.8 for a given neuron, then 0.8 would be the input fed to the next layer of neuron hidden layers. With this capability, if the output strength is set to 0.7, then even if RELU’s output is 1, the value passed will be 1 * 0.7 which is 0.7. For this example, even though the output of RELU is 0.8, the companion chip will multiply 0.8 * 0.7 and passes 0.56 as the value of the output of neuron to the next neuron hidden layer.

[0129] The above discusses a simple multiplication, but other algorithms may be used to generate the output of the neuron for the next layer. During the training and / or upon completion of the training, the value of the output strength of the cell can be programmed in the data array which could be in the same NAND Al Chip or in another nonvolatile chip. At some point, one can move all the strength data to a volatile memory to access faster as well.

[0130] Figure 4C depicts a memory mapped neural network hardware and software system 400C that is able to support the computational neuron as discussed herein. As illustrated, system 400C can have 128K columns, and 16K sense amps (1 sense amp for every 8 columns), giving 16K neurons per layer (NPL) and 16K inputs per neuron (IPN). In this example, the total hidden Layers = CHL * RHL = 8 X 200 Hidden Layers = 1.6K (HL). Total parameters in this example = Total Number of Connections = NPL X IPN X HL ((16 X 10³) X (16 X 10³) X (1.6 X 10³)) for a total of 410 X 10⁹. Total Neurons = NPL X HL ((16 X 10³) X (1.6 X 10³)) for a total of 25.6 X 10⁶.

[0131] As will be appreciated, in other embodiments a lossless parallel network can be constructed, with 8 parallel ANN provided using 8 column hidden layers (CHL). Such a system can support 50 X 10⁹ parameters and 3 X 10⁶ total neurons.

[0132] In another embodiment, sequential parallel networks can provide 4,8, or 16 additional parallel ANNs through IPN. In this embodiment, 8 Column Hidden Layers (CHL) can be used in conjunction with IK sense amps for total parameters = 200 X 10⁶ and total neurons = 200 X 10³

[0133] The training of the memory mapped neural networks such as discussed herein can be done using the computational neuron model described with respect to Figures 1A and IB. In some embodiments, due to slowness of the writing process in memory mapped neural network chips, a combination of volatile and non-volatile capabilities can be applied. Parallel approachesROOG-OOIOOWOcan also be applied to the memory mapped neural network. Instead of erasing to rewrite a part of the code, other planes or sections of the memory can be utilized. So, an erase operation does not need to happen, eliminating a slow process during the training process.

[0134] In some embodiments, the controller can mark a block as dirty, and the memory mapped neural network chip can have capabilities during training to not use certain blocks for determination of the value of a neuron. A group of blocks can be selected via stochastic gradient descent for training, and all of those blocks could be used as new input to the neuron, and a group of SGD blocks can be taken out of commission to be replaced in case of their parameters being modified and no longer being valid. This can also be done via the wordline voltages being set to zero for Higher than function and be driven to high value for the < function. The training mode can also be configured to allow certain capabilities to be turned on to improve the performance of the learning system.

[0135] In some embodiments, in effect an array can be broken to get more neurons. A column can have multiple neurons mapped to it. When a neuron is selected, only the cells of the wordlines associated with that neuron are able to conduct. Alternatively, other mechanisms can be used to shut off the conduction path of cells. Different schemes such as turning off the SGD or SGS in different parts of the array are also possible, translating into operation of different colors. So, when the color red is on, the only cells that can conduct are in the red zones.Therefore, even though all the cells are turned on due to wordlines of the upper wordlines turn on the neuron sections in the red zones and the bottom sections of those specific columns are not able to affect the current on the column which the sense amp uses for that column. In the middle of the array where there is another red section, the cells in that section are impacting the values of the neurons. So, while all the sense amplifiers get activated for all columns and all rows, only the colors that are selected are able to impact the neuron values.

[0136] In another embodiment, the portion representing the green parameters can be copied into X number of WLs in a tier, so the WL counter doesn’t need to stay the same for recurrent neural networks. X has to be decided, and in this case can be the maximum length of a sequence.

[0137] In still other embodiments, inputs can be a combination of new inputs from outside network, as well as previous outputs of neuron states. For example, when output of 2ndhidden layer is transferred to the controller to be used in hidden layer 3, a copy of that data is kept in registers for future use. Typically, a compiler can be used to let the controller know to keep theROOG-OOIOOWOoutput, or portion thereof, in local registers and use them as inputs to the layer that requires that past history.

[0138] In some embodiments, one can have X number of inputs coming from outside and Y coming from inside. The inside could be from the previous layer as may be the case for RNN’s or could be from layers older than the last layer. The assignment of which inputs are associated with old versus new is something that can be decided as necessary or could be hard coded. In other words, one can say that first Z inputs are from new material and Y from previous layers. Y could be the total number of neurons per last layer since one doesn’t know which ones of the outputs will need to be selected. For a fixed range, the values that the controller would select for the first set of inputs, for the range from previous output, would be designed in a way so as not to interfere negatively in the outcome. So, since there is no history from past words, the portions would need to act as don’t care with proper inputs.

[0139] As previously discussed, in some embodiments a computational neural network operable using the novel computational neurons described herein can be associated with various accompanying computer systems that are not based on machine learning, Al, or neural networks. For example, a companion chip such as discussed with respect to Figure 4A and based on an application specific integrated circuit (ASIC), a general processor, or other suitable logic components can be used to improve performance of the computational neural network operable using the novel computational neuron described herein. In one embodiment, an Al chip using novel computational neurons described herein can do the network and parameter calculations and matrix values, while a companion chip could be an ASIC that can do most of the multiplications or other math calculations. The companion chip can get the output of the layer sent to it in a serial fashion. The companion chip can also read the redundancy information that is stored in the memory Al chip during production.

[0140] In some embodiments, registers of a companion chip that hold the data from either outside or older outputs of the hidden layers can be used. When the inputs for the next hidden layers are sent, there could be data mixing from previous layers and data from the current layers.

[0141] Memory mapped neural network hardware can also send information regarding some matrices that are not output of Al network. In some embodiments, a companion chip such as discussed with respect to Figure 4A can take output of the current or previous hidden layers, and modify their content based on either the registers or data that is sent to the companion chip toROOG-OOIOOWOgenerate new data that is sent to the next layer as input.

[0142] Depending on how many registers are on the companion chip, the output that is sent to the chip for next hidden layer Al analysis and calculations can be combined with other data or manipulated by other matrices, deciding on how deep or forward layers can be. In other words, if output of a second hidden layer is kept in a register, one can forward that to many layers forward. If the recurrent neural network doesn’t know how many layers of output data can be stored, all layers can be stored and when input for layer n is created, any of the previous n registers that account for 1-n outputs can also contribute.

[0143] In some embodiments, when there are a deep neural network and its outputs are multiplied by 3 matrices, the actual multiplication or dot function could be done in the companion chip. The value of the resulting matrix can be used as input to the new type of ANN, and most of the advantages of the new neural network would be available.

[0144] For example, the value of different matrices can also come from other sources. The V matrix for each word can be a look up table which may be stored in the memory Al chip and can be given to the companion chip upon a request. The K matrix having a list of keywords would then ask a memory Al chip to provide the matrix associated with each one of the keywords. During an embedding process, the new matrix related to the sequence can then be forwarded to the memory Al chip discussed with respect to Figure 4A. Attention function can be pursued using the processes mentioned and are well understood using the new memory Al chip.

[0145] In other embodiments, output is calculated by the companion chip which determines the number of “passed conditions” through multiple reads of using multiple columns. The companion chip can then decide how to split the overall matching value to values on multiple neuron inputs. If the neuron can only calculate 4 bits of accuracy of higher or lower value, in the above case where the output of multiple neurons of 4 bit each can be accumulated, the companion chip can then split the summation value showing the value of how many nonmatching totals, into 4-bit inputs that can be fed into inputs of the neuron of next layer. It will be understood that 4 bit is used as an example, and that value can be done depending on the use case and application as well as accuracy needs of the application and the training program that utilizes such system.

[0146] In some embodiments, through techniques such as matrix decomposition as well as matrix factorization, it is possible to assemble larger matrices into smaller size matrices perROOG-OOIOOWOlimitation of the new neuron being able to perform multi-layer or have the companion chip do some multiplication of resulting matrices to get to larger matrices.

[0147] There will be parallelism in reading and getting inputs from a memory Al chip and calculations associated with matrix values. As an example, if one were to read entire 32K inputs from a neural network layer, when the first value is read for both the Al valuation and other associated values for that neuron, the mathematical activity such as multiplication and / or standardization can be done while other inputs are being read.

[0148] The sensitivity of Sense Amp may only allow 4 bit of accuracy for a given number of neuron inputs. However, through multiple readings, and passing the output values of neurons to either the companion chip, or an internal controller, the numbers can be added to get a total number of mismatches or matches by adding the different number together.

[0149] In some embodiments, one can also have multiple bitlines or columns representing part of a given neuron and their sense amp may show 4 bit sum of all the matches or mismatches, and the controller can add all of them together to get a sum which is bigger than what a single sense amp can do. The controller then can decide, based on the value of the sum, what value needs to be placed on the neuron of the next layer. In other words, if 8 bit accuracy is the total, it can decide what portion or how to provide levels for next neuron layer and algorithm to allocate.

[0150] In some embodiments, with applications in Al requiring fast or high bandwidth data, this new memory Al chip such as discussed herein may have need to access large amounts of data fast. This can be addressed by using high bandwidth memory (HBM) with the memory Al. This allows for buffered data from storage to be high speed available to the memory Al chip and its companion chip to do fast response operation such as is needed for real time Al systems like self-driving.

[0151] Other techniques can be used for improving speed. For example, through silicon vias (TSV) can be used to connect the HBM to either the memory Al chip or the companion chip. Alternatively, or in addition, an interposer can be used to connect the different chips of the chiplets. The chiplet could contain memory Al and its companion processor chip as well as more storage chips such as NAND Flash as well as HBM. A system level solution with elements described on the chiplet above can perform Al tasks at the required speed and making tradeoffs between the amount of memory Al as well as HBM size and external NAND Storage devices.

[0152] In some embodiments, the above ANN systems include a software stack necessaryROOG-OOIOOWOto perform operations for the given elements in the system. Such systems or subsystems can do tasks such as transformer functions or support long short-term memory (LSTM) or other recurrent processing concepts necessary for some Al tasks such as LLM’s.

[0153] Current inference of a large mode typically requires a large number of GPU’s, high bandwidth memories, and associated storage to go with it. For models of around 100 billion parameters, one needs 10’s to 100 millions of dollars for a datacenter to do not just processing, but to do it fast and manage the power and data channels as well as cooling. Advantageously a system of comparable functionality can be developed using only one chip for the memory Al chip and network calculations along with a companion processing chip to do the mathematical manipulations of the data being generated.

[0154] As mentioned previously, the memory Al chip such as described with respect to Figure 4A has all the parameters stored, as well as proper connectivity and routing of such signals and doing the operations of the neurons such as > or < functions. The companion chip can receive the output of the memory Al chip as well as content of different types of matrices such as Q matrix, K and V matrix values. It will then do the necessary functions such as dot product to evaluate the relationship between words in a prompt. The results are then sent back to the memory Al chip for further processing or into a feedforward neural network (FFNN). The two chips together manage to perform Al activities both during learning as well as Inference. The companion chip may also perform some functions that are used in traditional neural networks in case of hybrid networks.

[0155] In some embodiments, the output of all sense amplifier registers can be sent to be used as input of the next layer neurons. This allows the register outputs to be routed to the external companion chip. The chip can then decide:

[0156] Column redundancy to switch the outputs related to defective BL with the BL output from redundant BL.

[0157] The information about the column redundancy will have been transferred to the external controller.

[0158] Whether the external controller can override only the input register that is defective if the transfer to internal registers happened only. In other words, if the outputs of sense amplifiers don’t get routed, the external controller can reach in and change only the redundant registers.ROOG-OOIOOWO

[0159] In some embodiments, the companion chip can implement the values of each threshold depending on bias values. The companion chip can also manage “don’t care” activities. If there is a math function where an exact value is needed and it does not have RELU impact on the output, as an example, one would want to make sure A> B, then depending on threshold of next stage being very high for > or very low for < This can be done via training. In case of transformers, where there are a deep neural network and its outputs are multiplied by three matrices, the actual multiplication could be done in the companion chip. In some embodiments, value of the resulting matrix can be used as input to the new type of ANN, and most of the advantages of the new neural network would be had.

[0160] In some embodiments, at the companion chip, multiple neurons from the memory Al can be combined to create a super neuron based on summing the individual neurons of each being one column.

[0161] Companion chips can be used during training can get the strength setting and provide the multiplication or other functions to the output of the neuron cells to feed into the next neuron layer. The strength can either come from the same array that holds Al comparison circuitry on the memory Al chip, in a data section of that chip, or can come from a completely different chip, either volatile or nonvolatile chip. In some embodiments, the companion chip can get the bias values of given neurons from either the Al comparison circuitry, or from another chip that holds the bias values. These values are used to decide what level of voltages are applied to what wordline of the new neuron Layer as explained earlier.

[0162] Values transferred from a data portion of a nonvolatile chip can be transferred in parallel with the data from Al comparison circuitry portion of the chip. This operation can also be done in series depending on the timing requirements of the design. These values can also be transferred to the companion chip from a different nonvolatile chip or from the volatile memory that got the data earlier copied to it.

[0163] In some embodiments, values such as matrices that hold different matrix values, such as the ones needed for transformers like value vectors and Wq and Wk and Wv are transferred to the companion chip.

[0164] Values that modify the embedding of some vectors can be kept in the companion chip, or it can write them into volatile memory chips such as HBM for future use.

[0165] In some embodiments the companion chip as well as the Al comparison circuitryROOG-OOIOOWOand other nonvolatile and volatile memory chips can be done in a chiplet format. The channels of communication using TSV’s can also be designed individually for each depending on electrical and timing requirements of each chip.

[0166] In some embodiments, activities such as softmax can be done via the processing capabilities of the companion chip. For classification type problems, the softmax function calculates probability of an output being selected relative to all other outputs. In cases where the expected output is one hot, (one of the outputs is only selected), the output of the new neural network architecture may be 1 from the number of outputs. If one wants to see the output as probability function, the error would be Expected Value (one 1 and rest 0) - Output value. This error can be used in calculation for backward propagation to modify the values of parameters such as Bias and Tau (Thresholds). In this case, the output should be either 0 or 1 to calculate error and not probability.

[0167] In effect, an ANN is created where the output has a binarification at the end. In other words, a neuron that can become 0 or 1 with just 1 input. If the input is above a threshold, the output of that neuron will become 1 otherwise, 0. This neuron will set in front of the output layer, but it only gets one input from one of the neurons instead of fully connected where it gets input from all of the neurons of the output layer. Alternatively, one can use a SoftMax to decide the probability of an outcome in a classifier but use the output of the last layer that was regular value for calculating the error.

[0168] Figure 5 depicts steps in operation 500 of a memory device supporting computational neurons forming a neural network. In this embodiment, in a first step 510 a memory device is provided that has an array of memory cells configured in rows and columns, where each column represents a single neuron. In a second step 520, incoming values for each row by input values for each of the neurons are determined and selected. In a third step 530, operations by the memory device based on the incoming values of the row and the threshold value are programmed on a plurality of cells associated with the neuron, where the conduction of a cell represents a failure in the operation being checked. In a fourth step 540, a value from a sense amplifier is outputted in accordance with how many of the cells were determined to be conductive.

[0169] The training operation of the memory and neural network hardware described herein can be based at least in part on various mathematical modelling techniques. For example,ROOG-OOIOOWOoperation of mathematical models suitable for a computational neuron network can include use of sigmoid to determine higher or lower values of processed inputs after a neuron receives a plurality of input values. A sigmoid can be used to determine higher or lower values of processed inputs. In one embodiment, for each input value, a comparison is made to a threshold tau value. How many of the inputs are either higher or lower than the set thresholds can be summed and number of matches determined. In some embodiments a neuron output equal to{(£”<5[(— 1 + 2bi)(xi — zi)] -n) + m} X (1 / m) can be generated.

[0170] In more detail, and in other embodiments, operation of mathematical models suitable for a computational neuron network can include alternative use of sigmoid to determine higher or lower values of processed inputs. The sigmoid can be either 1 or 0 depending on the value of the sigmoid function. If the function value is >0, the output is 1, otherwise output is 0.

[0171] 8(xi — Ti)= 0 if xiO; 8(xi — zi)= 1 if xi > 0

[0172] For a given neuron with< n inputs, one can count number of times (xi > zi) by:

[0173] ”=0<5(xi — zi)

[0174] One can count number of times (xi < zi) by:

[0175] X"=05(Ti - xi)

[0176] For cases where b=l, checking to see if (xi > zi), one can get:

[0177] ^8(bi - 0.5) 8(xi - zi)

[0178] For cases where b=0, checking to see if (xi < zi), one can get:

[0179] ”<5(0.5 - bi) <5(zi - xi)

[0180] To sum up how many of the inputs are either higher or lower than thresholds set, one can add the two functions above:

[0181] £”[<5(bi - 0.5) 8(xi - ri)]+ [<5(0.5 - bi)<5(zi - xi)]

[0182] Simplification of above results in:

[0183] Number of matches = £” <5[(— 1 + 2bi)(xi — zi)]

[0184] Output= [(number of matches -n) + m] X (1 / m)

[0185] Where n was the total number of inputs to the neuron and m was RELU Sensitivity

[0186] Neuron Output ={(£” <5 [(—1 + 2bi)(xi — zi)] -n) + m] X (1 / m)

[0187] As will be understood, for learning, a gradient descent approach can be used.ROOG-OOIOOWO

[0188] In some embodiments, Tau-Bias (TB) neurons having two families of learnable parameters: thresholds T and biases b, can be used. During training, one can maintain continuous-valued copies TcontG [0,1] and bcontG [0,1], which are the targets of optimizer updates. However, since a forward pass operates on discretized versions the thresholds can be rounded to one decimal place and biases are binarized to {0,1}. Gradients are computed with respect to these discretized values during the backward pass, and the resulting updates are applied to the continuous parameters. To apply any gradient-based optimizer update, it is dE dErequired that — and —. Because the discretized predicate function is a step function (zero gradient almost everywhere), straight-through estimators (STEs) can be constructed that supply usable approximate gradients near each predicate’s decision boundary. In more detail, in the following derivation predicates evaluate non-matches — i.e., cases where the input fails to satisfy the threshold condition. Consequently, any count tallied below tracks failures rather than successes. This inversion simplifies the gradient expressions that follow; since the count as defined here is inversely correlated with the match count used elsewhere in the system.

[0189] NotationSymbol MeaningXj G [0,1] Input activationy G [0,1] Learnable threshold for neuron i, input jb[ j G {0,1} Learnable bias (predicate direction)Pi G {0,1} Predicate: 1 if satisfied, 0 otherwisea Scaling factor (typically 0.1)8 STE window half-widthdE Upstream gradient arriving at neuron i's outputdyi

[0190] Forward pass:count; = pij, zi = l - a - count;, yt= ReLU(z;)jThe predicate p;y = 1 when Xj < T;;- (if b;;- = 1) or Xj > T;;- (if b; j = 0).

[0191] Chain Rule StructureROOG-OOIOOWOConsider a three-layer network with layers I -> J K, where / feeds into J and J feeds into K. Each connection between layers carries its own T and b parameters. One can derive the gradient for Tjk— the threshold on the connection from neuron j in layer J to neuron k in layer K — as the motivating single-layer case. The chain rule factors the gradient into three terms:dE dE dykdcountfcdTjkdykdcountfcdjjkupstream ReLU + scale local STEdEThe first factor — is given — it arrives from whatever loss or downstream layer sits aboveneuron k. The second factor is the derivative through the ReLU and the linear scaling:^yk > f— a if county < 1 / adcountfe*-0 otherwiseThe third factor is the problematic one: the true derivative of the step-function predicate is zero almost everywhere. The STE approximations below replace this factor with a usable surrogate.

[0192] STE for T

[0193] The problem. The predicate pjkis a Heaviside step function of Tjkit flipskbetween 0 and 1 at the boundary Xj = Tjk. The true derivativeis a Dirac delta — zeroeverywhere except at the discontinuity, where it is undefined. No gradient flows.

[0194] Window approximation. One can introduce a proximity window of half-width £ > 0. When the input Xj lies within of the threshold Tyfc, a small perturbation of T would flip the predicate, so one can assign a unit-magnitude gradient. Outside this window, the predicate is insensitive to small T changes, so the gradient is zero:

[00195] = l[|xy- TJik| < £]

[0196] Sign determination. The sign depends on the predicate direction set by b] k:

[0197] When b] k— 0, the predicate tests Xj > jk. Increasing T raises the bar, making satisfaction harder. Near the boundary, count decreases, so the sign is — 1.

[0198] When b] k— 1, the predicate tests Xj < jk. Increasing T lowers the bar, making satisfaction easier. Near the boundary, count increases, so the sign is +1.r-1 bl k= 0

[0199] aj k= 2bjik- 1 = |+1 = ±

[0200] Assembled STE:ROOG-OOIOOWOd county (T)

[0201] — - - = W; / • O';kdTj,kJ-k

[0202] STE for b

[0203] The problem. The bias bjkis a discrete switch in {0,1} selecting which comparison direction the predicate uses. It has no continuous gradient.

[0204] Conceptual perturbation. One can reason about the effect of moving b from the “x > ” regime toward the “x < i” regime. Unlike T, no proximity window is needed — flipping b always changes the predicate’s truth value (except in the measure-zero case Xj = Tj'ic). The change in count depends on where Xj sits relative to

[0205] I f Xj > Tjkunder b = 0 the predicate is satisfied (contributes 1 to count); under b = 1 it is not. The flip removes a count, so the gradient is — 1.

[0206] If Xj < Tj. under b = 0 the predicate is unsatisfied; under b = 1 it is satisfied. The flip adds a count, so the gradient is +1.

[0207] Assembled STE:

[0208] = 1 - 2 • l[x. > T, J = I-1if*'>dbj,kL 7 J’J(+1 otherwise

[0209] Multi-Layer Backpropagation

[0210] Define the upstream signal / ? that accumulates the chain rule factors as gradients propagate backward through the I → J → K network. At the output layer K:

[0211] pk= -r~ -dyk3countte

[0212] For any earlier neuron a in layer with downstream neurons b in layer ℓ + 1:[100213J] ft = ft • dy& ) dcounta

[0214] As / ? propagates backward through the network, the gradient for any parameter at any depth reduces to:dE „ d county dE > > 5 county

[0215] - — = Py ’ -(^Tx,yb?x,ydl>x,y y &bx,y

[0216] Regardless of depth, each parameter gradient is its neuron’s accumulated / ? multiplied by the local STE factor derived above.

[0217] As used herein, data’ and ‘information’ can be used interchangeably (e.g., ‘data processing’ and ‘information processing’). A data structure is used in commerce to transform anROOG-OOIOOWOelectronic device for use as a specific machine as an article of manufacture. Data and information are physical objects, for example binary data (a ‘bit,’ usually signified with ‘0’ and ‘1’) enabled with two levels of voltage in a digital circuit or electronic component. As used herein, the term ‘process’ signifies an artificial finite ordered set of physical actions (‘action’ also signified by ‘operation’ or ‘step’) to produce at least one result. Some types of actions include transformation and transportation. An action is a technical application of one or more natural laws of science or artificial laws of technology. An action often changes the physical state of a machine, of structures of data and information, or of a composition of matter. The signifiers ‘algorithm,’ ‘method,’ ‘procedure,’ ‘(sub)routine,’ ‘protocol,’ ‘recipe,’ and ‘technique’ often are used interchangeably with ‘process,’ and 35 U. S. C. 100 defines a “method” as one type of process that is, by statutory law, always patentable under 35 U. S. C. 101.

[0218] As used herein, the term ‘rule’ signifies a process with at least one logical test (signified, e.g., by ‘IF test IS TRUE THEN DO process’). As used herein, a ‘grammar’ is a set of rules for determining the structure of information. Many forms of knowledge, learning, skills, and styles are authored, structured, and enabled - objectively - as processes and / or rules e.g., knowledge and learning as functions in knowledge programming languages.

[0219] As used herein, the term ‘component’ (also signified by ‘part,’ and typically signified by ‘element’ when described in a patent text or diagram) signifies a physical object that is used to enable a process in combination with other components. For example, electronic components are used in processes that affect the physical state of one or more electromagnetic or quantum particles / waves (e.g., electrons, photons) or quasiparticles (e.g., electron holes, phonons, magnetic domains) and their associated fields or signals.

[0220] As used herein, the term ‘processor’ or application specific integrated circuit (ASIC), signifies a tangible data and information processing machine for use in commerce that physically transforms, transfers, and / or transmits data and information, using at least one process. A processor consists of one or more modules, e.g., a central processing unit (‘CPU’) module; an input / output (‘I / O’ ) module, a memory control module, a network control module, and / or other modules. The term ‘processor’ can also signify one or more processors, or one or more processors with multiple computational cores / CPUs, specialized processors (for example, graphics processors or signal processors), and their combinations. Where two or more processors interact, one or more of the processors can be remotely located relative to the position of theROOG-OOIOOWOother processors. Where the term ‘processor’ is used in another context, such as a ‘chemical processor,’ it will be signified and defined in that context.

[0221] The ASIC or processor can comprise, for example, digital logic circuitry (for example, a binary logic gate), and / or analog circuitry (for example, an operational amplifier). The unit also can use optical signal processing, quantum operations, or a combination of technologies, such as an optoelectronic processor. For data and information structured with binary data, any processor that can transform data and information using the AND, OR and NOT logical operations (and their derivatives, such as the NAND, NOR, and XOR operations) also can transform data and information using any function of Boolean logic. A processor such as an analog processor, such as an artificial neural network, also can transform data and information.

[0222] The one or more processors also can use a process in a ‘cloud computing’ or ‘timesharing’ environment, where time and resources of multiple remote computers are shared by multiple users or processors communicating with the computers.

[0223] As will be understood, processors can run or communicate with Al system architectures, including neural network architectures suitable for use in the described systems.

[0224] As will be understood, a computer system 600 such as illustrated with respect to Figure 6 is suitable for interacting with or supporting Al system and other computational embodiments described in this disclosure and can include at least one computer which communicates with peripheral devices via bus subsystem. Typically, the computer system includes one or more processors (e g., a microprocessor, graphics processing unit, or digital signal processor), or its electronic processing equivalents, such as an Application Specific Integrated Circuit (‘ASIC’) or Field Programmable Gate Array (‘FPGA’). Typically, peripheral devices include a storage subsystem, comprising a memory subsystem and a file storage subsystem, user interface input devices, user interface output devices, and / or a network interface subsystem. The input and output devices enable direct and remote user interaction with the computer system. The computer system enables significant post-process activity using at least one output device and / or the network interface subsystem.

[0225] The computer system can be structured as a server, a client, a workstation, or any data processing machine with instructions that specify actions to be taken by that machine. The term ‘server,’ as used herein, refers to a computer or processor that typically performs processes for, and sends data and information to, another computer or processor. In some embodiments, theROOG-OOIOOWOcomputer system can send data or distribute processes to a computer cloud or other available collections of computing systems.

[0226] A computer system typically is structured, in part, with at least one operating system program. The computer system typically includes a Basic Input / Output System (BIOS) and processor firmware. The operating system, BIOS and firmware are used by the processor to structure and control any subsystems and interfaces connected to the processor.

[0227] Any embodiment is not limited to an electronic digital logic computer structured with programs or to an electronically programmable device. For example, the claimed inventions can use an optical computer, a quantum computer, an analog computer, or the like. Further, where only a single computer system or a single machine is signified, the use of a singular form of such terms also can signify any structure of computer systems or machines that individually or jointly use processes.

[0228] The network interface subsystem provides an interface to outside networks, including an interface to a communication network, and is coupled via communication network to corresponding interface devices in other computer systems or machines. Communication networks can comprise many interconnected computer systems, machines, and physical communication connections (signified by ‘links’). These communication links can be wireline links, optical links, wireless links (e.g., using the Wi-Fi or Bluetooth protocols), or any other physical devices for communication of information. A communication network can be any suitable computer network, for example a wide area network such as the Internet, and / or a local-to-wide area network such as Ethernet. The communication network is wired and / or wireless, and many communication networks use encryption and decryption processes, such as is available with a virtual private network. The communication network uses one or more communications interfaces, which receive data from, and transmit data to, other systems. Embodiments of communications interfaces typically include an Ethernet card, a modem (e.g., telephone, satellite, cable, or ISDN), (asynchronous) digital subscriber line (DSL) unit, Firewire interface, USB interface, and the like. Communication algorithms (protocols’) can be specified using one or communication languages, such as HTTP, TCP / IP, RTP / RTSP, IPX and / or UDP. The network interface may enable connectivity to one or more cloud computing systems.

[0229] User interface input devices can include an alphanumeric keyboard, a keypad, or pointing devices such as a mouse. Typically, the term ‘input device’ signifies all possible typesROOG-OOIOOWOof devices and processes to transfer data and information into a computer system or onto a communication network. User interface input devices typically enable a user to select objects, icons, text, and the like that appear on some types of user interface output devices, for example, a display subsystem.

[0230] User interface output devices can include a visual display subsystem, a printer, or a non-visual communication device such as audio and haptic devices. Typically, the term ‘output device’ signifies all possible types of devices and processes to transfer data and information out of a computer system to the user or to another machine or computer system. Such devices are connected by wire or wirelessly to a computer system.

[0231] The memory subsystem typically includes a number of memories including a main random-access memory (RAM) (or other volatile storage device) for storage of instructions and data during program execution and a read only memory (ROM) in which fixed instructions are stored. The file storage subsystem provides persistent storage for program and data files.

[0232] The bus subsystem provides a device for transmitting data and information between the various components and subsystems of the computer system. Although the bus subsystem is depicted as a single bus, alternative embodiments of the bus subsystem can use multiple buses.

[0233] The memory can include a non-transitory, processor readable data and information storage medium associated with file storage subsystem, and / or with network interface subsystem, and can include a data structure specifying a circuit design.

[0234] The Detailed Description signifies in isolation the individual features, structures, functions, or characteristics described herein and any combination of two or more such features, structures, functions or characteristics, to the extent that such features, structures, functions or characteristics or combinations thereof are enabled by the Detailed Description as a whole in light of the knowledge and understanding of a skilled person, irrespective of whether such features, structures, functions or characteristics, or combinations thereof, solve any problems disclosed herein, and without limitation to the scope of the Claims herein. When an embodiment comprises a particular feature, structure, function, or characteristic, it is within the knowledge and understanding of a skilled person to use such feature, structure, function, or characteristic in connection with another embodiment whether or not explicitly described, for example, as a substitute for another feature, structure, function, or characteristic.

[0235] In view of the Detailed Description, a skilled person will understand that manyROOG-OOIOOWOvariations of any embodiment can be enabled, such as function and structure of elements, described herein while being as useful as the embodiment. One or more elements of an embodiment can be substituted for one or more elements in another embodiment, as will be understood by a skilled person. Writings about any embodiment signify its use in commerce, thereby enabling other skilled people to similarly use this embodiment in commerce.

[0236] This Detailed Description is written to provide knowledge and understanding. Many modifications and other embodiments of the invention will come to the mind of one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is understood that the invention is not to be limited to the specific embodiments disclosed, and that modifications and embodiments are intended to be included within the scope of the appended claims. It is also understood that other embodiments of this invention may be practiced in the absence of an element / step not specifically disclosed herein.

[0237] Additionally, This Detailed Description is to be accorded the widest scope consistent with the disclosed principles and features. Without limitation, any and all equivalents described, signified, or incorporated by reference (or explicitly incorporated) in this patent application are specifically incorporated into the Detailed Description. In addition, any and all variations described, signified, or incorporated with respect to any one embodiment also can be included with any other embodiment. Any such variations include both currently known variations as well as future variations, for example any element used for enablement includes a future equivalent element that provides the same function, regardless of the structure of the future equivalent element.

[0238] It is intended that the domain of the set of claimed inventions and their embodiments be defined and judged by the following Claims and their equivalents. The Detailed Description includes the following Claims, with each Claim standing on its own as a separate claimed invention. Any embodiment can have more structure and features than are explicitly specified in the Claims.

Claims

1. ROOG-OOIOOWOCLAIMS1. A method for implementing a neuron computation model, comprising the steps having a neuron receive a plurality of input values;for each input value, making a comparison to a threshold tau value;for each successful comparison, having the neuron add a value to a number of matches, or for each unsuccessful comparison, having the neuron add a value to a number of mismatches; andbased on the number of matches or mismatches, generating a neuron output.

2. The method of claim 1, wherein the neuron output is set to between 0 and 1 inclusive.

3. The method of claim 1, wherein the comparison can be at least one of > or <4. The method of claim 1, wherein the comparison can be at least one of > or <, and a bias value is used to determine whether > or < is selected for comparison.

5. The method of claim 1, wherein the threshold tau value is set during training.

6. The method of claim 4, wherein the bias value is set during training.

7. The method of claim 1, wherein the neuron output is at least in part determined by the number of matches and a ReLU function.

8. The method of claim 1, wherein the neuron output is at least in part determined by a number of matches.

9. The method of claim 1, wherein the neuron output is further adjusted using a programmed output strength.

10. The method of claim 1, wherein the neuron output is further adjusted using a programmed output strength that is set during training.

11. The method of claim 1, wherein the neuron output is further adjusted by multiplication with a programmed output strength.

12. An Al system, comprisingan artificial neural network (ANN) having at least one a neuron able to receive a plurality of input values, wherein for each input value to the neuron, a comparison can be made to a threshold tau value; andfor each successful comparison, the at least one neuron adds a selected value to a number of matches, or for each unsuccessful comparison, having the neuron add a value to aROOG-OOIOOWOnumber of mismatches, and based on the number of matches generates a neuron output to one or more other neurons in the ANN.

13. The system of claim 12, wherein the ANN is a hybrid neural network comprising a conventional neural network connected to a neural network having neurons that, for each successful comparison, add a selected value to a number of matches, or for each unsuccessful comparison, having the neuron add a value to a number of mismatches, and based on the number of matches generate a neuron output.

14. The system of claim 12, wherein the neuron output is set to between 0 and 1 inclusive.

15. The system of claim 12, wherein the comparison can be at least one of > or <.

16. The system of claim 12, wherein the comparison can be at least one of > or <, and a bias value of 1 or 0 is respectively used to determine whether > or < is selected for comparison.

17. The system of claim 12, wherein the threshold tau value is set during training.

18. The system of claim 16, wherein the bias value is set during training.

19. The system of claim 12, wherein the ANN is supported on a memory device having an array of memory cells configured in rows and columns and where more than one row is selected in a given column and the sense amplifier detects and outputs a value representing how many of the row’s cells are conductive.

20. A memory device configured for operation in multiple modes, comprisingan array of memory cells configured in rows and columns;a plurality of sense amplifiers connected to the array of memory cells; wherein the array of memory cells has a first operational data storage and read mode, wherein circuitry performs a write operation of data to a memory cell by selecting one row from a multiple of rows and data is written and read from the memory cell using at least one sense amplifier attached to one of the columns; and whereinthe array of memory cells has a second operational MLP mode where more than one row is selected in a given column and the sense amplifier detects and outputs a value representing how many of the row’s cells are conductive.

21. The memory device of claim 20, wherein the memory device has a mode selector to switch between the first and second operational modes.ROOG-OOIOOWO22. The memory device of claim 20, wherein a single neuron is represented through a column of the array of memory cells.

23. The memory device of claim 22, wherein the sense amplifier is connected to the column to provide a sense amplifier output value corresponding to the output value for that neuron.

24. The memory device of claim 20, wherein the memory device supports a neural network comprising at least one neuron represented through a column of the array of memory cells.

25. The memory device of claim 20, wherein multiple operations can be performed in MLP mode where each sequential operation is related to determining the values of neurons represented through columns of the array of memory cells and arranged in one layer.

26. The memory device of claim 20, wherein values placed on selected rows are related to the values of sense amplifier from the previous operations27. The memory device of claim 20, wherein a neural network comprises at least one neuron represented through one column of the array of memory cells and input to a first hidden layer neuron is based on inputs given for a neural network.

28. A method for implementing a neural network, comprising the steps of providing a memory device having array of memory cells configured in rows and columns, where one neuron can reside on a single column, whereby:determining incoming values for each row by input values for each of the neurons selected;performing operations by the memory device based on the incoming values of the row and the threshold value programmed on a plurality of cells associated with the neuron, where the conduction of a cell represents a failure in the operation being checked;outputting a value from a sense amplifier in accordance with how many of the cells were determined to be conductive.

29. A method for implementing the neural network of claim 28, wherein there is no conduction path for rows that have cells on the column that are not a part of the neuron input.

30. A method for implementing the neural network of claim 28, wherein the intersection of rows and columns consists of a series of serially connected cells.ROOG-OOIOOWO31. A method for implementing the neural network of claim 28, wherein the operation being performed is < or >.

32. A method for implementing the neural network of claim 28, wherein a sense amplifier is attached to a column where there are multiple cells being checked, the sense amplifier output acting like an NOR gate33. A method for implementing the neural network of claim 28, wherein the operation being checked for the neuron is an opposite of the operation for a given input to a neuron.

34. A method for implementing the neural network of claim 28, wherein each operation being performed has one cell associated with it.

35. A method for implementing the neural network of claim 28, wherein only one operation is performed for each input.

36. A method for implementing the neural network of claim 28, wherein the same input can be inputted to multiple rows whereby multiple conditions can be performed on the same input.

37. A method for implementing the neural network of claim 28, wherein there are two inputs generated for each neuron input, one to check for each < or > condition.

38. A method for implementing the neural network of claim 28, wherein for < operation, where the neuron checks to see if the input of the neuron is < the threshold, the cell’s input would be the value of the input, and the threshold would be the actual threshold.

39. A method for implementing the neural network of claim 28, wherein for > operation, where the neuron checks to see if the input value is > the threshold of the cell, the second cell can be selected which can have (1 -input value) as the input to the cell and the threshold programmed into the cell would be (1 -Threshold condition).

40. A method for implementing the neural network of claim 39, wherein each cell from the serially connected cells not being checked is left in a conductive state, without respect to the input value of its row.

41. A method for implementing the neural network of claim 28, wherein conductivity level of each cell is controlled by controlling the gate of a series device in series with the string.

42. An Al system, comprisinga plurality of neurons forming at least a portion of an artificial neural network (ANN) having at least one a neuron able to receive a plurality of input values, wherein for each inputROOG-OOIOOWOvalue to the neuron, a comparison can be made to a threshold tau value and for each successful comparison, the neuron adds a selected value to a number of matches, and based on the number of matches generates a neuron output to one or more other neurons in the ANN;a memory device that supports multi-layer perceptron (MLP) operation and an array of memory cells configured in rows and columns, where each column represents a single neuron; anda companion ASIC chip connected to the memory device to provide computational support.

43. The AI system of claim 42, wherein the companion ASIC chip can provide initial inputs to the memory AI chip based on either inputs for training or inference, the memory AI chip then responding with the output of neurons based on the inputs provided to the companion ASIC chip.

44. The AI system of claim 42, wherein the companion ASIC chip can parse neuron outputs and provide values to the memory AI chip for next layer calculations.

45. The AI system of claim 42, wherein the companion ASIC chip can provide the inputs out of order to the memory AI chip.

46. The AI system of claim 45, wherein the companion ASIC chip can replace the output of one of the neurons with a redundant neuron output.

47. The AI system of claim 45, wherein the companion ASIC chip sets the memory AI chip in the memory mode and can read one parameter from the memory AI chip.

48. The AI system of claim 47, wherein the companion ASIC chip completes multiplication using the read parameter.

49. The AI system of claim 48, wherein the inputs to the memory AI chip are generated from the resulting values.

50. The AI system of claim 48, wherein the companion ASIC chip can perform the multiply operation and send resulting values to the memory AI chip for MLP processing.

51. A method for implementing a neural network system, comprising the steps of providing a memory AI chip comprising a neural network on a memory device having array of memory cells configured in rows and columns, where each column can represent a neuron and wherein for each input value to the respective column neuron, a comparison can be made to a threshold tau value; andROOG-OOIOOWOproviding computational math support to the memory AI chip with a companion ASIC.

52. A method for implementing the neural network system of claim 51, comprising providing inputs by the companion ASIC chip to the memory AI chip, and sending outputs from the memory AI chip back to the companion ASIC chip.

53. A method for implementing the neural network system of claim 51, comprising allowing the companion ASIC chip to take inputs from a neural network system as input to the memory AI chip to provide correct voltages on the associated rows.

54. A method for implementing the neural network of claim 53, comprising providing the companion ASIC chip take inputs from the memory AI chip that represents a current hidden layer and rearrange inputs to provide correct values to the memory AI chip as inputs to the next hidden layer.

55. A method for implementing the neural network system of claim 53, comprising providing output of some of the layers as be kept by the companion ASIC chip to be input to layers that are not the next hidden layer.

56. A method for implementing the neural network system of claim 53, comprising providing the output of the last layer to be utilized by the companion ASIC chip to implement multiply / accumulate (MAC) operation for the output layer.

57. A method for implementing the neural network system of claim 51, wherein LLMs are supported and that use the companion ASIC chip for attention and has the memory AI perform feed forward networks (FFNs).

58. A method for implementing the neural network system of claim 51, wherein vision and vision language models are supported and that use the companion ASIC chip for attention type activities and has the memory AI perform feed forward networks (FFNs).

59. A method for implementing the neural network of claim 51, wherein world models and video are supported and that use the companion ASIC chip for attention type activities and has the memory Al perform feed forward networks (FFNs).

60. A method for implementing the neural network system of claim 51, wherein an autonomous driving model is supported that use the companion ASIC chip for attention and has the memory AI chip perform feed forward networks (FFNs).

61. An AI system, comprisingROOG-OOIOOWOa plurality of neurons forming at least a portion of an artificial neural network (ANN) and having at least one neuron able to receive a plurality of input values;a memory device having array of memory cells configured in rows and columns, where each column represents a single neuron; anda companion ASIC chip connected to the memory device to provide computational support.

62. The AI system of claim 61, wherein the companion ASIC chip provides an address of cells needed for a next hidden layer.

63. The AI system of claim 61, wherein the companion ASIC chip provides a same address for the next hidden layer to create a recurrent neural network (RNN) where output of one layer is provided to the same existing layer that was previously evaluated with different inputs.

64. The AI system of claim 61, wherein multiple outputs read from different neurons can be added to increase neuron size.

65. The AI system of claim 61, wherein the companion ASIC chip can be incorporated into the same silicon as the memory device.

66. The AI system of claim 61, wherein the companion ASIC chip can be incorporated into the same package as the memory device.

Images