Crossbar array implementing a truth table

Abstract

A method is provided for preparing a trained crossbar array of a neural network. The method includes providing an input portion of a predefined truth table to a computer simulation of the crossbar array and generating an analog output value for the input portion of the truth table based on the simulated weights. The method further includes calculating a loss value from each of the analog output value and a predicted value for an output portion of the truth table and adjusting the simulated weights based on the calculated loss value. The method further includes re-providing the input portion of the predefined truth table to the computer simulation and recalculating the output value using the adjusted simulated weights until the analog output value generates a predicted value for the output portion of the truth table within a predefined error limit.

Description

Technical Field

[0001] The present invention generally relates to the use of crossbar arrays and neural networks for implementing complex truth tables, and more particularly to implementing crossbar arrays and neural networks instead of field programmable gate arrays (FPGAs) to provide complex truth tables and additional neural network functionality.

Background Art

[0002] One way to design combinational logic circuits is to interconnect logic gates, but digital logic circuits constructed using discrete logic gates have fixed functions defined at the time of manufacture. A field programmable gate array (FPGA) is an integrated circuit designed to be configured by a customer after manufacture. An FPGA includes an array of programmable logic blocks and a hierarchy of reconfigurable interconnects that enable the blocks to be wired together. Before an FPGA can be used, it is programmed to implement the desired function. A programmable logic array (PLA) is a type of programmable logic device used to implement combinational logic circuits. FPGAs can be used in the development of deep neural networks.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Combinational logic circuits are created from basic logic NAND gates, NOR gates, or NOT gates that are "combined" or connected together to produce more complex switching circuits. These logic gates are the building blocks of combinational logic circuits, including FPGAs.

[0004] A truth table is a tabular representation used in logic. In this case, each input variable has its own column, and each output variable also has its own column. The output column indicates the result of the logical operation represented by the table. Each row of the truth table contains one possible configuration of input variables and output values.

Means for Solving the Problem

[0005] According to one embodiment of the present invention, a method for preparing a trained crossbar array of a neural network is provided. The method includes providing an input portion of a predefined truth table to a computer simulation of the crossbar array, and generating analog output values for the input portion of the truth table based on the simulated weights. The method further includes calculating a loss value from each of the analog output values and a predicted value for an output portion of the truth table, and adjusting the simulated weights based on the calculated loss value. The method further includes re-providing the input portion of the predefined truth table to the computer simulation and recalculating output values using the adjusted simulated weights until the analog output values generate predicted values for the output portion of the truth table within a predefined error limit.

[0006] According to one embodiment of the present invention, there is provided a computer program product for training a crossbar array of a neural network, the computer program product comprising one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media. The program instructions include program instructions for providing an input portion of a predefined truth table to a computer simulation of the crossbar array, and program instructions for generating an analog output value for the input portion of the truth table based on the simulated weights. The program instructions further include program instructions for calculating a loss value from each of the analog output value and a predicted value for an output portion of the truth table, and program instructions for adjusting the simulated weights based on the calculated loss value. The program instructions further include program instructions for re-providing the input portion of the predefined truth table to the computer simulation and recalculating an output value using the adjusted simulated weights until the analog output value generates a predicted value for the output portion of the truth table within a predefined error limit.

[0007] According to one embodiment of the present invention, a computer system is provided for preparing a trained crossbar array of a neural network. The computer system includes one or more processors and a computer memory electronically coupled to the processors. The computer system includes a computer simulation stored in the computer memory that includes a model of the crossbar array, where the computer simulation receives an input portion of a predefined truth table, generates an analog output value for the input portion of the truth table based on simulated weights, calculates a loss value from each of the analog output value and a predicted value for an output portion of the truth table, adjusts the simulated weights based on the calculated loss value, provides the input portion of the predefined truth table again to the computer simulation, and is configured to recalculate the output value using the adjusted simulated weights until the analog output value generates a predicted value for the output portion of the truth table within a predefined error limit.

[0008] These and other features and advantages will become apparent from the following detailed description of its exemplary embodiments, which should be read in conjunction with the accompanying drawings.

[0009] The following description provides details of the preferred embodiments with reference to the following drawings.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0011] Various embodiments relate to implementing a crossbar array and a neural network instead of a field programmable gate array (FPGA) to provide complex truth tables and additional neural network functionality. The crossbar array can be used for machine learning and learning digital logic truth tables, where the crossbar array can be trained or programmed on a truth table data set. A truth table can include an input portion that provides values input to train a neural network and an output portion that represents predicted values generated at the output of the neural network. Not all combinations of input or output values can be predicted for a particular environment. However, learning digital logic truth tables can require 100% accuracy. In various embodiments, combinational logic blocks can be replaced with a crossbar array.

[0012] In various embodiments, there may be a noise margin at the output of a crossbar - array - based neural network. By using an inverter or a sense amplifier or both, it is possible to output digital logic of 0 and 1 despite variations and noise in the neural network output signal. This can be done for both digital inputs and digital outputs. Due to noise, a logic - high signal at the output of a driving device may reach the input of a receiving device at a lower voltage than that which triggers a high digital value. The noise margin is the total amount of noise that can be added to the worst - case output so that the signal can still be interpreted as a valid input. The I / O noise margins (low and high), i.e., NML and NMH, refer to the ability of a logic gate (e.g., an inverter) to handle input noise without generating an incorrect logic output. In various embodiments, the inverter can have a noise margin of less than 50% of Vdd (drain voltage), e.g., 2% to 49% of Vdd. In various embodiments, two or more inverters may be present in series.

[0013] In various embodiments, a neural network trained with a truth table can be reset and reprogrammed in this field either through retraining or through direct programming of new weights. This provides versatility in that by using a crossbar array for artificial neural network (ANN) applications, reprogrammable digital logic such as that of an FPGA can be implemented in the same neural network device package.

[0014] In various embodiments, training on a predefined truth table can be performed in a simulation environment using a model that mimics the signal noise associated with the crossbar array. The training can be performed offline using a graphics processing unit (GPU). The computer simulation and the weight values generated by the model can be directly loaded into the non-volatile memory cells of the crossbar array.

[0015] In various embodiments, the neural network can achieve training up to 100% accuracy by having variable layer sizes and depths, where the exact predicted output for each input truth table is generated by the crossbar array every time.

[0016] One example of an application / use to which the present invention can be applied, without limitation, includes those used for FPGAs, data centers, medical devices, automotive applications, edge computing, etc.

[0017] It should be understood that aspects of the present invention are described in terms of a given exemplary mechanism, but other mechanisms, structures, components, as well as process features and process steps, can be varied within the scope of the aspects of the present invention.

[0018] Referring now to like numbers to represent like elements or like elements, initially referring to FIG. 1, FIG. 1 is a cross-sectional view showing a crossbar array of an artificial neural network (ANN) trained with a truth table, according to one embodiment of the present invention.

[0019] In one or more embodiments, the crossbar array 110 can be implemented using non-volatile memory devices 120 electrically coupled between intersecting conductive lines, where the non-volatile memory devices 120 can be analog non-volatile memory devices 120 configured to store a range of weights, in which case multiple bits of weight data can be stored per cell / NVM device. Memory elements with low per-cycle variability or low noise in their stored weight values can be used to store multiple bits of weight information. The upper conductive lines 130 can be orthogonal to the lower conductive lines 140, or may intersect at an angle less than 90 degrees. By utilizing these non-volatile memory (NVM) arrays, calculations can be performed in parallel by storing weights and directly executing neural network calculations on the memory elements that store the weights, reducing the energy cost and latency cost associated with data movement.

[0020] In one or more embodiments, a pre-determined truth table 150 can be used to determine, through training, the weights stored by the non-volatile memory devices 120 of the crossbar array 110. The crossbar array can be directly trained using the truth table as an input, or a computer simulation that models the crossbar array can be used to realize the trained weights and program the weights into the crossbar array. The number of elements in each input set can determine the number of input nodes of the neural network and the number of weights used to calculate the set of output values, where the set of output values can include one or more values representing one or more states (e.g., 1 or 0), in which case the number of output values is determined by the truth table.

[0021] In various embodiments, the size and complexity of the input portion and the output portion of the truth table can determine the size and complexity of the neural network and the crossbar array utilized by the neural network.

[0022] In various embodiments, the non-volatile memory device 120 holds weights regarding the crossbar array 110 learned through training or programming. The weights can be constructed using one or more non-volatile memories. For example, if the phase change material (PCM) conductance is positive, negative weights can be constructed using other different pairs of conductance such that the overall weight is W = G+ - G-. At this time, if G+ > G-, the weight is positive, or if G+ < G-, the weight is negative. The weights can also be constructed using a larger conductance, W = F * (G+ - G-) + g+ - g- etc., where F * is any fixed conversion factor for expanding the overall dynamic range of the weights that can be represented.

[0023] FIG. 2 is a cross-sectional view showing a crossbar array of a neural network generating the output of a truth table according to an embodiment of the present invention.

[0024] In one or more embodiments, known values of the input portion of the truth table 150 can be input into a neural network having one or more crossbar arrays 110, and the output table 160 generated by the neural network can be compared with the intended output defined by the output portion of the truth table. By using the truth table input 150 and the truth table output 160, the neural network can be trained to learn the truth table with 100% accuracy. The training samples can cover, for example, up to 100% of the combinations of input / output values, in which case not all combinations are possible in the environment where the neural network will ultimately be implemented. The actual circuit may only encounter a small portion of the exhaustive list of possible combinations. Regarding the truth table, since all output values for all combinations of inputs can be known (ground truth), the neural network can be trained to 100% accuracy without overfitting.

[0025] In various embodiments, the training can be performed through software simulations of neural networks and crossbar arrays, which include noise simulation within the model. The weights generated by training the simulation can then be programmed / stored in the actual crossbar array.

[0026] In various embodiments, the truth table output 160 generated by the neural network can have one or more values regarding outputs representing different logic states. The truth table input 150 and the truth table output 160 can be a Boolean truth table or a more complex truth table. This may also be used for logics having three or more states (e.g., three-valued logic or multi-stage logic, etc.). The truth table output generated by the crossbar array may be sent to other digital circuits or input to another crossbar array.

[0027] In various embodiments, since the known values of the input truth table have a one-to-one mapping to the known values of the truth table output with an intended 100% accuracy, overfitting is not a problem, so a test set is not used to determine the accuracy of the neural network.

[0028] FIG. 3 is a top view showing a crossbar array of a neural network implementing an inverter at the output to generate digital 1s and 0s from a digital input, according to one embodiment of the present invention.

[0029] In one or more embodiments, the output can be digitized and processed by a digital circuit. Since the crossbar array performs calculations on analog signals, the output may be converted to a digital format that can be interpreted as a result or transmitted to the next crossbar array / layer of the neural network. By electrically coupling an inverter 180 to each crossbar output, intervening voltage values that could be misinterpreted by generating a voltage corresponding to a digital one or zero can be avoided. The inverter 180 can generate a digital output from the crossbar output even when there is noise in the crossbar output signal. In various embodiments, the crossbar array 110 can generate output signals / values within the noise margin. The inverter 180 can convert the output values within the noise margin to digital ones or zeros.

[0030] In various embodiments, an interface 170 is electronically coupled to and communicates electrically with the crossbar array 110 to sum the values generated by the analog non-volatile memory device 120 and apply a suitable activation function f to the summed output, where f can be a non-linear activation function. The interface 170 can perform a conversion on the analog values output by the crossbar array 110, where the conversion can map one representation of the input to another representation. Analog weights and analog signals can be used to accumulate errors in the crossbar array due to noise, non-linearity, and variability through the array for each neural network layer. The output representation can propagate as an input to another crossbar array. In various embodiments, the non-linear activation function can be, for example, a rectified linear unit (ReLU), a sigmoid function, or a hyperbolic tangent (tanh(x)).

[0031] A multiply-accumulate (MAC) unit can be used for vector-matrix multiplication. The multiply-accumulate hardware / unit / circuit / operation can calculate the product of two numbers and add the product to an accumulator.

[0032] MAC can be used in conventional ANN applications and in truth table applications, where the ANN (and thus MAC) can be used to learn digital logic truth tables.

[0033] In one or more embodiments, a crossbar array can replace combinational logic blocks used in an FPGA.

[0034] FIG. 4 is a block diagram / flow diagram showing the generation of adjusted simulated weights via computer simulation, according to one embodiment of the present invention.

[0035] In one or more embodiments, computer simulation can generate weights for a crossbar array of a neural network, where the computer simulation models the crossbar array and trains the neural network through backpropagation.

[0036] In block 410, the input portion of a predetermined truth table can be provided to a computer simulation of a crossbar array. The input values can be digital ones and zeros. In various embodiments, the weights of the computer simulation can be initialized with random weight values prior to training to avoid problems that can occur when all initial weights are set to one, zero, or the same value. The output portion of the predetermined truth table can be provided to the computer simulation for loss calculation.

[0037] In block 420, the computer simulation can generate an analog output value for the input portion of the truth table based on the simulated hardware and simulated weights for the crossbar array. The value output by the computer simulation can be an analog signal generated by matrix-vector multiplication (MVM).

[0038] In block 430, a loss function can be utilized to calculate a loss value from each of the analog output value and the predicted value from the output portion of the truth table. The output value can be an analog value or a digital value. In various embodiments, the loss function can be the mean squared error (MSE).

[0039] In block 440, the simulated weights can be adjusted based on the calculated loss value. The weights stored by the computer simulation of the crossbar array can be adjusted using backpropagation, utilizing the output value and the loss value calculated from the output portion of the truth table.

[0040] In block 450, the training can be repeated by re-providing the pre-determined input portion of the truth table to the computer simulation and recalculating the output value using the adjusted simulated weights until the analog output value generates a predicted value for the output portion of the truth table within a pre-defined error limit. In supervised learning, the network can be trained to make accurate predictions by repeatedly updating the weight matrix until the output provides the correct value. The original truth table can be provided to the neural network (NN), for example, within a range of about 10 times to about 1,000,000 times. In various embodiments, the training can be repeated until 100% of the output converges within any pre-defined error limit, for example, an error limit of 0%, in which case the output value matches 100% of the output portion of the truth table.

[0041] In one or more embodiments, the trained weights of the simulated crossbar array and neural network can be implemented in the field for the operations intended to utilize the truth table used for training.

[0042] In one or more embodiments, the crossbar array can be retrained with new simulated weights and results by providing a new truth table with known output values to a computer simulation. Before entering the new truth table, the weights of the crossbar array may be reset in a manner appropriate for the type of non-volatile memory utilized in the crossbar array (e.g., phase change, ferroelectric, magnetic, etc.).

[0043] FIG. 5 is a block diagram / flow diagram showing the training of a crossbar array according to one embodiment of the present invention.

[0044] In one or more embodiments, the neural network and crossbar array can be trained using a predetermined truth table (e.g., ground truth values) representing the inputs and predicted outputs.

[0045] In block 510, one or more crossbar arrays that will store the learned weights can be trained by providing the predetermined truth table to the input nodes of the neural network. The input values can be digital ones and zeros. In various embodiments, the crossbar array can be initialized with random weight values prior to training to avoid problems that may occur if the initial weights are all set to one, zero, or the same value. In various embodiments, multiple crossbar arrays may store the learned weights.

[0046] In block 520, the neural network can output a result value for the input truth table generated from the weights stored by the crossbar array. The value output by the crossbar array can be an analog signal, while the output generated by an inverter or a sense amplifier can be a digital signal.

[0047] In block 530, the output value can be compared with a known / predicted value for the input truth table. The output value can be an analog value or a digital value depending on whether the output value is output by the crossbar array or by an inverter.

[0048] In block 540, a loss can be calculated from the output value compared with the input truth table. In various embodiments, the analog value of the weights can be used to calculate the loss and train the weights, in which case the analog signal value can be compared with the analog value that generates the predicted digital output. In various embodiments, digital values can also be directly used for calculation by a loss function.

[0049] In various embodiments, the loss function can be the mean squared error (MSE), in which case one or more crossbar arrays can be trained to 100% accuracy by minimizing the loss function for the training data set.

[0050] In block 550, the weights stored by the crossbar array can be adjusted using the output value and the loss value calculated from the input truth table. The component by which the learned weights can be adjusted can be based on the loss value.

[0051] In block 560, the training can be repeated by re-providing the truth table to the neural network and recalculating the output values using the adjusted weights in the crossbar array. In various embodiments, the original truth table can be provided to the neural network (NN) in a range, for example, from about 10 times to about 1,000,000 times. In various embodiments, the training can be repeated until 100% of the output converges within an arbitrary error margin, for example, as determined by an inverter.

[0052] In block 570, the trained weights of the crossbar array and the neural network can be implemented within a field for the intended operation that utilizes the truth table used for training.

[0053] In block 580, the crossbar array can be retrained by providing a new truth table with known output values to the neural network and the crossbar array. Before entering the new truth table, the weights of the crossbar array may be reset in a manner appropriate for the type of non-volatile memory utilized in the crossbar array (e.g., phase change, ferroelectric, magnetic, etc.).

[0054] In various embodiments, a complete set of new weights trained on another crossbar array may be stored in the neural network where the field is implemented by rewriting the implemented crossbar array. The memory can be programmed or reprogrammed by applying voltage pulses of varying duration and current. These can be used to reset the memory. By then using additional pulses, the conductance of the non-volatile memory devices of the crossbar array can be adjusted incrementally.

[0055] In one or more embodiments, the neural network and crossbar array have a reset feature and associated circuitry to erase or reset the weights stored in the non-volatile memory device 120 of the crossbar array 110, such that the non-volatile memory device 120 of the crossbar array 110 can be reprogrammed or retrained with new weights in the field. Reprogramming the non-volatile memory device 120 of the crossbar array 110 may involve storing pre-trained weights determined offline in the non-volatile memory device 120 of the crossbar array 110, whereas retraining may involve re-inputting / inputting a new truth table for generating a new set of weights in the field, where the new truth table may be different from the original truth table used in the previous training.

[0056] FIG. 6 is a computer system related to the generation of adjusted simulated weights via computer simulation, according to one embodiment of the present invention.

[0057] In one or more embodiments, one or more processors 610 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), etc.) are electronically coupled to a computer memory 620 through a bus 630 and can communicate electrically with the computer memory 620. In various embodiments, a display screen 640 is electronically coupled to and can communicate electrically with one or more processors 610 and the computer memory 620 through the bus 630.

[0058] In various embodiments, a computer simulation 650 that models the crossbar array can be stored in the computer memory 620. In various embodiments, a truth table 660 that can be used to train a neural network through simulation can be stored in the computer memory 620.

[0059] As used herein, the term "hardware processor subsystem" or "hardware processor" can refer to a processor, memory, software, or combinations thereof that cooperate to perform one or more specific tasks. In advantageous embodiments, the hardware processor subsystem can include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The one or more data processing elements can include a central processing unit, a graphics processing unit, or a controller based on a separate processor or computing element (e.g., logic gates, etc.) or combinations thereof. The hardware processor subsystem can include one or more on-board memories (e.g., cache, dedicated memory arrays, read-only memories, etc.). In some embodiments, the hardware processor subsystem can include one or more memories that can be on-board or off-board, or dedicated for use by the hardware processor subsystem (e.g., ROM, RAM, basic input / output system (BIOS), etc.).

[0060] In some embodiments, the hardware processor subsystem can include and execute one or more software elements. The one or more software elements can include an operating system and / or one or more applications and / or specific code to achieve a specific result.

[0061] In other embodiments, the hardware processor subsystem can include dedicated specialized circuits that execute one or more electronic processing functions to achieve a specific result. Such circuits can include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and programmable logic arrays (PLAs) or combinations thereof.

[0062] These and other variations of the hardware processor subsystem are also contemplated by embodiments of the present invention.

[0063] The present invention may be a system, method, or computer program product at any possible technically detailed level of integration, or a combination thereof. The computer program product may include a computer-readable storage medium (or multiple computer-readable storage media) having computer-readable program instructions for causing a processor to implement aspects of the present invention.

[0064] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, punch cards, mechanically encoded devices such as raised structures in grooves having instructions recorded thereon, and any suitable combination of the foregoing. A computer-readable storage medium should not be construed as being a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses passing through an optical fiber cable), or electrical signals transmitted through a wire.

[0065] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or can also be downloaded from an external computer or an external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may comprise a copper transmission cable, an optical transmission fiber, a wireless transmission, a router, a firewall, a switch, a gateway computer, or an edge server, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each respective computing / processing device.

[0066] The computer-readable program instructions for carrying out the operations of the present invention may be source code or object code written in any combination of one or more programming languages, including assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or a combination of any of the foregoing with an object-oriented programming language such as Smalltalk, C++, and a procedural programming language such as the “C” programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit for performing aspects of the present invention.

[0067] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0068] These computer-readable program instructions may be provided to a computer processor or other programmable data processing apparatus, causing the instructions executed via the computer processor or other programmable data processing apparatus to create means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium having instructions stored therein that, when executed by a computer, a programmable data processing apparatus, or other devices or combinations thereof, cause the computer, programmable data processing apparatus, or other devices or combinations thereof to function in a particular manner so as to comprise a product comprising instructions for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0069] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices that execute the instructions to implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0070] References in the specification to "an embodiment" or "one embodiment" or other variations thereof of the present invention mean that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" and any other variations thereof that occur in various places throughout this specification are not necessarily all referring to the same embodiment.

[0071] For example, in the cases of "A / B", "A or B or both", and "at least one of A and B", it should be understood that the use of any of the following, namely " / ", "~ or... or both", and "at least one of ~", is intended to include the selection of only the first-listed option (A), or only the second-listed option (B), or the selection of both options (A and B). As a further example, in the cases of "A, B or C or a combination thereof" and "at least one of A, B and C", such language is intended to include the selection of only the first-listed option (A), or only the second-listed option (B), or only the third-listed option (C), or only the first and second-listed options (A and B), or only the first and third-listed options (A and C), or only the second and third-listed options (B and C), or the selection of all three options (A and B and C). This may be extended as long as many items are listed, as will be readily apparent to those skilled in this and related arts.

[0072] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart and block diagrams may represent a module, segment, or portion of instructions, which include one or more executable instructions for performing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed as one step, may be executed at the same time, may be executed in a partially or wholly temporally overlapping manner, or the blocks may be executed in the reverse order depending upon the functionality involved. It should also be noted that each block of the block diagrams or flowchart diagrams, or combinations of both, can be implemented by a special purpose hardware-based system that performs the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0073] Preferred embodiments of systems and methods (which are intended to be illustrative and not limiting) have been described, but it should be noted that modifications and variations can be made by those skilled in the art in light of the above teachings. Accordingly, it is to be understood that changes may be made in the particular embodiments disclosed within the scope of the invention as outlined by the appended claims. Having described the aspects of the invention with the detail and particularity required by patent law, the content desired to be claimed and protected by patent is set forth in the appended claims.

Claims

1. A computer method for preparing a trained crossbar array of a neural network, comprising: providing an input portion of a pre-determined truth table to a computer simulation of the crossbar array; generating an analog output value for the input portion of the truth table based on the simulated weights; calculating a loss value from each of the analog output value and a predicted value for an output portion of the truth table; adjusting the simulated weights based on the calculated loss value; re-providing the input portion of the pre-determined truth table to the computer simulation, and recalculating the output value using the adjusted simulated weights until the analog output value generates the predicted value for the output portion of the truth table within a pre-defined error limit.

2. The computer method according to claim 1, wherein each simulated analog output value includes an error of less than 49% of Vdd.

3. The computer method according to claim 2, wherein the pre-defined error limit is 0%.

4. The computer method according to claim 3, wherein the loss value is calculated using a mean squared error (MSE) loss function.

5. The computer method according to claim 4, further comprising programming one or more crossbar arrays with the adjusted simulated weights, wherein the programmed one or more crossbar arrays emulate a field programmable gate array (FPGA).

6. The computer method according to claim 5, further comprising resetting the weights of the one or more crossbar arrays.

7. The computer method according to claim 6, further comprising reprogramming the one or more crossbar arrays with different weights to emulate different field programmable gate arrays (FPGAs).

8. A computer program product for training a crossbar array of a neural network, comprising one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media, the program instructions comprising: Program instructions for providing an input portion of a pre-determined truth table to a computer simulation of a crossbar array, and program instructions for generating an analog output value for the input portion of the truth table based on the simulated weights, and program instructions for calculating a loss value from each of the analog output value and a predicted value for an output portion of the truth table, and program instructions for adjusting the simulated weights based on the calculated loss value, and program instructions for re-providing the input portion of the pre-determined truth table to the computer simulation and recalculating the output value using the adjusted simulated weights until the analog output value generates the predicted value for the output portion of the truth table within a pre-defined error limit, a computer program product. **Claim 9** The computer program product according to claim 8, wherein each analog output value includes an error of less than 49% of Vdd. **Claim 10** The computer program product according to claim 9, wherein the pre-defined error limit is 0%. **Claim 11** The computer program product according to claim 10, wherein the loss value is calculated using a mean squared error (MSE) loss function. **Claim 12** Further comprising programming one or more crossbar arrays with the adjusted simulated weights, wherein the programmed one or more crossbar arrays mimic a field programmable gate array (FPGA), the computer program product according to claim 11. **Claim 13** The computer program product according to claim 12, further comprising resetting the weights of the one or more crossbar arrays and reprogramming the one or more crossbar arrays with different weights to mimic different field programmable gate arrays (FPGAs). **Claim 14** A computer system for preparing a trained crossbar array of a neural network, comprising one or more processors, and a computer memory electronically coupled to the processor, and a computer simulation including a model of a crossbar array, receiving an input portion of a pre-determined truth table, Generate an analog output value for the input portion of the truth table based on the simulated weights, Calculate a loss value from each of the analog output value and a predicted value for the output portion of the truth table, Adjust the simulated weights based on the calculated loss value, A computer system comprising the computer simulation, configured to re-provide the input portion of the predefined truth table to the computer simulation and recalculate the output value using the adjusted simulated weights until the analog output value generates the predicted value for the output portion of the truth table within a predefined error limit.

15. The computer system according to claim 14, further comprising a digital logic truth table stored in the computer memory as a training data set.

16. The computer system according to claim 15, wherein each simulated analog output value includes an error that is less than 49% of Vdd.

17. The computer system according to claim 16, wherein the loss value is calculated using a mean squared error (MSE) loss function.

18. The computer system according to claim 16, wherein the predefined error limit is 0%.

19. The computer system according to claim 18, further comprising one or more crossbar arrays and an inverter at each output of the one or more crossbar arrays that generate a digital one or zero output from a noisy input signal.

20. The computer system according to claim 19, wherein the one or more crossbar arrays are configured to be reset and reprogrammed by applying a voltage pulse.

Images