SYSTEMS AND METHODS FOR EXPLOITING ECONOMICS

Abstract

Neuromorphic integrated circuit, including: a multi-layered neural network that exists in an analog A multiplier array of a plurality of two-quadrant multipliers is arranged in a memory sector of the neuromorphic integrated circuit, wherein at least one or more of the plurality of two-quadrant multipliers is a bias-free two-quadrant multiplier. wherein each multiplier of the multipliers is connected to ground and draws a small amount of current when input signal values ​​for input signals to transistors of the multiplier are approximately zero, weighting values ​​of the transistors of the multiplier are approximately zero, or a combination thereof is true, and where the efficiency in the neural network, in combination with the number of multipliers wired to ground, minimizes the energy consumption of the neuromorphic integrated circuit.

Description

PRIORITY

[0001] This application claims priority over US patent application no. 16 / 041,565 filed on July 20, 2018, and US preliminary patent application no. 62 / 535,705 entitled “Systems and Methods for Sparsity Exploiting” filed on July 21, 2017, which are hereby incorporated in their entirety by reference into this application. AREA

[0002] Embodiments of the disclosure relate to the field of neuromorphic computing. In particular, embodiments of the disclosure relate to systems and methods for promoting efficiency in a neural network of a neuromorphic integrated circuit and for minimizing the energy consumption of the neuromorphic integrated circuit. BACKGROUND

[0003] Conventional central processing units (CPUs) process instructions based on clocked time. Specifically, CPUs operate by transferring information at regular time intervals. Based on CMOS (Complementary Metal-Oxide-Semiconductor) technology, silicon-based chips with more than 5 billion transistors per chip can be manufactured using a process as small as 10 nm. Advances in CMOS technology have translated into advances in parallel data processing, which is ubiquitous in mobile phones and multi-processor personal computers.

[0004] However, as machine learning becomes the norm for numerous applications, including bioinformatics, image processing, video games, marketing, medical diagnostics, online search engines, and more, conventional CPUs often cannot provide sufficient processing power while keeping power consumption low. Specifically, machine learning is a subfield of computer science that deals with software capable of learning from data and making predictions. Furthermore, one branch of machine learning is deep learning, which aims to utilize deep (multi-layered) neural networks.

[0005] Current research aims to develop direct hardware implementations of deep neural networks, which may include systems that attempt to simulate silicon neurons (e.g., neuromorphic computing). Neuromorphic chips (e.g., silicon computer chips designed for neuromorphic computing) operate by processing instructions in parallel (as opposed to traditional sequential computers) using electrical pulses delivered at non-uniform intervals. As a result, neuromorphic chips require significantly less energy to process information, particularly artificial intelligence (AI) algorithms. To achieve this, neuromorphic chips can contain up to five times as many transistors as a conventional processor while consuming up to 2,000 times less energy.Therefore, the development of neuromorphic chips aims to provide a chip with enormous processing capabilities that consumes far less energy than conventional processors. Furthermore, neuromorphic chips have been developed to support dynamic learning in the context of complex and unstructured data. US 5,336,937 A describes an analog synapse circuit for an artificial neural network that uses two complementary tunneled floating-gate MOSFETs in an inverter configuration, with each MOSFET storing a weight value. This weight value is set by storing a charge injected into the floating gate via Fowler-Nordheim tunneling or other tunneling techniques, thereby shifting the device's threshold voltage. A programming line applies a current pulse to the MOSFET's floating gate to write or erase this stored charge, thereby adjusting the MOSFET's weight.The two MOSFETs are connected such that their gate electrodes and drain electrodes are connected to each other, providing a common gate and drain between the two MOSFETs. An input line is connected to the common gate, and an output line is connected to the common drain. The source electrodes of each MOSFET are connected to reference voltages. The synapse circuit can be used in either a forward or a feedback network and can be extended from two-quadrant to four-quadrant operation. The synapse provides a single output current line that is a function of the input voltage and the stored weights. Multiple such synapses can be configured in a network, with the output lines of each synapse connected to a current summation node at the input of a neuron.An active load at the neuron's input enables both excitatory and inhibitory output currents from the synaptic circuit. REAGEN, Brandon [et al.]: Minerva: Enabling low-power, highly accurate deep neural network accelerators. In: Proceedings / ACM / IEEE 43rd annual international symposium on computer architecture, June 18-22, 2016, pp. 267-278. - ISBN 978-1-4673-8948-8 describes Minerva, a highly automated co-design approach across algorithm, architecture, and circuit layers for optimizing DNN hardware accelerators.

[0006] There is a constant need for the development of neuromorphic chips with enormous processing capacities that consume far less energy than conventional processors. This paper presents systems and methods for promoting energy efficiency in neural networks of neuromorphic chips and for minimizing the energy consumption of these chips. SUMMARY

[0007] This document discloses a neuromorphic integrated circuit according to claim 1.

[0008] In some embodiments, each multiplier of the multipliers draws no current when the input signal values ​​for the input signals to the transistors of the multiplier are zero, the weighting values ​​of the transistors of the multiplier are zero, or a combination thereof is true.

[0009] In some embodiments, the weight values ​​correspond to synaptic weight values ​​between neuronal nodes in the neuronal network arranged in the neuromorphic integrated circuit.

[0010] In some embodiments, input signal values, multiplied by the weighting values, provide output signal values ​​that are combined to arrive at a decision by the neural network.

[0011] In some embodiments, the transistor of the two-quadrant multipliers includes a metal oxide semiconductor field-effect transistor (“MOSFET”).

[0012] In some embodiments, each two-quadrant multiplier of the two-quadrant multipliers has a difference structure configured to allow programmatic compensation for overages when one of two cells is set with a higher weight value than the intended one.

[0013] In some embodiments, the neuromorphic integrated circuit is configured for one or more application-specific standard products (“ASSPs”) selected from keyword spotting, speaker identification, one or more audio filters, gesture recognition, image recognition, video object classification and segmentation, and autonomous vehicles, including drones.

[0014] In some embodiments, the neuromorphic integrated circuit is configured to operate on battery power.

[0015] Furthermore, a method for a neuromorphic integrated circuit according to claim 9 is disclosed herein.

[0016] In some embodiments, each multiplier of the multipliers draws no current when the input signal values ​​for the input signals to the transistors of the multiplier are zero, the weighting values ​​of the transistors of the multiplier are zero, or a combination thereof is true.

[0017] In some embodiments, the method further includes tracking rates of change for the weighting values ​​of each multiplier of the multipliers during training and determining whether certain weighting values ​​tend towards zero and how quickly these certain weighting values ​​tend towards zero.

[0018] In some embodiments, the method further includes lowering the weighting values ​​towards zero for those weighting values ​​that tend towards zero during training as part of promoting parsimony in the neural network.

[0019] In some embodiments, the weighting values ​​correspond to synaptic weighting values ​​between neuronal nodes in the neuronal network of the neuromorphic integrated circuit.

[0020] In some embodiments, the method further includes incorporating the neuromorphic integrated circuit into one or more ASSPs selected from keyword spotting, speaker identification, one or more audio filters, gesture recognition, image recognition, video object classification and segmentation, as well as autonomous vehicles, including drones.

[0021] In some embodiments, the neuromorphic integrated circuit is configured to operate on battery power.

[0022] Furthermore, a method of a neuromorphic integrated circuit according to claim 14 is disclosed herein.

[0023] In some embodiments, each multiplier of the multipliers draws no current when the input signal values ​​for the input signals to the transistors of the multiplier are zero, the weighting values ​​of the transistors of the multiplier are zero, or a combination thereof is true.

[0024] In some embodiments, the method further includes setting a subset of the weight values ​​to zero before training the neural network, thereby further promoting parsimony in the neural network.

[0025] In some embodiments, training is performed using a training algorithm configured to reduce a substantial number of the input signal values, weight values, or combinations thereof for the multipliers towards zero, thereby enabling minimal energy consumption by the neuromorphic integrated circuit.

[0026] In some embodiments, training promotes parsimony in the neural network by minimizing a cost function that includes a set of non-zero weight values ​​as weight values.

[0027] In some embodiments, the method further includes minimizing a cost function using an optimization function, including gradient descent, backpropagation, or both. An estimate of the energy consumption of the neuromorphic integrated circuit is used as a component of the cost function.

[0028] In some embodiments, the weighting values ​​correspond to synaptic weighting values ​​between neuronal nodes in the neuronal network of the neuromorphic integrated circuit.

[0029] In some embodiments, the method further includes incorporating the neuromorphic integrated circuit into one or more ASSPs selected from keyword spotting, speaker identification, one or more audio filters, gesture recognition, image recognition, video object classification and segmentation, as well as autonomous vehicles, including drones.

[0030] In some embodiments, the neuromorphic integrated circuit is configured to operate on battery power. DRAWINGS

[0031] Embodiments of this disclosure are shown by way of example and without limitation in the figures of the accompanying drawings, in which the same reference numerals indicate similar elements and in which: Fig. Figure 1 provides a scheme that represents a System 100 for designing and updating neuromorphic integrated circuits (“ICs”) according to some embodiments. Fig. 2 provides a scheme that represents an analogous multiplier arrangement according to some embodiments. Fig. Figure 3 provides a schematic representation of an analog multiplier array according to some embodiments. Fig. Figure 4 provides a scheme representing a bias-free two-quadrant multiplier of an analog multiplier array according to some embodiments. DESCRIPTION TERMINOLOGY

[0032] In the following description, specific terminology is used to describe features of the invention. For example, in certain situations, the term "logic" may be representative of hardware, firmware, and / or software configured to perform one or more functions. As hardware, the logic may include a circuit with data processing or storage functionality. Examples of such circuits may include, but are not limited to, one or more processor cores, a programmable gate array, a microcontroller, a controller, an application-specific integrated circuit, a wireless receiver, transmitter and / or transceiver circuits, semiconductor memory, or combinational logic.

[0033] The term "process" can encompass an instance of a computer program (e.g., a collection of instructions, also referred to here as an application). In one embodiment, the process may involve the simultaneous execution of one or more threads (e.g., each thread can execute the same or different instructions concurrently).

[0034] The term "processing" can encompass the execution of a binary file or script, or the launching of an application in which an object is processed, with launching being interpreted as placing the application in an open state and, in some implementations, performing simulations of actions typical of human interactions with the application.

[0035] The term "object" generally refers to a collection of data, whether in transit (e.g., over a network) or at rest (e.g., stored), which often exhibits a logical structure or organization that allows it to be categorized or typed. The terms "binary file" and "binary" are used synonymously here.

[0036] The term "file" is used broadly to refer to any set or collection of data, information, or other content used with a computer program. A file can be retrieved, opened, stored, edited, or otherwise processed as a single entity, object, or unit. A file may contain other files and may contain related or unrelated content, or no content at all. A file may also have a logical format or be part of a file system with a logical structure or organization of multiple files. Files may have a name, sometimes simply called a "filename," and often have attached properties or other metadata. There are many types of files, such as data files, text files, program files, and directory files. A file may be created by a user of a computer device or by the computer device itself.Access to and / or operations on a file can be performed by one or more applications and / or the operating system of a computer device. A file system can organize the files of a computer device or storage device. The file system can enable the tracking of files and access to those files. A file system can also enable operations on a file. In some implementations, the operations on the file can include creating, modifying, opening, reading, writing, closing, and deleting files.

[0037] Finally, the terms "or" and "and / or," as used here, are to be interpreted as including or signifying one or a combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "The following: A; B; C; A and B; A and C; B and C; A, B, and C." An exception to this definition occurs only when a combination of elements, functions, steps, or actions is mutually exclusive in some way.

[0038] As now referred to Fig. Figure 1 shows a schematic representation of a system 100 for designing and updating neuromorphic ICs according to some embodiments. As shown, the system 100 can include a simulator 110, a neuromorphic synthesizer 120, and a cloud 130 configured for designing and updating neuromorphic ICs such as the neuromorphic IC 102. As further shown, designing and updating neuromorphic ICs can involve creating a machine learning architecture with the simulator 110 based on a specific problem. The neuromorphic synthesizer 120 can then transform the machine learning architecture into a netlist directed to the electronic components of the neuromorphic IC 102 and the nodes to which the electronic components are connected.Additionally, the Neuromorphic Synthesizer 120 can convert the machine learning architecture into a graphical data system (GDS) file that details the IC layout for the Neuromorphic IC 102. From the netlist and the GDS file for the Neuromorphic IC 102, the Neuromorphic IC 102 itself can be fabricated using current IC manufacturing technology. Once fabricated, the Neuromorphic IC 102 can be used to address the specific problem for which it was designed. While the initially fabricated Neuromorphic IC 102 may include initial firmware with user-defined synaptic weights between nodes, this initial firmware can be updated by the Cloud 130 as needed to adjust the weights. Because the Cloud 130 is configured to update the Neuromorphic IC 102's firmware, it is not required for day-to-day use.

[0039] Neuromorphic ICs like the neuromorphic IC 102 can be up to 100 times more energy-efficient than graphics processing units (GPUs) and up to 280 times more energy-efficient than digital CMOS solutions, with accuracies that meet or exceed comparable software solutions. Therefore, such neuromorphic ICs are suitable for battery-powered applications.

[0040] Neuromorphic ICs, such as the neuromorphic IC 102, can be configured for an ASSP (Automated Service Provider) application, including but not limited to keyword recognition, speaker identification, one or more audio filters, gesture recognition, image recognition, video object classification and segmentation, and autonomous vehicles, including drones. For example, if the specific problem is keyword finding, the Simulator 110 can create a machine learning architecture focused on one or more aspects of keyword finding. The Neuromorphic Synthesizer 120 can then transform the machine learning architecture into a netlist and a GDS file corresponding to a neuromorphic keyword-finding IC that can be manufactured using current IC manufacturing technology.Once the neuromorphic IC for keyword recognition is manufactured, it can be used to perform keyword recognition, for example, in a system or device.

[0041] Neuromorphic ICs, such as the neuromorphic IC 102, can be used in toys, sensors, wearables, augmented reality systems (“AR”) or devices, mobile systems or devices, household appliances, Internet of Things (“IoT”) devices, or hearables.

[0042] As now referred to Fig. Figure 2 shows a schematic representation of an analog multiplier array 200 according to some embodiments. Such an analog multiplier array can be based on a digital NOR flash array, in that a core of the analog multiplier array can be similar to or the same as a core of the digital NOR flash array. That is to say, at least the selection and read circuitry of the analog multiplier array differs from that of a digital NOR array. For example, the output current is routed as an analog signal to the next layer and not via bit lines going to a read amplifier / comparator to be converted into a bit. Word line analogs are driven by analog input signals rather than by a digital address decoder. Furthermore, the analog multiplier array 200 can be used in neuromorphic ICs such as the neuromorphic IC 102.For example, a neural network in the analog multiplier array 200 can be arranged in a memory sector of a neuromorphic IC.

[0043] Because the Analog Multiplier Array 200 is an analog circuit, its input and output current values ​​(or signal values) can vary over a continuous range, rather than simply being switched on or off. This is useful for storing weights (also called coefficients) of a neural network, as opposed to digital bits. During operation, the weights are multiplied by input current values ​​to provide output current values, which are combined to reach a decision for the neural network.

[0044] The analog multiplier array 200 can use a standard programming and erase circuit to generate tunneling and erase voltages.

[0045] As now referred to Fig. Figure 3 shows a schematic representation of an analog multiplier array 300 according to some embodiments. The analog multiplier array 300 can use two transistors (e.g., a positive metal-oxide-semiconductor field-effect transistor [“MOSFET”] and a negative MOSFET) to perform a two-quadrant multiplication of a signed weight (e.g., a positive or negative weight) and a non-negative input current value. When an input current value is multiplied by a positive or negative weight, the product or output current value can be either positive or negative. A positively weighted product can be stored in a first column (e.g., a column that contains I Out0+ in the analog multiplier array 300), and a negatively weighted product can be stored in a second column (e.g., a column that I Out0-(corresponds to 300 in the analog multiplier array). The above positively and negatively weighted products or output signal values ​​can be used as a differential current value to provide useful information for decision-making.

[0046] Since each output current of the positive or negative transistor is connected to ground and is proportional to the product of the input current value and the positive or negative weight, respectively, the current consumption of the positive or negative transistor is approximately zero when the input current values ​​or weights are zero or approximately zero. That is, when the input signal values ​​are '0' or when the weights are '0', no power is consumed by the corresponding transistors of the analog multiplier array 300. This is important because in many neural networks, a large proportion of the values ​​or weights, especially after training, are '0'. In this way, energy is saved during periods of inactivity and standstill. This contrasts with differential pairwise multipliers, which draw a constant current (e.g., through a terminal bias current I) regardless of the input signal. B ) to record.

[0047] As now referred to Fig. Figure 4 shows a schematic representation of a bias-free two-quadrant multiplier 400 of an analog multiplier array, such as the analog multiplier array 300, according to some embodiments. Since, as previously described, each output current of the positive transistor (e.g., M1 of the two-quadrant multiplier 400) or negative transistor (e.g., M2 of the two-quadrant multiplier 400) is proportional to the product of the input current value and the positive or negative weighting, respectively, the power consumption of the positive or negative transistor is approximately zero (or zero) when the input current values ​​or weightings are approximately zero (or zero). This contrasts with differential pairwise multipliers, which provide a constant current (e.g., through a terminal bias current I) regardless of the input signal. B ) to record.

[0048] Significant energy savings can be achieved by promoting power efficiency (many zeros) through training in neural networks composed of such unbiased two-quadrant multipliers. That is, a neural network arranged in an analog multiplier array of a number of two-quadrant multipliers in a memory sector of a neuromorphic integrated circuit can be trained to promote power efficiency within the neural network, thereby minimizing the power consumption of the neuromorphic IC. Before the neural network is trained, a subset of the weight values ​​can even be set to zero, further promoting power efficiency in the neural network and minimizing the power consumption of the neuromorphic IC. In fact, the power consumption of the neuromorphic IC can be minimized to such an extent that the neuromorphic IC can be powered by battery power.

[0049] Training the neural network can involve using a training algorithm configured to decrease a significant number of input current values, weight values, or combinations thereof for the number of multipliers towards zero, thereby promoting network efficiency and minimizing the power consumption of the neuromorphic IC. The training can be iterative, with the weight values ​​adjusted in each iteration. Furthermore, the training algorithm can be configured to track the rate of change of each multiplier's weight value towards zero.Rate of change of weight values ​​can be used to determine whether and how quickly certain weight values ​​are trending towards zero. This information can be used in training to accelerate the rate at which weight values ​​approach zero, for example, by programming them to approximately zero or zero. Furthermore, training the neural network and promoting parsimony can involve minimizing a cost function that includes a set of non-zero weight values. Minimizing the cost function can involve using an optimization function that incorporates gradient decay, backpropagation, or both. An estimate of the neuromorphic integrated circuit's power consumption can be used as a component of the cost function.

[0050] When programming a two-quadrant multiplier, such as the bias-free two-quadrant multiplier 400, it is common practice to clear each programmable cell (e.g., the cell with transistor M1 and the cell with transistor M2) to set the cells to an extreme weighting value before each cell is set to its target weighting value. Extending this to a full array, such as the analog multiplier array 300, all programmable cells in the full array are set to an extreme weighting value before each cell is set to its target weighting value. Setting the cells to their arbitrary weighting values ​​creates the problem of overshoots if one or more cells are set to a higher weighting value than the intended value.This means that all cells in the complete array must be reset to the single extreme weight value before the cells are reset to their target weight values. However, the differential structure of each of the bias-free two-quadrant multipliers of the analog multiplier arrays provided here allows for the compensation of such a programming overrun, thus avoiding the time-consuming process of deleting and resetting all cells in an array.

[0051] In an example of compensating for programming overruns, v i- and v i+ The two-quadrant multiplier of 400 is deleted to set the cells to an extreme weighting value. If v i- If, after deleting cells with excessively high weight values, a program is programmed to do so, v i+ It can be programmed with a larger weighting value than originally set, in order to change the weighting value of v.i- to compensate and achieve the originally defined effect. Therefore, the differential structure can be exploited to compensate for programming overruns without having to delete one or more cells and start over.

[0052] The above systems and methods promote efficiency in neural networks of neuromorphic ICs and minimize the energy consumption of the neuromorphic ICs, so that the neuromorphic ICs can be powered by battery current.

[0053] In the foregoing description, the invention is described with reference to specific exemplary embodiments thereof. However, it is obvious that various modifications and changes can be made to it without deviating from the broader meaning and scope of the invention as set out in the appended claims.

Claims

[1] Neuromorphic integrated circuit, comprising: a multi-layered neural network that exists in an analog A multiplier array of a plurality of two-quadrant multipliers is arranged in a memory sector of the neuromorphic integrated circuit, wherein at least one or more of the plurality of two-quadrant multipliers is a bias-free two-quadrant multiplier. wherein each multiplier of the multipliers is connected to ground and draws a small amount of current when input signal values ​​for input signals to transistors of the multiplier are approximately zero, weighting values ​​of the transistors of the multiplier are approximately zero, or a combination thereof is true, and where the efficiency in the neural network, in combination with the number of multipliers wired to ground, minimizes the energy consumption of the neuromorphic integrated circuit. [2] Neuromorphic integrated circuit according to claim 1, wherein each multiplier of the multipliers does not draw current when the input signal values ​​for the input signals to the transistors of the multiplier are zero, the weighting values ​​of the transistors of the multiplier are zero, or a combination thereof is true. [3] Neuromorphic integrated circuit according to claim 1, wherein the weighting values ​​correspond to synaptic weighting values ​​between neuronal nodes in the neuronal network arranged in the neuromorphic integrated circuit. [4] Neuromorphic integrated circuit according to claim 3, wherein input signal values ​​multiplied by the weighting values ​​provide output signal values ​​which are combined to arrive at a decision of the neural network. [5] Neuromorphic integrated circuit according to claim 1, wherein the transistor of the two-quadrant multipliers includes a metal oxide semiconductor field-effect transistor (“MOSFET”). [6] Neuromorphic integrated circuit according to claim 1, wherein each bias-free two-quadrant multiplier of the two-quadrant multipliers has a difference structure configured to allow programmatic compensation for exceedances when one of two cells is set with a higher weight value than the intended one. [7] Neuromorphic integrated circuit according to claim 1, wherein the neuromorphic integrated circuit is configured for one or more application-specific standard products (“ASSPs”) selected from keyword spotting, speaker identification, one or more audio filters, gesture recognition, image recognition, video object classification and segmentation, and autonomous vehicles, including drones. [8] Neuromorphic integrated circuit according to claim 1, wherein the neuromorphic integrated circuit is configured to operate using battery power. [9] Method of a neuromorphic integrated circuit, comprising: Training a multi-layered neural network located in a an analog multiplier array of a plurality of two-quadrant multipliers is arranged in a memory sector of the neuromorphic integrated circuit, wherein at least one or more of the plurality of two-quadrant multipliers is a bias-free two-quadrant multiplier, wherein each multiplier of the multipliers is connected to ground and draws a small amount of current when input signal values ​​for input signals to transistors of the multiplier are approximately zero, weighting values ​​of the transistors of the multiplier are approximately zero, or a combination thereof is true; and Promoting efficiency in the neural network by training with a training algorithm configured to reduce a substantial number of the input signal values, weight values, or their combination for the multipliers towards zero, thereby enabling minimal energy consumption by the neuromorphic integrated circuit. [10] Method according to claim 9, wherein each multiplier of the multipliers does not draw any current when the input signal values ​​for the input signals to the transistors of the multiplier are zero, the weighting values ​​of the transistors of the multiplier are zero, or a combination thereof is true. [11] The method of claim 9, further comprising: Tracking the rates of change for the weighting values ​​of each multiplier during training; and Determine whether certain weighting values ​​tend towards zero and how quickly these certain weighting values ​​tend towards zero. [12] The method of claim 9, further comprising: Lowering the weight values ​​towards zero for those weight values ​​that tend towards zero during training, in order to promote parsimony in the neural network. [13] Method according to claim 9, wherein the weighting values ​​correspond to synaptic weighting values ​​between neuronal nodes in the neuronal network of the neuromorphic integrated circuit. [14] Method of a neuromorphic integrated circuit, comprising: Training a multi-layered neural network located in a an analog multiplier array of a plurality of two-quadrant multipliers is arranged in a memory sector of the neuromorphic integrated circuit, wherein at least one or more of the plurality of two-quadrant multipliers is a bias-free two-quadrant multiplier, wherein each multiplier of the multipliers is connected to ground and draws a small amount of current when input signal values ​​for input signals to transistors of the multiplier are approximately zero, weighting values ​​of the transistors of the multiplier are approximately zero, or a combination thereof is true; Tracking the rate of change for the weighting values ​​of each multiplier during training; Determine whether certain weighting values ​​tend towards zero and how quickly these weighting values ​​tend towards zero; and Moving the weight values ​​towards zero for those weight values ​​that tend towards zero, thereby promoting parsimony in the neural network. [15] Method according to claim 14, wherein each multiplier of the multipliers does not draw any current when the input signal values ​​for the input signals to the transistors of the multiplier are zero, the weighting values ​​of the transistors of the multiplier are zero, or a combination thereof is true. [16] The method of claim 14, further comprising: Setting a subset of the weight values ​​to zero before training the neural network further promotes parsimony in the neural network. [17] Method according to claim 14, wherein the training is performed using a training algorithm configured to reduce a substantial number of the input signal values, weight values ​​or combinations thereof for the multipliers towards zero, thereby enabling minimal energy consumption by the neuromorphic integrated circuit. [18] Method according to claim 14, wherein the training promotes parsimony in the neural network by minimizing a cost function which includes a set of non-zero weight values ​​as weight values. [19] The method of claim 14, further comprising: Minimizing a cost function with an optimization function, including gradient decay, backpropagation or both gradient decay and backpropagation, where an estimate of the energy consumption of the neuromorphic integrated circuit is used as a component of the cost function. [20] The method of claim 14, further comprising: Integration of the neuromorphic integrated circuit into one or more application-specific standard products (“ASSPs”), selected from keyword spotting, speaker identification, one or more audio filters, gesture recognition, image recognition, classification and segmentation of video objects, and autonomous vehicles, including drones.

Images