Neural Network Accelerator

JP2024523538A5Pending Publication Date: 2024-11-29SYNTHARA AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023579442
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-06-28
Filing Date
2021-11-25
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing artificial intelligence systems face challenges in adapting to changing data processing requirements while minimizing energy consumption, particularly in safety-critical tasks and portable devices with limited power availability, necessitating efficient and timely neural network processing.

Method used

A neural network accelerator system comprising an array of computing elements with control, storage, and arithmetic circuits, capable of dynamic reconfiguration, performs matrix-vector operations with reduced power consumption by minimizing data transfers and leveraging in-memory calculations.

Benefits of technology

The system achieves efficient and timely processing with reduced power usage, suitable for real-time applications in sensor fusion and machine learning tasks, providing energy-efficient computation and adaptability to various computing tasks.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The computing element array system includes an array of computing elements connected by a plurality of connections. Each computing element has a control circuit, a storage circuit, and an arithmetic circuit, and each connection connects two computing elements. The storage circuit receives and stores a data packet including a data value and a target tag from one of the connections. The arithmetic circuit performs an operation on the data value to generate a processed data value. The target tag identifies the computing element that will perform the operation on the data value. The control circuit identifies the computing element from the target tag, and if the identified computing element matches the computing element, the arithmetic circuit processes the data value. The control circuit modifies the data packet to include the processed data value, and outputs the modified data packet on the connection.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to artificial intelligence process architectures, processing devices and processing methods, and in particular to artificial neural network hardware. [Background technology]

[0002] Sensor fusion is an important application for artificial intelligence systems, where devices such as smart wearable devices, augmented reality (AR) headsets, drones, industrial and automotive applications integrate many different sensors such as voice, camera, inertial and other sensor types. These sensors generate a variety of data that must be integrated (fused) into a common understanding and representation to address a specific task. Data processing has different complexities and requires different data bandwidths. As the data from different sensors changes over time, the processing and bandwidth requirements also change. Keeping up with these changes for different applications while minimizing energy consumption is challenging and current solutions are not adequately accommodating them.

[0003] Artificial neural networks have been proposed for decades and have been further developed in recent times, both as software simulations in conventional stored-program digital computers and as actual dedicated electronic hardware. A typical neural network has many layers of nodes with interconnections of various strengths that respond to a set of inputs. Both hardware and software neural networks are adaptive, meaning they can improve their results by varying the strength of their interconnections, i.e., by training them, by feeding them data that achieves known good outputs.

[0004] Modern neural networks can become very large and require large amounts of power.

[0005] Some applications, especially safety-critical tasks, require real-time processing to provide useful output in a useful time. Furthermore, applications in portable devices have limited power available. Therefore, there is a demand for neural networks that provide efficient and timely processing with low power and small hardware. Summary of the Invention [Problem to be solved by the invention]

[0006] Some applications, especially safety-critical tasks, require real-time processing to provide useful output in a useful time. Furthermore, applications in portable devices have limited power available. Therefore, there is a demand for neural networks that provide efficient and timely processing with low power and small hardware.

[0007] An object of the present invention is to provide a technique for solving the above-mentioned problems. [Means for solving the problem]

[0008] The present embodiments can provide, among other things, neural networks that operate with reduced power and bandwidth. Some embodiments comprise a neural network accelerator (e.g., a hardware circuit) designed for dynamic neural network processing workloads. The neural network can be a fixed hardware electronic system or can comprise a state machine whose operation can be reconfigured using variable parameters stored in the hardware, for example set in firmware or other software.

[0009] The embodiments disclosed herein include a real-time computing system, which may be a pattern matching system, a machine learning system, a neural network system, or an artificial intelligence system. The real-time computing system may be a neural network accelerator. According to the embodiments disclosed herein, the real-time computing system includes a hardware system that continuously monitors a variable input (e.g., a sensor input) and generates a monitor signal (e.g., an interrupt signal) responsive to the variable input. The sensor input includes a single value, a vector, or an array of values, such as an image.

[0010] The hardware system may be an electronic or optoelectronic computing system. The operation of the hardware system may be fixed or reconfigurable by stored parameters or firmware. For example, the operation of the hardware may be controlled and changed by storing variable values ​​in the hardware that can control and change the operation of the hardware, or by providing variables in the hardware's calculations or other software means. The disclosed embodiments may provide a hardware system with an array of in-memory matrix multipliers, each of which is a computing element. Each computing element comprises an array of computing elements connected by connections, each computing element including a control circuit, a storage circuit, and an operation circuit.

[0011] According to this embodiment, the computing element array system includes an array of computing elements connected by connections. Each computing element includes a control circuit, a storage circuit, and an arithmetic circuit. Each connection connects two computing elements. In each computing element in the computing element array, the storage circuit is operable to receive a data packet including a data value and a target tag from one of the connections and store the data packet. The arithmetic circuit is operable to perform an operation on the data value to generate a processed data value. The target tag specifies the computing element that performs the operation on the data value. The control circuit is operable to identify the computing element as the computing element specified by the target tag, and if the computing element identified by the target tag is a computing element, to enable the arithmetic circuit to process the data value, modify the data packet to include the processed data value, and enable output of the modified data packet on one of the connections.

[0012] The target tag may include one or more routing values ​​arranged in the routing value ordered list. The control circuitry may be operable to modify the routing values ​​in the routing value ordered list. The control circuitry is operable to remove a routing value from the routing value ordered list. The routing value ordered list includes a routing value list pointer, and the control circuitry is operable to modify the routing value list pointer. One or more of the routing values ​​may be a no-route value that identifies a computing element, and the control circuitry identifies the computing element with the no-route value. In some embodiments, a routing value list that is a null set (empty list) is equivalent to a no-route value. The computing element may include a self tag that describes the computing element. The target tag may include one or more computing element specific information. The computing element specific information may be arranged in the computing element ordered list. The computing element specific information may be an identification of the computing element, or an identification of a function of the computing element, such as an operation or calculation of the computing element. The control circuitry may identify the computing element by comparing the computing element specific information in the target tag to the self tag. The control circuitry may be operable to modify the computing element in the computing element ordered list. The control circuitry may be operable to remove a computing element from the computing element ordered list. The computing element ordered list may include a computing element list pointer, and the control circuitry may be operable to modify the computing element list pointer. The control circuitry is operable to modify a target tag to provide an end tag to the computing element ordered list. The target tag may specify multiple computing elements that perform sequential operations on the data value.

[0013] The array can be a two-dimensional grid, a two-dimensional array, a torus, a ring, a three-dimensional grid, or a three-dimensional array.

[0014] According to this embodiment, the computing element array system comprises a data control circuit connected to one or more computing elements in the array of computing elements. The data control circuit is operable to receive data packets from outside the array of computing elements and to transmit data packets outside the array of computing elements. The data control circuit may be connected to one or more computing elements in the array of computing elements via a connection. The data control circuit may be connected to one or more computing elements in the array of computing elements via a bus. The data control circuit may be operable to receive data packets from the array of computing elements and to return data packets to the array of computing elements. The data control circuit may be connected to one or more computing elements in the array of computing elements.

[0015] According to some embodiments, a data packet received by the data control circuit from a computing element in the array includes an end tag or a no root value, and the data control circuit is operable to transmit a data value from the computing element array in response to the end tag or the no root value. According to some embodiments of the present disclosure, a target tag of a data packet received by the data control circuit from a computing element in the array does not include an end tag or a no root value, and the data control circuit is operable to transmit the data value to a computing element in the array.

[0016] According to some embodiments, the connections are bidirectional connections. According to some embodiments, the connections are unidirectional connections. According to embodiments, one or more of the computing elements are connected by connections to two, three, four, or eight adjacent computing elements. According to some embodiments, the storage circuitry of one or more of the computing elements is operable to receive input data from one, two, three, four, or eight adjacent computing elements, and / or to output data to one, two, three, four, or eight adjacent computing elements.

[0017] According to some embodiments, the computation circuitry is operable to (i) compute a matrix-vector product operation, (ii) compute an addition, (iii) compute an accumulation, (iv) compute a multiplication, (v) compute a transform, (vi) perform a filter operation, (vii) perform a table lookup, or (viii) perform any combination of (i)-(vii). According to some embodiments of the present disclosure, one or more of the computing elements are configurable. According to some embodiments of the present disclosure, one or more of the computing elements are operable to store one or more configuration variables. According to some embodiments of the present disclosure, one or more of the configuration variables include a self tag, a computation constant, or an operational setting.

[0018] According to some embodiments of the present disclosure, the computing element is (i) a routing node, (ii) a memory node, (iii) operable to perform routing pass-through, (iv) operable to store one or more data values ​​for a specified period of time, or (v) any combination of (i)-(iv).

[0019] According to some embodiments of the present disclosure, the operational circuitry of a first computing element is operable to perform a first operation and the operational circuitry of a second computing element is operable to perform a second operation different from the first operation. The operational circuitry of one of the computing elements may be operable to perform two or more operations. The control circuitry may be operable to select an operation in response to a configuration variable.

[0020] According to some embodiments of the present disclosure, the control circuitry is operable to select an action responsive to the target tag or the routing value.

[0021] According to some embodiments of the present disclosure, two or more of the computing elements are synchronous, two or more of the computing elements are asynchronous, or both.

[0022] Each computing element may be directly or indirectly connected to the configuration control circuitry, for example via a serial bus, a parallel bus, a daisy chain connection, or the like.

[0023] The target tag can specify a mathematical function or a computation graph.

[0024] According to an embodiment of the present disclosure, a computing element array comprises an array of computing elements connected by connections, each computing element comprising a control circuit, a storage circuit, and an arithmetic circuit, each computing element operable to store a self-tag identifying an operation performed by the arithmetic circuit. For each computing element in the array of computing elements, the storage circuit may be operable to receive a data packet from one of the connections and store the input data, the data packet may include a data value and a target tag specifying an operation to be performed on the data value, the control circuit may be operable to compare the target tag with the self-tag, if the self-tag matches the target tag, the arithmetic circuit may be operable to process the data value, modify the data packet to include the processed data value, and enable output of the modified data packet on one of the connections, and the arithmetic circuit may be operable to process the data value to generate the processed data value. The target tag may be set from one or more routing values ​​arranged in a routing value order list. The control circuit may be operable to modify a routing value in the routing value order list. The control circuit may be operable to remove a routing value from the routing value order list. The routing value ordered list includes a routing value list pointer, and the control circuitry is operable to modify the routing value list pointer.

[0025] According to an embodiment of the present disclosure, a computing element array comprises an array of computing elements connected by connections, each computing element comprising a control circuit, a storage circuit, and an arithmetic circuit. For each computing element in the computing element array, the storage circuit may be operable to receive a data packet from one of the connections and store the data packet. The data packet may include a data value and a target tag specifying a routing value, one of the routing values ​​being a no-route value indicating that the arithmetic circuit operates on the data value, and the control circuit may be operable to identify the no-route value, and if the no-route value is identified, enable the arithmetic circuit to process the data value, modify the data packet to include the processed data value, and enable output of the modified data packet on one of the connections, and the arithmetic circuit may be operable to process the data value to generate the processed data value. The target tag may include one or more routing values ​​arranged in a routing value ordered list. The control circuit may be operable to modify the routing value in the routing value ordered list. The control circuit may be operable to remove a routing value from the routing value ordered list. The routing value ordered list includes a routing value list pointer, and the control circuitry is operable to modify the routing value list pointer.

[0026] According to embodiments of the present disclosure, the matrix-vector multiplier comprises an array of unit multipliers. Each unit multiplier may comprise a latch for storing a stored value, a capacitor, and a switch responsive to a switch input for conducting charge from the latch to the capacitor. The capacitors of the unit multipliers are electrically connected in parallel to provide an output charge value. The stored value may be an analog value, the switch input may be an analog input, and the unit capacitance multiplier may be an analog multiplier. According to some embodiments, the stored value is a digital value, the switch input is a digital input, and the unit multiplier is a digital multiplier. According to some embodiments, the switch input is a digital value, and the embodiment comprises a digital-to-analog converter connected to the switch for converting the switch input to an analog value. According to some embodiments, the output value is an analog charge value, and the embodiment comprises an analog-to-digital converter connected to the analog charge value for converting the analog charge value to an output digital value. The switch may comprise a bipolar transistor, the latch may comprise a capacitor, or both.

[0027] Embodiments of the present disclosure may include a conversion circuit connected to a switch input, a latch, a capacitor, or any combination of a switch input, a latch, and a capacitor.

[0028] According to an embodiment of the present disclosure, the matrix-vector multiplier comprises an array of digital unit multipliers. Each digital unit multiplier may comprise a latch to store a stored value, a resistor, and an array of switches responsive to the switch input to multiply the stored value and the switch input to provide a bit value. A bit adder may add the bit values ​​to generate a product of the stored value and the switch input. The bit adder may be a digital logic circuit.

[0029] According to an embodiment of the present disclosure, a matrix-vector multiplier comprises a vector input circuit providing N digital input values, an M×N array of stored digital weight values ​​stored in N rows of M columns, an M×N array of multipliers operable to multiply each column of the N stored digital weight values ​​with the N digital input values ​​to generate N products for each column M, and an accumulator operable to accumulate the digital values ​​of the N products for each column M to provide M accumulated values. Each of the N digital input values ​​is comprised of i bits, each of the stored digital weight values ​​is comprised of j bits, and each multiplier calculates a product of less than i+j bits. [Brief description of the drawings]

[0030] [Figure 1] FIG. 1 is a schematic diagram of a real-time computing system including a sensor and a hardware unit including a computing accelerator, according to an exemplary embodiment of the present disclosure. [Diagram 2] 1 is a functional schematic diagram of a computing accelerator according to an exemplary embodiment of the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of a matrix-vector product circuit according to an exemplary embodiment of the present disclosure. [Figure 4] 1 is a schematic diagram of a grid-connected computing accelerator including an array of computing elements according to an exemplary embodiment of the present disclosure. [Figure 5A] FIG. 2 is a schematic diagram of a computing element according to an exemplary embodiment of the present disclosure. [Figure 5B] FIG. 2 is a schematic diagram of a computing element according to an exemplary embodiment of the present disclosure. [Figure 6A] FIG. 2 is a schematic diagram of connections between computing elements according to an exemplary embodiment of the present disclosure. [Figure 6B] FIG. 2 is a schematic diagram of connections between computing elements according to an exemplary embodiment of the present disclosure. [Figure 7]1 is a schematic diagram of a torus-connected computing accelerator including an array of computing elements, according to an exemplary embodiment of the present disclosure. [Figure 8A] FIG. 2 illustrates a diagram of bits in a data packet according to an exemplary embodiment of the present disclosure. [Figure 8B] FIG. 2 illustrates a diagram of bits in a data packet according to an exemplary embodiment of the present disclosure. [Figure 8C] FIG. 2 illustrates a diagram of bits in a data packet according to an exemplary embodiment of the present disclosure. [Figure 9] FIG. 2 is a schematic diagram of an array of computing elements according to an exemplary embodiment of the present disclosure. [Figure 10] FIG. 2 is a schematic diagram of an array of computing elements illustrating data flow according to an exemplary embodiment of the present disclosure. [Figure 11A] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11B] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11C] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11D] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11E]1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11F] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11G] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11H] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11I] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11J] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11K] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11L] 1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 11M]1 is a contiguous target tag of a compute graph executed by an array of computing elements using self tags and computing element identifiers, according to an exemplary embodiment of the present disclosure. [Figure 12] FIG. 11 is a flow diagram illustrating the flow of data and computations of a computation graph in an array of computing elements corresponding to FIGS. 11A-11M, in accordance with an exemplary embodiment of the present disclosure. [Figure 13A] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13B] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13C] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13D] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13E] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13F] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13G] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13H]11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13I] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 13J] 11 is a diagram illustrating successive target tags of a compute graph executed by an array of computing elements using a no-path routing value, according to an exemplary embodiment of the present disclosure. [Figure 14] FIG. 13 is a flow diagram illustrating the flow of data and computations of a computation graph in an array of computing elements corresponding to FIGS. 13A-13J, according to an exemplary embodiment of the present disclosure. [Figure 15] FIG. 2 is a schematic diagram of an array of computing elements according to an exemplary embodiment of the present disclosure. [Figure 16] FIG. 2 is a schematic diagram of an array of computing elements according to an exemplary embodiment of the present disclosure. [Figure 17] FIG. 2 is a schematic diagram of an array of computing elements according to an exemplary embodiment of the present disclosure. [Figure 18] FIG. 2 is a schematic diagram of an array of computing elements according to an exemplary embodiment of the present disclosure. [Figure 19] FIG. 2 is a schematic diagram of an array of computing elements according to an exemplary embodiment of the present disclosure. [Figure 20] FIG. 2 is a simplified circuit diagram of a unit capacitance multiplier according to an exemplary embodiment of the present disclosure. [Figure 21] FIG. 2 is a simplified schematic diagram of an array of unit capacitance multipliers according to an exemplary embodiment of the present disclosure. [Figure 22] FIG. 2 is a simplified schematic diagram of an array of unit capacitance multipliers arranged to perform digital multiplication in accordance with an exemplary embodiment of the present disclosure. [Figure 23A] 1 is a unit logic cell according to an exemplary embodiment of the present disclosure. [Figure 23B] 1 is a circuit using unit logic cells to accumulate multiplied single-bit values ​​in accordance with an exemplary embodiment of the present disclosure. [Figure 23C] FIG. 1 illustrates a bit numbering diagram useful in understanding embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiment are merely examples, and are not intended to limit the technical scope of the present invention.

[0032] The features and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters identify corresponding elements throughout.

[0033] In the drawings, like reference numbers generally indicate identical, functionally similar, and / or structurally similar elements. The drawings are not necessarily drawn to scale.

[0034] [First embodiment] Certain embodiments of the present disclosure are directed to neural network, pattern matching, or machine learning computers that provide efficient and timely processing with reduced power and hardware requirements. Such embodiments may comprise a computing accelerator, e.g., a neural network accelerator, a pattern matching accelerator, a machine learning accelerator, or an artificial intelligence computing accelerator designed for static or dynamic processing workloads, e.g., but not limited to, sensor fusion. Dynamic workloads, in some embodiments, include at least two categories of tasks: always-on tasks that require continuous temporal monitoring, and interrupt-driven tasks that occur at specific times. In some cases, always-on tasks are relatively less computationally intensive, and interrupt-driven tasks are relatively more computationally intensive. Examples of always-on tasks include, but are not limited to, continuously classifying temporal signals from one or more sensors, e.g., biomedical sensors, motion sensors such as acceleration sensors, and audio sensors such as microphones, to provide motion detection and classification, and key phrase detection. Examples of interrupt-driven tasks may include, but are not limited to, image segmentation, speech translation, and object detection and identification. The always-on task being processed may monitor a continuous stream of input (e.g., sensor) data to detect trigger events and provide a trigger signal (e.g., an interrupt) in response to the trigger event for an interrupt-driven task being processed by another hardware or software system. The system of the present disclosure may be, for example, a computational accelerator connected to a conventional computer system or supercomputer via any conventional protocol or hardware interface, either serial or parallel.

[0035] The embodiment of the present disclosure shown in FIG. 1 comprises a hardware system 10 including a computing accelerator 20 that provides energy-efficient computation in response to an always-on, continuous, or continuously sampled signal, for example, from a sensor 30 providing a sensor signal 32. The hardware system 10 can comprise a microcontroller or computer (e.g., including a memory 12 and a processor 14) that can respond to the sensor signal 32 from the sensor 30 and communicate with the computing accelerator 20. According to some embodiments, the hardware system 10 can achieve detection of a trigger event to provide a trigger interrupt signal 22 while consuming only microwatt levels of power. According to embodiments of the present disclosure, the computing accelerator 20 can process a data set (e.g., calculate or compute a result by mathematically manipulating data values ​​in the data set) faster and with less power than a conventional stored-program digital computer (e.g., a computer including a software program stored in a memory (e.g., memory 12) that drives a central processing unit (e.g., processor 14)). According to an embodiment, the computing accelerator 20 can be a hardware accelerator. The operation of the hardware accelerator may be configurable, for example, using firmware or configuration variables. According to an embodiment, the computing accelerator 20 can compute neural network and machine learning tasks that are traditionally dominated by matrix-vector multiplication operations more efficiently and with less central processing unit-memory interaction or data transfer. The computing accelerator 20 can provide a useful combination of computing speed and adaptability to various computing tasks or functions.

[0036] The computing accelerator 20 may comprise an in-memory processing architecture that eliminates or reduces data transfers between the processor 14 (e.g., central processing unit or CPU) and the memory 12, as found in conventional stored program digital computers, to perform matrix-vector product operations. Because power usage in such conventional systems is dominated by memory-to-CPU data transfers, eliminating or reducing such data transfers may significantly reduce power usage in the hardware system 10. The hardware system 10 and the computing accelerator 20 may be implemented in one or more application specific integrated circuits (ASICs), may operate in real time in response to sensor 30 inputs, and may support a direct interface to raw analog data from the sensors 30 (e.g., different sensors 30A, 30B, 30C).

[0037] According to an embodiment of the present disclosure and as shown in FIG. 1 and FIG. 2, the hardware system 10 may comprise an array of in-memory matrix multipliers (e.g., an array of computing elements 70), may operate at sub-milliwatt energy levels, and may process input data synchronously or asynchronously. FIG. 2 is a functional illustration of the operation of the hardware system 10 and the computing accelerator 20, rather than a circuit representation. The computing accelerator 20 may comprise an in-memory matrix multiplier (MVM) 60, a non-linearity converter (NL) 62, an accumulator block (ACC) 64, and one or more memory nodes (MEM) 66. The hardware system 10 may comprise internal leakage, control, and clock gating logic 56, an interface circuit for the sensor 30, a sensor interface circuit 50 including, for example, an analog general purpose input / output circuit (GPIO) 52, and an automatic gain control (AGC) circuit 54 electrically connected to the analog sensor 30. The hardware system 10 and computing accelerator 20 can support low-pass recurrent neural networks (RNNs), fully connected neural network layers, gated RNNs such as long / short-term memory networks (LSTMs) or gated recurrent units, and one-dimensionally connected neural networks (1D-CNNs).

[0038] According to the present embodiment, the matrix vector multiplication calculation is based on an in-memory calculation module and architecture that performs the calculation efficiently with fewer data transfers, thereby reducing power consumption. 2(n is the order of the number of elements in the matrix). This embodiment has been modeled and simulations show that an array of computing elements 70 can operate at 1 Top / s / W (tera operations per second per watt) or less (e.g., 4 Top / s / W or less, 10 Top / s / W or less, 20 Top / s / W or less, 50 Top / s / W or less, or 100 Top / s / W or less). It may provide 4-100 Top / s / W. It may provide 50 GOps / s or less, 100 GOps / s or less, or 150 GOps / s or less.

[0039] Figure 3 shows an analog current-based implementation of a vector-matrix multiplication circuit. Each input line on the left carries a separate input data 40 value with a separate current (given voltage). Each output line across the bottom of the matrix has a resistor R WT The input current I IN The sum of the resistance R WT is the output current I OUT =V / R WT Provide a weight value associated with each input line to perform vector multiplication such that the total current on each output line, I OUT provides an output vector that is accumulated (added or summed) to provide a final weighted output value. The inputs, outputs, and accumulations may be analog or digital with corresponding digital-to-analog converters 97 and analog-to-digital converters 98 as shown in FIG.

[0040] The hardware implementation of the vector-matrix circuit must be sized to efficiently scale with the size of the input vectors. If the vector-matrix circuit is too large, much of the circuitry will go unused (e.g., some of the input lines will have low input currents I IN is zero, or the corresponding R WT , zero, or both). If the vector-matrix circuit is too small, vector multiplication must be decomposed into smaller vectors. Similarly, having too many small vectors leads to inefficiencies. By applying a nonlinear transformation (transformation) between the accumulation and the matrix-vector product operations, the circuit can better model the neuronal behavior.

[0041] According to a general embodiment of the present disclosure, and as shown in Figure 4 and in more detail in Figures 5A, 5B, 6A, and 6B, the computing accelerator 20 comprises an array of computing elements 70 connected by connections 48, as shown in Figure 4. Although the array in Figure 4 is illustrated as a two-dimensional grid, embodiments of the present disclosure are not so limited and may be a one-dimensional array, a two-dimensional array, a ring, a torus (as illustrated in Figure 7), a three-dimensional array, or other arrangement of connected computing elements 70.

[0042] As shown in Figure 5A, each computing element 70 comprises computing element control circuitry 78, storage circuitry 72, and operation (e.g., computation) circuitry 74 that performs an operation (e.g., a mathematical operation such as a matrix-vector product) on input data 40 stored in the storage circuitry 72. Figure 5B shows a computing element 70 that receives input data 40 for an operation from one or more connections 48 depending on the type of operation to be performed (e.g., addition or multiplication of two or more data values ​​D or vectors received separately from two or more connections 48).

[0043] The storage circuit 72 is operable to receive input data 40 from one of the connections 48 and store the received input data 40. The storage circuit 72 can be one or more shift registers, memories, or flip-flops. The input data 40 is information communicated over the connections 48, such as data values ​​and metadata describing operations and data routing through the array of computing elements 70. Both the connections 48 and the input data 40 are represented by connecting arrows indicating the source of the input data (e.g., output data 44 from an adjacent computing element 70) and the destination of the input data 40, as shown, for example, in FIG. 6A. FIG. 6B illustrates a bidirectional connection 48 between two adjacent computing elements 70, where separate wires are used for data communication. In some embodiments, the same wires are used to communicate data passed in both directions, for example, using tri-state drivers and storage circuits 72 that provide high resistance when not in use.

[0044] According to an embodiment of the present disclosure and as shown in Figures 6A and 6B, each computing element 70 can communicate with a neighboring computing element 70 using a signal handshake. The source (sending) computing element 70 can generate a request signal Req indicating that it is ready to transmit a data packet 80, and for example outputs the data packet 80 (e.g., serially or in parallel) on one or more wires connecting the neighboring computing elements 70. When the neighbor of the receiving computing element 70 receives the request signal Req, it inputs and stores the data packet 80 and provides an acknowledgement signal Ack that is sent back to the sending computing element 70, for example, on the wire (connection 48) connecting the neighboring computing elements 70. When the sending computing element 70 receives the acknowledgement Ack, the data packet 80 is no longer output from the sending computing element 70 and the transmission and reception of data is completed.

[0045] 4 illustrates an embodiment in which data control circuitry 24, including input control circuitry 26, receives input data 40 and communicates the input data 40 to one or more computing elements 70 via an input bus 42 connected to one or more computing elements 70, e.g., on the edge of an array of computing elements 70 in a grid configuration. Similarly, data control circuitry 24 may receive processed output data 44 from one or more computing elements 70 via an output bus 46 connected to one or more computing elements 70, e.g., on the edge of an array of computing elements 70 in a grid configuration, and include output control circuitry 28 that communicates the processed output data 44 to, e.g., memory 12 or processor 14 of hardware system 10. In some embodiments of the present disclosure and as illustrated in FIG. 7, data control circuitry 24 is connected to one or more computing elements 70, external sources of input data 40, and destinations of processed output data 44 by one or more connections 48. Data control circuitry 24 can be a type of computing element 70. In the embodiment shown in FIG. 7, the computing elements 70 are connected by connections 48 in a torus (a two-dimensional ring) with the data control circuits 24 connected in one ring of rows.

[0046] According to an embodiment of the present disclosure, the data packet 80 flows through the array of computing elements 70 to perform an operation on the data value D stored in the target tag T, as shown in FIG. 8A, in response to a routing value R (shown in FIG. 8B) communicated with the data value D. The routing value R dictates the path the data packet 80 takes through the array of computing elements 70. According to an embodiment of the present disclosure, the operation performed on the data value is determined by the operation of the computing element 70 in which the data value D is stored. The operation may be a mathematical operation that modifies the data value D, a timing operation (e.g., imparting a delay to the data packet 80 transmission between computing elements 70), a storage operation (e.g., memory) to store the data packet 80, or a routing operation to transmit the data packet 80 over a connection 48 (e.g., including a communication circuit).

[0047] According to some embodiments, each computing element 70 includes or stores a self-tag S that identifies the computing element 70 or its operation. The self-tag S need not be unique and can be the same for computing elements 70 performing the same operation. When the self-tag S of the computing element 70 matches the identifier C of the computing element 70, the target tag T of the data packet 80, the operation is performed. According to some embodiments, the self-tag S or the identifier C of the computing element 70 are not used. Instead, the routing value R encodes a no-route value N that, when encountered by the computing element 70, indicates that the operation of the computing element 70 should be performed on the data value D. The no-route value N of the routing value R is any value that the computing element 70 recognizes as an indication that the data packet 80 should not be sent on the connection 48, but rather enables the operation of the computing element 70 on the data value D of the data packet 80, a no-connection (e.g., no routing) value, or a zero count value of the remaining routing values ​​R (e.g., no routing values ​​R remain in the list). This name indicates that an operation is performed rather than routing the data packet 80 to a connection 48 (typically indicated by a routing value R). In some embodiments, the routing value R includes a counter that specifies the number of routing steps remaining, and the operation is performed when the counter is equal to zero. FIG. 8C illustrates a target tag T for such an embodiment. According to an embodiment of the present disclosure, identifying a computing element 70 on which the data packet 80 resides along with the computing element 70 specified by the target tag T in the data packet 80 includes (i) matching a self tag S stored on the computing element 70 to an identifier C of the computing element 70 (discussed with respect to FIGS. 11A-12), or (ii) finding a no-route N routing value R (discussed with respect to FIGS. 13A-4).Other routing values ​​R include a routing connection 48 identifier, a routing instruction, or a routing value count.

[0048] As shown in FIG. 8A, the input data 40 (and output data 44) are provided in data packets 80 including a data value D (e.g., a digital number of one or more bits) and a target tag T. As shown in FIG. 8B, the target tag T may include metadata specifying one or more computing element identifiers C, one or more routing values ​​R, one or more constants k used by the computation circuitry 74, and any other useful metadata. The target tag T may include any other packet header information useful in the operation of the computing element 70. As shown in FIG. 8C, the target tag T may include metadata specifying one or more routing values ​​R including a no-route value N, one or more constants k used by the computation circuitry 74, and any other useful metadata. The target tag T and the various values ​​within the target tag T may be digital bit strings or lists, with a prescribed number of bits for the various elements (e.g., computing element specific information, routing value R). The target tag may include a count value of the routing bits of the target tag T. The computing element identifiers C and the routing values ​​R are provided in an ordered list within the target tag T. In some embodiments, the ordered list includes a pointer (e.g., an address) that indicates the current, operable, or exposed value in the list, or the ordered list includes a counter that indicates the number of remaining values ​​in the list. If the counter is equal to zero, no further routing steps are available and the operation is performed. In some embodiments, the current, operable, or exposed value is the first value in the list, or the value at the head of the list, e.g., as specified by an order or address in the list.

[0049] In some embodiments, for example with a target tag T including a computing element identifier C, the computing element 70 stores a self-tag S that identifies the operation to be performed by the computing circuitry 74. The self-tag S can be one of one or more configuration variables V stored in the computing element 70. In some embodiments, the self-tag S is a configuration variable V that was loaded into the computing element 70 during an initialization step of the array of computing elements 70, for example from an external controller.

[0050] In some embodiments of the invention, the computing element control circuitry 78 is operable to receive a target tag T, parse the target tag T to extract a computing element 70 identifier C or a current routing value R, compare the computing element 70 identifier C of the target tag T to a self tag S or recognize a no-routing value N of the routing value R, process the data value D and route the data packet 80 in response to the routing value R if the comparison is a match, modify the target tag T, and enable corresponding operation on the computing element 70 to enable output of processed output data 44 or simply route the input data 40 if the comparison is not a match. The computing element control circuitry 78 enables the routing (pass-through) circuitry communicating input data 40 from one connection 48 to output data 44 on another connection 48, for example as shown by the dashed lines in Figures 5A and 5B. The operation circuitry 74 is operable to process the data value D (input data 40) to generate processed output data D (output data 44) in a process data packet 80 that is output to one of the connections 48, for example as output data 44.

[0051] The computing elements 70 may be digital computing elements 70 and may be any digital circuit. In some embodiments, the computing elements 70 may include analog circuitry. The computing element control circuitry 78 may similarly include digital circuitry, logic, storage elements such as flip-flops and latches, and may comprise a microcontroller, a state machine, or discrete control logic. The arithmetic circuitry 74 may comprise, for example, adders, multipliers, filters (e.g., first order filters), transformation circuits (e.g., linear or non-linear transformations performed by look-up tables, etc.). The arithmetic circuitry 74 may employ a constant k (e.g., a multiplication or addition constant) provided as a configuration variable V by the computing element control circuitry 78, which may be loaded into the particular computing element 70 before the input data 40 is provided to the computing element 70. In some embodiments, the constant k may be included in the target tag T, as shown in Figures 8B and 8C.

[0052] Under the control of the computing element control circuitry 78, the input data 40 may be loaded simultaneously into each storage circuit 72 in response to a common clock provided to each computing element 70. In some embodiments, alternating storage circuits 72 are controlled by a positive-going edge of the clock and a negative-going edge of the clock to avoid race conditions. The clock period cannot exceed the time required for the arithmetic circuitry 74 to process the data value D. In such embodiments, each computing element 70 is synchronous, performing calculations on different data elements simultaneously. In some embodiments, the computing elements 70 are asynchronous, operating independently and in response to data packets 80 transmitted from one computing element 70 to another. In either case, the data packets 80 may flow through the array of computing elements 70, allowing a large number of computing elements 70 to be included in the array and operating simultaneously, thereby achieving high computation and data rates. Additionally, computing elements 70 that are not being used at any one time to perform an operation are relatively quiescent, conserving power in the array of computing elements 70.

[0053] FIG. 9 illustrates an array of computing elements 70 with the operation of each computing element 70 labeled. The computing elements 70 need not all be identical and may have different operation circuitry 74. Some of the computing elements 70 may include operation circuitry 74 that calculates a matrix-vector product (e.g., using analog or digital hardware that implements the function as illustrated in the circuit of FIG. 3). Some of the computing elements 70 include addition operation circuitry 74 that adds two vectors. Some of the computing elements 70 include transformation operation circuitry 74, such as a linear or non-linear function implemented, for example, by a look-up table. In some embodiments, the computing element control circuitry 78 includes circuitry operable to determine whether an operation is possible (e.g., whether there is a match between the data packet 80 and the computing element 70) and allow the operation by the operation circuitry 74 if possible, or pass the data packet 80 to an adjacent computing element 70 if the operation is not possible, for example as a pass-through routing operation or function. Some computing elements 70 may be routing nodes that control the routing of data packets 80 or memory nodes that store or delay data packets 80, for example comprising an array of shift registers or SRAM (static random access memory), under the control of computing element control circuitry 78.

[0054] According to some embodiments of the present disclosure, each computing element 70 has a fixed computation function (e.g., an arithmetic circuit 74 that is only one of an adder, a multiplier, and a look-up table). According to some embodiments, the computing element 70 may comprise multiple arithmetic circuits (e.g., one or more of an adder, a multiplier, and a look-up table) and a configuration for selectively performing a desired computation stored in the computing element 70, e.g., received in a target tag T. For example, each computing element 70 may implement a state machine that is selected in response to a desired computation. The target tag T may specify one or more operations to be performed on a data value D. Upon completion of the computation, the target tag T may be modified. According to some embodiments, the computation of the computing element 70 may be configured to perform a selected or desired operation or function in response to a predefined computation graph, e.g., in response to a configuration variable V. According to some embodiments, the configuration variable V selects an operation or mode of operation. The configuration variable V that selects the operation may select, e.g., a mathematical operation. Here, the arithmetic circuitry 74 comprises hardware for computing different or multiple operations (e.g., first and second arithmetic circuits 74 which may be one or more of addition, an accumulator that calculates the sum of multiple values, multiplication, matrix-vector product, linear transformation, non-linear transformation), or the arithmetic circuitry 74 comprises hardware for a pass-through routing function where data packets 80 are simply communicated to different adjacent computing elements 70.

[0055] According to some embodiments of the present disclosure, two or more computing elements 70 connected to an array can input data 40 from any adjacent computing element 70 in the array, e.g., from the left, right, top, or bottom, via point-to-point connections 48, as shown in Figures 4 and 7. Computing elements 70 at the ends of the array can be connected to data control circuitry 24, including input control circuitry 26 and output control circuitry 28, e.g., via input bus 42 and / or output bus 46. The input bus 42 and output bus 46 can be (but are not necessarily) a common bus connected to one or more computing elements 70 at the ends of the array of computing elements 70, and are illustrated separately for ease of understanding. Thus, a computing element 70 within the array can be connected to four adjacent computing elements 70 via connections 48, and a computing element 70 at the ends of the array can be connected to two or three adjacent computing elements 70 via connections 48. In the embodiment of Figure 7, only the data control circuitry 24 is connected to the input and output buses 42, 46, and the input data 40 and output data 46 are communicated to the data control circuitry 24 via connections 48. The control circuits 78 within the computing elements 70 are communicatively connected in a daisy chain (or on a common bus) and can communicate configuration variables, including self-tags S, mathematical constants, and operating mode selections.

[0056] According to embodiments of the present disclosure, data packets 80 flowing through an array of computing elements 70 may be controlled, at least in part, by target tags 82 associated with each data packet 80. Thus, complex centralized control and communication mechanisms are not required.

[0057] According to an embodiment of the present disclosure, as shown in FIG. 8A, a data packet 80 includes a data value D (e.g., a numerical value to be mathematically operated on by the operation circuitry 74) and a target tag T. As shown in FIG. 8B, the target tag T may include a computing element 70 identifier C that identifies an associated operation on the data value D, and may include a routing value R. The target tag T indicates that the computing element 70 in which the target tag T resides performs the operation of the computing element 70, and thus identifies the computing element 70). The identification of the computing element 70 may include matching a label or operation of the computing element 70, or identifying a no-route value N routing value R (e.g., a null value or count that is not a connection 48). Thus, the target tag T may operate as a form of dynamic configuration variable that controls the operation of the operation circuitry 74 and the flow of data packets through the array of computing elements 70.

[0058] As shown in FIG. 9, each computing element 70 has an associated operation that it can perform on a data value D. In the illustration, computing elements 70 marked with a "*" can perform a matrix-vector product operation (referred to herein as a "multiplication") using an indicated weight W value (e.g., provided as a configuration variable, hardwired to the computing element 70, or provided as part of a data packet 80), computing elements 70 marked with a "+" can perform an addition or accumulation operation, and computing elements 70 marked with an "f" can perform a transformation (e.g., a lookup table, etc.). If the target tag T matches a computing element 70, as determined by, for example, the computing element control circuitry 78 matching a self tag S to a computing element identifier C or identifying a no-route N routing value R (e.g., a null value or a zero count of the routing value), the operation of the computing element 70 is performed on the data value D by the operation circuitry 74 as enabled by the computing element control circuitry 78. The target tag T may be modified as it passes through the computing elements 70, for example, a list of computing elements 70 by the computing element control circuitry 78 as an operation is completed, and a list of routing values ​​R as the data packet 80 is routed through the array of computing elements 70.

[0059] In embodiments of the present disclosure, an array of computing elements 70 may be initialized, e.g., in hardware, or by, e.g., processor 14 of hardware system 10, by sending a configuration variable V to each computing element 70 in the array. In some embodiments, the configuration variable V may include a self-tag S that identifies each computing element 70 and the associated operation (if the computing elements 70 do not have a hardwired operation identifier). In some embodiments, the self-tag S is not used and is not included in the configuration value V or hardwired into each computing element 70.

[0060] As shown in FIG. 10, an external compiler (e.g., including software executing on processor 14 of hardware system 10 shown in FIG. 1) constructs a set of data packets 80 including data values ​​D, computing element 70 identifiers C, if used, and routing information R, including no-path values ​​N, if used, to encode one or more operation graphs that specify one or more operations to be performed in a desired order on data values ​​D to be routed by routing values ​​R. Target tags T, if used, may include identifiers C of computing elements 70 and route values ​​R, for example, provided in a list in target tags T, that the operation at the next computing element 70 to be performed or route values ​​R to be used are the first values ​​in the associated list and thus exposed for the next step, the current values ​​for the next operation or connections 48 output. Data packets 80 are provided (e.g., by processor 14) to data control circuitry 24, which provides them to the particular computing element 70 over input bus 42 (as shown in FIG. 4) or over connections 48 (as shown in FIG. 7). Each receiving computing element control circuit 78 compares the computing element 70 identifier C specified in the target tag T with the self tag S or identifies a no-route N value R. If the computing element control circuit 78 finds a match, the indicated operation is performed, the data value D is processed, and the processed data value D is output to a connection 48, e.g., a hardware defined connection 48, a connection 48 defined by a configuration variable V, or a connection 48 defined by a routing value R, which may be deleted or modified to expose a new routing value R. (The actual value in the list may be deleted or overwritten, or in some embodiments a list pointer may be used to point to the next routing value R or identifier C.) If there is no match, the operation associated with the computing element 70 is not performed.Instead, the routing value R is removed from the list of routing values ​​in the target tag T, and the modified data packet 80 is output on the connection 48 indicated by the removed routing value R.

[0061] The compiler responsible for constructing the original data packet 80 sets a set of routing values ​​R and computing element 70 identifiers C (if used) to route the data packet 80 and perform the desired computation on the desired computing element 70. The end tag E can enable the processed output data 44 value D to be routed through the array as indicated by the routing value R without further processing by any arithmetic circuit 74 and transmitted through the output bus 46 to the data control circuit 24 as shown in FIG. 4 or to the connection 48 as shown in FIG. 7. The data packet 80 can be recirculated by the data control circuit 24 through the array of computing elements 70 to continue processing. Multiple data packets 80 can be introduced simultaneously into the array of computing elements 70 to keep the computing elements 70 busy and to use the computing elements 70 as efficiently as possible.

[0062] The example in Figure 10 is y=g(f((W 0 *x)+(W 1 *x))*W 2 ) where x is the input vector (input data 40), y is the output vector (processed output data 44), and the functions g and f are nonlinear transformations. As shown in FIG. 10, the data value D of a data packet 80 is calculated by the function (W 0 *x) and (W 19. The next operation, an addition requiring input data 40 from two computing elements 70, is provided to the top two left computing elements 70 of the array where the processed data packet 80 is then passed to the adjacent bottom computing element 70 as specified by the first routing value R. Since the receiving computing element 70 cannot perform the required next addition function (as shown in FIG. 9), the data packet 80 is routed to the next bottom computing element 70 as specified by the next routing value R, thus performing the pass-through function indicated by "P" in FIG. 10. Since the next operation is an addition requiring input data 40 from two computing elements 70, W 0 The data packet 80 of the product of S and T is propagated to the right (according to the next routing value R). The data packet 80 is then input from above and left by the next computing element 70, which performs the addition, modifies the target tag T to expose the next operation and routing value R, and passes the result of the addition to the computing element 70 which computes the functional transformation f. Again, since the exposed operations of the self tag S and the target tag T match, the functional transformation f is performed and the processed data value D is sent to the right (according to the next routing value R). The data packet 80 is then input from above and left by the next computing element 70, which performs the addition, modifies the target tag T to expose the next operation and routing value R, and passes the result of the addition to the computing element 70 which computes the functional transformation f. Again, the exposed operations of the self tag S and the target tag T match, so the functional transformation f is performed and the processed data value D is sent to the right (according to the next routing value R). 2 The result of the multiplication is passed right and then down twice, as indicated by the next routing value R, to a final matching computing element 70 which performs the functional transformation g(x). The processed output data 44 is conveyed to the data control circuitry 24 and removed from the arithmetic array.

[0063] In general, an operation may require input from one or more connections 48, and the computing element control circuitry 78 may validate input data 40 from any one or more of the connections 48. Additionally, input data 40 from different connections 48 (e.g., from data packets 80) may arrive at different times, and the computing element control circuitry 78 may delay an operation until all required data has arrived, for example, using a counter whose value is encoded in the target tag T and may be specified by a compiler. Additionally, in some embodiments, the computing element 70 may store data packets 80 internally while routing other data packets 80 that are not required for the next operation, for example, to a memory node. Thus, the data packets 80 may be efficiently routed without being delayed for unrelated operations to be performed.

[0064] The computation graph shown in Figure 10 may also be implemented using the toroidal array of computing elements 70 shown in Figure 7. In some such embodiments, a data packet 80 containing a data value D (input data 40) and a target tag T is compiled outside the array and communicated (e.g., via a bus) to a data control circuit 24 (e.g., a dedicated computing element 70). The data control circuit 24 communicates the data packet 80 to a computing element 70 in the array via a connection 48. For example, 0 , W 1, the first two multiplication operations may be performed by the leftmost and rightmost computing elements 70 in the top row of the array of computing elements 70. Then the processing data of the leftmost computing element 70 and the processing data of the rightmost computing element 70 are shifted down two computing elements 70, the rightmost data is shifted one computing element 70 to the right, and the processing data from the two multiplying computing elements 70 are added. The added data is shifted down one computing element 70, a transform operation is performed, and then processing is performed as described above.

[0065] As discussed above, the control flow of the data packet 80 can be implemented using the self tag S stored in each computing element 70 and the computing element identifier C communicated with the target tag T. FIGS. 11A-11J show simplified examples of the target tag T associated with a first data packet 80 in such an embodiment. FIGS. 13A-13J show an embodiment that does not use the self tag S and identifier C as in FIGS. 11A-11J, but uses a no-path N routing value R. As shown in FIG. 11A, the identifier C of the computing element 70 is annotated with a subscript indicating the function associated with the computing element 70, "*" (multiplication), "+" (addition), "f" (function transformation), "E" (end tag or no operation), and the processing order. The routing values ​​R are similarly marked in order and denoted with "B" (down), "L" (left), "R" (right), and "T" (up) corresponding to the connections 48 extending from the computing element 70 in the indicated direction. An "X" indicates an optional unused routing value R. The routing value may be a multi-bit binary value, for example a two-bit value indicating each direction of the connection 48. In some unidirectional arrangements, as shown in Figure 15 below, one bit is sufficient to select one of two possible connections 48.

[0066] The first operation "*" of the target tag T matches the top left computing element 70 and is executed. The computing element control circuit 78 then removes the operation and routing value "B" from the target tag T, fills the identifier C portion of the target tag T with another end tag and an optional routing value (denoted by "X"), and outputs the result of the multiplication on the bottom connection 48 with the updated target tag T of FIG. 11B. The update of the target tag T may be hardwired in digital logic, or executed as firmware in a microcontroller, or in other logic circuits, as will be apparent to those skilled in digital circuit technology. The next exposed operation is "+", which does not match the computing element 70, so only the routing value R of the data packet 80 is modified and output on the bottom connection 48, as shown in FIG. 11C. This computing element 70 matches the exposed operation "+", an addition operation is performed, and the result of the added data value D with the target tag T updated with both operation and routing value R is routed through the bottom connection 48, as shown in FIG. 11D. The next operation "f" also matches, the operation is performed, and the data packet 80 with the updated target tag T is output on the right connection 48, as shown in FIG. 11E. The data packet 80 is then routed up two computing elements 70 to encounter the multiplication computing element 70, as shown in FIG. 11F and FIG. 11G. After the multiplication operation is performed, the result is sent over the right connection 48, and the operation value and the routing value are updated, as shown in FIG. 11H. Since the next two computing elements 70 do not match the public function "f", the data packet 80 is sent twice over the bottom connection 48, as shown in FIG. 11I and FIG. 11J. After the "f" operation is performed, the data packet 80 is sent down again, to the top computing element 70 of the 4×4 array of computing elements 70, as shown in FIG. 11K.At this point, data packet 80 has again reached data control circuit 24, and since the exposed operation is the end tag, data control circuit 24 outputs the operation result.

[0067] The overall operational flow is shown in the flow diagram of FIG. 12 (corresponding to the array of computing elements 70 shown in FIG. 7). As shown in FIG. 12, in step 100, input data 40 is provided to the system, for example from a sensor 30 to a hardware system 10 (as shown in FIG. 1). The input data 40 is compiled in step 105 into a data packet 80, for example by a processor 14. The computing elements 70 are configured in step 110 (e.g., with settings such as self-tag S if not hardwired), and the data packet 80 is sent to the data control circuitry 24 in step 115. The data control circuitry 24 then sends the data packet 80 in step 120 to the array of configured computing elements 70, for example via an input bus 42 (shown in FIG. 4) or a connection 48 (shown in FIG. 7). In step 125, if the identifier C of the current exposed computing element 70 (e.g., the identifier C of the current computing element 70 at the top of the list of identifiers C of computing elements 70) matches the capabilities of the receiving computing element 70 (e.g., the identifier C of the computing element 70 matches the self-tag S), the operation is performed in step 130 and the identifier list is updated in step 135. In step 125, if the current identifier C does not match the self-tag S of the computing element 70, the operation is not performed. After updating the routing list in step 140, the data packet 80 is routed to the neighboring computing element 70 via the connection 48 indicated in the routing list in step 145. If the receiving computing element 70 is the data control circuit 24, the compute graph is completed in step 145 and the processed data is output in step 150. If the receiving computing element 70 is not the data control circuit 24 (or the data control circuit 24 does not detect the end tag E identifier C), the process continues until the processed data is communicated to the data control circuit 24.For example, data control circuitry 24 can route data from one side of the array to the other in the row to which it is connected.

[0068] 13A-13J show an embodiment that does not use a self-tag S and an identifier C as in FIGS. 11A-11J, but uses a no-path N routing value R. In an embodiment, the routing value R can be a multi-bit tag that indicates either a routing connection 48 or a performance operation (e.g., a no-path value N). In a non-limiting example, each routing value R has two bits, with 00 indicating a performance operation, 01 indicating routing to a first connection 48, 10 indicating routing to a second connection 48, and 11 indicating routing to a third connection 48 in a clockwise direction from the connection 48 on which the data packet 80 was received. (In this example, the data packet 80 is not returned through the connection 48 on which the data packet 80 was received, but in other examples, such return is possible, for example, with a three-bit routing value R.) The routing values ​​R are marked in order and are indicated with "B" (down), "L" (left), "R" (right), and "T" (up) corresponding to the connections 48 extending from the computing element 70 in the indicated direction. "X" indicates any routing value R that is not used. Using the wired value R (including the no-wired value N) provides advantages such as increased simplicity (since no self-tagging is required) and reduced circuitry. As will be appreciated by those familiar with digital circuit design techniques, various logic mechanisms can be provided to indicate that a computation is to be performed. For example, in some other embodiments, the routing value R is replaced with a null value (e.g., a no-routing value) after each transmission over the connection 48. When the list of routing values ​​R is filled with null values, the operation is performed. The embodiments of the present disclosure are not limited to the particular logical implementation of the routing value R and the no-route value N, or other indication that a computation is to be performed. According to the embodiments, all that is required is for the computing element 70 to recognize from the target tag T that a computing operation of the computing element 70 is to be performed. This recognition identifies the computing element 70 as the computing element 70 designated by the target tag T.

[0069] As shown in FIG. 13A, the first routing value R is “N”, so the operation of the first computing element 70 (multiplication indicated by “*” in FIG. 10) is performed, and the routing value R of “N” is removed from the routing value R list. Any value (e.g., “N” routing values) can be inserted at the bottom of the list. The next routing value R is “B” (shown in FIG. 13B), indicating that the result of the multiplication passes through the bottom connection 48 and the “B” routing value R is removed from the routing value list (shown in FIG. 13C). The next routing value R is also “B” (shown in FIG. 13C), indicating that the data packet 80 passes through the bottom connection 48 and the “B” routing value R is removed from the routing value list (shown in FIG. 13D). The next routing value R is “N”, so the operation of the computing element 70 (addition indicated by “+” in FIG. 10) is performed, and the routing value R of “N” is removed from the routing value R list. The next routing value R is "B" indicating that the result of the addition passes through the bottom connection 48 and the "B" routing value R is removed from the routing value list (shown in FIG. 13E with both "N" and "B" removed from the routing value R list). The next routing value R is again "N", so the operation of the computing element 70 is performed (the transformation indicated by "f" in FIG. 10) and the "N" routing value R is removed from the routing value R list. The next routing value R is "R" indicating that the result of the addition passes through the right connection 48 and the "R" routing value R is removed from the routing value R list (shown in FIG. 13F with both "N" and "R" removed from the routing value R list). The next two routing values ​​R are both "T", indicating that the data packet 80 passes through the top connection 48 and is presented to the multiplication computing element 70 and the "T" routing value R is removed (as shown in FIG. 13G). Since the next routing value R is again "N", the operation of the computing element 70 (multiplication indicated by "*" in FIG. 10) is performed and the routing value R of "N" is deleted from the routing value R list.The next routing value R is "R" indicating that the result of the addition is passed to the right connection 48 and the "B" routing value R is removed from the routing value R list (in FIG. 13H both "N" and "R" are shown removed from the routing value R list). The next two routing values ​​R are both "B" indicating that the data packet 80 is passed through the bottom connection 48 and presented to the transformation computing element 70 and the "B" routing value R is removed (as shown in FIG. 13I). The next routing value R is again "N" and therefore the operation of the computing element 70 is performed (the transformation indicated by "f" in FIG. 10) and the "N" routing value R is removed from the routing value R list. The next routing value R is "B" indicating that the result of the addition is passed through the bottom connection 48 and the "B" routing value R is removed from the routing value R list (in FIG. 13J both "N" and "B" are shown removed from the routing value R list). The next two routing values ​​R are both "R", indicating that the data packet 80 is passed through the right connection 48 and presented to the data control circuit 24. The data control circuit 24 detects a routing value of "N", indicating that the data control circuit 24 outputs the completed data packet 80. Processing of the data packet 80 is then completed and the data value D of the data packet 80 is output from the array of computing elements 70. FIGS. 13A-13J show a list of routing values ​​R where the active or exposed routing values ​​R are in the front or lowest address location of a memory or register. In some embodiments, the active or exposed routing values ​​R are in the rear or highest address location in a memory or register.

[0070] The overall data flow is shown in the flow diagram of FIG. 14 (corresponding to the array of computing elements 70 shown in FIG. 7). As shown in FIG. 14, in step 100, input data 40 is provided to the hardware system 10 (as shown in FIG. 1), for example from the sensor 30. The input data 40 is compiled in step 105 into a data packet 80, for example by the processor 14. The computing elements 70 are optionally configured (e.g., with configured values) in step 110, and the data packet 80 is sent to the data control circuitry 24 in step 115. The data control circuitry 24 then sends the data packet 80 in step 120 to the array of configured computing elements 70, for example via the input bus 42 (shown in FIG. 4) or the connection 48 (shown in FIG. 7). If the current computing element 70 detects the "N" routing value R, then the operation is performed in step 130. If the current computing element 70 does not detect the "N" routing value R, then the operation is not performed. The data packet 80 is routed to an adjacent computing element 70 via a connection 48 indicated in the routing list at step 145 after updating the routing list at step 140. If the receiving computing element 70 is a data control circuit 24 and the next routing value R is equal to "N", then the compute graph is completed at step 145 and the processed data is output at step 150. If the receiving computing element 70 is not a data control circuit 24 (or the data control circuit 24 does not detect an "N" routing value R), then the process continues until the processed data is communicated to the data control circuit 24. For example, the data control circuit 24 may route data from one side of the array to the other in the row to which the data control circuit 24 is connected.

[0071] Multiple sets of data packets 80 may be cycled through the array of computing elements 70 simultaneously, as dictated by the compiled data packets 80, allowing for efficient use of the computing elements 70. Because the computing elements 70 may be implemented directly in logic circuitry, without requiring a program machine stored on each computing element 70, input, processing, and output operations, including communication between one computing element 70 and another computing element 70 via connections 48, may be very fast, e.g., in nanoseconds, resulting in extremely fast and power efficient computing.

[0072] Embodiments of the present disclosure include a wide variety of computing element 70 configurations and connection 48 architectures. Similarly, embodiments of the present disclosure include a wide variety of computing elements 70, such as data control circuits 24 that interact with an external system (e.g., processor 14), computing elements 70 that include arithmetic circuits 74, computing elements 70 that function as memory nodes and can buffer or order data packets 80 (e.g., including a memory array such as a shift register or SRAM and a state machine controller) or routing nodes. FIG. 15 illustrates an embodiment including unidirectional connections 48 in a grid arrangement with computing elements 70 at opposite corners of the grid connected to data control circuits 24. Such an embodiment is simpler and requires fewer connections 48, but may require more routing than other arrangements such as FIG. 7. FIG. 16 illustrates a simpler one-dimensional array of computing elements 70 with fewer connections 48. In some embodiments, and as shown in FIG. 17, multiple data control circuits 24 can each control a subset of the array of computing elements 70. The computing elements 70 of each subset may be one- or two-dimensional arrays and may include various functions, such as matrix-vector product multipliers, accumulators, nonlinear transformers, and memory nodes. In some embodiments, the computing elements 70 are diagonally connected by bidirectional (optionally unidirectional) connections 48 to the eight nearest computing elements 70, as shown in FIG. 18. The diagonal connections 48 may reduce the number of routing communication steps. In some such embodiments, the routing value R may be stored as a 3-bit value. In some configurations, the data packets 80 may be routed using different paths to control the arrival times of the data packets 80.

[0073] 19 shows an embodiment in which relatively simple message passing circuits 71 are connected by connections 48, with each message passing circuit 71 capable of passing data packets 80 to one or more computing elements 70 connected to the message passing circuit 71. This arrangement may simplify the computing elements 70.

[0074] In an embodiment of the present disclosure, multiple data packets 80 may circulate within the array of computing elements 70. However, when any one or more of the computing elements 70 are not computing, routing, or otherwise active, they may enter a reduced power quiescent state to conserve power of the computing accelerator 20.

[0075] According to an embodiment of the present disclosure, the circuitry of the computing element 70 may be an integrated circuit containing digital logic, for example, made in a semiconductor foundry using photolithography processes on a silicon substrate or other semiconductor material and assembled using printed circuit board technology. The computing element 70 may be comprised of a single integrated circuit. A computing element array system, for example the computing accelerator 20, may be operated by providing appropriate electrical signals that power the circuitry comprising the computing accelerator 20 and provide the input data 40 and the target tag T to the computing accelerator 20.

[0076] According to an embodiment of the present disclosure, a matrix vector multiplier (MVM) 60 can perform a matrix vector multiplication operation in a computing element 70 (e.g., as shown in FIG. 3 ), either as a digital operation, or with an analog operation, or a combination thereof. Such an MVM 60 can comprise digital circuits and logic for performing the indicated operations, or can comprise analog circuits, or may comprise a combination of digital and analog circuits. Such a dot product matrix multiplication is a typical operation in neural network computation. In some embodiments, the MVM 60 includes a multi-bit capacitor array and an analog-to-digital converter 98 that provides a multiply-accumulate (MAC) function and creates the basis for improved power efficiency of the in-memory computation architecture of the computing element 70 in the hardware system 10. The multi-bit capacitor array, referred to herein as a qMAC 61 (where “q” refers to the charge Q in the multi-bit capacitor array), can implement the dot product operation in the MVM 60 of the computing element 70. Such a dot product multiplies each member of a one-dimensional vector with N input values ​​by each member of a column of an MxN array with N values, and accumulates (sums) the multiplied values ​​to result in an output vector with M values. The MxN array serves as a weight matrix in the neural network's calculations. If M is equal to 1, corresponding elements of each one-dimensional Nx1 array are multiplied and their values ​​are added to obtain a single value. In some embodiments of the present disclosure, the dot product operation is performed at least partially in the analog domain (e.g., with charges in one or more capacitors) to reduce power and improve the speed of the multiplication, resulting in improved performance and power efficiency.

[0077] As shown in the embodiment of FIG. 20, the unit capacitance multiplier 90 includes a latch 92 that stores a value, such as a weight value W. The value stored in the latch 92 may be an analog value, such as a voltage stored in a capacitor, or a digital value, such as a binary value stored in a flip-flop or digital latch. A switch 94 (e.g., a transistor) connects the latch 92 to the capacitor 96. The switch 94 may be a digital binary switch operable in two states, or an analog switch with continuous control, such as in response to a voltage on a bipolar transistor. A switch input 93 (e.g., input 40) controls the switch 94 to turn on and conduct charge from the latch 92 to the capacitor 96. If the latch 92 stores a zero value or the switch input 93 is zero, no charge is conducted. Charge is conducted to the capacitor 96 only if both the latch 92 and the switch input 93 have values ​​greater than zero. In the simplest case, the latch 92 and the switch 94 are single bits. In some embodiments, latch 92 and switch 94 have voltages that represent multiple bits or are continuous voltages that charge capacitor 96 with a substantially continuous charge. The output of capacitor 96 (charge Q) is equivalent to switch input 93 multiplied by latch 92 such that Q=W*X (and corresponds, for example, to the inset of FIG. 3).

[0078] The complete MVM 60 may include multiple unit-capacitance multipliers 90 that generate the qMAC 61. The accumulation (sum) of an array of unit-capacitance multipliers 90 (e.g., representing the column of multipliers in FIG. 3) is obtained by accumulating all of the charges Q by shorting capacitors 96 as shown in FIG. 21. This accumulation is nearly instantaneous (because charges propagate very quickly along conductors) and requires very little power, thereby conserving energy. The inputs 40 (e.g., 40A, 40B, 40C, 40D) may be converted from digital to analog values ​​(e.g., voltages) using digital-to-analog converters (as shown in FIG. 3), and the output values ​​may be converted from analog (e.g., charge) values ​​to digital values ​​using analog-to-digital converters. In some embodiments, the qMAC 61 is an analog array multiplier. The input data 40 and the output data 44 may be converted from one mathematical space to another (e.g., from linear to nonlinear or vice versa) by optional conversion circuitry 68. This transformation may be specifically selected to complement the characteristics of switch 94 and latch 92 to enable the multiplication operation.

[0079] In some embodiments, each multiplication is performed as a digital operation, and each unit multiplication element of FIG. 21 is a digital multiplier, for example as shown in FIG. 22. In some such embodiments, latch 92 stores a binary value, switch input 93 is a binary value, and switch 94 is a binary switch. When latch 92 and switch input 93 are both high (a positive voltage, true value, or a 1 value), charge is deposited on capacitor 96. The charges of each bit of the product are summed, for example, on capacitor 96. The bits of the resulting digital value can be accumulated, assuming a square matrix with the missing unit multipliers also labeled, as shown in FIG. 23C, for example using digital circuitry as shown in FIG. 23A and FIG. 23B (where the bits are B and B respectively). yx22 ) (y is the row and x is the bit of the row of the unit multiplier of FIG. 22 ). FIG. 23A shows a logical exclusive-or (XOR) function and a logical AND function (along with an OR function) that is used repeatedly in FIG. 23B to accumulate each bit of the lower 3 bits of the multiplication. Those familiar with digital logic design can use similar or similar concepts to design logic circuits that calculate bit values ​​of digital multiplications with any number of bits. In some embodiments, the capacitors in FIG. 20 and FIG. 21 can be replaced with high value resistors and the output of each unit multiplier can be fed directly to the circuit of FIG. 23B as a voltage (pulled by a resistor to ground as a zero value if the latch 92 or switch 94 is low (having a low voltage) or pulled up as a one value if both are high (having a high voltage)) without capacitive summation or analog-to-digital or digital-to-analog conversion.

[0080] According to embodiments of the present disclosure, the use of digital 1-bit multipliers as shown in FIG. 20 provides advantages over the use of conventional digital logic AND gates. Such embodiments can operate in a lower voltage region of the switches 94 (e.g., transistors) with reduced leakage compared to digital gates or logic circuits, thereby reducing power usage. Because unit multipliers 90 are used repeatedly at high speed and frequency, such power reduction can significantly reduce the power used by MVM 60. For example, a 4×4 multiplier array as shown in FIG. 21 using the digital multiplier structure of FIG. 22 would require an array of 4×4 unit multipliers 90, or 256 (16×16) multipliers to perform each multiplication. Since the number of bits is typically much larger than 4, the number of unit multipliers would be correspondingly large, as would the power savings provided by embodiments of the present disclosure. For example, 8-bit values ​​would require 4096 (64 x 64) multipliers, 16-bit values ​​would require 65536 (256 x 256) multipliers, and 32-bit values ​​would require 1048576 (1024 x 1024) multipliers.

[0081] According to an embodiment of the present disclosure, MVM 60 can calculate only the most significant bits of the multiplication between an input vector (e.g., input data 40) and a weight matrix. Since neural nets can only respond very weakly to very small changes in the input data, the least significant bits of the matrix-vector multiplication can be ignored. Thus, the least significant bits of the calculation as shown in FIG. 22 can be ignored, reducing the size and power of the circuitry used to perform the multiplication.

[0082] In some embodiments of the present disclosure, only a subset of accumulated values ​​or vectors are processed. In networks where matrices are repeatedly processed, the total number of calculations and energy used are reduced by pruning or otherwise downsampling the results from one or more vector product multiplications (e.g., selecting a subset of less than all of the accumulated values ​​or vectors). In some embodiments, results that are zero or relatively small compared to other accumulated values ​​or vectors are set to zero or excluded from subsequent calculations.

[0083] Having described specific implementations of the embodiments, it will now be apparent to those of ordinary skill in the art that other implementations incorporating the concepts of the present disclosure may be used. Accordingly, the present disclosure should not be limited to specific implementations, but rather should be limited only by the spirit and scope of the following claims.

[0084] Throughout this specification, where devices and systems are described as having, including, or comprising certain elements, or where processes and methods are described as having, including, or comprising certain steps, it is further contemplated that there are devices and systems of the disclosed technology that consist essentially of, or consist of, the disclosed elements, and that there are processes and methods according to the disclosed technology that consist essentially of, or consist of, the disclosed processing steps.

[0085] It should be understood that the order of steps or order for performing certain actions is immaterial so long as the disclosed technology remains operable. Moreover, in some circumstances, two or more steps or actions may be performed simultaneously. Although the present disclosure has been described in detail with particular reference to certain embodiments thereof, it will be understood that variations and modifications can be effected within the spirit and scope of the following claims. [Explanation of symbols]

[0086] B bit C Element Identifier Calculation / Identifier D Data Value E end tag k constant N Non-root O operation Q Charge R Routing value S Self Tag T Target Tag W weight 10 Hardware System 12. Memory 14 processors 20 Computing Accelerators 22 Interrupt Signal 24 Data control circuit 26 Input control circuit 28 Output control circuit 30, 30A, 30B, 30C Sensors 32 Sensor Signal 40, 40A, 40B, 40C, 40D, 40N Input / Input data / Input value 42 Input Buses 44 Output / Output Data 46 Output Buses 48 Connections 50 Sensor interface circuit 52 General-purpose input / output circuit 54 Automatic Gain Circuit 56 Internal Logic and Clock Gating 60 In-memory matrix multiplier 61 qMAC 62 Nonlinear Transformers 64 Accumulator 66 memory nodes 68 Conversion Circuit 70 Computing Elements 72 Storage Circuit 74 Arithmetic circuit 78 Control Circuit / Computing Element Control Circuit 80 Data Packets 90 Unit Capacity Multiplier / Unit Multiplier 92 Latch 93 Switch Input 94 Switch 95-bit adder 96 Capacitor 97 Digital to Analog Converter 98 Analog to Digital Converter 100 Steps to provide input data 105 Steps for compiling a data packet 110 Steps for Configuring Computing Elements 115 providing a data packet to a data control circuit. 120 Transmitting a data packet to a computing element 125. A step of determining whether the exposed operation corresponds to a computing element step. 127 Step to determine whether the published routing value matches N 130 Calculation Steps 135 Operation List Update Step 140 Routing List Update Steps 145 Determining whether the computing element corresponds to the data control circuit. 147 determining whether the computing element is a data control circuit and the routing value matches N 150 Steps to output processed data values

Claims

1. 1. A computing element array system comprising a computing element array including a plurality of computing elements connected by a plurality of connections, Each computing element includes a control circuit, a memory circuit, and an arithmetic circuit; Each of the plurality of connections connects two of the computing elements. At each computing element in the computing element array, the storage circuitry is operable to receive a data packet from one of the plurality of connections, the data packet including a data value and a target tag, and to store the data packet; the operation circuitry is operable to perform an operation on the data values ​​to generate processed data values; the target tag designates a computing element that should perform an operation on the data value; the control circuitry is operative to determine whether the computing element is specified by the target tag, to enable the arithmetic circuitry at the computing element specified by the target tag to perform an operation on the data value, to modify the data packet to include the processed data value, and to output the modified data packet to one of the plurality of connections, A computing element array system in which the array is a two-dimensional grid having one or more unidirectional connections.

2. the computing element includes a self tag that describes the computing element; the target tag includes one or more computing element specific information; 2. The computing element array system of claim 1, wherein the control circuit identifies the computing element by comparing the computing element identification information with the self tag.

3. The computing element array system of claim 1 or 2, wherein the control circuitry is operable to modify the target tag to provide an end tag in the computing element ordered list.

4. The computing element array system of claim 1 , wherein the target tag designates a number of the computing elements that perform a sequential operation on the data value.

5. a data control circuit coupled to at least one of the plurality of computing elements; The computing element array system of any one of claims 1 to 4, wherein the data control circuitry is operable to receive data packets from outside the computing element array and to transmit data packets to outside the computing element array.

6. 6. The array of computing elements of claim 1, wherein the computing elements are (i) routing nodes, (ii) memory nodes, (iii) operable to perform routing pass-through, (iv) operable to store one or more data values ​​for a specified period of time, or (v) any combination of (i)-(iv).

7. 7. The array of computing elements of claim 1, wherein the operational circuitry of a first computing element is operable to perform a first operation and the operational circuitry of a second computing element is operable to perform a second operation different from the first operation.

8. The array of computing elements of any preceding claim, wherein two or more computing elements are synchronous.

9. The array of computing elements of any preceding claim, wherein two or more computing elements are asynchronous.

10. 10. The array of computing elements of claim 1, wherein the storage circuitry of one or more computing elements is operable to receive input data from two adjacent computing elements, or to output data to one or two adjacent computing elements, or both.

11. The computing element array of claim 1, wherein the target tag specifies a mathematical function or a computational graph.

12. A computing element array including a plurality of computing elements connected by a plurality of connections, Each computing element includes a control circuit, a memory circuit, and an arithmetic circuit; each computing element is operable to store a self-tag that identifies an operation performed by said computing circuitry; At each computing element in the computing element array, the storage circuitry is operable to receive a data packet from one of the plurality of connections and to store the data packet; the data packet includes a data value and a target tag, the target tag specifying an operation to be performed on the data value; the control circuitry is operable to compare the target tag with the self tag, and if the self tag matches the target tag, to cause the operational circuitry to process the data value, modify the data packet to include the processed data value, and output the modified data packet on one of the connections; The arithmetic circuitry is an array of computing elements operable to process the data values ​​to generate processed data values, A computing element array system in which the array is a two-dimensional grid having one or more unidirectional connections.

13. A computing element array including a plurality of computing elements connected by a plurality of connections, Each computing element includes a control circuit, a memory circuit, and an arithmetic circuit; At each computing element in the computing element array, the storage circuitry is operable to receive a data packet from one of the plurality of connections and to store the data packet; the data packet includes a data value and a target tag that specifies a routing value; one of the routing values ​​is a no-route value indicating that the computation circuit operates on the data value; the control circuitry is operable to identify the no-route value, and if the no-route value is identified, to cause the operational circuitry to process the data value, modify the data packet to include the processed data value, and output the modified data packet over one of the connections; The operational circuitry is an array of computing elements operable to process data values ​​to generate processed data values, A computing element array system in which the array is a two-dimensional grid having one or more unidirectional connections.

14. The computing element array system of claim 5, wherein the data control circuitry is connected to the computing elements at opposing corners of the grid connected to the data control circuitry.

15. The computing element array system of claim 5 , wherein a plurality of said computing elements are not all identical and have different operational circuitry.

16. A computing element array system comprising a computing element array including a plurality of computing elements connected by a plurality of connections, comprising: Each computing element includes a control circuit, a memory circuit, and an arithmetic circuit; Each of the plurality of connections connects two of the computing elements. At each computing element in the computing element array, the storage circuitry is operable to receive a data packet from one of the plurality of connections, the data packet including a data value and a target tag, and to store the data packet; the operation circuitry is operable to perform an operation on the data values ​​to generate processed data values; the target tag designates a computing element that should perform the operation on the data value; The control circuit determining whether the computing element matches the computing element specified by the target tag; enabling said computation circuitry to process said data value at said computing element designated by said target tag; modifying the data packet to include the processed data value; a computing element array system operative to output the modified data packet to one of the plurality of connections, the computing element array system comprising: A computing element array system in which the computing elements are not all identical and have different operational circuitry.