Implicit vector concatenation in 2D mesh routing

Abstract

Implicit Vector Concatenation in 2D Mesh Routing An array of neural cores is provided, each neural core comprising ordered input lines, ordered output lines, and synapses, each of the synapses operatively coupled to one of the input lines and one of the output lines. A plurality of signal lines is provided, at least one of the signal lines disposed along each dimension of the array of neural cores. A plurality of routers is provided, each of the routers operatively coupled to one of the neural cores and to at least one of the signal lines along each dimension of the array of neural cores. Each of the routers selectively routes signals from the at least one signal line to its associated neural core. Each of the routers selectively routes signals from its associated neural core to the at least one signal line. The routers segment the ordered input lines and the ordered output lines into segments and route signals for each segment independently.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to multi-core neural networks, and more particularly, to implicit vector concatenation within 2D mesh routing.

Summary of the Invention

[0002] According to one aspect of the present invention, a system is provided that includes an array of neural cores, the array having at least one dimension. Each neural core includes a plurality of ordered input lines, a plurality of ordered output lines, and a plurality of synapses, each synapse being operably coupled to one of the plurality of input lines and one of the plurality of output lines. A plurality of signal lines are provided. At least one of the plurality of signal lines is disposed along each dimension of the array of neural cores. A plurality of routers are provided, each router being operably coupled to (i) one of the plurality of neural cores and (ii) at least one of the signal lines along each dimension of the array of neural cores. Each of the plurality of routers is configured to selectively route signals from at least one signal line to its coupled neural core. Each of the plurality of routers is configured to selectively route signals from its coupled neural core to at least one signal line. The plurality of routers are configured to segment the plurality of ordered input lines and the plurality of ordered output lines into a plurality of segments and to route the signals of each of the plurality of segments independently.

[0003] According to another aspect of the present invention, a method for operating a multicore neural network is provided. A signal is received by multiple routers via multiple signal lines. The signal is selectively routed by each of the multiple routers from at least one signal line to a neural core. The signal is selectively routed by the multiple routers from the neural core to multiple signal lines. The multiple routers are configured to segment multiple ordered input lines and multiple ordered output lines into multiple segments and to route the signals of each of the multiple segments independently. The neural core comprises multiple ordered input lines, multiple ordered output lines, and multiple synapses, each of which synapses is operably coupled to one of the multiple input lines and one of the multiple output lines. The multiple signal lines are arranged along each dimension of an array of neural cores having the neural core. [Brief explanation of the drawing]

[0004] [Figure 1] An exemplary array of neural cores according to embodiments of the present disclosure is shown.

[0005] [Figure 2] This figure shows an upstream / downstream router for circuit-switched connections of neuron circuits according to an embodiment of the present disclosure.

[0006] [Figure 3] An embodiment of the present disclosure shows an upstream / downstream router configured using a crossbar array.

[0007] [Figure 4] An array of interconnected routers according to an embodiment of the present disclosure is shown.

[0008] [Figure 5A] An embodiment of the present disclosure shows an upstream / downstream router including a border guard circuit at the core end. [Figure 5B]An embodiment of the present disclosure shows an upstream / downstream router including a border guard circuit at the core end.

[0009] [Figure 6] An exemplary analog fabric according to an embodiment of the present disclosure is shown.

[0010] [Figure 7A] This embodiment of the disclosure illustrates the routing of data vectors on an analog fabric. [Figure 7B] This embodiment of the disclosure illustrates the routing of data vectors on an analog fabric. [Figure 7C] This embodiment of the disclosure illustrates the routing of data vectors on an analog fabric.

[0011] [Figure 8] This shows a segmented fabric configuration according to an embodiment of the present disclosure.

[0012] [Figure 9] This illustrates the implicit coupling according to the embodiments of the present disclosure.

[0013] [Figure 10] This illustrates the implicit partitioning according to embodiments of the present disclosure.

[0014] [Figure 11] The following describes the data collection according to the embodiments of this disclosure.

[0015] [Figure 12] This is a detailed diagram of an exemplary border guard circuit according to an embodiment of the present disclosure.

[0016] [Figure 13] Figure 12 shows an exemplary tile according to this disclosure that uses the circuit.

[0017] [Figure 14]Schematic circuit diagram of an exemplary border guard circuit according to an embodiment of the present disclosure.

[0018] [Figure 15] A method of operating a multi-core neural network according to an embodiment of the present disclosure is shown.

[0019] [Figure 16] A computing node according to an embodiment of the present disclosure is shown.

Mode for Carrying Out the Invention

[0020] An artificial neural network (ANN) is a distributed computing system composed of a large number of neurons interconnected via connection points called synapses. Each synapse encodes the strength of the connection between the output of one neuron and the input of another neuron. The output of each neuron is determined by the set of inputs received from other neurons connected to it. Therefore, the output of a given neuron is based on the outputs of the connected neurons from the previous layer and the strength of the connections determined by the synaptic weights. The ANN is trained to solve a specific problem (e.g., pattern recognition) by adjusting the synaptic weights so that a specific class of inputs produces a desired output.

[0021] ANN can be implemented on various types of hardware, including a crossbar array, also known as a cross-point array or cross-wire array. A basic crossbar array configuration includes a set of conductive row lines and a set of conductive column lines formed to intersect the set of conductive row lines. The intersections between the two sets of lines are separated by cross-point devices. The cross-point devices function as weighted connections of the ANN between neurons.

[0022] In various embodiments, non-volatile memory-based crossbar arrays, or crossbar memories, are provided. Multiple junctions are formed by row lines intersecting column lines. Resistive memory elements, such as non-volatile memory, are in series with a selector at each junction coupling between one of the row lines and one of the column lines. The selector may be a volatile switch or a transistor, various types of which are known in the art. It will be understood that various resistive memory elements, including memristors, phase-change memories, conductive bridge RAMs, and spin-transfer torque RAMs, are suitable for the uses described herein.

[0023] A certain number of synapses are provided on a core, and then multiple cores can be connected to provide a complete neural network. In such embodiments, interconnections between cores are provided, for example, to transmit the output of a neuron on one core to another core via a packet-switched network or a circuit-switched network. In a packet-switched network, power and speed are sacrificed because address bits must be transmitted, read, and processed, but greater flexibility of interconnection can be achieved. In a circuit-switched network, address bits are not required, so flexibility and reconfigurability must be achieved by other means.

[0024] In various exemplary networks, multiple cores are arranged in an array on a chip. In such embodiments, the relative positions of the cores may be referred to by basic directions (north, south, east, west).

[0025] In the context of a physical array of neural cores comprising analog memory elements, this disclosure provides flexible routing of signals between array cores for neuromorphic computation. In various embodiments, the data carried by these neural signals is encoded over the pulse duration carried by each line using a digital voltage level suitable for buffering or other forms of digital signal reconstruction. In other embodiments, the data may be directly time-multiplexed as binary pulses representing digital position bits and coordinated with the edges of a local or transfer clock signal used to latch the digital position bits at the destination.

[0026] Within such a core array, multiply-accumulate operations can be performed efficiently in a single step, but the problem remains of transmitting output data from an output peripheral at the end of one core to an input peripheral at the end of several other cores. To implement flexible and reconfigurable hardware, the ability to support arbitrary routing paths from any one core to any other core is desired. Deep learning applications do not require the full flexibility provided by packet-switched networks, but it is desirable to support both single-cast and multicast of data from an output neuron of one core to associated input neurons of multiple other array cores.

[0027] In various embodiments of this disclosure, each neuron on the end of each core is connectable to a dedicated routing fabric for that particular neuron. The routing fabric comprises a mesh of lines, buffers, and switches associated with a particular neuron within the entire data vector corresponding to all neurons at the core end. While the routing fabric for a single neuron is described in various examples herein, it will be understood that all neurons (or elements of the data vector) move in parallel along their respective dedicated routing lines. In various embodiments, one or more control lines control all or substantial portions of the parallel lines simultaneously. In other embodiments, a register of mask bits enables masked control of a subset of the parallel lines.

[0028] The objective of analog AI is to perform parallel vector multiplication operations by introducing excitation vectors into multiple row lines in order to perform multiplication and accumulation operations across the entire matrix of stored weights encoded in the conductance values ​​of analog non-volatile resistive memory. As discussed above, the array-integrated upstream / downstream router method for circuit-switched parallel connections is advantageous for such applications. This scheme enables massively parallel transmission and buffering of signal vectors between analog AI tiles and computing cores, either in duration form or as raw digital data.

[0029] Furthermore, it is necessary to address situations where vectors that must arrive at a specific destination originate from multiple sources, and each source contains only a subset of the entire long vector. Therefore, this disclosure provides implicit vector concatenation within 2D mesh routing.

[0030] Specifically, this disclosure describes a method for implicitly concatenating many short vectors acquired from different sources across a 2D mesh and delivering them as a long concatenated vector to one or more destinations. One advantage of this method is that a long data vector can be concatenated without the need to sequentially send sub-vectors one after another to a common staging point where the concatenation is performed by digital circuitry.

[0031] Because each column and row has its own dedicated router circuit with a dedicated path on the 2D router mesh, implicit concatenation requires that some parts of the border guard circuit (e.g., vector elements 0-31) force new data onto the 2D mesh (e.g., send this data north on the mesh), while other parts of the border guard circuit (e.g., vector elements 32-511) have the ability to buffer existing data received (e.g., from the south) so that it can be transmitted in parallel with the newly forced data.

[0032] Such capability enables the implicit concatenation of LSTM vectors obtained from multiple computing units located at different positions. For example, a vector of length 512 could be implicitly concatenated across four computing cores, such as 128 data elements coerced from computing core 1, another 128 from computing core 2, and so on. This vector can be transferred to multiple tiles and cores with dedicated SRAM, and in the context of a bidirectional LSTM, for example, it may be necessary to store the currently concatenated vector for later use.

[0033] To avoid the possibility that two data elements that need to be concatenated might attempt to use the same dedicated router line within the router mesh, care must be taken to structure calculations so that all data to be concatenated aligns to unique columns (rows) in the 2D mesh router. Furthermore, the control circuits that control the operation of border guards must allow for sufficiently fine-grained control to pass or force appropriate portions of the data vectors to the router lines in order to support this implicit concatenation.

[0034] Referring here to Figure 1, an exemplary array of neural cores according to an embodiment of the present disclosure is shown. Array 100 includes a plurality of cores 101. The cores within array 100 are interconnected by lines 102, as will be further described below. In this example, the array is two-dimensional. However, it will be understood that the present disclosure may be applied to one-dimensional or three-dimensional arrays of cores. Core 101 includes a non-volatile memory array 111 that implements synapses as described above. Core 101 includes a west side and a south side, each functioning as an input and the other as an output. The west side includes a support circuit 112 dedicated to the entire side of core 101, a shared circuit 113 dedicated to a subset of rows, and a per-row circuit 114 dedicated to individual rows. The south side similarly includes a support circuit 115 dedicated to the entire side of core 101, a shared circuit 116 dedicated to a subset of columns, and a per-column circuit 117 dedicated to individual columns. It should be understood that the naming convention of West / South is merely adopted to facilitate the reference of relative locations and is not intended to restrict the direction of input and output.

[0035] It will be understood that, while operating as a classifier, the core array can be trained using various methods known in the art. Certain algorithms may be suitable for specific tasks such as image recognition, speech recognition, and language processing. A training algorithm derives a pattern of synaptic weights that converge towards an optimal solution to a given problem during the learning process. Backpropagation is one algorithm suitable for supervised learning, where known correct outputs are available during the learning process. The goal of such learning is to obtain a system that generalizes data that was not available during training.

[0036] Generally, during backpropagation, the network output is compared to a known correct output. An error value is calculated for each neuron in the output layer. The error values ​​are propagated backward, starting from the output layer, to determine the error value associated with each neuron. The error values ​​correspond to each neuron's contribution to the network output. The weights are then updated using the error values. By correcting stepwise in this way, the network output is adjusted to match the training data. During backpropagation, the data vector may move between cores in the opposite direction to the direction used during forward propagation. Therefore, if during forward propagation the data vector was passed, for example, from the south side of core 3 to the west side of core 4, during backpropagation it may be necessary to pass the data vector in the reverse direction, i.e., from the west side of core 4 to the south side of core 3.

[0037] With backpropagation applied, the ANN quickly achieves high accuracy for most examples in the training set. The majority of training time is spent further improving the accuracy of this test. During this time, the large number of training data examples are rarely corrected, as the system has already learned to recognize these examples. Generally, the performance of an ANN tends to improve with the size of the dataset, which can be explained by the fact that larger datasets contain more boundary examples between different classes on which the ANN is being trained.

[0038] Therefore, during training, the array 100 may be provided with example data and example labels. The inferred classification may be provided as output. Based on the inferred classification, weight overrides may be provided to the core array. The updated weights may then be read from the array.

[0039] Referring now to Figure 2, an upstream / downstream router for circuit-switched connections of neuron circuits according to an embodiment of the present disclosure is shown. In this embodiment, the switch is configured at the end of the array, and the buffer is located within the array. This circuit-switched router addresses the need to transmit the entire vector of output data to another core. In some embodiments, the data is encoded as duration. In other embodiments, the data is encoded as binary. Synchronization signals, such as a transfer clock signal that provides an edge indicating the start of duration-encoded data, or an edge for latching digital position bits, may also be transmitted along with the data vector.

[0040] Lines 201-204 provide connections between cores within the array and correspond to line 102 in Figure 1. As discussed above, each line corresponds to a relative direction on the chip. Each is paired with transmission gates 205-208.

[0041] Transmission gates 209 and 210 (labeled ROW_connect and COLUMN_connect) are provided at the entry points to the row 211 and column 212 neuron circuits of each core. This allows each neuron circuit to be connected (or disconnected) to the associated router lines via the transmission gates. In some embodiments, the transmission gates are implemented as PFETs and NFETs connected source-to-source and drain-to-drain using complementary drive signals. In exemplary embodiments, the entry points to the columns correspond to the south side of the core, and the entry points to the rows correspond to the west side of the core.

[0042] Transmission gates 213...216 surround buffer 217. By correctly configuring transmission gates 205-208 between cores (labeled West, East, South, and North); transmission gates 209, 210 at the entrances of each neuron circuit (labeled ROW_connect and COLUMN_connect); and transmission gates 213...216 surrounding buffers (labeled DOWN_in, UP_in, DOWN_out, and UP_out), this router can be used to provide per-neuron circuit-switched routing from a neuron circuit at any end of one array core to a neuron circuit at any other array core.

[0043] In a multicore environment, router lines (e.g., individual elements of line 102, 201...204) pass near each neuron circuit (e.g., individual elements of array 111, 211, 212) and connect via transmission gates (e.g., 205...208) to router lines of the next core (north, south, east, or west) and to buffer circuits (e.g., 217) located in each core.

[0044] In various embodiments, for simplicity, the north and west router wiring are wired to each other, and the south and east router wiring are wired to each other. In such embodiments, a buffer circuit can be used to enhance the signal either downstream (north / west to south / east) or upstream (reverse).

[0045] Referring to Figure 3, the router in Figure 2 is generalized to the case of multiple neurons, and switches and buffers are shown in relation to a crossbar array. In this figure, transmission gates 301-304 are provided between adjacent cores (labeled West, East, South, and North) and core 305. In various embodiments, transmission gates 301...304 are coupled to all arriving neuron routers so that transmission is selected based on the entire core. Router 306 (shown in a simplified diagram) may be configured as described in relation to Figure 2 to provide circuit-switched routing for each neuron in core 305.

[0046] As described, in some embodiments, the buffer circuit can be organized so that its position is aligned to either the row or column position of the associated neuron circuit, or both.

[0047] Therefore, the duration or binary data (encoded data) generated in the surrounding area is handled by border guard circuits on the wiring that crosses the core array. These signals may be buffered or blocked by various other border guard circuits until they reach the target core.

[0048] In various embodiments, buffer circuits and their transmission gates are located within the array at periodic positions, i.e., positions correlated with column positions and independent of row positions (or vice versa, if necessary). Transmission gates between arrays are located at the outer ends of the neuron circuit, while transmission gates at the entry points of the neuron circuit can be included in appropriate parts of the neuron circuit. As shown in Figure 3, using two north-south lines eliminates the need to place buffer circuits in the same location within a given row, as the north-south lines transmit signals to the physical locations of the buffers. In some embodiments, this can be reversed, with two east-west lines, so that the buffer positions are aligned with row positions rather than column positions.

[0049] Referring to Figure 4, an array of interconnected routers according to an embodiment of the present disclosure is shown. In this example, each of routers 401-406 corresponds to an instance of router 200 and is located on an adjacent core.

[0050] The gates in this example are configured for multicast routing. The outputs of the southern column neurons of core 401 are routed to the western row neurons of both nearby and far adjacent cores. Transmission gates that are closed to apply this routing network are shown in light gray, while open transmission gates remain their original color. Note that signals are buffered when passing through any array core in the upstream / downstream direction, but signals passed through hardcoded connections (e.g., north to west or vice versa, or south to east or vice versa) are not buffered. Closed transmission gates are shown in gray.

[0051] Here, column neurons on the south side of core 401 drive the router fabric. An open transmission gate in core 404 allows this signal vector to be transmitted and buffered as it passes through core 404, but is ignored by the row and column neurons of core 404. However, because transmission gates are open between cores 404 and 405, between 405 and 402, and between 402 and 403, the signal vector can reach the row neurons on the western end of core 403. At the same time, the signal vector is buffered by a buffer in core 405 and by an open transmission gate in the border guard circuit between cores 405 and 406 (described further below), allowing the same signal vector to reach the row neurons on the western end of core 406. As a result, the data vector is multicast from the south side of core 401 to the western sides of both cores 403 and 406, using two buffering stages along the way.

[0052] Referring to Figures 5A and 5B, an upstream / downstream router including a shared border guard circuit is shown. In this embodiment, both the switch and buffer are located within the border guard at the core end. This allows elements of a data vector encoded as one or more pulse durations to be transferred from one or more source cores to one or more destination cores.

[0053] In this example, the border guard circuit includes essential transmission gates and buffers. Therefore, no additional circuitry is located within the core itself. Such an embodiment requires a slightly larger area because it requires four inverters instead of two, but it offers advantages in terms of end-to-end buffering. Furthermore, this configuration allows row or column neurons at the ends of a core to receive input from the router fabric via a CONNECT_TO_OUT switch through an adjacent core, while another signal simultaneously passes through its array core. In such a situation, by closing the transmission gates INSIDE and CONNECT_TO_IN, the row or column neuron can receive data from either inside or outside the array core portion of the router mesh, or not receive data from either, without interrupting the transmission of neuron data downstream (using the PASS_IN transmission gate) or upstream (through the PASS_OUT transmission gate).

[0054] Core 501 includes a shared circuit 502 that performs the function of a border guard, allowing signals to enter and exit the corresponding ends of the core. Higher level wiring 503 is provided at the pitch to interconnect the cores (e.g., corresponding to lead 102). The border guard shared circuit allows signals to enter and exit from the western and southern ends of core 501.

[0055] Referring to Figure 5B, the south border guard circuit 504 on core 501 uses buffering to connect the output signal 505 to wiring 503. The signal 505 is then routed to the destination core 506 via one or more intermediate routers. The west border guard circuit 507 connects the input signal 505 to core 506.

[0056] Referring to Figure 6, an exemplary analog fabric according to an embodiment of the present disclosure is shown. In various embodiments, a 2D mesh (drawn by vertical lines) is provided on top of each component. In various embodiments, the components include computing cores (CC, 601) and analog tiles (tiles, 602). In this exemplary embodiment, each component has 512 vertical lines (channels) and 512 horizontal channels.

[0057] Data routing on each component is controlled using 512 border guards (BGs, 603) in four sets. Each border guard includes a driver and a switch, as described elsewhere in this specification. In this diagram, BGs of the same color can transmit data to each other.

[0058] Referring to Figures 7A to 7C, exemplary routing of data vectors on an analog fabric according to embodiments of the present disclosure is shown. In this example, one producer generates data vectors d1…d512. All BGs on the edge work together to distribute the data vector from a single producer for the entire data vector. Multiple consumers may receive the vector, but each consumer is required to consume the entire data vector.

[0059] Referring to Figure 8, a segmented fabric configuration according to an embodiment of the present disclosure is shown. Each end border guard is segmented into N groups. In this exemplary figure, N=4. Each segment is independently controlled, and its operation is software programmable.

[0060] BGs can also be interleaved between segments. For example, BG0, BG4, and BG8 may be located in physical segment 1.

[0061] In various embodiments, the control circuit and mask register are shared across the BG within each end segment.

[0062] Referring to Figure 9, implicit concatenation according to an embodiment of the present disclosure is shown. In this example, several producers 901-904 are provided. All producers can transmit simultaneously or individually, and either choice is supported using masks and control circuits. The segments transmitted by producers 901-904 are concatenated into a concatenation vector 905.

[0063] Referring to Figure 10, an implicit partitioning according to an embodiment of the present disclosure is shown. In this example, vector 1005 is partitioned across multiple segments and routed to multiple subconsumers 1001-1004.

[0064] Referring to Figure 11, collection according to an embodiment of the present disclosure is shown. In this example, vector 1001 is collected from several other tiles. This will be seen as just one example of the many data manipulation operations that are made possible by the present disclosure.

[0065] Referring to Figure 12, a detail diagram of an exemplary border guard circuit according to an embodiment of the present disclosure is provided. A given task is defined with respect to a valid pair of RCV_operations and DRV_operations. The receive operation is RCV_{N|S|OUT}, and the drive operation is DRV_{N|S|IN}. In various embodiments, the north driver and south driver have buffering and tristate.

[0066] Referring to Figure 13, an exemplary tile using the circuit in Figure 12 is shown. Based on the local mask bits, routing task A or task B is selected. For example, TASK_A is RCV_S and DRV_N; and TASK_B is RCV_OUT and DRV_N. This technique allows for fine-grained control across all channels. In various embodiments, the mask is loaded using a 2D mesh or using a scan chain. In this example, it is RCV_A_{N|S|OUT} and RCV_B_{N|S|OUT}; DRV_A_{N|S|IN} and DRV_B_{N|S|IN}.

[0067] Referring to Figure 14, a schematic circuit diagram of an exemplary border guard circuit according to an embodiment of the present disclosure is provided.

[0068] Referring to Figure 15, a method for operating a multicore neural network according to an embodiment of the present disclosure is shown. In 1501, a signal is received by multiple routers via multiple signal lines. In 1502, the signal is selectively routed by each of the multiple routers from at least one signal line to a neural core. In 1503, the signal is selectively routed by the multiple routers from the neural core to multiple signal lines. The multiple routers are configured to segment multiple ordered input lines and multiple ordered output lines into multiple segments and to route the signals of each of the multiple segments independently. The neural core comprises multiple ordered input lines, multiple ordered output lines, and multiple synapses, each synapse being operably coupled to one of the multiple input lines and one of the multiple output lines. The multiple signal lines are arranged along each dimension of the array of neural cores having the neural core.

[0069] Referring here to Figure 16, a schematic diagram of an example computing node is shown. Computing node 10 is merely an example of a suitable computing node and is not intended to imply any limitation on the scope of use or functionality of the embodiments described herein. Notwithstanding this, computing node 10 can implement and / or perform any of the functions described above.

[0070] Computing node 10 includes a computer system / server 12 that can operate in a number of other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments and / or configurations that may be suitable for use with computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and similar ones.

[0071] The computer system / server 12 can be described in the general context of computer system executable instructions, such as program modules, that are executed by the computer system. Generally, a program module may include routines, programs, objects, components, logic, data structures, etc., that perform a specific task or implement a specific abstract data type. The computer system / server 12 may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules may reside on both local and remote computer system storage media, including memory storage devices.

[0072] As shown in Figure 16, the computer system / server 12 within the computing node 10 is shown in the form of a general-purpose computing device. The components of the computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components, including the system memory 28, to the processor 16.

[0073] Bus 18 represents one or more of several types of bus structures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. Examples, but not limited to, such architectures include the Industry Standard Architecture (ISA) bus, Microchannel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, Peripheral Component Interconnect (PCI) bus, Peripheral Component Interconnect Express (PCIe), and Advanced Microcontroller Bus Architecture (AMBA).

[0074] The computer system / server 12 typically includes various computer system-readable media. Such media can be any available media accessible by the computer system / server 12, and include both volatile and non-volatile media, and removable and non-removable media.

[0075] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. As merely an example, the storage system 34 may be provided for reading from and writing to a non-removable non-volatile magnetic medium (not shown, commonly referred to as a “hard drive”). Not shown, a magnetic disk drive may be provided for reading from and writing to a removable non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive may be provided for reading from and writing to a removable non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical medium. In such examples, each may be connected to the bus 18 by one or more data medium interfaces. As further illustrated and described below, the memory 28 may include at least one program product having a set of program modules (e.g., at least one) configured to perform the functions of embodiments of the present disclosure.

[0076] A program / utility 40 having a set (at least one) of program modules 42 may be stored in memory 28, as well as, but not limited to, an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methodologies of the embodiments described herein.

[0077] The computer system / server 12 may also communicate with one or more external devices 14, such as a keyboard, pointing device, display 24, etc.; one or more devices that allow a user to interact with the computer system / server 12; and / or any device that allows the computer system / server 12 to communicate with one or more other computing devices (e.g., a network card, modem, etc.). Such communication can be done via an input / output (I / O) interface 22. Furthermore, the computer system / server 12 may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet), via a network adapter 20. As shown in the illustration, the network adapter 20 communicates with other components of the computer system / server 12 via a bus 18. It should be understood that other hardware and / or software components may be used in conjunction with the computer system / server 12, although these are not shown in the illustration. Examples, but not limited to, include microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data archive storage systems, etc.

[0078] This disclosure may be embodied as a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium (or more mediums) having computer-readable program instructions for causing a processor to execute aspects of this disclosure.

[0079] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exclusive list of more specific examples of computer-readable storage mediums includes portable computer diskettes, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved raised structures on which instructions are recorded, and any suitable combination of the devices described herein. The computer-readable storage mediums used herein should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0080] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0081] The computer-readable program instructions for performing the operations of the Disclosure may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++, or similar, and conventional procedural programming languages ​​such as the C programming language or similar. The computer-readable program instructions may run as a standalone software package, entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or such connection may be made to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing state information of computer-readable program instructions for personalizing the electronic circuit in order to perform an aspect of the present disclosure.

[0082] Aspects of this disclosure are described herein with reference to flowcharts or block diagrams of methods, apparatus (systems), and / or computer program products according to embodiments of this disclosure. It will be understood that each block in a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0083] These computer-readable program instructions are provided to the processors of general-purpose computers, dedicated computers, or other programmable data processing devices, which can generate machines that, as a result, create means for instructions executed via the processor of a computer or other programmable data processing device to perform functions / operations specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in computer-readable storage media that can instruct computers, programmable data processing devices, and / or other devices to function in a particular way, and as a result, the computer-readable storage media storing the instructions comprises a product containing instructions that perform the modes of functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0084] Computer-readable program instructions can also be loaded into a computer, other programmable data processing device, or other device and cause a series of operable steps on the computer, other programmable device, or other device to generate a computer implementation process, the resulting instructions executed on the computer, other programmable device, or other device to implement a function / operation specified in one or more blocks of a flowchart and / or block diagram.

[0085] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction having one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions shown within a block may be performed in a different order than shown in the figure. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or the blocks may be executed in reverse order depending on the related functions. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs a specified function or operation, or a combination of dedicated hardware and computer instructions.

[0086] The descriptions of the various embodiments of this disclosure are presented for illustrative purposes only and are not intended to be exhaustive or limit to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the embodiments described. The terms used herein have been selected to best describe the principles of the embodiments, their practical applications, or any technical improvements beyond the technology available on the market, or to enable other those skilled in the art to understand the embodiments disclosed herein.

Claims

1. It is a system, An array of neural cores, the array having at least one dimension, wherein each neural core of the array of neural cores comprises a plurality of ordered input lines, a plurality of ordered output lines, and a plurality of synapses, each of which is operably coupled to one of the plurality of ordered input lines and one of the plurality of ordered output lines; Multiple signal lines, wherein at least one of the multiple signal lines is arranged along each dimension of the array of neural cores; and Multiple routers, each of the said multiple routers, One of the multiple neural cores, and The array of neural cores is operably coupled to at least one of the plurality of signal lines along each of the dimensions, where Each of the plurality of routers is configured to selectively route signals from at least one signal line to its coupled neural core. Each of the plurality of routers is configured to selectively route signals from its coupled neural core to at least one signal line. Each of the aforementioned routers has multiple border guard circuits, Each of the plurality of border guard circuits is coupled to the end of a neural core in the array of neural cores. Each of the aforementioned multiple border guard circuits includes multiple segments, The plurality of routers are configured to independently route the signals between different neural cores via the plurality of segments. Equipped with, A system in which each of the aforementioned neural cores is configured to concatenate signals routed from segments of border guard circuits coupled to other neural cores as a combined input vector.

2. The system according to claim 1, wherein the plurality of segments are of equal size.

3. The system according to claim 1, wherein the routing of signals in the plurality of routers can be configured by the routing mask in each neural core of the array of neural cores.

4. The system according to claim 1, wherein the array of neural cores has two dimensions.

5. The system according to claim 1, wherein the plurality of signal lines include one signal line corresponding to each of the plurality of ordered input lines of the array of neural cores, and one signal line corresponding to each of the plurality of ordered output lines of the array of neural cores.

6. The system according to claim 5, wherein the plurality of signal lines are arranged in a plane parallel to the array of neural cores.

7. The system according to claim 5, wherein the plurality of signal lines are arranged within a mesh.

8. The system according to claim 1, wherein each of the plurality of routers is configured to selectively bypass its coupled neural core.

9. The system according to claim 1, wherein each of the plurality of routers is configured to transmit signals in two directions along at least one of the plurality of signal lines.

10. The system according to claim 1, further comprising a plurality of digital buffers, each of which is operably coupled to one of the plurality of routers, and each of which is configured for signal restoration.

11. The system according to claim 1, wherein the plurality of synapses of the array of neural cores are configured as a trained neural network.

12. The system according to any one of claims 1 to 11, wherein the first direction along the plurality of signal lines corresponds to forward propagation, and the second direction along the plurality of signal lines corresponds to backward propagation.

13. It is a method, The stage where signals are received by multiple routers via multiple signal lines; The step of each of the plurality of routers selectively routing the signal from at least one signal line to a neural core in the array of neural cores; In the step where the plurality of routers selectively route signals from the neural core to the plurality of signal lines, The plurality of routers are configured to segment the plurality of border guard circuits into a plurality of segments and to independently route the signals between the neural core and different neural cores through the plurality of segments. The neural core comprises a plurality of ordered input lines coupled to the plurality of border guard circuits, a plurality of ordered output lines coupled to the plurality of border guard circuits, and a plurality of synapses, each of the plurality of synapses being operably coupled to one of the plurality of ordered input lines and one of the plurality of ordered output lines. The plurality of signal lines are arranged along each dimension of the array of neural cores having the neural cores. Equipped with, A method in which each of the arrays of neural cores is configured to concatenate signals routed from segments of border guard circuits coupled to different neural cores as a combined input vector.

14. The method according to claim 13, wherein the plurality of segments are of equal size.

15. The method according to claim 13, wherein the routing of signals in the plurality of routers can be configured by the routing mask in each neural core of the array of neural cores.

16. The method according to claim 13, wherein the array of neural cores has two dimensions.

17. The method according to any one of claims 13 to 16, wherein one signal line corresponds to each of the plurality of ordered input lines of the array of neural cores, and one signal line corresponds to each of the plurality of ordered output lines of the array of neural cores.

18. The method according to claim 17, wherein the signal lines are arranged in a plane parallel to the array of neural cores.

Images