Runtime reconfigurable neural network processor core

Abstract

A hardware neural network processor is provided. The neural core includes a weight memory, an activation memory, a vector-matrix multiplier and a vector processor. The vector-matrix multiplier is adapted to receive a weight matrix from the weight memory, receive an activation vector from the activation memory, and calculate a vector-matrix product of the weight matrix and the activation vector. The vector processor is adapted to receive one or more input vectors from one or more vector sources, and perform one or more vector functions on the one or more input vectors to generate an output vector. In some embodiments, a programmable controller is adapted to configure and operate the neural core.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a hardware neural network processor, and more particularly to a neural network processor core that is reconfigurable at runtime. Summary of the invention

[0002] According to an embodiment of the present disclosure, a neural core is provided. The neural core includes a weight memory, an activation memory, a vector-matrix multiplier and a vector processor. The vector-matrix multiplier is suitable for receiving a weight matrix from the weight memory, receiving an activation vector from the activation memory, and calculating a vector-matrix product of the weight matrix and the activation vector. The vector processor is suitable for receiving one or more input vectors from one or more vector sources, and performing one or more vector functions on one or more input vectors to generate output vectors. In some embodiments, a programmable controller is suitable for configuring and operating the neural core. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0004] Figure 1 A neural core according to an embodiment of the present disclosure is shown.

[0005] Figure 2 The following diagram shows the logical operations performed by the inference processing unit according to an embodiment of the present disclosure.

[0006] Figure 3 A neural core and associated network are shown according to an embodiment of the present disclosure.

[0007] Figure 4A -B is a kernel timing diagram of an exemplary spatial and filter inner loop according to an embodiment of the present invention.

[0008] Figure 5 is a detailed view of a vector unit according to an embodiment of the present disclosure.

[0009] Figure 6 A method for configuring a neural core according to an embodiment of the present disclosure is shown.

[0010] Figure 7 A method for configuring a neural core according to an embodiment of the present disclosure is shown.

[0011] Figure 8 Depicted is a compute node according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] An artificial neuron is a mathematical function whose output is a nonlinear function of a linear combination of its inputs. Two neurons are connected if the output of one neuron is the input of the other neuron. A weight is a scalar value that encodes the strength of the connection between the output of one neuron and the input of another neuron.

[0013] A neuron computes its output (called an activation) by applying a nonlinear activation function to a weighted sum of its inputs. A weighted sum is an intermediate result computed by multiplying each input by its corresponding weight and accumulating the products. A partial sum is a weighted sum of a subset of the inputs. The weighted sum of all inputs can be computed in stages by accumulating one or more partial sums.

[0014] A neural network is a collection of one or more neurons. Neural networks are often divided into groups of neurons called layers. A layer is a collection of one or more neurons that all receive input from the same layer and all send output to the same layer, and generally perform similar functions. An input layer is a layer that receives input from a source outside the neural network. An output layer is a layer that sends output to a destination outside the neural network. All other layers are intermediate processing layers. A multi-layer neural network is a neural network with more than one layer. A deep neural network is a multi-layer neural network with many layers.

[0015] A tensor is a multidimensional array of values. A tensor block is a contiguous subarray of the elements in a tensor.

[0016] Each neural network layer is associated with a parameter tensor V, a weight tensor W, an input data tensor X, an output data tensor Y, and an intermediate data tensor Z. The parameter tensor contains all the parameters that control the activation functions of the neurons in the layer. The weight tensor contains all the weights that connect the inputs to the layer. The input data tensor contains all the data that the layer uses as input. The output data tensor contains all the data that the layer computes as output. The intermediate data tensor contains any data that the layer produces as intermediate computations, such as partial sums.

[0017] The data tensors (input, output, and intermediate) of a layer can be 3-dimensional, where the first two dimensions can be interpreted as encoding spatial locations, while the third dimension can be interpreted as encoding different features. For example, when the data tensor represents a color image, the first two dimensions encode the vertical and horizontal coordinates within the image, and the third dimension encodes the color at each location. Each element of the input data tensor X can be connected to each neuron via a separate weight, so the weight tensor W usually has 6 dimensions, concatenating the 3 dimensions of the input data tensor (input row a, input column b, input feature c) with the 3 dimensions of the output data tensor (output row i, output column j, output feature k). The intermediate data tensor Z has the same shape as the output data tensor Y. The parameter tensor V concatenates the 3 output data tensor dimensions with an additional dimension o that indexes the parameters of the activation function σ.

[0018] The elements of the layer's output data tensor Y can be calculated as in Equation 1, where the neuron activation function σ is configured by a vector V[i,j,k,:] of activation function parameters, and the weighted sum can be calculated as in Equation 2.

[0019] Y[i,j,k]=σ(V[i,j,k,:]; Z[i,j,k]) Equation 1

[0020]

[0021] To simplify notation, the weighted sum in Equation 2 may be referred to as the output, which is equivalent to using a linear activation function Y[i,j,k]=σ(Z[i,j,k])=Z[i,j,k], while it should be understood that the same statement applies without loss of generality when different activation functions are used.

[0022] In various embodiments, the computation of the output data tensor as described above is broken down into smaller problems. Each problem can then be solved in parallel on one or more neural cores or on one or more cores of a conventional multi-core system.

[0023] Now refer to Figure 1 , depicts a neural core according to an embodiment of the present disclosure. The neural core 100 is a splicable computing unit that computes a block of output tensors. The neural core 100 has M inputs and N outputs. In various embodiments, M=N. To compute the output tensor block, the neural core multiplies the M×1 input tensor block 101 with the M×N weight tensor block 102, and accumulates the products into the weighted sum stored in the 1×N intermediate tensor block 103. The O×N parameter tensor block contains O parameters that specify each of the N neuron activation functions that are applied to the intermediate tensor block 103 to produce the 1×N output tensor block 105.

[0024] Multiple neural cores can be tiled in a neural core array. In some embodiments, the array is two-dimensional.

[0025] A neural network model is a set of constants that collectively specify the entire computation performed by a neural network, including a graph of connections between neurons and the weights and activation function parameters of each neuron. Training is the process of modifying a neural network model to perform the desired function. Inference is the process of applying a neural network to inputs to produce outputs without modifying the neural network model.

[0026] An IPU is a type of processor that performs neural network inference. A neural inference chip is a specific physical instance of an IPU.

[0027] Figure 2At a high level the logical operations performed by the inference processing unit are shown. At 201, the input activation (X) is assigned to the core. At 202, the layer instructions are assigned to the core. At 203, the layer weights (W) and / or parameters are assigned to the core. At 204, the matrix weights (W) are read. At 205, the activation vector (X) is read. At 206, the vector-matrix product (Z=X) is calculated. T W). At 207, the partial sum vector (Z) is read and / or selected. src ). At 208, the vector-vector function Z is calculated. tgt =f(Z src At 209, the partial sum vector (Z tgt ). At 210, the activation function is calculated (Y=f(Z)). At 211, the activation vector is written (Y→X). At 212, the layer activations (X) are redistributed among the cores. At 213, the output activations (X) are collected from the cores.

[0028] As shown, steps 201...213 are repeated on all input frames. Steps 202...212 are repeated on all layers of the neural network. Steps 204...211 are repeated on all layer outputs. Steps 204...209 are repeated on all layer inputs.

[0029] See now Figure 3 , shows an exemplary neural core and associated network according to an embodiment of the present disclosure. Figure 1 Core 301 implemented as described is interconnected with other cores via networks 302 ... 304. In this embodiment, network 302 is responsible for distributing weights and / or instructions, network 303 is responsible for distributing partial sums, and network 304 is responsible for distributing activations. However, it should be understood that different embodiments of the present disclosure may combine these networks, or further separate them into multiple additional networks.

[0030] See also Figure 3 In step 201, input activation (X) is distributed to core 301 from outside the core through activation network 304 to activation memory 305. In step 202, layer instructions are distributed to core 301 from outside the core through weight / instruction network 302 to instruction memory 306. In step 203, layer weights (W) and / or parameters are distributed to core 301 from outside the core through weight / instruction network 302 to weight memory 307 and / or parameter memory 308.

[0031] At step 204, the weight matrix (W) is read from the weight memory 307 by the vector matrix multiplication (VMM) unit 309. At step 205, the activation vector (V) is read from the activation memory 305 by the vector matrix multiplication (VMM) unit 309. Then, at step 306, the vector matrix multiplication (VMM) unit 309 calculates the vector-matrix product Z=XT W and provides the result to the vector-vector unit 310. In step 207, the vector-vector unit 310 reads the additional partial sums from the partial sum memory 311 and receives the additional partial sums from the outside of the core through the partial sum network 303. In step 208, the vector-vector unit 310 calculates the vector-vector operation from these source partial sums. For example, the partial sums can be summed in sequence. In step 209, the obtained target partial sum is written to the partial sum memory 311, sent to the outside of the core through the partial sum network 303, and / or fed back for further processing by the vector-vector unit 310.

[0032] At step 210, the partial and results from the vector-vector unit 310, after all computations for the inputs of a given layer are completed, are provided to the activation unit 312 for computing the output activations. At step 211, the activation vector (Y) is written to the activation memory 305. At step 212, the layer activations (including the results written to the activation memory) are redistributed from the activation memory 305 across the cores through the activation network 304. On reception, they are written to the local activation memory of each receiving core. After processing of a given frame is completed, the output activations are read from the activation memory 305 and sent out of the core through the network 304.

[0033] See also Figure 4A -B, provides a kernel timing diagram for an exemplary spatial inner loop and filter inner loop. At 401, the activation memory is preloaded with activations X. At 402, microcode instructions are loaded. At 403, the weight tensor W is loaded. At 404-5, the activation block and weight block are read. At 406, the partial sum block is calculated. At 407-9, the partial sum block is transferred, stored and / or accumulated. At 410, the activation function is calculated. At 411, the activation is written back to the activation memory. It should be understood that the steps in these timing diagrams correspond to the steps in the reference sequence diagram. Figure 2 The steps described. Cycle through the depicted space Figure 4A In , for each weight block W0…W1, an activation block X0…X3 is loaded, allowing the computation of the partial sum block Z 00 …Z 03 ,Z 10 …Z 13 . In the depiction filter loop Figure 4B In , weight blocks W0…W3 are loaded for each activation block X0…X1, allowing the computation of the partial sum block Z 00 …Z 30 ,Z 01 …Z 31 .

[0034] Thus, in operation, the core control microengine (e.g., 313) coordinates the data movement and calculation of the core. The microengine issues a read activation memory address operation to load the input activation block into the vector-matrix multiplication unit. The microengine issues a read weight memory address operation to load the weight block into the vector-matrix multiplication unit. The microengine issues a calculation operation to the vector-matrix multiplication unit, causing the vector-matrix multiplication unit to calculate the array calculation portion and block.

[0035] The microengine issues one or more of a partial sum read / write memory address operation, a vector calculation operation, or a partial sum communication operation to perform one or more of the following: read partial sum data from a partial sum source; perform calculations using a partial sum arithmetic unit; or write partial sum data to a partial sum destination. Writing partial sum data to a partial sum destination may include communicating externally to the core through a partial sum network interface or sending partial sum data to an activated arithmetic unit.

[0036] The microengine issues an activation calculation operation, so that the activation arithmetic unit calculates the output activation block. The microengine issues a write activation memory address, and the output activation block is written to the activation memory through the activation memory interface.

[0037] Accordingly, various sources, destinations, address types, computation types, and control components are defined for a given core.

[0038] The sources of the vector-vector unit 310 include the vector-matrix multiplication (VMM) unit 309, the activation memory 305, constants from the parameter memory 308, the partial sum memory 311, the partial sum result from the previous cycle (TGT partial sum); and the partial sum network 303.

[0039] The targets of the vector-to-vector unit 310 include the partial sum memory 311 , the partial sum result of the subsequent cycle (SRC partial sum), the activation unit 312 , and the partial sum network 303 .

[0040] Thus, a given instruction may read or write from activation memory 305, read from weight memory 307, or read or write from partial sum memory 311. Computational operations performed by the core include vector-matrix multiplications performed by VMM unit 309, vector (partial sum) operations performed by vector unit 310, and activation functions performed by activation unit 312.

[0041] Control operations include a program counter and a loop and / or sequence counter.

[0042] Thus, steps 204…211 can be mapped to the following operations. In steps 204, 205, 207, 209, 211, memory operations are issued to read weights from addresses in weight memory, read parameters from addresses in parameter memory, read activations from addresses in activation memory, and read / write partial sums to addresses in partial sum memory. In steps 206, 208, 210, compute operations are issued to perform vector-matrix multiplications, vector-vector operations, and activation functions. At steps 207, 209, communication operations are issued to select vector-vector operands, route messages on partial sum networks, and select partial sum targets. The loop on the layer output (in steps 204…211) and the loop on the layer input (in steps 204…209) are controlled by control operations that specify program counters, loop counters, and sequence counters.

[0043] See also Figure 5 , provides a detailed view of a vector unit slice according to an embodiment of the present disclosure. The vector unit has many vector unit slices to operate on all elements of the vector in parallel, so only one slice is shown here. In some embodiments, all slices of the vector unit perform the same operation on their corresponding vector elements in parallel at the same time. Source 501 corresponds to the source described above. Specifically, the weight memory 307 is accessed at the read address W_RD and the activation memory 305 is accessed at the read address act_RD to provide the input of the vector-matrix multiplication, and the result of the vector-matrix multiplication then serves as the source of the vector-vector unit. The parameter memory 308 is accessed at the read address NRN_RD to provide parameters to the vector-vector unit. The partial sum memory 311 is accessed at the read addresses P0_RD and P1_RD to retrieve partial sum operands for the vector-vector unit. The on-chip network 303 is used to provide partial sum input to the vector-vector unit. As shown in the figure, input is received from four cardinal directions (north, south, east and west). However, it will be appreciated that various network configurations according to the present disclosure can be used to allow input to be received from any core.

[0044] Operation 502 includes a compute operation (COMPUTE_OP) and a bypass operation (BYPASS_OP). The compute operation maps the source 501 to the compute functional units 503, 504 (A and B) and specifies the computation to be performed by the compute functional units. The bypass operation (BYPASS_OP) maps the source 501 to the bypass functional units 505, 506 (C and D). The result of the computation or bypass constitutes the intermediate source 507.

[0045] Operations 508 on intermediate source 507 include output target operations (OUT_TGT_OP) and activate output target operations (ACT_TGT_OP). Output target operations map intermediate source 507 to output target 509. Activate output target operations map intermediate source 507 to activate output target 510 and specify the computation to be performed by the activation function unit.

[0046] The result of the activation function is written to the activation memory 305 at the write address act_WR. The partial sum is written to the partial sum memory 311 at the write addresses P0_WR and P1_WR. The network output is sent through the network 303. As shown, the output is sent to four cardinal directions, namely north, south, east and west. However, it will be appreciated that various network configurations according to the present disclosure can be used to allow outputs to be sent to any core.

[0047] It will be understood that a given core may include multiple functional units. Thus, for simplicity of description, Figure 5 Depicts a slice of a vector-vector unit.

[0048] Referring to Table 1, an exemplary part and data operation format is provided. In Table 1, bypass operations include BYPASS_C_OP and BYPASS_D_OP corresponding to the individual bypasses 505, 506 described above. Target output operations include OUT_TGT_OP, COND_TGT_OP, and ACT_TGT_OP. Compute operations follow ACT_TGT_OP.

[0049]

[0050]

[0051] Table 1

[0052] Table 2 lists the sources in the above examples.

[0053]

[0054]

[0055] Table 2

[0056] Table 3 lists the intermediate sources in the above embodiments.

[0057] name describe resA Result A resB Result B resC Result C resD Result D Z Zero Vector

[0058] Table 3

[0059] Table 4 lists the output targets in the above embodiment.

[0060] name describe Po0 Part and memory, write port 0 Po1 Partial and memory, write port 1 No NOC North Output Port So NOC South Output Port Eo NoC East Output Port I Network-on-Chip West Output Port

[0061] Table 4

[0062] Table 5 lists the activation targets in the above embodiment.

[0063] name describe Ao Output activation function

[0064] Table 5

[0065] Table 6 lists the vector calculation operations in the above embodiment.

[0066] name index describe VADD_A 7 ResA=SRC_A0+(SRC_A1+SRC_A2) VADD_B 8 ResB=(SRC_B0+SRC_B1)+SRC_B2 VSUB_B 9 ResB=(SRC_B0-SRC_B1)+SRC_B2 VMAC_B 10 ResB=(SRC_B0[32b]*SRC_B1[8b])+SRC_B2 VCMP_B 11 ResB=GT(SRC_B0,SRC_B1)+SRC_B2 VMAX_B 12 ResB=MAX(SRC_B0,SRC_B1)+SRC_B2 VABS_B 13 ResB=ABS(SRC_B0)+SRC_B2 VLIM_B 14 ResB=(IMM_4b_1<<4)+IMM_4b_0+SRC_B2 VNOT_B 15 ResB=(~SRC_B0)+SRC_B2 VAND_B 16 ResB=(SRC_B0&SRC_B1)+SRC_B2 VOR_B 17 ResB=(SRC_B0|SRC_B1)+SRC_B2 VXOR_B 18 ResB=(SRC_B0^SRC_B1)+SRC_B2

[0067] Table 6

[0068] Referring to Table 7, a summary of the various reconfigurable elements of the neural core according to different embodiments of the present disclosure is provided.

[0069]

[0070]

[0071] Table 7

[0072] refer to Figure 6 , illustrates a method for operating a neural core according to an embodiment of the present disclosure. At 601, a weight matrix is ​​received from a weight memory. At 602, an activation vector is received from an activation memory. At 603, a vector-matrix product of the weight matrix and the activation vector is calculated. At 604, one or more vector functions are performed on the vector-matrix product to produce an output vector. At 605, an activation function is applied to the output vector to determine a result. At 606, the result is provided to the activation memory.

[0073] See also Figure 7 , a method for configuring a neural core according to an embodiment of the present disclosure is shown. At 701, a programmable controller maps one or more of a plurality of vector sources to a vector processor. At 702, the programmable controller maps the vector processor to one or more of a plurality of vector targets. At 703, the programmable controller instructs the vector processor to perform a vector function on inputs from one or more of the plurality of sources and provide results to one or more of the plurality of vector targets.

[0074] Thus, the present disclosure provides a neural network processor core that is reconfigurable at runtime. Such a core is reconfigurable through a programmable controller. In some embodiments, the reconfiguration is performed concurrently with the calculations in the pipeline.

[0075] See now Figure 8, a schematic diagram of an example of a computing node is shown. Computing node 10 is only one example of a suitable computing node and is not intended to impose any limitations on the scope of use or functionality of the embodiments described herein. Regardless, computing node 10 is capable of being implemented and / or performing any of the functions set forth above.

[0076] In computing node 10 there is a computer system / server 12, which may operate with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices, and the like.

[0077] Computer system / server 12 may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally speaking, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. Computer system / server 12 may be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

[0078] like Figure 8 As shown, the computer system / server 12 in the computing node 10 is shown in the form of a general-purpose computing device. The components of the computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples different system components including the system memory 28 to the processor 16.

[0079] Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, Peripheral Component Interconnect (PCI) bus, Peripheral Component Interconnect Express (PCIe), and Advanced Microcontroller Bus Architecture (AMBA).

[0080] Computer system / server 12 typically includes a variety of computer system readable media. Such media can be any available media that can be accessed by computer system / server 12, and it includes volatile and nonvolatile media, removable and non-removable media.

[0081] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The computer system / server 12 may also include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown, and generally referred to as a "hard drive"). Although not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical drive for reading and writing to a removable non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media may be provided. In such an instance, each may be connected to the bus 18 via one or more data media interfaces. As will be further depicted and described below, the memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of embodiments of the present disclosure.

[0082] A program / utility 40 having a set (at least one) of program modules 42, as well as an operating system, one or more application programs, other program modules, and program data, may be stored in memory 28 by way of example and not limitation. Each or some combination of the operating system, one or more application programs, other program modules, and program data may include an embodiment of a networking environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0083] The computer system / server 12 may also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24; with one or more devices that enable a user to interact with the computer system / server 12; and / or with any device that enables the computer system / server 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communications may occur through an input / output (I / O) interface 22. Furthermore, the computer system / server 12 may communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other components of the computer system / server 12 through a bus 18. It should be understood that, although not shown, other hardware and / or software components may be used in conjunction with the computer system / server 12. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archiving storage systems, etc.

[0084] In various embodiments, one or more inference processing units (not shown) are coupled to bus 18. In such embodiments, the IPU may receive data from or write data to memory 28 via bus 18. Likewise, the IPU may interact with other components via bus 18 as described herein.

[0085] The present disclosure may be embodied as a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium (or medium) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0086] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device (such as a punch card or a raised structure in a groove with instructions recorded thereon), and any suitable combination of the above. As used herein, a computer-readable storage medium should not be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by a fiber optic cable), or an electrical signal transmitted by a wire.

[0087] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0088] The computer-readable program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented Smalltalk, C++ and other programming languages) and conventional procedural programming languages ​​(such as "C" programming language) or similar programming languages. The computer-readable program instructions may be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (for example, using an Internet service provider through the Internet). In certain embodiments, an electronic circuit (including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA)) may be executed by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit, so as to perform aspects of the present disclosure.

[0089] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0090] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine that, when executed by a processor of the computer or other programmable data processing apparatus, creates a device for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, a programmable data processing apparatus, and / or other device to function in a particular manner, such that a computer-readable storage medium having instructions stored therein includes an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0091] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other apparatus that causes a series of operating steps to be performed on a computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the different embodiments of the present disclosure. To this end, each box in the flowchart or block diagram may represent a part of a module, segment or instruction, which includes one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box may not occur in the order marked in the figure. For example, depending on the functions involved, the two boxes shown in succession can actually be executed substantially simultaneously, or these boxes can sometimes be executed in the opposite order. It will also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented by a system based on special-purpose hardware, and the system based on special-purpose hardware performs a specified function or action or performs a combination of special-purpose hardware and computer instructions.

[0093] Descriptions of different embodiments of the present disclosure have been given for illustrative purposes, but these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements over existing technologies on the market, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A neural core device, comprising: Weight memory; Activate memory; a vector-matrix multiplier adapted to receive a weight matrix from a weight memory, receive an activation vector from an activation memory, calculate a set of partial products by vector-matrix products of the weight matrix and the activation vector, and output the set of partial products; a vector processor adapted to receive one or more input vectors including said set of partial products from one or more vector sources including said vector-matrix multiplier, and to perform one or more vector functions on the one or more input vectors to produce an output vector; A microengine device operatively coupled to the vector processor, the microengine device being adapted to: Map one or more vector sources to vector processors, Mapping vector processors to one or more vector targets, instructs the vector processor to perform one or more vector functions, and An output vector is provided to the one or more vector destinations.

2. The neural core device of claim 1, further comprising: an activation unit operatively coupled to the vector processor and adapted to: Applies an activation function to the result from the vector processor.

3. The neural core device of claim 1, wherein: Vector sources include parts and memories, networks, registers, or parameter memories.

4. The neural core device of claim 2, wherein: Vector targets include activations of memory, sections and memories, registers, or networks.

5. The neural core device of claim 1, wherein: A vector processor is adapted to apply one or more constants to the result.

6. The neural core device of claim 1, configured to accumulate partial sums.

7. The neural nucleus device of claim 1, wherein: The microengine device is further adapted to instruct the vector-matrix multiplier to read the weight matrix from the weight memory, read the activation vector from the activation memory, and calculate the vector-matrix product of the weight matrix and the activation vector.

8. The neural core device of claim 4, wherein: The activation function is configurable.

9. The neural nucleus device of claim 8, wherein: The microengine device is further adapted to instruct the activation unit to calculate the activation function and provide a result to the activation memory.

10. The neural nucleus device of claim 9, wherein: Weight Matrix is ​​a subarray of the neural network weight matrix; An activation vector is a subarray of the neural network's activation vector.

11. The neural nucleus device of claim 10, wherein: A vector-matrix product of the weight matrix and the activation vector is provided for accumulation, the accumulation producing a vector-matrix product of the neural network weight matrix and the neural network activation vector.

12. The neural core device of claim 1, configured to compute a neural network function having inputs, parameters, and outputs.

13. The neural core device of claim 12, wherein: The weight matrices and / or activation vectors have configurable sizes.

14. The neural core device of claim 12, wherein: Neural network inputs, parameters, and / or outputs have configurable sizes.

15. The neural core device of claim 12, wherein: Neural network functions are configurable.

16. The neural core device of claim 1, configured to compute a neural network function in conjunction with a plurality of additional neural cores interconnected via a network.

17. The neural core device of claim 1, configured to compute a portion of a neural network function.

18. The neural nucleus device of claim 17, wherein: The portion of the neural network function is configurable.

19. A method comprising: The microengine device instructs the vector-matrix multiplier to receive the weight matrix from the weight memory; Instructing a vector-matrix multiplier to receive an activation vector from an activation memory by the microengine device; The microengine device instructs the vector-matrix multiplier to calculate the vector-matrix product of the weight matrix and the activation vector; Instructing the vector-matrix multiplier, by the microengine device, to perform one or more vector functions on the vector-matrix product to generate an output vector; Instructing the activation unit, by the microengine device, to apply the activation function to the output vector to determine a result; The microengine device instructs the activation unit to provide the result to the activation memory and stores the activation vector and the result at the same time.

20. The method of claim 19, wherein: The weight matrix is ​​a subarray of the neural network weight matrix, and the activation vector is a subarray of the neural network activation vector, the method further comprising: The result is accumulated with the additional result to produce the vector-matrix product of the neural network weight matrix and the neural network activation vector.

21. A method comprising: mapping, by the microengine device, one or more of the plurality of vector sources to a vector processor; mapping, by the microengine device, the vector processor to one or more of the plurality of vector targets; A vector processor is directed by the microengine device to perform a vector function on inputs including a set of partial products from one or more of the plurality of vector sources and provide results to one or more of the plurality of vector destinations.

Images