Configurable processor element array for implementing convolutional neural networks

Abstract

The present disclosure relates to an array of configurable processor elements for implementing a convolutional neural network. An example apparatus disclosed includes an array of processor elements including rows each having a first number of processor elements and columns each having a second number of processor elements. The example apparatus also includes configuration registers to store descriptors for configuring the array of processor elements to implement a layer of a convolutional neural network based on a dataflow schedule corresponding to one of a plurality of tensor processing templates, some of the processor elements to be configured to implement the one of the tensor processing templates based on the descriptors to operate on input activation data and filter data associated with the layer of the convolutional neural network to generate output activation data associated with the layer of the convolutional neural network. The example apparatus also includes a memory to store the input activation data, the filter data, and the output activation data.

Description

Technical Field

[0001] This disclosure generally relates to neural networks, and more specifically to configurable arrays of processor elements for implementing convolutional neural networks. Background Technology

[0002] Neural networks have been and continue to be adopted as a fundamental technological solution in a wide range of technological fields, such as facial recognition, speech recognition, navigation, market research, and a few others. Consequently, the field of neural networks has been and continues to grow rapidly in terms of inference algorithm development and the development of hardware platforms used to implement these evolving inference algorithms. The network layers of neural networks, such as deep learning convolutional neural networks, have many possible tensor shapes, and their dimensions continue to change as existing neural network inference algorithms are modified and / or new neural network inference algorithms are developed. Summary of the Invention

[0003] According to one aspect of this disclosure, an apparatus for implementing a convolutional neural network is provided, the apparatus comprising: an array of processor elements including rows and columns, each row having a first number of processor elements and each column having a second number of processor elements; a configuration register for storing a plurality of descriptors for configuring the array of processor elements to implement layers of the convolutional neural network based on a data flow schedule corresponding to one of a plurality of tensor processing templates, some of the processor elements being configured, based on the descriptors, to implement the one of the plurality of tensor processing templates to operate on input activation data and filter data associated with the layer of the convolutional neural network to produce output activation data associated with the layer of the convolutional neural network; and a memory for storing the input activation data, the filter data, and the output activation data associated with the layer of the convolutional neural network.

[0004] According to another aspect of this disclosure, a computer-readable medium is provided, including computer-readable instructions that, when executed, cause at least one processor to perform at least the following operations: writing a first set of descriptors to a configuration register to configure an array of processor elements to implement a first layer of a convolutional neural network based on a first data flow schedule corresponding to a first tensor processing template among a plurality of tensor processing templates, the array including rows and columns, each row having a first number of processor elements and each column having a second number of processor elements, the first set of descriptors being used to configure some of the processor elements to implement the first tensor processing template among the plurality of tensor processing templates to implement the first layer of the convolutional neural network. The first layer of the convolutional neural network is operated on with associated input activation data and filter data to produce output activation data associated with the first layer of the convolutional neural network; and a second set of descriptors is written to the configuration register to configure an array of processor elements to implement a second layer of the convolutional neural network based on a second data flow schedule corresponding to a second tensor processing template among the plurality of tensor processing templates. The second set of descriptors is used to configure some of the processor elements to implement the second tensor processing template among the plurality of tensor processing templates to operate on the input activation data and filter data associated with the second layer of the convolutional neural network to produce output activation data associated with the second layer of the convolutional neural network.

[0005] According to another aspect of this disclosure, a method for implementing a convolutional neural network is provided, the method comprising: using at least one processor to write a first set of descriptors into a configuration register to configure an array of processor elements to implement a first layer of the convolutional neural network based on a first data flow schedule corresponding to a first tensor processing template among a plurality of tensor processing templates, the array including rows and columns, each row having a first number of processor elements, each column having a second number of processor elements, the first set of descriptors being used to configure some of the processor elements to implement the first tensor processing template among the plurality of tensor processing templates to apply input activation data and overcurrent protection associated with the first layer of the convolutional neural network. The process involves manipulating filter data to generate output activation data associated with the first layer of the convolutional neural network; and using the at least one processor to write a second set of descriptors into the configuration register to configure an array of processor elements to implement the second layer of the convolutional neural network based on a second data flow schedule corresponding to the second tensor processing template among the plurality of tensor processing templates. The second set of descriptors is used to configure some of the processor elements to implement the second tensor processing template among the plurality of tensor processing templates to manipulate the input activation data and filter data associated with the second layer of the convolutional neural network to generate output activation data associated with the second layer of the convolutional neural network.

[0006] According to another aspect of this disclosure, an apparatus for implementing a convolutional neural network is provided, the apparatus comprising: an array of processor elements including rows and columns, each row having a first number of processor elements and each column having a second number of processor elements; a configuration means for configuring the array of processor elements based on a plurality of descriptors to implement layers of the convolutional neural network based on a data flow schedule corresponding to one of a plurality of tensor processing templates, the descriptors being configured to configure some of the processor elements to implement the one of the plurality of tensor processing templates to operate on input activation data and filter data associated with the layer of the convolutional neural network to produce output activation data associated with the layer of the convolutional neural network; and means for storing the input activation data, the filter data, and the output activation data associated with the layer of the convolutional neural network. Attached Figure Description

[0007] Figure 1 This is a block diagram of an example configurable processor element array for implementing a convolutional neural network in accordance with the teachings of this disclosure.

[0008] Figure 2 It shows the result of Figure 1 Example tensor operations performed by a configurable array of processor elements.

[0009] Figure 3-4 It shows that it should be made by Figure 1 The configurable processor element array includes example processor elements implementing example tensor processing templates and corresponding example data stream scheduling.

[0010] Figure 5 It shows the result of Figure 1 An example operational pipeline implemented using a configurable array of processor elements.

[0011] Figure 6 yes Figure 1 A block diagram of example processor elements included in the configurable processor element array.

[0012] Figure 7-12 It shows Figure 1 Example operation phases of a configurable processor element array and example transitions between operation phases.

[0013] Figure 13A -B shows the support Figure 1 An example hardware architecture for the external portion and accumulation of configurable processor element arrays.

[0014] Figure 14 It shows the support Figure 1 An example hardware architecture for element-wise operations in a configurable array of processor elements.

[0015] Figure 15 It shows the support Figure 1 An example hardware architecture for maxpooling operations in a configurable array of processor elements.

[0016] Figure 16-19 This illustrates the configuration based on four corresponding example tensor processing templates. Figure 1 Four example data flow scheduling implementations of configurable processor element arrays.

[0017] Figure 20-21 It shows Figure 16 The first example of data stream scheduling is the data partitioning and chunking operation.

[0018] Figure 22-25 This shows an example set of configuration register descriptor values ​​that will be used for configuration. Figure 1 Configurable processor element arrays to achieve Figure 16-19 The corresponding example is data stream scheduling.

[0019] Figure 26This is a flowchart illustrating example computer-readable instructions that can be executed to operate... Figure 1 and / or Figure 6 Configurable array of processor elements.

[0020] Figure 27 This is a block diagram of an example processor platform, which is constructed to include... Figure 1 and / or Figure 6 The example can be configured as an array of processor elements and constructed to execute Figure 26 Example computer-readable instructions for operation Figure 1 and / or Figure 6 Configurable array of processor elements.

[0021] The accompanying drawings are not to scale. Generally, the same reference numerals will be used throughout one or more drawings and the accompanying written description to refer to the same or similar parts, elements, etc.

[0022] When identifying multiple elements or components that can be individually mentioned, descriptors such as "first," "second," "third," etc., are used herein. Unless otherwise specified or understood based on their context of use, such descriptors are not intended to assign any meaning of priority or chronological order, but merely serve as labels to refer to multiple elements or components respectively, to facilitate understanding of the disclosed examples. In some examples, the descriptor "first" may be used to refer to an element in the detailed description, while different descriptors such as "second" or "third" may be used in the claims to refer to the same element. In such cases, it should be understood that such descriptors are used merely for the convenience of referring to multiple elements or components. Detailed Implementation

[0023] This paper discloses an example configurable processor element array for implementing convolutional neural networks. As mentioned above, the neural network field has experienced and continues to experience rapid growth in terms of inference algorithm development and the development of hardware platforms for implementing evolving inference algorithms. The network layers of neural networks, such as deep learning convolutional neural networks, have many possible tensor shapes, and their dimensions continuously change as existing neural network inference algorithms are modified and / or new neural network inference algorithms are developed. To accommodate the rapid evolution of neural networks, the hardware platforms used to implement neural networks need to be configurable to support varying sizes of network layer tensor shapes. Previous neural network platforms used field-programmable gate arrays (FPGAs) to provide this configurability, rather than application-specific integrated circuits (ASICs), because reconfiguring network layer tensor shapes in ASIC implementations could require replacing the ASIC, and ASIC design cycles could be lengthy. Therefore, by the time ASIC-based solutions for specific deep learning inference algorithms enter the market, the inference algorithms may have evolved, rendering the ASIC-based solutions obsolete. However, FPGAs lack the processing performance and energy efficiency of ASICs.

[0024] In contrast to previous neural network hardware platforms, the exemplary configurable processor element arrays disclosed herein offer FPGA-like configurability while maintaining the energy efficiency of ASICs. The disclosed exemplary configurable processor element arrays enable runtime configuration of computations with different tensor shapes, adaptable to the rapidly evolving field of neural network algorithms, which include network layers with widely varying tensor sizes, while retaining the performance and energy efficiency offered by ASICs.

[0025] The disclosed example configurable processor element array is based on an array of software-configurable processor elements (PEs), also referred to herein as processing elements or raw kernel modules, which can perform convolutional computations (such as filter weights, input activations, and / or output activations) on tensor data of flexible shapes to implement a given layer of a neural network. As disclosed in more detail below, the microarchitecture of the example PEs included in the configurable processor element array can be reconfigured at runtime (e.g., based on software-programmable configuration registers) to implement successive layers of a given neural network, or to implement other neural networks. In some examples, the PEs utilize activation and weight reuse for energy efficiency by placing a distributed local storage device close to the computational units included within the PE itself.

[0026] As disclosed in more detail below, the flexibility of the disclosed example PE in supporting variable tensor shape computation in hardware is based on decomposing the tensor computation associated with a given layer of the neural network into one of a set of possible tensor processing templates. Examples of such tensor processing templates include, but are not limited to, vector-vector, vector-matrix, and matrix-matrix tensor processing templates. As disclosed in more detail below, the example PE is controlled to support specific tensor computations in hardware based on a set of configuration descriptors, which are initialized at the start of execution of a given layer of the neural network. Thus, the example PE disclosed herein can be implemented as a pure hardware solution (e.g., via an ASIC), but it exposes hardware configuration registers to software, which enables the software to configure the tensor data flow for a given network layer during runtime. Therefore, the example PE disclosed herein, and the associated arrangement of the PEs arranged in the example configurable processor element array disclosed herein, enable the flexible data flow of convolutional neural network layers to be executed in a hardware accelerator without performance penalty due to, for example, the need to offload any work to an external processor or software.

[0027] The example configurable processor element arrays disclosed herein offer numerous benefits compared to existing hardware platforms used to implement convolutional neural networks. For example, the configurable processor element arrays can be implemented using ASICs instead of FPGAs, thus exhibiting improved performance and power consumption relative to existing platforms. The energy-efficient nature of the example configurable processor element arrays disclosed herein enables further use of machine learning accelerators in a wide range of applications such as facial recognition, speech recognition, navigation, market research, etc. The energy-efficient nature of the example configurable processor element arrays disclosed herein also enables the adoption of machine learning accelerators in applications where machine learning techniques cannot be utilized due to the relatively high power consumption exhibited by existing neural network hardware platforms, such as Internet of Things (IoT) applications, drone (e.g., unmanned vehicle) applications, etc.

[0028] Switch to the attached image. Figure 1 A block diagram of an example configurable processor element array 100 for implementing a convolutional neural network according to the teachings of this disclosure is shown. Figure 1 Example configurable processor element array 100 includes example PEs 105a-i arranged as an array comprising example rows 110a-c and example columns 115a-c, wherein each row in rows 110a-c has a first number of PEs and each column in columns 115a-c has a second number of PEs. The first number of PEs in rows 110a-c and the second number of PEs in columns 115a-c can be the same or different. In the example shown, the first number of PEs in rows 110a-c and the second number of PEs in columns 115a-c are the same and... Figure 1The value is marked as "N". For example, N can be 16 or some other value.

[0029] Figure 1 The example configurable processor element array 100 also includes an example configuration register 120, which may be implemented by, for example, one or more hardware registers, arrays, memory, data units, etc., or any combination thereof. Configuration register 120 configures the array of PE 105a-i to implement a given layer of the example convolutional neural network based on data flow scheduling. In the illustrated example, the data flow scheduling corresponds to one of a set of possible tensor processing templates supported by PE 105a-i. As disclosed in more detail below, configuration register 120 accepts a set of descriptors that configure some of the PE 105a-i to implement one of the possible tensor processing templates to operate on input activation data and filter data associated with a given layer of the convolutional neural network to produce output activation data associated with the given layer of the convolutional neural network. As disclosed in more detail below, configuration register 120 may accept a new set of descriptors to reconfigure the array of PE 105a-i to implement subsequent layers of the convolutional neural network. For example, the new set of descriptors may be the same as a previous set of descriptors applied to configuration register 120. By keeping the descriptors identical in such examples, some of the PEs 105a-i can be configured to implement the same tensor processing template as the previous neural network layer. In other examples, the new set of descriptors can differ from the previous set of descriptors applied to configuration register 120. By using different descriptors in such examples, some of the PEs 105a-i can be configured to implement another of the possible tensor processing templates to operate on the input activation data and filter data associated with subsequent layers of the convolutional neural network to produce output activation data associated with subsequent layers of the convolutional neural network. Thus, configuration register 120 is an example of a means for configuring an array of PEs 105a-i based on multiple descriptors to implement layers of a convolutional neural network based on a data flow schedule corresponding to one of the multiple tensor processing templates. Moreover, PEs 105a-i are examples of means for operating on the input activation data and filter data associated with layers of a convolutional neural network based on tensor processing templates to produce output activation data associated with layers of a convolutional neural network.

[0030] Figure 1The illustrated example includes an example configuration loader 122 that loads a set of descriptors into a configuration register 120. In some examples, the configuration loader 122 includes a compiler that converts descriptions of layers of a convolutional neural network to be implemented by the configurable processor element array 100 into a dataflow schedule corresponding to a selected template from a set of possible tensor processing templates. The compiler in such an example may utilize one or more criteria (such as, but not limited to, execution time, memory utilization, number of PEs to be activated, etc.) to select the tensor processing template to use for constructing the dataflow schedule. Additionally, the compiler in such an example may then convert the resulting dataflow schedule into a set of descriptors to be written into the configuration register 120. In some examples, the configuration loader 122 is composed of one or more processors (such as those combined below) Figure 27 The example processor 2712 shown in the example processor platform 2700 discussed herein is an implementation of this example. Thus, the configuration loader 122 is an example of a means for determining descriptors and / or writing / loading descriptors into the configuration register 120.

[0031] Figure 1 The example configurable processor element array 100 also includes an example memory 125 that stores input activation data, filter data, and output activation data associated with a given layer of the convolutional neural network being implemented in PE105a-i. In the illustrated example, memory 125 is implemented using a group of static random access memory (SRAM). However, in other examples, memory 125 may be implemented using other amounts and / or types of memory and / or one or more combinations thereof. Thus, memory 125 is an example of a means for storing input activation data, filter data, and output activation data associated with a layer of a convolutional neural network.

[0032] Figure 1 The example configurable processor element array 100 also includes an example tensor data allocation unit 130, which reads data from memory 125 and writes data to PE 105a-i. Based on a tensor processing template configured by a set of descriptors for a given neural network layer to be implemented by PE 105a-i, the tensor data allocation unit 130 also accepts data from PE 105a-i and stores the data in memory 125. An example implementation of the tensor data allocation unit 130 is described in U.S. Patent Application Serial No. 16 / 456,707, filed June 28, 2019, which is incorporated herein by reference.

[0033] Possible tensor processing templates provide different ways to decompose the overall tensor operations to be performed by the configurable processor element array 100 to implement a given neural network layer, so that the overall tensor operations can be implemented by a combination of PEs 105a-i included in the configurable processor element array 100. Figure 2 The diagram illustrates an example global tensor operation 200 to be performed by a configurable processor element array 100 to implement a given neural network layer. Figure 2 The examples illustrate the symbols that will be used throughout this disclosure.

[0034] Example tensor operation 200 corresponds to a neural network layer, where a set of input data 205 (also called input activation data 205 or input activation 205) is convolved with a set of filter kernels 210 (also called filter weights 210 or simply weights 210) to produce a set of output data 215 (also called output activation data 215 or output activation 215). In the example shown, the input activations 205 are arranged with I in the x-dimensional... x I of element y-dimensional y The elements and the input activation data I c An array of different channels. Dimension I x I y and I c They can be the same or different, and can be any one or more values. For example, if the neural network layer corresponding to tensor operation 200 is the input layer (e.g., the first layer) of an image processing neural network, then I x and I y The dimension can correspond to the number of pixels in the rows and columns of the input image, respectively, and I c Dimension can correspond to the number of channels in image data, such as the 3 channels of image data represented in red-blue-green (RGB) format. As another example, if the neural network layer corresponding to tensor operation 200 is an intermediate layer (e.g., the second layer) of an image processing neural network, then I... x and I y The dimension can correspond to the number of pixels in the rows and columns of the image being processed, respectively, and I c The dimension can correspond to the number of different filters that are convolved with the input activation data of the previous neural network layer, such as 64 filters or some other number of filters.

[0035] In the example shown, dimension I is processed using a set of filters 210. x multiply by I y multiply by I c The input activation data is 205. In the example shown, filter 210 is arranged with F in the x-dimensional... xEach element (e.g., weight), F in the y-dimensional dimension y Each element (e.g., weights) and I in the channel dimension c An array of elements, the latter being the number of channels I of the input activation data. c Same. For example, F x Dimensions and F y The dimensions can each correspond to 3 and 3, such that the 3x3 filter 210 is convolved with each input activation data element and its neighboring elements. Of course, the filter 210 can have parameters for F... x Dimensions and F y Other values ​​of the dimension, and F x Dimensions and F y Dimensions can be the same as or different from each other.

[0036] Figure 2 Example tensor operation 200 involves convolving each filter in filter 210 with input activation data 205, and summing (accumulating) the resulting data along the channel dimension (Ic) to produce output activation data 215. For example, a given filter 210a in the filter is convolved with a given portion 220 of the input activation data 205 centered on a given input activation data element 225. As shown, the results along each channel dimension are summed (e.g., corresponding to Ic). c (Accumulation in the x-dimensional direction) to produce output activation data element 230 at the array position corresponding to the array position of input activation data element 225. In the example shown, each filter in filter 210 convolves with the input activation data 205 to produce output activation data 215, which is arranged with O in the x-dimensional direction. x O in the y-dimension of an element y Each element and the output activation data O c An array of different channels. x Dimensions and O y Dimensions can have the same or different values, and can be related to I. x Dimensions and I y Dimensions are the same or different. c The dimension can correspond to the number of different filters 210 that are convolved with the input activation data 205.

[0037] Other terms used in this disclosure are as follows. n This refers to the batch size. For example, if a configurable processor element array 100 is to implement a convolutional neural network to process images, then O nThis refers to the number of images to be processed in parallel. The abbreviation "IF" refers to the input activation data, "FL" refers to the filter data (e.g., weights), and "OF" refers to the output activation data. Additionally, the term "Psum" refers to a portion of the result in the convolution operation, and is described in more detail below.

[0038] exist Figure 3 and Figure 4 The text shows the components to be generated by... Figure 1 The configurable processor element array 100 includes example tensor processing templates 305, 310, and 315 implemented by example PEs 105a-i, and corresponding example data flow schedules 405, 410, and 415. Tensor processing template 305 is an example of a vector-to-vector tensor processing template. Tensor processing template 310 is an example of a vector-to-matrix tensor processing template. Tensor processing template 315 is an example of a matrix-to-matrix tensor processing template. Data flow schedule 405 represents mapping a portion of example tensor operations implementing a given layer of an example convolutional neural network to one of PEs 105a-i according to vector-to-vector tensor processing template 305. Data flow schedule 410 represents mapping a portion of example tensor operations implementing a given layer of an example convolutional neural network to one of PEs 105a-i according to vector-to-matrix tensor processing template 310. Data flow schedule 415 represents mapping a portion of example tensor operations implementing a given layer of an example convolutional neural network to one of PEs 105a-i according to matrix-to-matrix tensor processing template 315. Example configurable processor element array 100 can support other tensor processing templates, such as scalar-vector processing templates.

[0039] Figure 3 and Figure 4 The example shown uses the symbol "DT" d / j / kThe symbol “DT” is used to define a specific tensor processing template. In this notation, “DT” refers to a data type that can be “I” for the input activation data to be processed by the PE according to the defined template, or “O” for the output activation data to be produced by the PE according to the defined template. The symbol “d” represents a dimension and can be “x”, “y”, or “c”. The symbol “j” represents the number of elements of data type “DT” on dimension “d” to be processed by a given PE according to the defined template. The symbol “k” represents the number of PEs involved in processing / producing elements of data type “DT” on dimension “d” to produce the total tensor operation output of a given neural network layer being implemented according to the defined template. In the example shown, the symbol “k” is set to a dash (-) when referring to the template of a single PE being applied to PE 105a-i. In the example shown, when a specific data type and / or dimension is omitted, it is assumed that the template specifies that the PE will process / produce one (1) element of that data type on that dimension.

[0040] For example, the tensor processing template 305 in the example shown is composed of the symbol "O x / 1 / -”O y / 1 / -" and "I c The " / 64 / -" definition specifies that the PE configured according to this template will process I c 64 IF data elements in dimension O x and O y OF data is generated at one (1) location in the dimension. The tensor processing template 310 of the example shown is composed of the symbol "O c / 8 / -" and "I c The " / 8 / -" definition specifies that the PE configured according to this template will process I c Eight (8) IF element data in dimension O x and O y A (1) position in dimension O c OF data is generated at eight (8) locations in the dimension. The tensor processing template 315 of the example shown is composed of the symbol "O x / 8 / -”I c / 8 / -” and “O” c The " / 8 / -" definition specifies that the PE configured according to this template will process in O x The eight (8) positions of the dimension and I c IF data elements at eight (8) positions in dimension O c The eight (8) positions of the dimension and O x and O y OF data is generated at one (1) location in the dimension.

[0041] As shown in example data flow scheduling 405, vector-to-vector tensor processing template 310 can be used to configure some of the PEs 105a-i to implement data flow scheduling corresponding to the portion of performing tensor operations, such as by multiplying vectors by vectors, in a given O x and O y The O is calculated by accumulating multiple (e.g., 64 in this example) filtered IF data elements (e.g., multiplied by the corresponding IF data) along the Ic dimension at the location. x and O y Scheduling of one data element at a given location. As shown in example data flow scheduling 410, vector-matrix tensor processing template 310 can be used to configure some of the PEs 105a-i to implement data flow scheduling corresponding to the portion of multiplying a vector and a matrix for performing tensor operations, such as by filtering the corresponding FL data from a first number of filters (e.g., 8 in this example) at a given O. x and O y To calculate the O, accumulate a second number (e.g., 8 in this example) of IF data elements along the Ic dimension at the location. x and O y The scheduling of the first number of data elements in the Oc dimension at the location. As shown in the example data flow scheduling 415, the matrix-matrix tensor processing template 315 can be used to configure some of the PE 105a-i to implement data flow scheduling corresponding to the matrix-matrix multiplication for performing tensor operations, such as by filtering the corresponding FL data from a third number of filters (e.g., 8 in this example) in O. x Each of the second number of positions in the dimension (e.g., 8 in this example) but at the same O y To calculate those O, accumulate the third number of IF data elements in the Ic dimension at the location. x Position and O y The scheduling of the first number (e.g., 8 in this example) of data elements in the Oct dimension at the location.

[0042] After mapping a specific data stream of a convolutional neural network layer to one of the possible tensor templates (e.g., by the compiler), it will be represented by the notation "DT". d The macro-level instruction represented by " / j / k" is broken down into several micro-instructions that can be processed by a given PE using a flexible PE pipeline. Figure 5 It shows the result of Figure 1An example operation pipeline 500 is implemented in the configurable processor element array 100 using PEs. Example pipeline 500 represents the decomposition of macro-granular instructions into multiple simpler micro-instructions, such as Config, Load, Compute, Accumulate, and Drain. In the illustrated example, the same set of micro-instructions can be used to implement different macro-instructions. For this purpose, fields within the micro-instructions change to accommodate different possible tensor processing templates (e.g., vector-to-vector tensor processing template, vector-to-matrix tensor processing template, matrix-to-matrix tensor processing template, scalar-to-vector tensor processing template, etc.).

[0043] return Figure 1 Example configurable processor element array 100 performs computations on IF, FL, and OF tensor data (and Psum tensor data as described below) based on a data flow schedule configured for the current layer of a convolutional neural network, wherein the data flow schedule is projected into one of a vector-to-vector tensor processing template, a vector-to-matrix tensor processing template, a matrix-to-matrix tensor processing template, or a scalar-to-vector tensor processing template. As described above, configurable processor element array 100 includes a configuration register 120 for accepting configurable descriptors that control the data flow corresponding to one of a set of possible tensor processing templates. Configurable processor element array 100 also includes an array of PEs 105a-i, arranged as an NxN grid of each PE 105a-i (e.g., where N=16 or some other value).

[0044] The configurable processor element array 100 of the illustrated example also includes column buffer stores 135a-c, which are used to buffer data between the SRAM bank of memory 125 and the local register file store within PE 105a-i, wherein each column buffer store in column buffer stores 135a-c is associated with a corresponding column in columns 115a-c of PE 105a-i. In the illustrated example, column buffers 135a-c also include various example output data processors 138a-c, which are capable of performing truncation and / or corrected linear unit (ReLU) operations on the data being output from PE 105a-i for storage in memory 135. The configurable processor element array 100 of the illustrated example includes example dedicated buses 140a-c, which are used to move IF, FL, and OF tensor data between the array of PE 105a-i and column buffers 135a-c, respectively.

[0045] As illustrated in the examples, each PE in PE 105a-i includes example register file (RF) local storage 145a-c for storing the IF, FL, and OF tensor data of that PE, respectively. Each PE in PE 105a-i also includes: an example multiplication and accumulation (MAC) unit 150 (which can be piped) for performing multiplication and accumulation operations on the IF and FL data to be processed by the PE; an example element-wise computation unit 155 for performing element-wise operations on the IF data to be processed by the PE; and an example max-pooling unit 160 with an example pool register 165 for performing max-pooling operations to produce OF tensor data associated with that PE. Each PE in PE 105a-i also includes one or more example configuration registers 170 and an example finite state machine (FSM) 175. FSM 175 manages the following: (i) loading IF and FL tensor data from RF storage 145a-b into different computation units 150-160 within PE; (ii) ordering computations within the respective computation units 150-160; (iii) providing control signals for scheduling the accumulation of partial sums within PE according to the configured data flow; (iv) providing control signals for transferring partial and OF tensor data to and from PE for accumulating partial sums across different processing iterations and / or across PE; and (v) extracting complete OF tensor data from PE into the SRAM buffer of memory 135 via column buffers 135a-c, wherein truncation and / or ReLU operations can be performed to reduce the OF tensor data from one size (e.g., 32-bit data) to a different (e.g., smaller) size (e.g., 8 bits) before storing it in the SRAM of memory 125 for the next layer of computation.

[0046] Table 1 below describes a set of descriptor fields for the example, which support flexible data flow scheduling by controlling the appropriate ordering of various computational stages of the input tensor data within PE 105a-i according to one or a set of possible tensor processing templates.

[0047] Table 1 The descriptor fields in Table 1 are applied to each PE included in the configurable processor element array 100. Thus, although each active PE in a PE will operate on different blocks of the total amount of IF and FL data for a given network layer, the amount of data operated by each active PE in a PE will be similar.

[0048] In Table 1, the stride descriptor field represents the parameters of the convolutional neural network. The IcPF descriptor field is the Ic partitioning factor, indicating how many PEs are working on a data partition in a given Ic dimension. Therefore, this field indicates how many PEs have partial sums that need to be accumulated along the Ic dimension. The PEColActv descriptor field indicates which columns 115a-c of PEs 105a-i are active in the configurable processor element array 100. The PERowActv descriptor field indicates which rows 110a-c of PEs 105a-i are active in the configurable processor element array 100. The OpPEColActv descriptor field indicates which of columns 115a-c will have the output of the currently implemented network layer. The OpPERowActv descriptor field indicates which of rows 110a-c will have the output of the currently implemented network layer. For example, the IcPF descriptor field indicates when the Ic dimension is partitioned across multiple PEs 105a-i. In this case, some of the PE 105a-i will only contribute to the output data, and the OpPEColActv and OpPERowActv descriptor fields indicate which PE105a-i will have the final output data after the partial sums are accumulated.

[0049] In Table 1, the TotalWrIFRF descriptor field indicates how many IF data points are to be written to PE 105a-i. The TotalWrFLRF descriptor field indicates how many FL data points are to be written to PE 105a-i. The TotalWrOFRF descriptor field indicates how many OF data points are to be written to PE 105a-i.

[0050] In Table 1, the StAddrIFRF descriptor field indicates the starting address of IF RF storage 145a. The LenAddrIFRF descriptor field indicates how many IF data points are accessed during a computation cycle. For example, consider tensor processing template 315, which has 8 filter channels (FLs) and each channel will process 8 IF data points on a different Ix dimension. The LenAddrIFRF descriptor field will indicate that each group of 8 IF data points will be processed by a different filter channel. The Reset2StartIF descriptor field indicates whether PE 105a-i will be reset to the starting address in IF RF storage 145a when the value of the LenAddrIFRF descriptor field is reached, or whether PE 105a-i should continue to increment through IF RF storage 145a. The IncCycIFRF descriptor field indicates the number of computation cycles after which the starting address of IF RF storage 145a will be incremented.

[0051] Similarly, the StAddrFLRF descriptor field indicates the starting address of FL RF storage 145b. The LenAddrFLRF descriptor field indicates how many FL data points to access during a computation cycle. The Reset2StartFL descriptor field indicates whether PE 105a-i will be reset to the starting address in FL RF storage 145b when the value of the LenAddrFLRF descriptor field is reached, or whether PE 105a-i should continue to increment through FL RF storage 145b. The IncCycFLRF descriptor field indicates the number of computation cycles after which the starting address of FL RF storage 145b will be incremented.

[0052] Similarly, the StAddrOFRF descriptor field indicates the starting address of OF RF storage 145c. The LenAddrOFRF descriptor field indicates how many OF data points to access during a computation cycle. The Reset2StartOF descriptor field indicates whether PE 105a-i will be reset to the starting address in OF RF storage 145c when the value of the LenAddrOFRF descriptor field is reached, or whether PE 105a-i should continue to increment via OF RF storage 145c. The IncCycOFRF descriptor field indicates the number of computation cycles after which the starting address of OF RF storage 145c will be incremented.

[0053] In Table 1, the BlocksPERF descriptor field indicates how many computational work blocks PE 105a-i performed, where one work block corresponds to computing one output point (or a partial sum associated with a given output point). The NumPEComp descriptor field indicates how many cycles are required to process the amount of data fed into PE 105a-i for processing according to the configured tensor processing template. For example, vector-to-vector tensor processing template 305 (which will utilize 64 FL data elements to process I...) c 64 IF data elements in dimension O x and O y (OF data is generated at one location in the dimension) will utilize 64 cycles, which corresponds to the time used to generate I c The 64 multiplication and summation operations involve multiplying 64 IF data elements with 64 FL data elements in a dimension and then summing the results.

[0054] In Table 1, the IcMapDirX descriptor field is a Boolean value (e.g., True or False) indicating whether to map the partition of the IC dimension across rows 110a-c of PE 105a-i. The IcMapDirY descriptor field is a Boolean value (e.g., True or False) indicating whether to map the partition of the IC dimension across columns 115a-c of PE 105a-i. These descriptor fields indicate how to share partial sums across PE 105a-i.

[0055] In Table 1, the NumIncStAddr descriptor field, IncStAddrPerBlockIFRF descriptor field, and StepIFRF descriptor field are used to specify how to move FL data with Fx and Fy dimensions on IF data to produce OF data.

[0056] In Table 1, the ExtPsum descriptor field is a Boolean value (e.g., True or False) that indicates whether the configured tensor processing template involves partial sums. If the value is False, each PE can run autonomously to output a given OF data point. If the value is True, partial sums will be used to generate OF data.

[0057] In Table 1, the OFGenStartNthBlock and PsumLoadStartNthBlock descriptor fields specify the number of times the configured tensor processing template is executed to generate the OF data points for the implemented neural network layer, as well as previously computed portions and when they will be reloaded for further accumulation. For example, if the current network layer has 256 Ic dimensions and the configured tensor processing template processes 64 Ic dimensions, the configured tensor processing template will be executed 4 times to process all 256 Ic dimensions to determine the OF data points for the current neural network layer.

[0058] In Table 1, the LinesPsumPerLoad descriptor field specifies the size of the Psum to be loaded to accumulate partial sums based on the configured tensor processing template (e.g., in the SRAM row). The LinesTotalPsum descriptor field specifies the number of Psums to be loaded to compute OF data points.

[0059] In Table 1, the ReLU descriptor field is a Boolean value (e.g., True or False) indicating whether the ReLU operation is active for the current neural network layer being implemented. The ReluThreshold descriptor field specifies the threshold to be used for the ReLU operation.

[0060] In Table 1, the EltWise descriptor field is a Boolean value (e.g., True or False) indicating whether the element-wise operation is active for the current neural network layer being implemented. The Drain2FLSRAM descriptor field is used in conjunction with the element-wise operation.

[0061] In Table 1, the Maxpool descriptor field is a Boolean value (e.g., True or False) that indicates whether the max pooling operation is active for the current neural network layer being implemented.

[0062] exist Figure 6 It shows Figure 1 A block diagram of one of the example implementations of PE 105a-i. For convenience, Figure 6 The block diagram illustrates an example implementation of PE 105a. However, Figure 6 The example implementation can be used to implement any PE in PE 105a-i. Figure 6 Example PE 105a includes a set of configuration registers 170 that accept values ​​of the descriptors shown in Table 1, which are updated at the beginning of each layer of the convolutional neural network being processed by PE 105a. In the example shown, a set of descriptor fields applied to configuration register 170 are programmed via configuration loader 122 to implement data flow scheduling based on tensor processing templates to process the IF and FL tensor data of the current layer (L) of the implemented convolutional neural network. For example, FSM 175 uses this set of programmed descriptor fields to perform data redirection during loading, computation, and dumping operations to be performed on the input tensor data. Thus, configuration registers 170 in each PE of PE 105a-i are examples of means for configuring an array of PE 105a-i based on multiple descriptors to implement layers of a convolutional neural network based on data flow scheduling corresponding to one of multiple tensor processing templates.

[0063] Figure 6Example PE 105a also includes FSM 175. In the example shown, FSM 175 includes an internal counter and logic for generating the following control signals: (i) read and write control signals for driving the IF, FL, and OF register files 145a-c; and (ii) multiplexer control signals for routing data from register files 145a-c to the appropriate one of MAC compute unit 150, element-wise compute unit 155, or max-pool compute unit 160 based on the type of operation to be performed on the tensor data of the current layer of the implemented convolutional neural network by PE 105a (e.g., multiplication and accumulation of MAC unit 150, comparison of max-pooling unit 160, etc.). In the example shown, to generate read and write control signals to the IF, FL, and OF register files 145a-c, FSM 170 uses the descriptor field "StAddr". <if fl of>RF”、"LenAddr <if fl of>RF”、"Reset2Start <if fl of>”、"IncCyc <if fl of>RF is used to generate relevant control signals. Internally, a counter... ifcount , wcount and ofcount Track the addresses / indices of the IF, FL, and OF register files 145a-c, which are determined by the number of input activations and weights required to compute each OF point (or pSum) during a computation block. <lenaddrif fl>The "RF" descriptor field is incremented or reset. The number of blocks (set by the "BlocksPERF" descriptor field) determines the total number of points (or pSum) to be written to OF register file 145c. The data flow for a given neural network layer (whether IF, FL, or OF static) is incremented or reset by the aforementioned counter and "Reset2Start". <if fl of>"The signals generated by the descriptor field are controlled internally." StAddr <if fl of>The "RF" descriptor field tracks the starting address of each register file in register files 145a-c for each new compute block. These internal structures and associated control logic included in the FSM 170 support flexible data flow scheduling in the PE 105a.

[0064] exist Figure 6 In the illustrated example, PE 105a includes example shared computing logic 605, which is shared among MAC computing unit 150, element-wise computing unit 155, and max-pooling computing unit 160 to achieve efficient reuse of hardware resources. Example shared computing logic 605 includes example multiplier 610, example adder 615, and example comparator 620, as well as associated example multiplexer control logics 625, 630, 635, 640, 645, 650, 655, and 660 (collectively referred to as multiplexer control logics 625-660), which route appropriate tensor data to one or more of elements 610-615 to implement processing by MAC computing unit 150, element-wise computing unit 155, or max-pooling computing unit 160. In the illustrated example, the default configuration of the multiplexer control logics 625-660 of shared computing logic 605 is to implement max-pooling computing unit 160. The descriptor fields "Eltwise" and "Maxpool" are used to reconfigure the shared computation logic 605 to implement the element-wise computation unit 155 and the max-pooling computation unit 160, respectively.

[0065] Figure 6 The example PE 105a includes RF local storage 145a-c. The example shown includes three RF 145a-cs for storing IF, FL, and OF tensor data, respectively. In the example shown, each of the RF 145a-cs is implemented using a set of 1-read-1-write registers that support simultaneous reading from and writing to one register. In the example shown, the tensor data stored in the IF and FL RF 145a-b is 8 bits wide (although other example implementations may support other widths), and the tensor data stored in the OF RF 145c is 32 bits wide (although other example implementations may support other widths) to accommodate the partial and accumulation characteristics of the data stream scheduling, where it is not possible to accumulate all input channels in a single processing iteration / block; therefore, partial sums are sent out from the PE 105a and sent back at a later point in time to complete the final OF tensor data computation.

[0066] At the output of IF RF 145a, example multiplexer logic 625 includes a 1:3 multiplexer that redirects IF tensor data to one of MAC computation unit 150, element-wise computation unit 155, or max-pooling computation unit 160. At the output of FL RF 145b, example multiplexer logic 630 includes a 1:2 multiplexer that redirects FL tensor data to one of MAC computation unit 150 or element-wise computation unit 155, because max-pooling computation unit 160 does not operate on the data contained in FL RF 145b. At the input of OF RF 145c, example multiplexer logic 635 includes a 1:2 multiplexer on the write path to OF RF 145c to store the output of one of MAC computation unit 150, element-wise computation unit 155, or max-pooling computation unit 160. Additional storage in the form of pool register 165 is used to store intermediate results of max-pooling computation unit 160.

[0067] Figure 6 Example PE 105a is configured to support both internal and external partial sum accumulation. PE 105a can accept partial sums from its neighboring PEs in the horizontal direction (pSumX) or vertical direction (pSumY). In some examples, PE 105a cannot accept partial sums from other PEs in other directions. The programmable descriptor field applied to configuration register 170 can be used to specify the direction of internal accumulation via example "accum_dir" signal 665. Example "accum_Nbr" control signal 670 is used to identify whether the partial sum accumulation is within PE 105a or among PEs including PE 105a and allowed neighboring PEs. For external partial sum accumulation, one set of values ​​is stored in "ext_pSum" register 675, while a second set of values ​​exists in OF RF 145c. Example multiplexer control signal "en_ext_pSum" 680 is used to select between internal and external partial sum accumulation.

[0068] Figure 7-12 It shows Figure 1 Examples include example operation phases supported by the configurable processor element array 100, and example allowed transitions between operation phases supported by the configurable processor element array 100. (Example...) Figure 7 As shown in the example state transition diagram 700, the example operation phases supported by the configurable processor element array 100 include an example configuration phase 705, an example loading phase 710, an example computation phase 715, an example accumulation phase 720, an example external part and accumulation phase 725, and an example retrieval phase 730 (also known as an example discharge phase 730). In the configuration phase 705 (where the example is in...), Figure 8 In the diagram (shown in more detail), descriptor values ​​for the configuration registers 120 (or, in some examples, stored in memory 125) of the configurable processor element array 100 applied to the current neural network layer being implemented (and subsequent neural network layers in some examples) are moved to the configuration registers 170 of PEs 105a-i, and the FSM 175 of PEs 105a-i is configured based on those descriptors. For example, descriptor values ​​are loaded into the configuration registers 170 of some PEs in PEs 105a-i, which will compute to one of the possible tensor processing template types (e.g., vector-vector, vector-matrix, matrix-matrix, scalar-vector, etc.).

[0069] In loading phase 710 (examples are in...) Figure 9 In the calculation phase 710 (examples of which are shown in more detail below), tensor data is loaded from memory 125 into RF 145a-c of PE 105a-i. For example, IF, FL, or OF tensor data is transferred from memory 125 to local RF storage 145a-c within some PEs of PE 105a-i via column buffer 135a-c. Figure 10 In (shown in more detail below), arithmetic operations (e.g., MAC, element-wise, or max pooling) are performed on the tensor data in RF 145a-c of some PEs residing in PE 105a-i. For example, some PEs in PE 105a-i can compute MAC operations to generate partial sum (Psum) or final OF tensor data for the currently implemented convolutional neural network layer. The internal accumulation stage 720 and the external partial sum accumulation stage 725 (examples of which are shown below) Figure 12 (Shown in more detail) are optional stages that may or may not exist for a given data stream schedule of the current network layer L configured to implement a convolutional neural network. In the example shown, the inner accumulation stage 720 corresponds to the inner accumulation stage, where the portions of adjacent PEs working on separate input channels of the same OF tensor data are accumulated. The accumulation direction is constrained to be horizontal or vertical. In the outer portion and accumulation stage 725, the portions of the sum computed earlier in time but which must be evicted from the local PE RF 145c are fed back to the PE for accumulation to generate the final OF tensor output. In the retrieval stage 730 (an example of which is shown in...), the retrieval stage... Figure 11 (shown in more detail below), some and / or final OF tensor data are transferred from the local PE RF 145c of some of the PEs in PE 105a-i to the respective column buffers 135a-c corresponding to those PEs in order to be moved into memory 125.

[0070] The permissible transitions between configuration phase 705, loading phase 710, calculation phase 715, internal accumulation phase 720, external part, accumulation phase 725, and retrieval phase 730 are determined by... Figure 7 The state transition diagram 700 is represented by directed lines. In the example state transition diagram 700 shown, the configuration phase 705, loading phase 710, computation phase 715, and retrieval phase 730 are mandatory, while the internal accumulation phase 720 and the external accumulation phase 725 depend on the specific data flow schedule being implemented. The example state transition diagram 700 begins with the configuration phase 705, in which the configuration register 120 of the configurable processor element array 100 is populated with descriptor fields, and then the configuration register 170 of each PE in PE 105a-i is populated with descriptor fields. Then, the process transitions to the loading phase 710, in which the IF and FL tensor data are moved from memory 125 to PE RF145a-b of each PE in PE 105a-i that is active for the currently implemented convolutional neural network layer.

[0071] In the example shown, a transition from loading phase 710 to computation phase 715 is permitted. From computation phase 715, processing can transition to any of the following: loading phase 710, computation phase 715, accumulation phase 720, outer part and accumulation phase 725, and retrieval phase 730. For example, processing can remain in computation phase 715 and continue computation, or processing can return to loading phase 710 to load new IF / FL tensor data into PEs 105a-i. This is typically the case when there is no Ic partition in the dataflow schedule of the currently implemented neural network layer. If there is an Ic partition in the dataflow schedule of the currently implemented neural network layer, then processing transitions from computation phase 715 to inner accumulation phase 720 or outer part and accumulation phase 725, depending on whether all Ic processes are partitioned between adjacent PEs 105a-i in the dataflow schedule of the current neural network layer or between different processing iterations performed by the same PE 105a-i.

[0072] If the final OF result is available during calculation phase 715, the process transitions to retrieval phase 730. In internal accumulation phase 720, once the final OF result is available, the process may transition to retrieval phase 730, or, if this is the last round of internal accumulation before the start of external accumulation phase 725, the process transitions to external accumulation phase 725. From external accumulation phase 725, the process may transition to loading phase 705 to retrieve additional portions and data from memory 125, or, once the final OF result is available, the process may transition to retrieval phase 730 to transfer the OF data to memory 125.

[0073] exist Figure 13A -B shows Figure 1 The example configurable processor element array 100 supports an example hardware architecture that supports external partial sum accumulation. In some dataflow scheduling, the accumulation of the filtered input channels (Ic) of the IF tensor data is not completed in a single processing iteration. Instead, a portion of the input channels is fed into the IF RF 145a of a given PE 105a-i, and the calculated partial sum is extracted into memory 125. Then, at a later point in time when the remaining input channels have been accumulated, the partial sum is fed back into the OF RF 145c of the given PE 105a-i. To maintain the accuracy of the final convolution result, the example configurable processor element array 100 does not perform truncation or ReLU on the partial sum data. For example, the partial sum data output as the MAC unit 150 of a given PE 105a-i has 32-bit precision (or some other precision in other examples). During normal operating modes (e.g., without partial sums), the loading and unloading data paths for each tensor data point have 8-bit precision (or some other precision in other examples). To support external partitioning and accumulation, the configurable processor element array 100 includes example bypass data paths 1305a-c, which support direct read and write access to partitions and data between column buffers 135a-c and memory 125 with the original precision of the partitions and data (32 bits in the illustrated example). Additionally, in the illustrated example, by bypassing the OF out-of-flight multiplexing logic 1310 included between column buffer 1305a and memory 125, a bypass data path for a given column buffer (such as bypass data path 1305a of column buffer 135a) divides the 32-bit wide data path into 1-byte blocks between column buffer 1305a and memory 125.

[0074] Back Figure 1 For example, although the input IF and FL tensor data are 8-bit precision (or some other precision in other examples), the output of the MAC within the PE is 32-bit precision (or some other, larger precision in other examples) to handle the accumulation and prevent precision loss. However, since the OF tensor data generated by a given neural network layer (L) is used as the IF tensor data for a subsequent neural network layer (L+1), the configurable processor element array 100 includes example output data processors 138a-c associated with corresponding column buffers 135a-c. Output data processors 138a-c perform a truncation operation to adjust the bit precision of the accumulated OF tensor data values ​​to 8 bits before writing them to memory 125. Furthermore, if a ReLU operation is to be performed by a given neural network layer, the output data processors 138a-c perform the ReLU operation, which results in bit precision adjustment to generate the final OF tensor data. Thus, before writing data to the SRAM buffer, the output data processor 138a-c applies saturation truncation or ReLU to the 32-bit OF tensor data output from the corresponding column buffer 135a-c. The ReLU threshold used by the output data processor 138a-c can also be adjusted via the "ReluThreshold" descriptor in Table 1.

[0075] exist Figure 14 It shows Figure 1 The example provided is an example hardware architecture supporting element-wise operations in the configurable processor element array 100. Some residual neural networks, such as ResNet, employ element-wise operations, such as adding tensor data elements from two convolutional layers of the neural network. To support element-wise operations while leveraging hardware resource reuse, the configurable processor element array 100 routes OF tensor data elements from two different layers to a given PE in PEs 105a-i by reusing existing load and discharge paths. For example, the configurable processor element array 100 routes OF tensor data from the first layer in the layer to IF RF 145a of a given PE 105a-i, and routes OF tensor data from the second layer in the layer to FL RF 145b of a given PE 105a-i. Thus, IF RF 145a and FL RF 145b will contain OF tensor data from two different layers. The "Eltwise" programmable descriptor field in Table 1 is set to "True" to indicate that element-wise operations are activated, and the eltwise enable signal is used to bypass the MAC operation within a given PE 105a-i, instead performing element-wise operations (e.g., addition or maximum) on the first OF tensor data stored in IF RF 145a and the second OF tensor data stored in FL RF 145b.

[0076] exist Figure 15 It shows Figure 1 The example hardware architecture supporting max-pooling operation is provided in the configurable processor element array 100. Max-pooling operation is widely used in many deep neural networks (DNNs) to prune the size of the generated feature maps. To support max-pooling operation, the configurable processor element array 100 also reuses load and dump paths so that the OF data of the network layer to be max-pooled is stored in the IF RF 145a of a given PE 105a-i. The pool register 165 of the given PE105a-i is used to keep track of the current maximum value, and subsequent OF points of the layer to be max-pooled will be compared with the current maximum value.

[0077] Figure 16-25 An example use case is shown, in which a configurable processor element array 100 is configured to operate according to four (4) different data stream schedules to implement layers of a residual neural network such as ResNet. Figure 16-19 Example pseudocode representing different dataflow scheduling implemented by the configurable processor element array 100 in these examples is shown. As described in more detail below, the four (4) different dataflow scheduling shown in these examples are based on corresponding four (4) different tensor processing templates. In the following examples, the array of PE 105a-i included in the configurable processor element array 100 is assumed to be N x N = 16 x 16, which totals 256 PE 105a-i. However, these and other example use cases can be implemented with arrays of PE 105a-i with different dimensions.

[0078] Figure 16 Example pseudocode for the first example data stream scheduling 1600 is shown, which processes IF tensor data and FL tensor data to produce OF tensor data for example layers of a residual neural network. Figure 16 In the example shown, the volume of the IF tensor data to be processed has 56 elements in the Ix dimension, 56 elements in the Iy dimension, and 64 elements in the Ic dimension. The volume of the tensor data to be produced has 56 elements in the Ox dimension, 56 elements in the Oy dimension, and 256 elements in the Oc dimension, corresponding to the 256 different filters (FL data) to be applied to the IF tensor data. The example data stream scheduling includes example internal processing loop 1605, which maps 8 partitions of 1-element Ox data and 2 partitions of 32-element Ic data to rows 16 (110a-c) of the PE 105a-i array, and maps 14 partitions of 2-element Oy data to columns 14 (115a-c) of the PE 105a-i array. Therefore, each PE 105a-i in the 16 x 14 section of the PE 105a-i array takes one (1) point of Ox, two points of Oy, and 32 input channel (Ic) points, and generates a partial sum for two (2) OF points belonging to one (1) output channel (Oc). Thus, each PE 105a-i processes 64 IF points for 32 different Ics and 32 FL points for 32 different Ics, while generating two (2) different OF points belonging to a single Oc. Note that since the Ic partition factor along PE column 115a-c is two (2), this means that two (2) PEs in adjacent rows 110a-c are working to generate the final OF point at that position in the OF output data volume. Therefore, the internal summation of the partial sums of two (2) PE 105a-i across adjacent rows 110a-c is used to generate the final OF point at that position in the OF output data. This results in eight (8) PEs producing the final OF points within a given column 115a-c of the array of PEs 105a-i, and a total of 112 PEs 106a-i (8 x 14 columns per column), which produce the final OF points obtained from the inner processing loop 1605. Thus, the inner loop 1605 produces an OF data volume with eight (8) elements in the Ox dimension, 28 elements in the Oy dimension, and one (1) element in the Oc dimension. The example data stream schedule includes an example outer processing loop 1610, which performs 256 iterations in the Oc dimension, seven (7) iterations in the Ox dimension, and two (2) iterations in the Oy dimension, producing a final OF data volume of 56x56x256 OF points. Since the IF data is reused by the outer loop 1610, the data stream 1600 is input activation stationary. Since the data stream accumulates Ic data elements on the same Oc dimension, data stream 1600 corresponds to the vector-vector tensor processing template.

[0079] exist Figure 20 The text vividly depicts Figure 16 The example data stream scheduling 1600 includes example data partitioning and chunking. Figure 21 The text vividly depicts the efforts to achieve... Figure 16 The example data stream schedules 1600 and the example convolution operation is performed by the PE 105a-i array. Figure 22 The example values ​​for the configuration descriptors shown in Table 1 can be used to configure the example configurable processor element array 100 to implement... Figure 16 Example data stream scheduling 1600.

[0080] Figure 17 Example pseudocode for a second example data stream scheduling 1700 is shown, which processes IF tensor data and FL tensor data to produce OF tensor data for example layers of a residual neural network. Figure 17 In the example shown, the volume of the IF tensor data to be processed has 28 elements in the Ix dimension, 28 elements in the Iy dimension, and 128 elements in the Ic dimension, and the volume of the OF tensor data to be generated has 28 elements in the Ox dimension, 28 elements in the Oy dimension, and 512 elements in the Oc dimension, which correspond to 512 different filters (FL data) to be applied to the IF tensor data. The example data stream schedule 1700 includes an example internal processing loop 1705, which maps 16 partitions of 8-element Oc data and 16 partitions of 8-element Ic data to 16 rows 110a-c and 16 columns 115a-c of the array of PE 105a-i, respectively. Each PE 105a-i takes eight (8) input channel (Ic) points and eight (8) output channel (Oc) points to generate eight (8) OF data points. Therefore, each PE 105a-i operates on eight (8) IF data points of eight (8) different Ics and 64 FL points to be applied to the eight (8) different Ic data points to produce eight (8) different Oc data points. Thus, the inner loop 1705 produces an OF data volume with one (1) element in the Ox dimension, one (1) element in the Oy dimension, and 8 x 16 = 128 elements in the Oc dimension. The example data flow schedule includes an example outer processing loop 1710, which performs 28 iterations in the Ox dimension, 28 iterations in the Oy dimension, and four (4) iterations in the Oc dimension. Since the 16 partitions of the IC data map to 16 columns 115a-c, the final OF data is determined by accumulation along the PE row direction (e.g., PE(i, 15) for "i=0 to 15"), and the OF data is extracted from the last PE column 115c. Because the FL data is reused by outer loop iterations on the Oy and Ox dimensions, example data stream schedule 1700 is weighted stationary. Furthermore, because data stream 1700 accumulates IC data across different OC dimensions, data stream 1700 corresponds to a vector-matrix tensor processing template. Figure 23 The example values ​​for the configuration descriptors shown in Table 1 can be used to configure the example configurable processor element array 100 to implement... Figure 17 Example data flow scheduling 1700.

[0081] Figure 18 Example pseudocode for a third example data stream schedule 1800 is shown, which processes IF tensor data and FL tensor data to generate OF tensor data for example layers of a residual neural network. Example data stream 1800 includes an example internal processing loop 1805 that maps two (2) partitions of 8-element Ic data and eight (8) partitions of 1-element Ox data along columns 115a-c of the array of PEs 105a-i, and maps 16 partitions of 8-element Oc data along rows 110a-c of the array of PEs 105a-i. Thus, each PE processes 1x7x8 volume of OF data by processing 7x8 volume of IF data and 8x8 volume of FL points, generating 56 partial and OF data points. Example dataflow 1800 also includes example outer processing loop 1810, in which, after each interval of 32 iterations in the Ic dimension, the partial sums of two (2) adjacent PE105a-i along the horizontal direction are internally accumulated to generate the final OF data point. Since new IF and FL data points are fed into PE 105a-i (Ic in the outer loop) in each iteration, and the partial sums are fixed in PE 105a-i, dataflow scheduling 1800 is output activation stationary. In addition, because dataflow 1800 performs accumulation on IC data points in different Ox and different OC dimensions, dataflow 1800 corresponds to a matrix-matrix tensor processing template. Figure 24 The example values ​​for the configuration descriptors shown in Table 1 can be used to configure the example configurable processor element array 100 to implement... Figure 18 Example data stream scheduling 1800.

[0082] Figure 19 Example pseudocode for the fourth example data stream schedule 1900 is shown, which processes IF tensor data and FL tensor data to produce OF tensor data for example layers of residual neural networks. Data stream schedule 1900 is tailored for neural network layers using 3x3 filters (while the other example data streams 1600-1800 correspond to neural network layers using 1x1 filters). Example data stream 1900 includes example inner processing loop 1905, which maps 14 partitions of 4-element Oy data along columns 115a-c in the array of PE 105a-i, and maps eight (8) partitions of 1-element Ox data and two (2) partitions of 16-element Oc data along rows 110a-c in the array of PE 105a-i. Therefore, each PE 105a-i processes 1x4x16 volume data and consumes 18 IF data points (since the weight dimension is 3x3, producing 1x4 volume OF data involves 3x6 volume IF data, corresponding to 18 IF points) and 16 FL data points to produce 64 partial sums. Example data stream 1900 also includes example outer processing loop 1910, in which the final OF points are generated when all nine (9) FL data points (corresponding to 3x3 filters) and 64 Ic data points have been accumulated within a given PE 105a-i. Since Ic exists in outer processing loop 1910, data stream scheduling 1900 is an example of output activation fixed scheduling. In addition, because data stream 1900 feeds filter points one after another, and each computation involves multiplying a scalar (filter) by multiple input activation points, data stream 1900 corresponds to a scalar-vector tensor processing template. Figure 25 The example values ​​for the configuration descriptors shown in Table 1 can be used to configure the example configurable processor element array 100 to implement... Figure 19 Example data flow scheduling 1900.

[0083] Despite Figure 1-25 The diagram illustrates an example of how the configurable processor element array 100 can be implemented, but it can be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Figure 1-25 One or more of the elements, processes, and / or devices shown. Additionally, it can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Figure 1-25 Examples of example PE 105a-i, one or more example configuration registers 120, example memory 125, example tensor data allocation unit 130, example column buffer storage 135a-c, example output data processor 138a-c, example bus 140a-c, example RF storage 145a-c, example MAC unit 150, example element-wise computation unit 155, example max pooling unit 160, example pool register 165, one or more example configuration registers 170, example FSM 175, example shared computation logic 605, example multiplier 610, example adder 615, example comparator 620, example multiplexer control logic 625-660, example register 675, and / or more generally example configurable processor element array 100. Therefore, for example, the example PE 105a-i, one or more example configuration registers 120, example memory 125, example tensor data allocation unit 130, example column buffer storage 135a-c, example output data processor 138a-c, example bus 140a-c, example RF storage 145a-c, example MAC unit 150, example element-wise computation unit 155, example max pooling unit 160, example pool register 165, one or more example configuration registers 170, and example FSM can be implemented by one or more analog or digital circuits, logic circuits, one or more programmable processors, one or more programmable controllers, one or more graphics processing units (GPUs), one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more programmable logic devices (PLDs), one or more field-programmable gate arrays (FPGAs) and / or one or more field-programmable logic devices (FPLDs). 175, example shared computational logic 605, example multiplier 610, example adder 615, example comparator 620, example multiplexer control logic 625-660, example register 675 and / or any of the more general example configurable processor element array 100.When reading any device or system claim covering purely software and / or firmware implementations of this patent, examples of configurable processor element array 100, example PE 105a-i, one or more example configuration registers 120, example memory 125, example tensor data allocation unit 130, example column buffer storage 135a-c, example output data processor 138a-c, example bus 140a-c, example RF storage 145a-c, example MAC unit 150, example element-wise computation unit 155, example max pooling unit 160, example pool register 165, one or more example configuration registers 170, and example FSM are included. 175, at least one of the following: example shared computing logic 605, example multiplier 610, example adder 615, example comparator 620, example multiplexer control logic 625-660, and / or example register 675, is explicitly defined as including a non-transitory computer-readable storage device or disk containing the software and / or firmware, such as memory, digital universal disc (DVD), optical disc (CD), Blu-ray disc, etc. Additionally, the example configurable processor element array 100 may include, except... Figure 1-25 Those other than or in place of those shown Figure 1-25 The elements, processes, and / or devices shown may include one or more of the elements, processes, and devices shown, and / or may include more than one of any or all of the elements, processes, and devices shown. As used herein, the phrase "in communication" (including variations thereof) includes direct communication and / or indirect communication via one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or continuous communication, but additionally includes selective communication at periodic intervals, scheduled intervals, non-periodic intervals, and / or one-off events.

[0084] exist Figure 26 The diagram illustrates example hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof used to implement an example configurable processor element array 100. In these examples, machine-readable instructions can be those provided to a computer processor (such as those combined below). Figure 27 The processor 2712 shown in the example processor platform 2700 discussed in this study executes one or more executable programs or one or more portions of executable programs. These one or more programs or portions thereof may be embodied in a non-transitory computer-readable storage medium (such as a CD-ROM, floppy disk, hard disk drive, DVD, Blu-ray disk). TM The software is located on a Blu-ray disc or in memory associated with the processor 2712, but all or one or more programs and / or portions thereof may alternatively be executed by a device other than the processor 2712 and / or embodied in firmware or dedicated hardware. Additionally, although references... Figure 26 The flowchart shown describes one or more example programs, but many other methods for implementing the example configurable processor element array 100 can be used instead. For example, refer to... Figure 26 The flowchart shown can change the execution order of blocks, and / or can modify, eliminate, combine some of the described blocks and / or subdivide them into multiple blocks. Additionally or alternatively, any or all blocks can be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware.

[0085] The machine-readable instructions described herein can be stored in one or more of the following formats: compressed, encrypted, segmented, packaged, etc. The machine-readable instructions described herein can be stored as data (e.g., portions of instructions, code, code representations, etc.) that can be used to create, manufacture, and / or produce machine-executable instructions. For example, machine-readable instructions can be segmented and stored on one or more storage devices and / or computing devices (e.g., servers). Machine-readable instructions may require one or more of the following to be installed, modified, adapted, updated, combined, supplemented, configured, decrypted, decompressed, unpacked, distributed, and redistributed so that they can be directly read and / or executed by computing devices and / or other machines. For example, machine-readable instructions can be stored in multiple parts that are individually compressed, encrypted, and stored on separate computing devices, wherein these parts, when decrypted, decompressed, and combined, form a set of executable instructions that implement programs such as those described herein. In another example, machine-readable instructions can be stored in a state where they are readable by a computer but require the addition of libraries (e.g., dynamic link libraries), software development kits (SDKs), application programming interfaces (APIs), etc., to execute these instructions on a specific computing device or other device. In another example, it may be necessary to configure the machine-readable instructions (e.g., settings are stored, data is entered, network addresses are logged, etc.) before they can be executed in whole or in part. Therefore, the disclosed machine-readable instructions and / or one or more corresponding programs are intended to include such machine-readable instructions and / or one or more programs regardless of their specific format or state when stored or otherwise in a static or transmissive state.

[0086] In another example, machine-readable instructions may be stored in a state where they are readable by a computer, but require the addition of libraries (e.g., dynamic link libraries), software development kits (SDKs), application programming interfaces (APIs), etc., to execute these instructions on a specific computing device or other device. In another example, the machine-readable instructions may need to be configured (e.g., settings are stored, data is entered, network addresses are logged, etc.) before they can be executed, wholly or partially, of the machine-readable instructions and / or one or more corresponding programs. Therefore, the disclosed machine-readable instructions and / or one or more corresponding programs are intended to include such machine-readable instructions and / or one or more programs, regardless of their specific format or state when stored or otherwise in a static or transceived state.

[0087] The machine-readable instructions described in this article can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, machine-readable instructions can be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, Hypertext Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0088] As described above, executable instructions (e.g., computer and / or machine-readable instructions) stored on non-transitory computer and / or machine-readable media (such as hard disk drives, flash memory, read-only memory, optical disks, digital universal disks, caches, random access memory, and / or any other storage device or disk that stores information for any duration (e.g., for an extended period of time, permanently, for a short period of time, temporarily buffered, and / or cached)) can be used to implement Figure 26 Example process. As used herein, the term non-transitory computer-readable medium is explicitly defined as including any type of computer-readable storage device and / or storage disk and excluding propagation signals and transmission media. Furthermore, as used herein, the terms "computer-readable" and "machine-readable" are considered equivalent unless otherwise stated.

[0089] "Comprising" and "including" (and all their forms and tenses) are used herein as open-ended terms. Therefore, whenever a claim uses any form of "comprising" or "including" (e.g., including, comprising, including, including, having, etc.) as a preamble or in any kind of claim statement, it will be understood that additional elements, terms, etc., may be present that do not fall outside the scope of the corresponding claim or statement. As used herein, when the phrase "at least" is used as a transitional term, for example, in the preamble of a claim, it is as open-ended as the terms "comprising" and "including". The terms "and / or" when used, for example, in the form of "A, B, and / or C", refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, and (7) A and B and C. As used herein in the context of describing structures, components, projects, objects, and / or things, the phrase "at least one of A and B" is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, projects, objects, and / or things, the phrase "at least one of A or B" is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the conduct or execution of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A and B" is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing the conduct or execution of processes, instructions, actions, activities and / or steps, the phrase "at least one of A or B" is intended to refer to an implementation including any one of: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0090] As used herein, singular references (e.g., "a," "an," "first," "second," etc.) do not exclude multiple entities. As used herein, the term "a" or "an" entity refers to one or more of that entity. The terms "a" (or "an"), "one or more," and "at least one" are used interchangeably herein. Furthermore, although listed separately, multiple means, elements, or method actions may be implemented by, for example, a single unit or processor. Moreover, although individual features may be included in different examples or claims, these features may be combined, and inclusion in different examples or claims does not imply that a combination of features is not feasible and / or advantageous.

[0091] Depend on Figure 26 The flowchart shown illustrates example program 2600, which can be executed to operate... Figure 1 The example allows configuring the processor element array 100 to implement layers of a convolutional neural network. Referring to the foregoing figures and associated written description, Figure 26 Example program 2600 begins execution at block 2605, where configuration loader 122 executes instructions (e.g., compiler, software, etc.) to load input data (IF data) and filter data (FL data) corresponding to the convolutional neural network to be implemented by configurable processor element array 100 into memory 125 of configurable processor element array 100. At block 2610, configuration loader 122 executes instructions (e.g., compiler, software, etc.) to write descriptors to configuration register 120 to configure configurable processor element array 100 to implement the first layer of the convolutional neural network based on a given data flow schedule corresponding to one of the possible tensor processing templates, as described above. Thus, block 2610 corresponds to an example of configuration phase 705 described above. At block 2615, PEs 105a-i of configurable processor element array 100 load descriptor values ​​into their respective configuration registers 170. Thus, block 2615 corresponds to an example of loading phase 710 described above. At block 2620, as described above, PE 105a-i of the configurable processor element array 100 performs computational operations on the input data and filter data corresponding to the current neural network layer according to the configured descriptor. As described above, the computational operations performed at block 2620 may include, for example, MAC operations, element-wise operations, max-pooling operations, inner part-and-accumulate operations, outer part-and-accumulate operations, etc. Thus, block 2620 corresponds to an example of computation stage 715, accumulation stage 720, and / or outer part-and-accumulate stage 725 described above. At block 2625, as described above, PE 105a-i stores the output data (OF data) determined in block 2620 for the current neural network layer in the memory of the configurable processor element array 100. Thus, block 2625 corresponds to an example of retrieval stage 730 described above.

[0092] At box 2630, configuration loader 122 executes instructions (e.g., compiler, software, etc.) to determine whether to implement another layer (e.g., a second layer) of the neural network. If another neural network layer is to be implemented ("Yes" at box 2630), control returns to box 2610, where configuration loader 122 executes instructions (e.g., compiler, software, etc.) to write another set of descriptors to configuration register 120 to configure configurable processor element array 100 to implement the next (e.g., second) layer of the convolutional neural network based on a given data flow schedule corresponding to one of the possible tensor processing templates, as described above. As described above, the tensor processing template configured by configuration loader 122 at box 2610 for the next (e.g., second) layer of the convolutional neural network and the resulting associated data flow schedule can be the same as or different from the tensor processing template configured for the first layer of the convolutional neural network during the previous iteration at box 2610. The control then moves on to box 2615 and subsequent boxes to implement the next (e.g., second) layer of the convolutional neural network.

[0093] However, if no other neural network layers are to be implemented ("No" at box 2630), then at box 2635, the configurable processor element array 100 causes its PEs 105a-i to perform any final partial and cumulative operations (see, for example, example dataflow schedules 1600-1900 described above), and then writes the final output data (OF data) into the memory 125 of the configurable processor element array 100. Example program 2600 then terminates.

[0094] Figure 27 This is a block diagram of an example processor platform 2700, which is configured to execute... Figure 26 Instructions to achieve Figure 1-25 Configurable processor element array 100. Processor platform 2700 can be, for example: a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., cellular phone, smartphone, such as iPad). TM Tablet computers, personal digital assistants (PDAs), internet devices, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, headphones or other wearable devices, or any other type of computing device.

[0095] The processor platform 2700 shown in the example includes a processor 2712. The processor 2712 in the example shown is hardware. For example, the processor 2712 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor 2712 can be a semiconductor-based (e.g., silicon-based) device. In the example shown, the hardware processor 2712 is implemented... Figure 1 Configuration loader 122.

[0096] The processor 2712 of the illustrated example includes local memory 2713 (e.g., cache). The processor 2712 of the illustrated example communicates with main memory, including volatile memory 2714 and non-volatile memory 2716, via link 2718. Link 2718 may be implemented by a bus, one or more point-to-point connections, or a combination thereof. Volatile memory 2714 may be implemented by synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of random access memory device. Non-volatile memory 2716 may be implemented by flash memory and / or any other desired type of memory device. Access to main memory 2714 and 2716 is controlled by a memory controller.

[0097] The processor platform 2700 shown in the example also includes interface circuitry 2720. Interface circuitry 2720 can be implemented using any type of interface standard, such as an Ethernet interface, Universal Serial Bus (USB), Bluetooth® interface, Near Field Communication (NFC) interface, and / or PCI Fast Interface.

[0098] In the example shown, one or more input devices 2722 are connected to interface circuitry 2720. The one or more input devices 2722 allow a user to input data and / or commands into processor 2712. One or more input devices can be implemented using, for example, an audio sensor, microphone, camera (still or video), keyboard, buttons, mouse, touchscreen, trackpad, trackball, tracking bar (such as isopoints), speech recognition system, and / or any other human-machine interface. Additionally, many systems, such as processor platform 2700, can allow users to control the computer system and provide data to the computer using physical gestures (such as, but not limited to, hand or body movements, facial expressions, and facial recognition).

[0099] The processor platform 2700 also includes a configurable processor element array 100, which communicates with other elements of the processor platform 2700 via a link 2718. For example, the configurable processor element array 100 can obtain input IF data from one or more of the input devices 2722 via interface circuitry 2720, implement layers of a convolutional neural network as described above to process the input IF data, and output the resulting OF data to output device 2724 via interface circuitry 2720.

[0100] One or more output devices 2724 are also connected to the interface circuitry 2720 of the example shown. The output devices 2724 may be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), cathode ray tube displays (CRTs), in-situ switching (IPS) displays, touchscreens, etc.), haptic output devices, printers, and / or one or more speakers. Therefore, the interface circuitry 2720 of the example shown typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0101] The interface circuitry 2720 in the example shown also includes communication devices (such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces) to facilitate data exchange with external machines (e.g., any kind of computing device) via network 2726. Communication can be via, for example, Ethernet connections, digital subscriber line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, line-to-line wireless systems, cellular telephone systems, etc.

[0102] The processor platform 2700 shown in the example also includes one or more mass storage devices 2728 for storing software and / or data. Examples of such mass storage devices 2728 include floppy disk drives, hard disk drives, optical disk drives, Blu-ray disc drives, redundant array of independent disks (RAID) systems, and digital universal disc (DVD) drives. In some examples, one or more mass storage devices 2728 implement the memory 125 of the configurable processor element array 100. Alternatively, in some examples, volatile memory 2714 implements the memory 125 of the configurable processor element array 100.

[0103] and Figure 26 The corresponding machine-executable instructions 2732 can be stored in a mass storage device 2728, a volatile memory 2714, a non-volatile memory 2716, a local memory 2713, and / or on a removable non-transitory computer-readable storage medium (such as a CD or DVD 2736).

[0104] As will be appreciated from the preceding description, example configurable processor element arrays for implementing convolutional neural networks have been disclosed. The disclosed configurable processor element arrays provide a low-cost, programmable deep neural network (DNN) hardware solution that supports flexible data flow scheduling mapping by mapping the data flow of a given neural network layer to one of vector-vector, vector-matrix, matrix-matrix, or scalar-vector macroinstruction tensor processing templates. The disclosed configurable processor element arrays can provide flexibility similar to that of FPGAs while maintaining the energy efficiency of ASIC hardware accelerators. Moreover, the disclosed configurable processor element arrays are not limited to specific register files or memory sizes or arrangements, and therefore can be adopted in a wide range of machine learning accelerator designs. Furthermore, the disclosed configurable processor element arrays can be used to develop DNN accelerators that utilize energy efficiency derived from data reuse. The disclosed configurable processor element arrays thus represent one or more improvements in the functionality of computer technology.

[0105] The foregoing disclosure provides example solutions for implementing convolutional neural networks using the disclosed array of configurable processor elements. Further examples are disclosed herein, including subjects such as: an apparatus for implementing a convolutional neural network; a non-transitory computer-readable medium including instructions that, when executed, cause at least one processor to configure the apparatus to implement a convolutional neural network; and a method for configuring the apparatus to implement a convolutional neural network. The disclosed examples can be implemented individually and / or in one or more combinations.

[0106] Example 1 is an apparatus for implementing a convolutional neural network. The apparatus of Example 1 includes an array of processor elements comprising rows and columns, each row having a first number of processor elements and each column having a second number of processor elements. The apparatus of Example 1 also includes configuration registers for storing a plurality of descriptors for configuring the array of processor elements to implement layers of the convolutional neural network based on a data flow schedule corresponding to one of a plurality of tensor processing templates. Some of the processor elements are configured, based on the descriptors, to implement one of the plurality of tensor processing templates to operate on input activation data and filter data associated with the layers of the convolutional neural network to produce output activation data associated with the layers of the convolutional neural network. The apparatus of Example 1 also includes memory for storing the input activation data, filter data, and output activation data associated with the layers of the convolutional neural network.

[0107] Example 2 includes the subject of Example 1, wherein the layer is the first layer of a convolutional neural network, the plurality of descriptors is a first plurality of descriptors, one of the plurality of tensor processing templates is the first of the plurality of tensor processing templates, and the configuration register is reconfigurable to store a second plurality of descriptors for configuring an array of processor elements to implement a second layer of the convolutional neural network based on a second data flow schedule corresponding to the second of the plurality of tensor processing templates, the second of the plurality of tensor processing templates being different from the first of the plurality of tensor processing templates.

[0108] Example 3 includes the subject of Example 2, wherein multiple tensor processing templates include vector-to-vector templates, vector-to-matrix templates, and matrix-to-matrix templates.

[0109] Example 4 includes the subject of any one of Examples 1 to 3, wherein the first processor element in the array of processor elements includes: (i) an input activation register file for storing first input activation data to be processed by the first processor element; (ii) a filter register file for storing first filter data to be processed by the first processor element; (iii) an output activation register file for storing first output activation data to be generated by the first processor element based on the first input activation data and the first filter data; and (iv) a finite state machine for controlling the operation of the first processor element to implement one of a plurality of tensor processing templates.

[0110] Example 5 includes the subject of Example 4, wherein the configuration register is a first configuration register, and the first processor element further includes a second configuration register for storing at least some of the descriptors and for configuring a finite state machine.

[0111] Example 6 includes the subject of Example 4, wherein the first processor element further includes: (i) a multiplication and accumulation unit for performing multiplication and accumulation operations on the first input activation data and the first filter data; (ii) an element-wise computation unit for performing element-wise operations on the first input activation data; (iii) a max-pooling unit for performing a max-pooling operation to produce the first output activation data; and (iv) control logic that can be configured by a finite state machine to control the operations of the multiplication and accumulation unit, the element-wise operation unit, and the max-pooling unit.

[0112] Example 7 includes the subject of any one of Examples 1 through 6, where the first quantity equals the second quantity.

[0113] Example 8 includes the subject of any of Examples 1 through 7, and also includes a processor for executing computer instructions to write multiple descriptors into configuration registers.

[0114] Example 9 includes the subject of Example 8, wherein the layer is the first layer of a convolutional neural network, the plurality of descriptors is a first plurality of descriptors, one of the plurality of tensor processing templates is the first of the plurality of tensor processing templates, and the processor writes a second plurality of descriptors to a configuration register, the second plurality of descriptors being used to configure an array of processor elements to implement a second layer of the convolutional neural network based on a second data flow schedule corresponding to the second of the plurality of tensor processing templates, the second of the plurality of tensor processing templates being different from the first of the plurality of tensor processing templates.

[0115] Example 10 is a non-transitory computer-readable medium comprising computer-readable instructions that, when executed, cause at least one processor to perform at least the following operations: (i) writing a first set of descriptors to a configuration register to configure an array of processor elements to implement a first layer of a convolutional neural network based on a first data flow schedule corresponding to the first of a plurality of tensor processing templates, the array comprising rows and columns, each row having a first number of processor elements and each column having a second number of processor elements, the first set of descriptors being used to configure some of the processor elements to implement the first of a plurality of tensor processing templates to implement a first layer of a convolutional neural network. (ii) The input activation data and filter data associated with the first layer of the network are manipulated to produce output activation data associated with the first layer of the convolutional neural network; and (ii) a second set of descriptors is written to a configuration register to configure an array of processor elements to implement the second layer of the convolutional neural network based on a second data flow schedule corresponding to the second of a plurality of tensor processing templates, the second set of descriptors being used to configure some of the processor elements to implement the second of a plurality of tensor processing templates to manipulate the input activation data and filter data associated with the second layer of the convolutional neural network to produce output activation data associated with the second layer of the convolutional neural network.

[0116] Example 11 includes the subject of Example 10, wherein the second of the multiple tensor processing templates is different from the first of the multiple tensor processing templates.

[0117] Example 12 includes the subject of Example 11, wherein multiple tensor processing templates include vector-to-vector templates, vector-to-matrix templates, and matrix-to-matrix templates.

[0118] Example 13 includes the subject of any one of Examples 10 to 12, wherein the instructions, when executed, also cause at least one processor to write a third set of descriptors to a configuration register to configure an array of processor elements to implement a third layer of a convolutional neural network. The third set of descriptors is used to configure some of the processor elements to perform at least one of an element-wise operation or a max-pooling operation.

[0119] Example 14 includes the subject of any one of Examples 10 to 13, where the first quantity equals the second quantity.

[0120] Example 15 is a method for implementing a convolutional neural network. The method of Example 15 includes: writing a first set of descriptors to a configuration register by executing instructions with at least one processor to configure an array of processor elements to implement a first layer of the convolutional neural network based on a first data flow schedule corresponding to the first of a plurality of tensor processing templates. The array includes rows and columns, each row having a first number of processor elements and each column having a second number of processor elements. The first set of descriptors is used to configure some of the processor elements to implement the first of the plurality of tensor processing templates to operate on input activation data and filter data associated with the first layer of the convolutional neural network to produce output activation data associated with the first layer of the convolutional neural network. The method of Example 15 further includes: writing a second set of descriptors into a configuration register by executing instructions with at least one processor to configure an array of processor elements to implement a second layer of a convolutional neural network based on a second data flow schedule corresponding to the second of a plurality of tensor processing templates, the second set of descriptors being used to configure some of the processor elements to implement the second of the plurality of tensor processing templates to operate on input activation data and filter data associated with the second layer of the convolutional neural network to produce output activation data associated with the second layer of the convolutional neural network.

[0121] Example 16 includes the subject of Example 15, wherein the second of the multiple tensor processing templates is different from the first of the multiple tensor processing templates.

[0122] Example 17 includes the subject of Example 16, wherein multiple tensor processing templates include vector-to-vector templates, vector-to-matrix templates, and matrix-to-matrix templates.

[0123] Example 18 includes the subject of any of Examples 15 through 17, and further includes writing a third set of descriptors to a configuration register to configure an array of processor elements to implement a third layer of a convolutional neural network. The third set of descriptors is used to configure some of the processor elements to perform at least one of an element-wise operation or a max-pooling operation.

[0124] Example 19 includes the subject of any one of Examples 15 through 18, where the first quantity equals the second quantity.

[0125] Example 20 is an apparatus for implementing a convolutional neural network. The apparatus of Example 20 includes an array of processor elements, the array comprising rows and columns, each row having a first number of processor elements and each column having a second number of processor elements. The apparatus of Example 20 also includes means for configuring the array of processor elements based on a plurality of descriptors to implement a layer of the convolutional neural network based on a data flow schedule corresponding to one of a plurality of tensor processing templates. The descriptors are used to configure some of the processor elements to implement one of the plurality of tensor processing templates to operate on input activation data and filter data associated with the layer of the convolutional neural network to produce output activation data associated with the layer of the convolutional neural network. The apparatus of Example 20 also includes means for storing the input activation data, filter data, and output activation data associated with the layer of the convolutional neural network.

[0126] Example 21 includes the subject of Example 20, wherein the layer is a first layer of a convolutional neural network, the plurality of descriptors are a first plurality of descriptors, one of the plurality of tensor processing templates is the first of the plurality of tensor processing templates, and the configuration means will configure an array of processor elements based on a second plurality of descriptors to implement a second layer of the convolutional neural network based on a second data flow schedule corresponding to the second of the plurality of tensor processing templates, the second of the plurality of tensor processing templates being different from the first of the plurality of tensor processing templates.

[0127] Example 22 includes the subject of Example 21, wherein multiple tensor processing templates include vector-to-vector templates, vector-to-matrix templates, and matrix-to-matrix templates.

[0128] Example 23 includes the subject of any one of Examples 20 to 22, where the first quantity equals the second quantity.

[0129] Example 24 includes the subject of any of Examples 20 to 23, and also includes means for loading multiple descriptors into an array for configuring processor elements.

[0130] Example 25 includes the subject of Example 24, wherein the layer is a first layer of a convolutional neural network, the plurality of descriptors is a first plurality of descriptors, one of the plurality of tensor processing templates is the first of the plurality of tensor processing templates, and the means for loading will load a second plurality of descriptors into the means for configuration to configure an array of processor elements to implement a second layer of the convolutional neural network based on a second data flow schedule corresponding to the second of the plurality of tensor processing templates, the second of the plurality of tensor processing templates being different from the first of the plurality of tensor processing templates.

[0131] Although certain example methods, apparatuses, and articles of manufacture have been disclosed herein, the scope of this patent is not limited thereto. Rather, this patent covers all methods, apparatuses, and articles of manufacture that fairly fall within the scope of the claims of this patent.< / if> < / if> < / lenaddrif> < / if> < / if> < / if> < / if>

Claims

1. One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations including: For the first layer of the neural network, multiple descriptors are generated, and for the second layer of the neural network, multiple other descriptors are generated. The first layer has a first input tensor, and the second layer has a second input tensor. The processor circuitry is configured using the plurality of descriptors to execute the first layer by performing a plurality of operations on a first set of sub-tensors, wherein the first set of sub-tensors has different portions of the first input tensor; and The processor circuitry is configured using the plurality of other descriptors to execute the second layer by performing a plurality of other operations on the second set of sub-tensors, wherein the second set of sub-tensors has different portions of the second input tensor. The subtensors in the first group of subtensors have different shapes from the subtensors in the second group of subtensors.

2. The one or more non-transitory computer-readable media according to claim 1, wherein, The first or second layer is a convolutional layer.

3. One or more non-transitory computer-readable media according to claim 2, wherein, Configuring the processor circuitry using the plurality of descriptors includes: The plurality of descriptors are stored in one or more registers or one or more memories of the processor circuit.

4. One or more non-transitory computer-readable media according to claim 1, wherein, The subtensor in the first group of subtensors or the subtensor in the second group of subtensors is a multidimensional tensor.

5. One or more non-transitory computer-readable media according to claim 1, wherein, The processor circuit includes one or more multiply-accumulate units.

6. One or more non-transitory computer-readable media according to claim 1, wherein, The processor circuitry is configured to read the first set of subtensors from memory based on the plurality of descriptors, or to read the second set of subtensors from memory based on the plurality of other descriptors.

7. One or more non-transitory computer-readable media according to claim 1, wherein, The second input tensor is the output tensor of the first layer of the neural network.

8. A calculation method, comprising: For the first layer of the neural network, multiple descriptors are generated, and for the second layer of the neural network, multiple other descriptors are generated. The first layer has a first input tensor, and the second layer has a second input tensor. The processor circuitry is configured using the plurality of descriptors to execute the first layer by performing a plurality of operations on a first set of sub-tensors, wherein the first set of sub-tensors has different portions of the first input tensor; and The processor circuitry is configured using the plurality of other descriptors to execute the second layer by performing a plurality of other operations on the second set of sub-tensors, wherein the second set of sub-tensors has different portions of the second input tensor. The subtensors in the first group of subtensors have different shapes from the subtensors in the second group of subtensors.

9. The calculation method according to claim 8, wherein, The first or second layer is a convolutional layer.

10. The calculation method according to claim 9, wherein, Configuring the processor circuitry using the plurality of descriptors includes: The plurality of descriptors are stored in one or more registers or one or more memories of the processor circuit.

11. The calculation method according to claim 8, wherein, The subtensor in the first group of subtensors or the subtensor in the second group of subtensors is a multidimensional tensor.

12. The calculation method according to claim 8, wherein, The processor circuit includes one or more multiply-accumulate units.

13. The calculation method according to claim 8, wherein, The processor circuitry is configured to read the first set of subtensors from memory based on the plurality of descriptors, or to read the second set of subtensors from memory based on the plurality of other descriptors.

14. The calculation method according to claim 8, wherein, The second input tensor is the output tensor of the first layer of the neural network.

15. A computing system, comprising: The memory stores instructions; as well as One or more processors, The instruction is responsive to execution by the one or more processors, causing the one or more processors to perform the computation method according to any one of claims 8 to 14.

16. A computing device comprising means for performing the computing method according to any one of claims 8 to 14.

17. A computer program product comprising instructions that, in response to execution by one or more processors, cause the one or more processors to perform the computation method according to any one of claims 8 to 14.

Images