Reconfigurable hardware buffers in neural network accelerator framework

Abstract

Embodiments of the present disclosure relate to reconfigurable hardware buffers in a neural network accelerator framework. A convolution accelerator framework (CAF) has a plurality of processing circuits including one or more convolution accelerators, a reconfigurable hardware buffer configurable to store data of a variable number of input data lanes, and a flow switch coupled to the plurality of processing circuits. The reconfigurable hardware buffer has a memory and control circuitry. The number of the variable number of input data lanes is associated with an execution epoch. During processing of the execution epoch, the flow switch streams data of the variable number of input data lanes between a processing circuit of the plurality of processing circuits and the reconfigurable hardware buffer. The control circuitry of the reconfigurable hardware buffer configures the memory to store the data of the variable number of input data lanes, the configuration including allocating a portion of the memory to each of the variable number of input data lanes.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to neural networks. More specifically, but not exclusively, the present disclosure relates to utilization of hardware resources in a neural network accelerator framework. BACKGROUND

[0002] Known computer vision, speech recognition, and signal processing applications benefit from the use of convolutional neural networks (CNNs). A CNN is a computer-based tool that processes large amounts of data and adaptively "learns" by merging the closest related features in the data to make broad predictions about the data and improve predictions based on reliable conclusions and new merges. The CNN is arranged in multiple "layers," and different types of predictions are made on each layer.

[0003] For example, if multiple two-dimensional pictures of a human face are provided as input to a CNN, the CNN will learn various features of the face, such as edges, curves, angles, points, color contrasts, highlights, dark spots, etc. These one or more features are learned at one or more first layers of the CNN. Then, at one or more second layers, the CNN will learn various identifiable features of the face, such as eyes, eyebrows, forehead, hair, nose, mouth, cheeks, etc.; each identifiable feature can be distinguished from all other features. That is, the CNN learns to identify and distinguish an eye from a brow or any other facial feature. At one or more third and subsequent layers, the CNN learns whole human faces and higher-order features, such as race, gender, age, emotional state, etc. In some cases, the CNN can even be taught to identify a specific identity of a person. For example, a random image can be identified as a human face, and the human face can be identified as Orlando Bloom, Andrea Bocelli, or some other identity.

[0004] In other examples, a CNN can be provided with multiple pictures of animals and taught to identify lions, tigers, and bears; the CNN can be provided with multiple pictures of cars and taught to identify and distinguish different types of vehicles; many other CNNs can also be formed and trained. CNNs can be used to learn word patterns in sentences, identify music, analyze personal shopping patterns, play video games, create traffic routes, and CNNs can also be used for many other learning-based tasks. SUMMARY

[0005] Techniques and systems for implementing convolutional neural networks are described herein.

[0006] In an embodiment, a convolution accelerator framework (CAF) includes a plurality of processing circuits, a reconfigurable hardware buffer, and a flow switch. The plurality of processing circuits includes one or more convolution accelerators, the reconfigurable hardware buffer is configurable to store data for a variable number of input data lanes, and the flow switch is coupled to the plurality of processing circuits. The reconfigurable hardware buffer has a memory and control circuitry. The number of the variable number of input data lanes is associated with an execution epoch. In operation, during processing of the execution epoch, the flow switch streams data for the variable number of input data lanes between a processing circuit of the plurality of processing circuits and the reconfigurable hardware buffer, and the control circuitry of the reconfigurable hardware buffer configures the memory to store data for the variable number of input data lanes during processing of the execution epoch, the configuring including allocating a portion of the memory to each of the variable number of input data lanes.

[0007] In an embodiment, the memory includes an input buffer memory, a mirror buffer, and a random access memory (RAM), and allocating a portion of the memory to an input data lane of the variable number of input data lanes includes determining a size of an input buffer of the input buffer memory to allocate to the input data lane, determining a word size of the mirror buffer, and determining a size of a sub-RAM of the RAM to allocate to the input data lane. In an embodiment, the word size of the mirror buffer is a function of the size of the sub-RAM allocated to the lane. In an embodiment, the sub-RAM is configured as a first-in-first-out memory. In an embodiment, the control circuitry of the reconfigurable hardware buffer reconfigures the memory in operation between processing of a first execution epoch and processing of a second execution epoch. In an embodiment, the variable number is an element of the set of numbers {0, 1, 2, 3, 4}.

[0008] In an embodiment, the reconfigurable hardware buffer includes an output interface, and the control circuitry, in operation, configures the output interface to provide a variable number of output data lanes. In an embodiment, configuring the output interface, in operation, to provide an output data lane of the variable number of output data lanes includes providing a mirrored output of an input data lane of the variable number of input data lanes. In an embodiment, configuring the output interface, in operation, to provide an output data lane of the variable number of output data lanes includes providing a shaped output of an input data lane of the variable number of input data lanes. In an embodiment, in operation, the shaped output is provided as an input to a convolution accelerator of the one or more convolution accelerators via the flow switch, where the convolution accelerator, in operation, performs a dilated convolution operation of an execution epoch using the shaped output. In an embodiment, in operation, the variable number of input data lanes is equal to the variable number of output data lanes. In an embodiment, in operation, the variable number of input data lanes is less than the variable number of output data lanes. In an embodiment, the control circuitry includes control logic and a plurality of counters, and, in operation, the control logic uses the counters to track storage of data of the variable number of input data lanes in the memory and control provision of the stored data to the variable number of output data lanes.

[0009] In an embodiment, a system for implementing a neural network (NN), such as a convolutional neural network (CNN), includes a host processor, a main memory, and an accelerator framework (AF), such as a convolutional accelerator framework (CAF). The convolutional accelerator framework includes a plurality of processing circuits, a reconfigurable hardware buffer, and a flow switch. The plurality of processing circuits includes one or more accelerators, such as one or more convolution accelerators. The reconfigurable hardware buffer is configurable to store data of a variable number of input data lanes, the reconfigurable hardware buffer having a memory and control circuitry. The flow switch is coupled to the plurality of processing circuits and the reconfigurable hardware buffer. In operation, a number of the variable number of input data lanes is associated with an execution epoch of the CNN; during processing of the execution epoch, the flow switch streams data of the variable number of input data lanes between a processing circuit of the plurality of processing circuits and the reconfigurable hardware buffer; and the control circuitry of the reconfigurable hardware buffer configures the memory of the reconfigurable hardware buffer to store the data of the variable number of input data lanes during processing of the execution epoch, the configuring including allocating a portion of the memory of the reconfigurable hardware buffer to each input data lane of the variable number of input data lanes.

[0010] In an embodiment, the memory of the reconfigurable hardware buffer includes input buffer memory, mirror buffer, and random access memory (RAM), and allocating a portion of the memory of the reconfigurable hardware buffer to an input data lane of a variable number of input data lanes includes determining a size of an input buffer of the input buffer memory allocated to the input data lane, determining a word size of the mirror buffer, and determining a size of a sub-RAM of the RAM allocated to the input data lane. In an embodiment, the size of the mirror buffer allocated to the input data lane is a function of the determined size of the sub-RAM. In an embodiment, the reconfigurable hardware buffer includes an output interface, and the control circuitry, in operation, configures the output interface to provide a variable number of output data lanes. In an embodiment, configuring the output interface, in operation, to provide an output data lane of a variable number of output data lanes includes providing a shaped output of an input data lane of a variable number of input data lanes. In an embodiment, the system includes an integrated circuit including a host processor, a main memory, and a convolution accelerator framework.

[0011] In an embodiment, a method includes convolver accelerator framework to process an execution epoch of a convolutional neural network, the convolver accelerator framework having a plurality of processing circuits, a flow switch, and a reconfigurable hardware buffer. The processing of the execution epoch includes configuring a variable number of input data lanes of the reconfigurable hardware buffer of the convolver accelerator framework, the number of the variable number of input data lanes being associated with the execution epoch, the configuring including allocating a portion of a memory of the reconfigurable hardware buffer to each of the variable number of input data lanes; streaming data of the variable number of input data lanes between processing circuits of the plurality of processing circuits and the reconfigurable hardware buffer via the flow switch; and storing the data of the variable number of input data lanes in the memory of the reconfigurable hardware buffer based on the allocation of the portion of the memory to the variable number of input data lanes. In an embodiment, the memory includes an input buffer memory, a mirror buffer, and a random access memory (RAM), and allocating the portion of the memory to an input data lane of the variable number of input data lanes includes determining a size of an input buffer of the input buffer memory for allocation to the input data lane, determining a word size of the mirror buffer, and determining a size of a sub-RAM of the RAM for allocation to the input data lane. In an embodiment, the method includes reconfiguring the memory of the reconfigurable hardware buffer between the processing of a first execution epoch and the processing of a second execution epoch. In an embodiment, the reconfigurable hardware buffer includes an output interface, and the method includes configuring the output interface to provide a variable number of output data lanes. In an embodiment, configuring the output interface to provide an output data lane of the variable number of output data lanes in operation includes providing a shaped output of an input data lane of the variable number of input data lanes. In an embodiment, the method includes using a counter of the reconfigurable hardware buffer to track storage of the data of the variable number of input data lanes in the memory and to control provision of the stored data to the variable number of output data lanes.

[0012] In an embodiment, content of a non-transitory computer-readable medium causes one or more hardware processors of a convolution accelerator framework to perform a method comprising: processing an execution epoch of a convolutional neural network, the processing of the execution epoch comprising: configuring a variable number of input data lanes of a reconfigurable hardware buffer of the convolution accelerator framework, the number of the variable number of input data lanes being associated with the execution epoch, the configuring comprising allocating a portion of a memory of the reconfigurable hardware buffer to each of the variable number of input data lanes; streaming data of the variable number of input data lanes between processing circuits of a plurality of processing circuits and the reconfigurable hardware buffer via a stream switch of the convolution accelerator framework; and storing the data of the variable number of input data lanes in the memory of the reconfigurable hardware buffer based on the allocation of the portion of the memory to the variable number of input data lanes. In an embodiment, the memory comprises an input buffer memory, a mirror buffer, and a random access memory (RAM), and allocating the portion of the memory to an input data lane of the variable number of input data lanes comprises determining a size of an input buffer of the input buffer memory for allocation to the input data lane, determining a word size of the mirror buffer, and determining a size of a sub-RAM of the RAM for allocation to the input data lane. In an embodiment, the reconfigurable hardware buffer comprises an output interface, and the method comprises configuring the output interface to provide a variable number of output data lanes. In an embodiment, configuring the output interface to provide an output data lane of the variable number of output data lanes in operation comprises providing a shaped output of an input data lane of the variable number of input data lanes. In an embodiment, the content comprises instructions executed by the one or more processors of the convolution accelerator framework.

[0013] The tools and methods discussed in this disclosure set forth one or more aspects and embodiments of a convolution accelerator in which a memory used as a reconfigurable hardware buffer can be used to store partial data and reduce the likelihood of concurrency and deadlock issues.

[0014] The innovations described in this disclosure are new and useful and are not, as of this time, known, conventional or customary in the silicon manufacturing industry. Some parts of the innovations described herein can create something more than the known building blocks used in a new and useful way to combine other structures and limitations. Embodiments improve known computing systems that, when unprogrammed or differently programmed, do not perform or provide the particular reconfigurable features claimed herein.

[0015] The embodiments described herein use computerized techniques to improve the technology of convolution accelerators, but other techniques and tools are still available to provide reconfigurable interconnects. Thus, the claimed subject matter does not exclude all or any substantial part of the field of silicon fabrication or reconfigurable interconnect technology.

[0016] These features, as well as others, and attendant advantages, will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings, in which:

[0017] This Summary is provided to introduce certain concepts in a simplified form that are further described below in the DETAILED DESCRIPTION. This Summary does not identify key or essential features of the claimed subject matter and neither is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 depicts a process by which a convolutional neural network is used to classify an input image via operations on successive feature maps using convolutions, subsampling, and fully connected layers;

[0019] Figure 2 is a functional block diagram of an embodiment of an electronic device or system implementing an artificial neural network (ANN) employing one or more convolution accelerators in accordance with one or more embodiments disclosed herein;

[0020] Figure 3 is a functional block diagram of an embodiment of an electronic device or system implementing an artificial neural network employing one or more convolution accelerators in accordance with one or more embodiments disclosed herein;

[0021] Figure 4 is a conceptual diagram used to illustrate the organization of data along three spatial dimensions into execution epochs and channel data blocks;

[0022] Figure 5 is a conceptual diagram used to illustrate the data to be considered in a convolution for various example dilution rates d;

[0023] Figure 6 is a conceptual diagram used to illustrate possible deadlock and data loss situations that can arise during various operations performed by a convolutional neural network;

[0024] Figure 7 is a conceptual diagram used to illustrate the storage of intermediate accumulation results in system memory during processing of a CNN execution epoch on data;

[0025] Figure 8is a conceptual diagram for illustrating an example of storing intermediate accumulation results in reconfigurable hardware buffers during processing of a data execution phase by a CNN;

[0026] Figure 9 is a conceptual diagram for illustrating a fork and join in a dataflow chain;

[0027] Figure 10 is a conceptual diagram for illustrating a fork and join in a dataflow chain, where reconfigurable hardware buffers are employed to buffer data in the dataflow chain;

[0028] Figure 11 is a functional block diagram of an embodiment of a convolution accelerator framework including reconfigurable hardware buffers;

[0029] Figure 12 and Figure 13 is a conceptual diagram for illustrating a dataflow process that solves concurrency issues and simplifies execution phases by using reconfigurable hardware buffers;

[0030] Figure 14 is a conceptual diagram for illustrating an example buffer set for storing partial data output from an early process of a dataflow for use in a subsequent process;

[0031] Figure 15 is a conceptual diagram for illustrating an example buffer set for storing partial data output from an early operation of a dataflow for use in a subsequent operation, and corresponding shaping circuitry for shaping the data;

[0032] Figure 16 is a functional block diagram of an embodiment of a reconfigurable hardware buffer that can be employed in a hardware accelerator framework;

[0033] Figure 17 is a conceptual diagram for illustrating an example of determining a word size of a mirror buffer of a reconfigurable hardware buffer during a data execution phase;

[0034] Figure 18 is a conceptual diagram for illustrating data in a mirror buffer, and an output counter and sub-RAM allocated to a data lane associated with an operation;

[0035] Figure 19 is a conceptual diagram for illustrating an embodiment of a sub-RAM in a first-in-first-out (FIFO) buffer;

[0036] Figure 20 is a conceptual diagram for illustrating an example of determining a size of an output sub-RAM for allocation to a data output lane associated with a subsequent operation;

[0037] Figure 21is a conceptual diagram illustrating an example of reorganizing data stored in a mirror buffer and one or more output sub-RAMs to provide a shaped output, such as an output that has had a dilution factor applied; and

[0038] Figure 22 Embodiments of a method of processing an execution phase of a CNN using a convolution accelerator framework with reconfigurable hardware buffers are shown. DETAILED DESCRIPTION

[0039] In the following description, certain specific details are set forth in order to provide a thorough understanding of various embodiments of devices, systems, methods, and articles of manufacture. However, those skilled in the art will understand that other embodiments can be practiced without some or all of these details. In other instances, well known structures and methods have not been described in detail in order to avoid unnecessarily obscuring the description of the embodiments.

[0040] Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise," "comprising," and the like are to be construed in an open, inclusive sense, as

[0041] References to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" or "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment or embodiments, nor are they necessarily referring to the same technical feature, structure, or characteristic. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0042] The headings provided herein are merely for convenience and do not interpret the scope or meaning of the disclosure.

[0043] The size and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes of various elements and angles are not drawn to scale, some of which can be exaggerated, and positions are not necessarily drawn to scale to make the drawings easier to understand. Also, the particular shapes of the elements as drawn are not intended to convey any information regarding the actual shape of the particular elements, and have been selected solely for the ease of recognition in the drawings.

[0044] A convolutional neural network (CNN) is a type of deep neural network (DNN) having one or more layers each performing a convolution on a 3-dimensional (3D) tensor of feature data (denoted as width x height x depth). Typically, the convolution operation is associated with the bulk of the processing effort, often performing a large number of multiply-accumulate (MAC) operations per inference.

[0045] Specialized convolution accelerators aim to more efficiently process the convolution operations, such as by exploiting a higher level of data parallelism than standard processor cores. Many CNNs also include fully connected (FC) layers, where the classical 3D convolution is transformed into a vector by a matrix operation on a 1 x 1 x depth tensor of feature data.

[0046] Figure 1 A process is depicted in which an input image 101 is classified using a CNN by alternately operating on successive feature maps to derive a classification output for the input image through a fully connected layer. In particular, a plurality of first feature maps 105a are generated from the input image 101 by a first convolutional layer using kernels 110. A plurality of second feature maps 105b are then generated by a first subsampling layer of the CNN, which are in turn used to generate a larger plurality of third feature maps 105c by a second convolutional layer of the CNN. In the depicted embodiment, a second subsampling layer of the CNN generates another larger plurality of fourth feature maps 105d, which are accepted as input by a fully connected layer of the CNN to generate an N-dimensional output classification vector 120. The CNN can be used to classify acoustic scenes, recognize activities based on sensor data, classify heartbeat signals, recognize faces or facial expressions, etc., for example.

[0047] Figure 2 is a functional block diagram of an embodiment of an electronic device or system 200 to which described embodiments can be applied. The system 200 includes one or more processing cores or circuitry 202. The processing core 202 can include, for example, one or more processors, state machines, microprocessors, programmable logic circuitry, discrete circuitry, logic gates, registers, etc., and various combinations thereof. The processing core can control overall operation of the system 200, execution of applications by the system 200, etc.

[0048] The system 200 includes one or more memories, such as one or more volatile and / or non-volatile storage devices, which can store, for example, all or part of instructions and data relating to control of the system 200, applications, and operations performed by the system 200, etc. As shown, the system 200 includes one or more cache memories 204, one or more main memories 206, and one or more secondary memories 208, each including memory management circuitry or MMU 210.

[0049] The system 200 can include one or more sensors 220 (e.g., accelerometers, pressure sensors, temperature sensors, etc.), one or more interfaces 230 (e.g., wireless communication interfaces, wired communication interfaces, etc.), one or more artificial neural network (ANN) circuits (such as a convolutional neural network (CNN) accelerator circuit 240 as shown, and other circuitry 250 which can include other functional circuitry, antennas, power supplies, etc.), and a main bus system 260. The main bus system 260 can include one or more data, address, power and / or control buses that couple to the various components of the system 200. The system 200 can also include additional bus systems, such as a bus system 262 that communicatively couples the cache memory 204 and the processing core 202, a bus system 264 that communicatively couples the cache memory 204 and the main memory 206, a bus system 266 that communicatively couples the main memory 206 and the processing core 202, and a bus system 268 that communicatively couples the main memory 206 and the secondary memory 208.

[0050] Figure 3 is a functional block diagram of an embodiment of an electronic device or system 300 to which the described embodiments can be applied. The system 300 can be used, for example, to implement an ANN to classify sensor data. Note that Figure 2 embodiments of Figure 3 embodiments, and can include a system on a chip (SoC).

[0051] The system 300 includes a global memory 302 that can be used, for example, as a main memory for ANN 303 processes or clusters and host system 304 processes or clusters (see Figure 1 main memory 206 of the described embodiments). The global memory 302 includes memory management circuitry 306 and one or more shared memory arrays 308. The memory management circuitry 306 employs one or more memory management routines in operation to allocate regions of the global memory 302, such as regions of the shared memory arrays 308 to various processes executed by the system 300, and to control access to the global memory 302, such as access to regions of the shared memory arrays 308.

[0052] As shown, the ANN 303 includes one or more data movers 310, one or more memory bridges 320, one or more sensors 330 and corresponding sensor interfaces 332, one or more convolution accelerator engines 340, and one or more fully connected engines 350, which can be implemented and operated in other conventional manners to produce a classification output 360. As shown, the convolution accelerator engines 340 include a plurality of multiply-accumulate (MAC) circuits in a MAC cluster 342.

[0053] In operation, the data mover 310 moves data streams between the IO (e.g., sensor interface 332), the memory hierarchy (e.g., global memory 302, memory bridge 320), the convolution accelerator 340, and the fully connected engine 350.

[0054] In embodiments, the convolution accelerator engine 340 can include an array of clustered multiply-accumulate (MAC) units or circuits 342 that, in operation, implement convolution layer processing in the ANN. The convolution accelerator engine can include a row buffer for fetching multiple feature map data words in parallel with a single memory access. A register-based kernel buffer providing multiple read ports can be used, while multiple multi-bit fixed-point MAC units or circuits perform multiple MAC operations per clock cycle. A tree of adders can be employed to sum the MAC results for each kernel column.

[0055] In some embodiments, the system 200 or system 300 can include more components than illustrated, can include fewer components than illustrated, can separate illustrated components into separate components, can combine illustrated components, etc., as well as various combinations of the above. For example, in some embodiments, the main memory 206 and the secondary memory 208 can be combined into a single memory. In another example, in some embodiments, the convolution accelerator engine 340 and the fully connected engine 350 can be combined.

[0056] Convolution layers typically perform convolution operations between inputs and convolution kernels, non-linear activation functions (such as rectifiers), and max-pooling operations, which are typically the most computationally demanding. In practice, due to the large number of operations to be performed, these operations cannot typically be completed at once. Operations can be developed in several stages, and partial data can be organized accordingly.

[0057] The amount of data to be processed can be divided using a channel data partitioning strategy, and execution epochs are employed to organize processing to subdivide the overall operation into smaller groups in terms of time and space. A streaming memory access concept can be employed. For example, data to be processed can be organized in execution epochs and channel data blocks along three spatial dimensions, as conceptually described in Figure 4 The data blocks can be created taking into account the number of channels and the size of the incoming data. Elements can be written to memory, for example, in the order of depth (z), width (x), and height (y). Various parameters and definitions can be employed to indicate data to be processed in an execution epoch. Partial data resulting from different data blocks can be reorganized and used for subsequent processing, such as processing of subsequent execution epochs.

[0058] Various convolution operations can be employed, such as a dilated convolution operation (also referred to as a void convolution), in which a dilation rate d determines the spacing between the two values to be considered in the convolution operation. The dilated convolution operation can be employed to increase the considered receptive field. Figure 5 Data to be considered in the convolution is conceptually illustrated for various example dilation rates d. As shown, when d is 1, all data values are considered, when d is 2, every other data value is considered, and when d is 3, every two data values are considered.

[0059] Figure 6 Possible deadlock and data loss situations that can arise during various operations performed by a convolutional neural network, such as convolutions, pooling, arithmetic operations, etc., are conceptually illustrated. A possible deadlock can occur when data from a previous process (e.g., one or more outputs of a convolution accelerator 340, etc.) is not ready in time to be used as input data for a current process (e.g., used as input to a convolution accelerator of the current process or input to another convolution accelerator). Figure 3 A possible deadlock can occur when data from a previous process is not ready in time to be used as input data for a current process (e.g., used as input to a convolution accelerator of the current process or input to another convolution accelerator). As shown, a wait state can be applied to the current process until the data is ready.

[0060] A data loss situation can occur when data from a current process is ready before a subsequent process is ready to use the data. As shown, the data can be stored in a buffer to prevent data loss before the subsequent process is ready to process the data. Figure 6

[0061] Figure 7 is a conceptual diagram illustrating an example of storing intermediate accumulation results in system memory during processing of an epoch of data by CNN 700. Figure 7 The data flow chain of FIG. 7B illustrates a data flow of a loop or iterative process in which the output of the convolution accelerator 740 is provided as input to the convolution accelerator for a subsequent round of processing. The data flow chain includes a fork in which the output of the convolution accelerator 740 is also provided to other functional circuitry 790 of the accelerator framework, as shown provided to the addition engine and then to the activation engine. The feature data tensor having a large number of channels is processed by buffering the intermediate accumulation results output from the convolution accelerator engine 740 in a software buffer of system memory (see memory 302 of FIG. 3B) using two stream engines 710 (see data processing program 310 of FIG. 3B) so that the data is ready to be input to the convolution accelerator in the next round of processing. Figure 3 Figure 3

[0062] Figure 8 is a conceptual diagram illustrating an example of storing intermediate accumulation results output from the convolution accelerator engine 840 in a reconfigurable hardware buffer during processing of an epoch of data by CNN 800.​​​Figure 8 dataflow chain (similar to Figure 7 dataflow chain) illustrates a loop or iterative process in which the output of the convolution accelerator 840 is provided as input to the convolution accelerator for a subsequent round of processing. The dataflow chain includes a fork in which the output of the convolution accelerator 840 is also provided to other functional circuitry 890 of the accelerator framework, as shown to the addition engine and activation engine. The feature data tensor with a large number of channels is processed by buffering the intermediate accumulation results in reconfigurable hardware buffers 870. The reconfigurable hardware buffers help avoid the need to buffer intermediate results in system memory when sufficient storage space can be allocated in the reconfigurable hardware buffers 870. Fewer stream engines can be needed to be used during processing of an execution epoch (which helps to use the stream engines for other tasks). As shown, compared to the example of Figure 7 two stream engines 710 / 810 are avoided, in which software buffers in system memory are used to store intermediate results, and through Figure 8 embodiments of

[0063] Figure 9 is a conceptual diagram illustrating a fork and join in a dataflow chain 900. As shown, the input feed is processed by convolution accelerator engine 940a, and the output of convolution accelerator engine 940a is forked and provided (all or in part) to convolution accelerator engines 940b and 940c. The output of convolution accelerator engines 940b and 940c is joined again (all or in part) and provided as input to convolution accelerator 940d. For example, if the timing of the output of convolution accelerator engines 940b and 940c is not aligned in time, the chain 900 can cause concurrency and deadlock issues related to the input of convolution accelerator 940d.

[0064] Figure 10is a conceptual diagram illustrating the concept of forking and joining in a dataflow chain 1000, where reconfigurable hardware buffers 1070 are used to buffer data in the dataflow chain, e.g., until data from a process in the chain is ready to be joined with the buffered data. As shown, input is fed to a convolution accelerator engine 940a, and the output of the convolution accelerator engine 940a is forked and provided (in whole or in part) to convolution accelerator engines 940b and 940c. The output of convolution accelerator 940b is buffered in a reconfigurable hardware buffer 1070, and then joined with the output of convolution accelerator 940c and provided as input to convolution accelerator 940d. The reconfigurable hardware buffer helps avoid concurrency and deadlock issues related to the input to convolution accelerator 940d, e.g., if the output of convolution accelerator 940b is ready before the output of convolution accelerator 940c it is to be joined with is ready.

[0065] Other forked and joined chains can be employed in some embodiments, and reconfigurable hardware buffers can buffer the output of multiple convolution accelerators in the chain. For ease of illustration, Figure 9 and Figure 10 A dataflow chain with multiple convolution accelerator engines is illustrated. Various different types of engines (e.g., addition engines, activation engines, etc., in various combinations) can be employed in the dataflow chain, and loop or iteration operations can be performed.

[0066] Figure 11 is a functional block diagram of an embodiment of a convolution accelerator framework (CAF) 1100 that includes a reconfigurable hardware buffer 1170. As shown, the convolution accelerator framework 1100 also includes a flow switch 1180, a control interface 1182 (e.g., an advanced high-performance bus (AHB) coupled to AHB slaves), a system bus interface 1184, one or more DMA engines 1186 (e.g., DMA controllers), one or more external device interfaces 1188, one or more processing modules or circuits 1190 (e.g., data compression / decompression modules, pooling function modules, activation function modules, adders, etc.), one or more convolution accelerators 1192, and clock control circuitry 1194.

[0067] The flow switch 1180 is, e.g., a reconfigurable unidirectional interconnect fabric formed of a plurality of unidirectional flow links. The flow links can be arranged to transmit multi-bit data streams from the accelerators, interfaces, and other logic modules or circuits of the convolution accelerator framework 1100 to the flow switch 1180, and from the flow switch 1180 to the accelerators, interfaces, and other logic modules or circuits of the convolution accelerator framework 1100.

[0068] The system bus interface 1184 provides an interface to other circuitry of a system that includes the convolution accelerator framework 1100, such as a host system and global memory. SeeFigure 3 Host system 304 and global memory 302.

[0069] DMA engine 1186 is arranged to provide a bidirectional channel for an input data stream, an output data stream, or both. Large amounts of data can be transferred into CAF 1100, out of CAF 1100, or both.

[0070] In one example embodiment, one or more DMA engines 1186 are connected to flow switch 1180 through one input and one output port. DMA engine 1186 can be configured in either input or output mode. DMA engine 1186 can be configured to packetize and send data to any address location accessible on any bus, or other address location. DMA engine 1186 can additionally or alternatively be configured to unpack and extract data and convert the unpacked data into a data stream.

[0071] Figure 12 and Figure 13 is a conceptual diagram illustrating a dataflow graph employing reconfigurable hardware buffers to resolve concurrency issues and simplify epochs. Figure 12 An implementation of a process using two epochs is illustrated, and for convenience will be referred to as Figure 11 described. As shown, the first epoch (Epoch 1) performs a convolution operation (Convolution 1), for example, using one of convolution accelerators 1192 of Figure 11 The first epoch (Epoch 1) then performs an activation / rectification function (Activation / Rectification 1) on the results of the convolution operation (Convolution 1), for example using activation function module or circuit 1190 of Figure 11 The second epoch (Epoch 2) performs an addition operation (Addition 1) with the results of the convolution operation (Convolution 1) and the results of the activation / rectification operation (Activation / Rectification 1) as inputs.

[0072] In embodiments of Figure 12 Because the outputs of the convolution and activation / rectification operations are not necessarily ready at the same time, in embodiments of Figure 3the global memory 302). Another DMA output operation stores the output of the Activation / Rectification 1 operation, for example, by using the DMA 2 1186 to store the result in the system memory (see Figure 3 the global memory 302). One of the operations DMA retrieves the stored output of the Convolution 1 operation, for example, by using the DMA 3 1186 to retrieve the stored result from the system memory (see Figure 3 the global memory 302). Another operation DMA retrieves the stored output of the Activation / Rectification 1 operation, for example, by using the DMA 4 1186 to retrieve the stored result from the system memory (see Figure 3 the global memory 302). Another operation DMA retrieves the stored output of the Activation / Rectification 1 operation, for example, by using the DMA 4 1186 to retrieve the stored result from the system memory (see

[0073] As shown in Figure 13 the reconfigurable hardware buffer structure (e.g., the reconfigurable hardware buffer 1170 of Figure 11 the shortest path among the two paths, this helps to avoid splitting the computation into two consecutive steps, reducing the number of cycles to perform the desired operation and the need for additional memory buffers and system memory accesses. The graph of Figure 12 the graph in Figure 13 the graph in

[0074] Even for operations different from the convolution, partial data resulting from intermediate computations cannot always be passed to the subsequent stages if these stages are not ready to process the partial data. Such data flow can lead to deadlocks, starvation, and concurrency errors in general.

[0075] Such errors can be solved using a scheduling method together with multiple small buffering structures storing partial data from early processes or circuits in the data flow and outputting the partial data to the subsequent processes or circuits when the processes are ready for the data. Indeed, along a single execution phase, different processes or circuits can take advantage of having dedicated buffers. Therefore, different buffers for different processes or circuits can be employed to store, reorganize, and read the data to be used by the subsequent processes or circuits. Figure 14An example set of buffers (buffers 1 to n) is conceptually illustrated that can be used in a convolution accelerator framework to store partial data output from earlier processes or circuits of a data stream for use in later processes or circuits.

[0076] As mentioned above, in some cases, reorganization or reshaping of data can be employed before partial data output by an operation is processed by a subsequent operation. For example, when a subsequent operation performs a dilated convolution on the output data of a previous operation, the output data can be reshaped, e.g., decimated. Figure 15 An example set of buffers (buffers 1 to n) for storing partial data output from earlier operations of a data stream and corresponding reshaping circuits (reshaping circuits 1 to n) for reshaping data for use in later operations are conceptually illustrated. In embodiments, the reshaping circuits can be implemented using buffer memory that stores the partial output data and controls readout of the data from the buffer memory to reshape the data.

[0077] Data organization and operation scheduling in CNN architectures are used to facilitate correctness of execution, as well as control timing performance and shutdown of execution. In CNNs, several structures typically process small data blocks or data in data blocks along the channel dimension, even subdividing these operations into smaller operations to obtain suitable dimensional data batches that can accelerate overall performance of a HW accelerator framework. Incorrect scheduling can result in concurrent errors in execution, leading to errors in the resulting output, and can even cause the data stream chain to stall and starve.

[0078] In particular, in the case of data stream oriented architectures, several operations can need to be considered in scheduling and processing (e.g., chaining, forking, stall propagation, etc.). Deadlock and starvation problems are typically caused by loops in processing and can cause operations to hang when there is not enough buffer space.

[0079] Circuits that use iterative loops to perform processing, subdivide data stream channels, split operations into smaller blocks and subsequently join partial data (e.g., convolution accelerator circuits) are typical examples of structures that can benefit from utilizing reconfigurable hardware buffers of a convolution accelerator framework in processing.

[0080] As mentioned above, these problems can be solved in software by scheduling the operations to be performed. The architectures typically considered follow the monolithic approach and thus the buffer dimensions and management are decided a priori and the flexibility of the architecture can be sacrificed to support simpler scheduling strategies. However, the inventors have realized that, for monolithic architectures, due to the flexibility of the modular structure, in the case of the modular approach, a reconfigurable buffering structure can be employed to solve the data flow timing problems. The reconfigurable buffering structure can also be configured to shape data for use in, for example, dilated convolutions, which typically pose difficult scheduling problems for software solutions.

[0081] In particular, finding a software solution using a scheduler and fixed buffer dimensions is not always a trivial task and determining such a solution typically requires a significant amount of time and computational resources. Moreover, if the sequence of operations needs to be split across different execution epochs, this can result in longer execution times and more external memory accesses, even if no concurrency problems occur.

[0082] A hardware structure capable of alleviating these problems, such as a reconfigurable hardware buffer, can facilitate improved data organization, shorter execution times, fewer external memory accesses, easier data shaping, etc., while also simplifying the scheduling design and reducing the time and resources used to execute it.

[0083] A reconfigurable hardware buffer can be a HW architecture configured to receive one or more data streams, reorganize the data into a suitable form, and provide the data as output of the buffer in the original format or after a shaping operation.

[0084] Considering the different number of active stream lanes, a shaping operation can be performed to adapt the connection between the two components of the convolution accelerator framework, change the shape of the input data to adapt it to different computations, and optionally organize the data so as to provide a viable stream for subsequent operations, such as a dilated convolution operation. Otherwise, a mirroring operation mode can be used to reproduce the input on the output without changing the shape of the data.

[0085] In an embodiment, up to 8 circuits (e.g., convolution accelerators, pooling circuits, activation circuits, etc.) can be coupled to a reconfigurable buffer (e.g., through a stream switch), which can have read / write priority from the first connection to the last. For example, up to 4 circuits can be coupled to the input of the reconfigurable hardware buffer, and up to 4 circuits can be coupled to the output of the reconfigurable hardware buffer. A circuit can be coupled to both the input and output of the reconfigurable hardware buffer. In this way, for example, if different buffer streams are ready at the same time, the stream associated with the highest priority (lowest index) will be read first. A memory module or circuit used as a multi-FIFO structure can be employed to store data waiting to be read, while a counter and logic system can be employed to reproduce or conveniently shape the input.

[0086] Figure 16 An embodiment of a reconfigurable hardware buffer unit or circuit 1600 that can be employed in a convolution accelerator framework, such as the accelerator framework 1100 of Figure 11 is shown. The accelerator framework 1100 itself can be used as a CNN accelerator in a system, such as the CNN accelerator 240 of Figure 2

[0087] As shown, the reconfigurable hardware buffer 1600 includes buffers 1602 that are reconfigurably organized as a plurality of input buffers 1604, input counters 1606, mirror buffers 1608 that are reconfigurably organized to store one or more data streams 1610, output count circuitry illustrated as a plurality of output counters 1612, output RAMs 1614 that are reconfigurably organized into one or more sub-RAMs 1616, output interfaces 1618, and control circuitry 1620. The mirror buffers 1608 can be implemented using a set of one or more support buffers. As shown, the control circuitry 1620 includes a processing core 1622 and control registers 1624 to store control information (e.g., information to configure the buffers 1602, mirror buffers 1608, and RAMs based on, for example, scheduling information associated with an execution epoch). For convenience, examples of the operation of the reconfigurable hardware buffer 1600 of Figure 11 will be described with reference to the accelerator framework 1100 of Figure 16

[0088] ​​The input buffer 1604 receives one or more input data streams of one or more data lanes, e.g., from the output of other components of the accelerator framework 1100, such as from the output of the convolution accelerator 1192, through the stream switch 1180. An input counter or register 1606 can be used to track the number of incoming samples and the input data format (e.g., for use in a mirror operation mode). In embodiments, the input buffer 1604 and input counter 1606 can be combined into an input interface that can also receive control information, such as configuration and timing information associated with an execution epoch.

[0089] The resources of the reconfigurable hardware buffer 1600 can be allocated to an operational data lane for which data is to be stored for subsequent operations. Data for a data lane is stored in the input buffer 1604 of a determined size that is allocated to the operational data lane. The data is transferred from the input buffer into the mirror buffer 1608, and can subsequently be transferred into the sub-RAM 1616 of the RAM 1614 of a determined size that is allocated to the data lane. One or more input counters 1606 and one or more output counters 1612 can also be allocated to the data lane. Note that some determined sizes can be fixed, while others can vary. For example, the size of the input buffer can be fixed, while the word size of the mirror buffer and the size of the sub-RAM (e.g., the width of the sub-RAM) can vary.

[0090] The counts and shaping information stored in the output counter 1612 can be used by the output interface 1618 to control the output of data from the mirror buffer or sub-RAM 1614 or both via the interface 1618. The interface 1618 can be controlled to provide, for example, the appropriate timing, mirroring of the output, or dilution of the execution data for use by a subsequent operation, e.g., a subsequent empty convolution operation in the case of execution dilution. The counters 1606, 1612 can be sized according to the maximum frame size to be considered. For example, for a raw frame of 2 32 samples, a 32-bit counter can be used, or for a raster scan frame, a 2 16 x 2 16 .

[0091] In embodiments, the reconfigurable hardware buffer can be configured to have up to 4 input streams or data lanes, up to 4 output streams or data lanes, and can include 96 x 256-bit memory and 32-bit counters.

[0092] An example mirror operation will be described with reference to Figures 16-20 in which the output provided by the reconfigurable hardware buffer mirrors the input.

[0093] Figure 17is a conceptual diagram illustrating examples of how the mirror buffer 1608 stores data for a data lane associated with an operation, according to the dimension of the data lane. Assume that the memory word size is 12 bytes, the mirror buffer adapts the input data before writing the data to memory. As shown, the reconfigurable hardware buffer 1600 can be configured to store data for a single lane input, corresponding to 8 data bits; the reconfigurable hardware buffer 1600 can be configured to store data for two input lanes, corresponding to 16 data bits. The reconfigurable hardware buffer 1600 can be configured to store data for three input lanes, corresponding to 24 data bits. If the frame ends before completing the write of a vector (e.g., a 96-bit vector), any remaining bits in the word can be padded with zero values. Figure 17 The arrows in indicate the transfer of a single word from the mirror buffer to the sub-RAM assigned to the lane.

[0094] Figure 18 is a conceptual diagram illustrating data stored in a mirror buffer (mirror i) and an output counter (outcnt i) and a sub-RAM (subRAM i) assigned to a data lane associated with an operation. When valid data associated with an operation is read from the mirror buffer, the output counter associated with the mirror buffer is incremented by 1 to account for the new data. The output counter can be incremented even when data is not transferred from the mirror buffer to the sub-RAM associated with the data lane.

[0095] The reconfigurable hardware buffer 1600 can account for the line type of the incoming data associated with a mirror operation. For example, in the case of a raw data line type, an output counter 1612 with 32 bits can use all bits as a single counter. When new data from an input buffer associated with a lane is stored in a mirror buffer associated with the lane, the value of the counter is incremented, as follows:

[0096] Counter value = a 31 a 30 a 29 … a2a1a0 + 1

[0097] In the case of a raster or single-raster data line type, 16 bits of the register can be used to track the incoming column, and 16 bits of the register can be used to track the incoming row. When new data from an input buffer associated with a lane is stored in a mirror buffer associated with the lane before the end of the input row, the value of the counter is incremented, as follows:

[0098] Counter value = r 15 r 14 r 13 … r2r1r0c 15 c 14c 13 …c2c1c0+1

[0099] When new data from an input buffer associated with a lane is stored in a mirror buffer associated with the lane and the input row ends, the value of the counter is incremented, as follows:

[0100] Counter value = r 15 r 14 r 13 …r2r1r0c 15 c 14 c 13 …c2c1c0+2 16

[0101] Figure 19 is a conceptual diagram illustrating an embodiment of a sub-RAM in a logical form for a first-in-first-out (FIFO) buffer that facilitates generating appropriate outputs based on a counter and shaping information.

[0102] Figure 20 is a conceptual diagram illustrating an example of data being written and read out of an interface that provides data for an output lane associated with a subsequent operation, assuming a word size of twelve bytes. For example, when the data output lane is a mirrored output of an input lane, an example of Figure 20 may be employed. As shown, when the reconfigurable hardware buffer 1600 is configured to output data for a single lane (corresponding to 8 data bits), an output sub-RAM having a first size can be allocated for that lane; when the reconfigurable hardware buffer 1600 is configured to output data for two input lanes (corresponding to 16 data bits), an output sub-RAM having a second size can be allocated for that lane; and when the reconfigurable hardware buffer 1600 is configured to output data for three input lanes (corresponding to 24 data bits), an output sub-RAM having a third size can be allocated for that lane.

[0103] Note that more than one output lane associated with one or more subsequent operations can be allocated to an input operation, for example, when one subsequent operation will use mirrored data and another subsequent operation will use dilated data from the operation whose data is being stored.

[0104] Figure 21 is a conceptual diagram illustrating an example of reorganizing data stored in a mirror buffer and one or more output sub-RAMs to provide a shaped output, such as an output to which a dilution factor has been applied. Such an output can be provided to, for example, a convolution accelerator circuit 1192 that performs a dilutive convolution operation. Data coming out of the mirror and sub-RAM buffer structure can be reorganized using a dilution factor d in order to obtain an output stream suitable for a particular application by taking the correct data out of the structure and ignoring other data that is not needed to participate in the convolution operation. For Figure 21The example shown has a dilution factor of 2.

[0105] Figure 22 An embodiment of method 2200 for processing the execution period of a CNN using a convolutional accelerator framework with reconfigurable hardware buffers is shown, and embodiments thereof can be used, for example... Figure 2 System 200 or Figure 3 The system 300 adopts this. For convenience, the reference will be... Figure 11 The convolutional accelerator framework 1100 and Figure 16 The reconfigurable hardware buffer 1600 is described as Figure 22 Method 2200.

[0106] Method 2200 begins at 2202. Method 2200 may, for example, respond to processing from the host system (see...). Figure 3 The host system (304) receives a request for the execution of the CNN and begins. Method 2200 proceeds from 2202 to 2204.

[0107] At 2204, method 2200 parses control information associated with the execution period to generate configuration information to allocate resources of the convolutional accelerator framework 1100 to the execution period, including allocating resources of the reconfigurable hardware buffer 1170 to data channels of the execution period. The configuration information may include: a data stream chain, wherein various operations of the data stream chain are assigned to various components of the convolutional accelerator framework 1100, such as to one or more convolutional accelerators 1192 or other functional circuitry 1190; configuration information for configuring stream switching 1180 to route data streams or channels of the data stream chain between the various components performing the various operations and the reconfigurable hardware buffers; and configuration information for allocating resources of the reconfigurable hardware buffers 1170, 1600 to the data streams or channels to temporarily buffer or store data output by the operations of the data stream chain for subsequent use by the operations of the data stream chain; and any shaping information for the data streams or channels. Method 2200 proceeds from 2204 to 2206.

[0108] In step 2206, method 2200 allocates resources of the convolutional accelerator framework 1100 to the execution period based on configuration information. For the reconfigurable hardware buffers 1170, 1600, the configuration information may include: the number of input channels; the buffer index associated with each of the one or more input data channels; the size of the input buffer 1604 allocated to the input data channel; the word size of the mirror buffer; the size of the RAM allocated to the input data channel (e.g., the number of sub-RAMs 1614); the one or more output data channels associated with the input data channel; and any shaping and information associated with the output data channels. The size information may be based, for example, on the number of data channels of the data stream chain to be temporarily stored by the reconfigurable hardware buffers 1170, 1600 during processing in the execution period, as described above regarding... Figures 16 to 21 This is under discussion. Default and fixed sizes can be used. For example, in some embodiments, the input buffer allocated to the data channel can have a fixed size. Method 2200 proceeds from 2206 to 2208.

[0109] In 2208, method 2200 uses the allocated resources to process the execution period. For example, refer to Figure 11 , Figure 13 and Figure 16 The convolution accelerator 1192 of the accelerator frame 1100 can perform convolution on the input data. The result of the convolution can be provided to the activation circuit 1190 and stored in a reconfigurable hardware buffer 1170. The stored result and the output of the activation circuit are provided to the addition circuit 1190 of the accelerator frame 1100. Method 2200 proceeds from 2208 to 2210.

[0110] In 2210, method 2200 may optionally perform other processing, such as returning the result of the execution period to the requesting host, other processing associated with the execution period, instructing the accelerator framework 1100 to handle processing of another execution period, etc.

[0111] Embodiments of methods for processing execution epochs using a convolutional accelerator framework with reconfigurable hardware buffers may include... Figure 22 Additional actions not shown may be omitted. Figure 22 All the actions shown can be performed in various orders. Figure 22 The actions shown can be modified in various aspects. For example, action 2204 can be omitted from method 2200 when it is not necessary to parse configuration information from control information. In another example, method 2200 can be modified to process multiple periods using the same configuration information, such as for an iterative process, or modified to store the output of one period in a reconfigurable hardware buffer for use in subsequent periods.

[0112] Some embodiments can take the form of or include a computer program product. For example, according to one embodiment, a computer-readable medium comprising a computer program is provided that is adapted to cause one or more processing devices to perform one or more of the methods or functions described above. The medium can be a physical storage medium such as a ROM chip, or a disk such as a digital versatile disk (DVD-ROM), a compact disk (CD-ROM), a hard disk, memory, a network, or a portable media article to be read by an appropriate drive, including encoding in one or more barcodes or other related codes, stored on one or more such computer-readable media and read by an appropriate reader device.

[0113] Further, in some embodiments, some or all of the methods and / or functionality can be implemented or provided in other ways, such as at least partially in firmware and / or hardware, including but not limited to one or more application-specific integrated circuits (ASICs), digital signal processors, discrete circuitry, logic gates, standard integrated circuits, controllers (e.g., by execution of appropriate instructions, and including microcontrollers and / or embedded controllers), field- programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and the like, as well as devices employing RFID technology and various combinations thereof.

[0114] In the foregoing description, certain specific details are set forth in order to provide a thorough understanding of various disclosed embodiments. However, one skilled in the relevant art will recognize that embodiments can be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with electronic and computing systems, including client and server computing systems and networks, have not been shown or described in detail in order to avoid unnecessarily obscuring descriptions of the embodiments.

[0115] Unless the context requires otherwise, throughout the specification and claims which follow, the word "comprise" and variations thereof (such as "comprises" and "comprising") are to be construed in an open, inclusive sense, that is as "including, but not limited to."

[0116] References throughout this specification to "one embodiment" or "an embodiment" and variations thereof mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0117] As used in this specification and the appended claims, the singular forms“a,”“an” and“the” include plural referents unless the content and context clearly dictates otherwise. It should also be noted that the conjunction“and” and“or” are generally used in the broadest sense to include both an inclusive and exclusive disjunction (as appropriate) unless the content and context clearly dictates otherwise. Furthermore, combinations of“and” and“or” are intended to cover embodiments including all relevant items or ideas as well as one or more other alternative embodiments including less than all relevant items or ideas.

[0118] The title and summary of the disclosure provided herein are provided merely for convenience and do not limit or construe the scope or meaning of the embodiments.

[0119] The various embodiments described above can be combined to provide further embodiments. If the concepts of various patents, applications, and publications are needed to provide further embodiments, aspects of the embodiments can be modified.

[0120] These and other changes can be made to the embodiments in light of the above Detailed Description. In general, the selected terms used in the following claims are not to be construed as limiting the claims to the specific embodiments disclosed in the specification and claims, but rather have their full scope encompassing all embodiments and equivalent concepts that are within the scope of the claims. Therefore, the claims are not limited to the disclosure.

Claims

1. A convolution accelerator framework (CAF) comprising: a plurality of processing circuits including one or more convolution accelerators; a reconfigurable hardware buffer configurable to store data for a variable number of input data lanes, the reconfigurable hardware buffer having: a memory; and control circuitry; and a flow switch coupled to the plurality of processing circuits and the reconfigurable hardware buffer, wherein in operation, the variable number of input data lanes is associated with an execution epoch; during processing of the execution epoch, the flow switch streams data for the variable number of input data lanes between a processing circuit of the plurality of processing circuits and the reconfigurable hardware buffer; and the control circuitry of the reconfigurable hardware buffer configures the memory to store data for the variable number of input data lanes during processing of the execution epoch, the configuring including allocating a portion of the memory to each input data lane of the variable number of input data lanes, wherein the memory includes an input buffer memory, a mirror buffer, and a random access memory (RAM), wherein allocating a portion of the memory to an input data lane of the variable number of input data lanes includes determining an input buffer of the input buffer memory for allocation to the input data lane, determining a size of a sub-RAM of the RAM for allocation to the input data lane, and determining a word size of the mirror buffer, wherein the mirror buffer is reconfigurably organized to store one or more data streams and is configured to adapt input data prior to writing the data to the memory.

2. The convolution accelerator framework of claim 1, wherein the word size of the mirror buffer is a function of the variable number of input data lanes stored in the reconfigurable hardware buffer during processing of the execution epoch.

3. The convolution accelerator framework of claim 1, wherein in operation, the sub-RAM is configured as a first-in-first-out memory.

4. The convolution accelerator framework of claim 1, wherein the control circuitry of the reconfigurable hardware buffer reconfigures the memory in operation between processing of a first execution epoch and processing of a second execution epoch.

5. The convolution accelerator framework of claim 1, wherein the variable number is an element of the set of numbers {0, 1, 2, 3, 4}.

6. The convolution accelerator framework of claim 1, wherein the reconfigurable hardware buffer includes an output interface and the control circuitry configures the output interface in operation to provide a variable number of output data lanes. provides a mirrored output of an input data lane of the variable number of input data lanes.

7. The convolution accelerator framework of claim 6, wherein, in operation, configuring the output interface to provide output data lanes of the variable number of output data lanes comprises: provides a shaped output of an input data lane of the variable number of input data lanes.

8. The convolution accelerator framework of claim 6, wherein, in operation, configuring the output interface to provide output data lanes of the variable number of output data lanes comprises: ​ 9. The convolution accelerator framework of claim 8, wherein in operation, the shaped output is provided as input to a convolution accelerator of the one or more convolution accelerators via the flow switch, wherein the convolution accelerator, in operation, performs a dilated convolution operation of the execution epoch using the shaped output.

10. The convolution accelerator framework of claim 6, wherein in operation, a variable number of the input data lanes is equal to a variable number of the output data lanes.

11. The convolution accelerator framework of claim 6, wherein in operation, a variable number of the input data lanes is less than a variable number of the output data lanes.

12. The convolution accelerator framework of claim 6, wherein the control circuitry comprises control logic and a plurality of counters, and in operation, the control logic uses the counters to track storage of data of the variable number of input data lanes in the memory and to control provision of the stored data to the variable number of output data lanes.

13. A system for implementing a neural network (NN), the system comprising: a host processor; a host memory; and an accelerator framework (AF) comprising: a plurality of processing circuits including one or more accelerators; a reconfigurable hardware buffer configurable to store data of a variable number of input data lanes, the reconfigurable hardware buffer having: a memory; and control circuitry; and a flow switch coupled to the plurality of processing circuits and the reconfigurable hardware buffer, wherein in operation, a number of the variable number of input data lanes is associated with an execution epoch of the NN; during processing of the execution epoch, the flow switch streams data of the variable number of input data lanes between a processing circuit of the plurality of processing circuits and the reconfigurable hardware buffer; and the control circuitry of the reconfigurable hardware buffer configures the memory of the reconfigurable hardware buffer to store data of the variable number of input data lanes during processing of the execution epoch, the configuring including allocating a portion of the memory of the reconfigurable hardware buffer to each input data lane of the variable number of input data lanes, wherein the memory of the reconfigurable hardware buffer includes an input buffer memory, a mirror buffer, and a random access memory (RAM), wherein allocating a portion of the memory of the reconfigurable hardware buffer to an input data lane of the variable number of input data lanes includes determining an input buffer of the input buffer memory for allocation to the input data lane, determining a word size of the mirror buffer, and determining a size of a sub-RAM of the RAM for allocation to the input data lane; wherein the mirror buffer is reconfigurably organized to store one or more data streams and is configured to adapt input data prior to writing the data to the memory. ​ 14. The system of claim 13, wherein the size of the mirror buffer is a function of the variable number of input data lanes stored in the reconfigurable hardware buffer during processing of the execution epoch.

15. The system of claim 13, wherein the reconfigurable hardware buffer comprises an output interface, and the control circuitry, in operation, configures the output interface to provide a variable number of output data lanes.

16. The system of claim 15, wherein in operation, configuring the output interface to provide an output data lane of the variable number of output data lanes comprises: providing a shaped output of an input data lane of the variable number of input data lanes.

17. The system of claim 13, comprising an integrated circuit comprising the host processor, the main memory, and the accelerator framework.

18. A method comprising: processing an execution epoch of a convolutional neural network using a convolutional accelerator framework having a plurality of processing circuits, a flow switch, and a reconfigurable hardware buffer, the processing of the execution epoch comprising: configuring a variable number of input data lanes of the reconfigurable hardware buffer of the convolutional accelerator framework, a number of the variable number of input data lanes being associated with the execution epoch, the configuring comprising allocating portions of a memory of the reconfigurable hardware buffer to each of the variable number of input data lanes; streaming data of the variable number of input data lanes between processing circuits of the plurality of processing circuits and the reconfigurable hardware buffer via the flow switch; and storing data of the variable number of input data lanes in the memory of the reconfigurable hardware buffer based on the allocation of the portions of the memory to the variable number of input data lanes, wherein the memory comprises an input buffer memory, a mirror buffer, and a random access memory (RAM), and allocating a portion of the memory to an input data lane of the variable number of input data lanes comprises determining an input buffer of the input buffer memory for allocation to the input data lane, determining a word size of the mirror buffer, and determining a size of a sub-RAM of the RAM for allocation to the input data lane; wherein the mirror buffer is reconfigurably organized to store one or more data streams, and is configured to adapt input data prior to writing the data to the memory.

19. The method of claim 18, comprising reconfiguring the memory of the reconfigurable hardware buffer between processing of a first execution epoch and processing of a second execution epoch.

20. The method of claim 18, wherein the reconfigurable hardware buffer comprises an output interface, and the method comprises configuring the output interface to provide a variable number of output data lanes.

21. The method of claim 20, wherein configuring the output interface to provide an output data lane of the variable number of output data lanes comprises: providing a shaped output of an input data lane of the variable number of input data lanes.

22. The method of claim 20, comprising using a counter of the reconfigurable hardware buffer to track storage of data of the variable number of input data lanes in the memory, and to control provision of stored data to the variable number of output data lanes.

23. A non-transitory computer readable medium having content that causes one or more hardware processors of a convolution accelerator framework to perform a method comprising: processing an execution epoch of a convolutional neural network, the processing of the execution epoch comprising: configuring a variable number of input data lanes of a reconfigurable hardware buffer of the convolution accelerator framework, the number of the variable number of input data lanes being associated with the execution epoch, the configuring comprising allocating a portion of a memory of the reconfigurable hardware buffer to each of the variable number of input data lanes; streaming data of the variable number of input data lanes between processing circuits of the plurality of processing circuits and the reconfigurable hardware buffer via a stream switch of the convolution accelerator framework; and storing data of the variable number of input data lanes in the memory of the reconfigurable hardware buffer based on the allocation of the portion of the memory to the variable number of input data lanes, wherein the memory comprises an input buffer memory, a mirror buffer, and a random access memory (RAM), and allocating a portion of the memory to an input data lane of the variable number of input data lanes comprises determining an input buffer of the input buffer memory for allocation to the input data lane, determining a word size of the mirror buffer, and determining a size of a sub-RAM of the RAM for allocation to the input data lane; and wherein the mirror buffer is reconfigurably organized to store one or more data streams, and is configured to adapt input data prior to writing the data to the memory.

24. The non-transitory computer readable medium of claim 23, wherein the reconfigurable hardware buffer comprises an output interface, and the method comprises configuring the output interface to provide a variable number of output data lanes.

25. The non-transitory computer-readable medium of claim 23, wherein configuring the output interface to provide an output data lane of the variable number of output data lanes comprises: providing shaped output of an input data lane of the variable number of input data lanes.

26. The non-transitory computer readable medium of claim 23, wherein the content comprises instructions executed by the one or more processors of the convolution accelerator framework.

Images