Processor system and method for increasing data transfer bandwidth during execution of scheduled parallel processes - Patents.com

Abstract

A broadcast subsystem of a processor system includes a set of broadcast buses, each broadcast bus in the set of broadcast buses electrically coupled to a subset of primary memory units in the set of primary memory units; primary memory unit queues configured to store a first set of data transfer requests associated with the set of primary memory units and electrically coupled to the data buffer; and a broadcast scheduler electrically coupled to the primary memory unit queues, electrically coupled to the set of broadcast buses, and configured to transfer source data from the data buffer to a target subset of primary memory units in the set of primary memory units via the set of broadcast buses based on the set of data transfer requests stored in the primary memory unit queues.

Description

[Technical field]

[0001] The present invention relates generally to the field of integrated circuit design, and more particularly to a novel and useful processor system and method for increasing data transfer bandwidth during the execution of scheduled parallel processes.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 071,874, filed August 28, 2020, which is incorporated by reference in its entirety.

[0003] This application is related to U.S. Patent No. 10,474,464, filed July 3, 2018, U.S. Patent Application No. 17 / 127,904, filed December 18, 2020, U.S. Patent Application No. 17 / 211,707, filed March 24, 2021, U.S. Patent Application No. 17 / 331,585, filed May 26, 2021, and U.S. Patent Application No. 17 / 331,590, filed May 26, 2021, each of which is incorporated by reference in its entirety. [Brief description of the drawings]

[0004] [Figure 1] FIG. 1 is a schematic diagram of a processor system. [Diagram 2] FIG. 2 is a flow chart of the method. [Diagram 3] FIG. 3 is a flow chart of one embodiment of the method. [Figure 4] FIG. 4 is a flow chart of one embodiment of the method. [Diagram 5] FIG. 5 is a flow chart of one embodiment of the method. [Figure 6] FIG. 6 is a flow chart of one embodiment of the method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] The following description of the embodiments of the present invention is not intended to limit the present invention to those embodiments, but is intended to enable a person skilled in the art to make and use the present invention. The aspects, configurations, examples, embodiments, and examples described herein are optional and are not limited to only the aspects, configurations, examples, embodiments, and examples they describe. The invention described herein can include any and all permutations of those aspects, configurations, examples, embodiments, and examples.

[0006] 1. Processor System As shown in FIG. 1, the processor system 100 includes a direct memory access core 110 including a data buffer 112, a set of primary memory units 120, and a broadcast subsystem 130. The broadcast subsystem 130 includes a set of broadcast buses 132, where each broadcast bus 132 in the set of broadcast buses 132 is electrically coupled to a subset of the primary memory units 120 in the set of primary memory units 120, a primary memory unit queue 134, and a broadcast scheduler 136. The primary memory unit queues 134 are configured to store a first set of data transfer requests associated with the set of primary memory units 120 and are electrically coupled to the data buffers 112. The broadcast scheduler 136 is electrically coupled to the primary memory unit queues 134, electrically coupled to the set of broadcast buses 132, and configured to transfer source data from the data buffers 112 to a target subset of the primary memory units 120 in the set of primary memory units 120 via the set of broadcast buses 132 based on the set of data transfer requests stored in the primary memory unit queues 134.

[0007] 2. Method 2, method S100 is performed by a neural network in a processor system 100 that includes a shared memory unit 140, a set of primary memory units 120, and a set of processing units 150. Method S100 includes, in block S110, storing a first weight tensor including the set of weight tensor partitions at a first source address and storing a first input tensor larger than the first weight tensor at a second source address in shared memory unit 140. Method S100 also includes, in block S120, broadcasting the first input tensor from the second source address to a first relative destination address in the set of primary memory units 120. Method S100 further includes, for each processing unit 150 in the set of processing units 150, transferring a weight tensor partition in the set of weight tensor partitions from a first source address to a first destination address in the primary memory unit 120 of the processing unit 150 in block S130, and generating, in the processing unit 150, an output tensor partition of the first output tensor based on the first input tensor and the weight tensor partition in block S140. Method S100 further includes, in the shared memory unit 140, storing the second weight tensor in a third source address and storing a second input tensor that includes the set of input tensor partitions, the second input tensor being smaller than the second weight tensor, in a fourth source address in block S150. Method S100 further includes, in block S160, broadcasting the second weight tensor from the third source address to a second relative destination address in the set of primary memory units 120.Further, method S100 includes, in block S170, transferring, for each processing unit 150 in the set of processing units 150, an input tensor partition in the set of input tensor partitions from a fourth source address to a second destination address in the primary memory unit 120 of the processing unit 150, and, in block S180, generating, in each processing unit 150 in the set of processing units 150, an output tensor partition of a second output tensor based on the second weight tensor and the input tensor partition.

[0008] 3. Application Generally, a multi-core processor system (hereinafter, "processor system 100") can simultaneously broadcast data read from a shared memory unit 140 (i.e., L2 memory) of the processor system 100 to multiple primary memory units 120, each corresponding to one processing unit 150 (i.e., processor core) of the processing system, thereby increasing memory transfer bandwidth, shortening processing time, and reducing power consumption of the processor system 100 during scheduled parallel processing (e.g., evaluation of a convolutional neural network (hereinafter, "CNN")). More specifically, the processor system 100 includes a broadcast subsystem 130 configured to concurrently transfer data from the shared memory unit 140 to the primary memory unit 120 of the processor system 100, thereby reducing the rate of read / write requests performed at the shared memory unit 140 for a given amount of transferred memory, and a memory management subsystem configured to utilize the reduction in read / write requests at the shared memory unit 140 to reduce power consumption of the shared memory unit 140 by selectively transitioning inactive memory modules of the shared memory unit 140 to a lower power state. Thus, the processor system 100 can quickly and efficiently perform complex parallel processing in edge computing, low power or offline applications where cloud-based resources may not be available or practical.

[0009] Additionally, processor system 100 includes a direct memory access core 110 (hereinafter “DMA core 110”) configured to receive (e.g., from a control processor or queue processor) a set of control signals specifying data transfer operations between shared memory unit 140 and primary memory unit 120 of processor system 100. To this end, DMA core 110 issues read or write requests (i.e., data transfer requests) indicating source and destination addresses, respectively, to initiate a transfer or broadcast between shared memory unit 140 and the set of primary memory units 120. Thus, DMA core 110 can retrieve target data from shared memory unit 140 (e.g., into local data buffer 112 within DMA core 110) and direct those target data to broadcast subsystem 130 for delivery to the indicated destination address or addresses in the set of primary memory units 120.

[0010] To transfer target data from the shared memory unit 140 to multiple primary memory units 120 simultaneously, the broadcast subsystem 130 includes a broadcast-enabled interconnect (e.g., one or more broadcast buses 132, a crossbar interconnect, or a network-on-chip) that connects to an exclusive set of primary memory units 120 of the processor system 100. The broadcast-enabled interconnect can operate in two data transfer modes: unicast (i.e., transfer target data to a single primary memory unit 120 to which the interconnect is connected) or multicast (i.e., transfer target data to all of the primary memory units 120 to which the interconnect is connected). In one example, the processor system 100 includes eight processing units 150 with corresponding primary memory units 120, and the broadcast subsystem 130 includes two broadcast buses 132 that function as the broadcast-enabled interconnect, with each broadcast bus 132 connected to four of the primary memory units 120. In this example, the processor system 100 may multicast target data over both broadcast buses 132, multicast over a first broadcast bus 132, unicast over a second broadcast bus 132, or unicast over both broadcast buses 132. Additionally, the processor system 100 may transfer the same set of target data over both broadcast buses 132, or transfer different sets of target data over each broadcast bus 132.Thus, by combining the above-mentioned transfer options, the processor system 100 can transfer a single set of target data to one, two, four, five, or eight primary memory units 120 simultaneously, or transfer two different sets of target data simultaneously, such that each set of target data can be transferred to a separate primary memory unit 120, each set of target data can be transferred to a separate group of four primary memory units 120, or one set of target data can be transferred to a single primary memory unit 120 and the other set of target data can be transferred to a separate group of four primary memory units 120. By facilitating this flexibility, the broadcast subsystem 130 reduces idle time of the processing units 150 due to serial transfers and allows for tight synchronization of output-fixed parallel processes running across the set of processing units 150. However, because the broadcast subsystem 130 includes only two broadcast buses 132, the broadcast system can maintain a small on-silicon space footprint.

[0011] As a result of the increased data transfer bandwidth between the shared memory unit 140 and the primary memory unit 120 enabled by the broadcast subsystem 130, the processor system 100 can issue fewer read / write requests to the shared memory unit 140 per unit of transferred memory, thereby allowing a longer downtime of the memory modules of the shared memory unit 140. Taking advantage of this increased downtime, the memory management subsystem can reduce the power consumption of the shared memory unit 140 through the shared memory unit 140 being divided into discrete memory modules, a conflict resolution scheduler configured to analyze a queue of data transfer requests to the shared memory unit 140, detect collisions in the queue of the shared memory unit, and reorder or suspend requests to resolve the collisions, and a power management unit configured to track the idle factor of each memory module and selectively switch the memory modules into a sleep mode based on the idle factor of each memory module. In one aspect, the conflict resolution scheduler and the power management unit are hardware implemented finite state machines or microprocessors embedded within the processor system 100.

[0012] In one application, the processor system 100 can reduce power consumption and inference time of a statically scheduled CNN executed on the processor system 100 according to the method S100. In this application, the statically scheduled CNN is characterized by an output fixed data flow. However, while operating according to the output fixed data flow defined by the static scheduling of the CNN, the processor system 100 can either broadcast the input tensors of each layer and unicast multiple partitions of the weight tensors of each layer (i.e., input broadcast data flow), or the processor system 100 can broadcast the weight tensors of each layer and unicast multiple partitions of the input tensors of each layer (i.e., weight broadcast data flow). Thus, prior to execution of the CNN by the processor system 100, a cooperating scheduling application can identify a first subset of layers in the CNN that execute more efficiently according to the input broadcast data flow and identify a second subset of layers in the CNN that execute more efficiently by the weight broadcast data flow to generate a hybrid data flow schedule that includes both input broadcast layers and weight broadcast layers in a single static scheduling of the CNN. Thus, the processor system 100 can leverage its large data parallelism, low power consumption and small on-silicon footprint in combination with hybrid dataflow scheduling to rapidly execute statically scheduled CNNs in edge computing environments.

[0013] 4. Processor System Generally, as shown in Figure 1, processor system 100 is a multi-core processor circuit that includes a set of processing units 150, a set of memory components in a memory hierarchy for each processing unit 150 of processor system 100, such as a main memory, a shared memory unit 140 (i.e., L2 memory), a set of primary memory units 120 (i.e., L1 memory), a DMA core 110, a control processor (e.g., a queue processor described in U.S. patent application Ser. No. 17 / 211,707), a broadcast subsystem 130, and a memory management subsystem. Processor system 100 can be configured to execute statically or dynamically scheduled parallel processes, such as inference generation via statically scheduled CNN.

[0014] Processor system 100 includes a set of interconnects, an address interconnect, control lines, interrupt lines, and hardware-implemented queues connecting each of these components to enable communication of data between a control processor and DMA core 110, between DMA core 110 and shared memory unit 140, between DMA core 110 and a set of primary memory units 120, and / or between DMA core 110 and main memory. That is, processor system 100 includes DMA core 110 that receives control signals specifying data transfers and / or data broadcasts from shared memory unit 140 to the set of primary memory units 120. Processor system 100 may also include a control interconnect between the control processor and the set of processing units 150 for issuing instructions to those processing units 150.

[0015] 4.1 DMA Core: Generally, DMA core 110 is configured to receive control signals from a control processor. Each control signal indicates a data transfer operation between memory components of processor system 100. DMA core 110 is further configured to issue a data transfer request (e.g., a read request or a write request) corresponding to the indicated memory component. For example, DMA core 110 may receive a data transfer operation specifying a data transfer from shared memory unit 140 to a set of primary memory units 120. In this example, DMA core 110 may issue a read request to a memory management subsystem, which then forwards the read request to shared memory unit 140. Shared memory unit 140 may then respond to the read request by transferring the requested target data to a data buffer 112 within DMA core 110. In response to receiving the target data in data buffer 112, DMA core 110 may issue a write request for the target data to broadcast subsystem 130 to initiate the transfer of the target data to one or more primary memory units 120 within the set of primary memory units 120. In this manner, DMA core 110 coordinates efficient data transfers between memory components of processor system 100 .

[0016] More specifically, to transfer data between memory components, DMA core 110 may issue a read request to a source memory component specifying a source memory address of the target data, store the target data in internal data buffer 112, issue a write request to one or more destination memory components specifying a destination memory address or a relative destination memory address (for a broadcast operation), and enable access to those target data in internal data buffer 112 by broadcast subsystem 130 (for writing to a set of primary memory units 120) or by memory management subsystem (for writing to shared memory unit 140). When performing a broadcast operation, DMA core 110 may issue a write request indicating a relative destination memory address, which indicates a single memory address for the broadcast operation. In one aspect, because each primary memory unit 120 in the set of primary memory units 120 defines the same memory address, broadcast subsystem 130 may write the target data from DMA core 110 to the same location in each primary memory unit 120 specified in the write request. In one example, DMA core 110 can issue a write request to a set of primary memory units 120 specifying a single memory address and a set of primary memory unit 120 identifiers. The identifiers specify which primary memory units 120 the write request is intended for. Broadcast subsystem 130 can then receive this write request and identify the broadcast bus 132 and data transfer mode of their interconnects to transfer the target data to a destination address within the correct primary memory units 120.

[0017] In one aspect, processor system 100 includes a set of multiple in-line DMA engines (i.e., DMA contexts) operating as DMA core 110 to process multiple transfer instructions simultaneously. This allows DMA core 110 to issue transfer requests between the same pair of data components simultaneously to support the increased data bandwidth enabled by the broadcast and memory management subsystems. In one embodiment, DMA core 110 includes eight DMA engines: a first set of four DMA engines dedicated to transferring data between shared memory unit 140 and the set of primary memory units 120, and a second set of four DMA engines dedicated to transferring data between main memory and shared memory unit 140. In another embodiment, DMA core 110 does not include a DMA engine configured to transfer data directly between main memory and the set of primary memory units 120.

[0018] In another aspect, the processor system 100 includes a DMA core 110 such as a tensor traversal engine described in U.S. Patent Application No. 17 / 331,585 and U.S. Patent Application No. 17 / 331,590, each of which is incorporated by reference in its entirety. More specifically, the processor system 100 may include a DMA core 110 including a set of tensor traversal engines configured to issue data transfer requests specifying multi-dimensional data transfer operations. In this aspect, each tensor traversal engine may perform stride data transfer operations across multiple dimensions (e.g., according to a data access pattern) and perform inline data decompression, data unrolling, or data transposition. Thus, in an aspect in which the processor system 100 includes a set of tensor traversal engines as the DMA core 110, the processor system 100 may leverage those multi-dimensional data transfer operations to improve memory management and broadcasting in either the memory management subsystem or the broadcast subsystem 130.

[0019] 4.2 Processing Unit and Primary Memory Unit Generally, the processor system 100 includes a set of processing units 150 configured to perform computational steps on data transferred to a set of primary memory units 120 in order to execute the scheduled parallel processes. More specifically, the processor system 100 includes a set of processing units 150, each corresponding to one primary memory unit 120 from which the processing units 150 can read input data and write output data. Thus, by transferring data to the primary memory units 120 corresponding to the processing units 150, the processor system 100 can make accessible the inputs required to compute the output or intermediate data of the scheduled parallel processes.

[0020] In one aspect, the processor system 100 includes a set of heterogeneous processing units 150 that perform specific computational tasks. For example, the processor system 100 can issue control signals to a CPU, a GPU, or a specialized deep learning processor (hereinafter, "DLP") included in the multi-core processor system 100.

[0021] In another embodiment, the processor system 100 includes a set of processing units 150 that are specifically configured for edge execution of CNN or other deep artificial neural networks, as described in more detail in U.S. Patent No. 10,474,464. In one example, the processor system 100 includes eight DLPs as the set of processing units 150, with each DLP corresponding to one primary memory unit 120.

[0022] A control processor communicates with this set of processing units 150 via a control interconnect and can send and receive control signals to each processing unit 150. Thus, the control processor can dispatch instructions to each processing unit 150, register when each instruction has been executed by a processing unit 150, and track the execution order for the instructions.

[0023] 4.3 Broadcast Subsystem Generally, the broadcast subsystem 130 includes a set of hardware components at an interface between the DMA core 110 and the set of primary memory units 120 for simultaneously broadcasting data read from the shared memory unit 140 to multiple primary memory units 120. More specifically, the broadcast subsystem 130 includes a set of primary memory unit queues 134, a broadcast scheduler 136, and a broadcast bus 132 that functions as a broadcast-enabled interconnect. Thus, the broadcast system modifies the functionality of the processor system 100 to support broadcast operations between the shared memory unit 140 of the processor system 100 and the primary memory units 120 of the processor system 100. The individual components of the broadcast system are described further below.

[0024] 4.3.1 Primary Memory Unit Queue Generally, primary memory unit queue 134 is a hardware-implemented queue that tracks read and write requests made from DMA core 110 to primary memory units 120 and buffers these sequential data transfer operations. More specifically, primary memory unit queue 134 is a first-in, first-out queue (hereinafter "FIFO queue") electrically coupled to DMA core 110 and to the set of broadcast buses 132. Each element of primary memory unit queue 134 stores a data transfer request from DMA core 110, including a read request or a write request. Thus, primary memory unit queue 134 serves as a request buffer for the set of primary memory units 120 corresponding to the set of processing units 150.

[0025] In one aspect, the processor system 100 includes a primary memory unit queue 134 that defines a subqueue corresponding to each primary memory unit 120. More specifically, the processor system 100 can include a primary memory unit queue 134 that includes a set of primary memory unit 120 subqueues, with each primary memory unit 120 subqueue in the set of primary memory unit 120 subqueues corresponding to a primary memory unit 120 in the set of primary memory units 120. In this aspect, the broadcast scheduler 136 can aggregate similar write requests (i.e., write requests for the same target data) for multiple primary memory units 120 into a single broadcast operation, thereby reducing overhead in the DMA core 110 (e.g., by the DMA core 110 issuing serial write requests that the broadcast subsystem 130 can aggregate into a broadcast or multicast operation). Thus, upon issuing a transfer request for a first primary memory unit 120, the processor system 100 can add a subqueue to the primary memory unit queue 134 that corresponds to the first primary memory unit 120 having the request.

[0026] In another aspect, the processor system 100 includes a primary memory unit queue 134 defining a single subqueue, and a DMA core 110 configured to issue write requests including a set of primary memory unit 120 identifiers indicating a subset of target primary memory units 120 within the set of primary memory units 120, and to receive data corresponding to the write requests. In this aspect, the broadcast scheduler 136 may utilize a logical table to schedule unicast, multicast and / or broadcast operations based on the subset of target primary memory units 120 indicated by the write requests in the primary memory unit queue 134, as described further below.

[0027] In yet another aspect, the processor system 100 includes a primary memory unit queue 134 that defines each broadcast bus 132 subqueue of the broadcast system. More specifically, the primary memory unit queue 134 may include a set of broadcast bus 132 subqueues, where each broadcast bus 132 subqueue in the set of broadcast bus 132 subqueues corresponds to a broadcast bus 132 in the set of broadcast buses 132. For example, in an aspect where the broadcast system includes two broadcast buses 132, the primary memory unit queue 134 may include two subqueues. In this aspect, the processor system 100 includes a DMA core 110 configured to issue write requests indicating unicast or multicast transfer operations to the broadcast bus 132. Additionally or alternatively, the processor system 100 may include a DMA core 110 configured to issue write requests to subqueues corresponding to the broadcast bus 132, where each write request specifies a subset of the primary memory units 120 in the set of primary memory units 120 corresponding to the broadcast bus 132 of the subqueue.

[0028] 4.3.2 Broadcast-Capable Interconnect Generally, the broadcast subsystem 130 includes a set of broadcast buses 132 operating as a broadcast-enabled interconnect configured to transfer data in parallel to multiple primary memory units 120 in the set of primary memory units 120. More specifically, the broadcast subsystem 130 includes a set of broadcast buses 132, each of which is connected to an exclusive subset of the primary memory units 120 and configured to operate two data transfer modes: a unicast mode and a multicast mode. In particular, each broadcast bus 132 in the set of broadcast buses 132 is configured to operate according to one of a set of data transfer modes including a unicast mode to transfer source data to one primary memory unit 120 in the subset of primary memory units 120 and a multicast mode to transfer source data to each primary memory unit 120 in the subset of primary memory units 120. Thus, the processor system 100 can transfer data from the shared memory unit 140 to multiple primary memory units 120 simultaneously or can perform multiple serial data transfer operations in parallel using separate broadcast buses 132.

[0029] As shown in Figure 3, when operating in unicast mode, the broadcast bus 132 forwards data to a single target primary memory unit 120 to which it is connected based on the target primary memory unit 120 indicated in the associated write request (stored in the primary memory unit queue 134), and forwards the data to the destination address indicated in the associated write request. As shown in Figure 4, when operating in multicast mode, the broadcast bus 132 forwards data to all primary memory units 120 to which it is connected, and forwards the data to the same relative destination address in each of the primary memory units 120 to which it is connected.

[0030] In one aspect, the broadcast subsystem 130 includes two broadcast buses 132, each connected to four processing units 150, for a total of eight processing units 150. More specifically, the broadcast subsystem 130 may include a set of broadcast buses 132 including a first broadcast bus 132 electrically coupled to a first subset of the primary memory units 120 in the set of primary memory units 120 and a second broadcast bus 132 electrically coupled to a second subset of the primary memory units 120 in the set of primary memory units 120. In this aspect, the broadcast subsystem 130 may maintain a small on-silicon footprint (compared to a broadcast subsystem 130 including four or eight broadcast buses 132) while allowing for much greater parallelization (e.g., two simultaneous serial data streams over each broadcast bus 132) than a single broadcast bus 132 connected to all eight processing units 150. In one example of this embodiment, a first subset of primary memory units 120 may include a first set of four primary memory units 120, and a second set of primary memory units 120 may include a second set of four primary memory units 120. This exemplary embodiment allows for broadcast functionality of the full set of eight processing units 150, while also maintaining the flexibility to unicast in parallel within the two sets of four processing units.

[0031] Additionally or alternatively, the broadcast subsystem 130 may include a set of broadcast buses 132 (e.g., a pair of broadcast buses 132) disposed or connected to each distinct group of four primary memory units 120. In this alternative example of an exemplary embodiment, the broadcast subsystem 130 may perform multiple broadcast operations to the distinct groups of four primary memory units 120 simultaneously or substantially simultaneously.

[0032] 4.3.3 Broadcast Scheduler In general, the broadcast scheduler 136 serves as an interface between the primary memory unit queues 134 and the set of broadcast buses 132. In particular, the broadcast scheduler 136 may access the primary memory unit queues 134 to read the earliest (i.e., first-in) data transfer request or the earliest set of data transfer requests in the primary memory unit queues 134 to coordinate data transfer operations requested in one or more data transfer requests over the set of broadcast buses 132. More specifically, the broadcast scheduler 136 is configured to, for the first-in data transfer request or the earliest subset of data transfer requests in the set of data transfer requests stored in the primary memory unit queues 134, select a subset of the broadcast buses 132 in the set of broadcast buses 132 for transferring source data to the target subset of the primary memory units 120 according to the first-in data transfer request, and select a data transfer mode (e.g., unicast or multicast) for each broadcast bus 132 in the subset of broadcast buses 132. The processor system 100 may then broadcast the target data from the data transfer operation over the selected broadcast bus 132 and write the target data to the target subset of the primary memory units 120 .

[0033] The broadcast scheduler 136 may be a hardware implemented finite state machine or a microprocessor embedded within the processor system 100 configured to read the data transfer requests in the primary memory unit queue 134. In one aspect, the broadcast scheduler 136 is configured to reorder the data transfer requests in the primary memory unit queue 134 to more efficiently aggregate similar requests in the primary memory unit queue 134 for transfer via broadcast or multicast operations. For example, the broadcast scheduler may identify a set of similar transfer requests in the primary memory unit queue 134, reorder the primary memory unit queue 134 such that the set of similar transfer requests are contiguous in the primary memory unit queue 134, and execute the set of similar transfer requests as a broadcast or multicast operation to a target set of the primary memory units 120.

[0034] In one aspect where the write requests stored in the primary memory unit queue 134 identify a target set of primary memory units 120 for the data transfer operation, the broadcast scheduler 136 may utilize a look-up table or integrated logic circuit to select a subset of the broadcast buses 132 and the data transfer mode of those broadcast buses 132. In one example where the processor system 100 includes eight primary memory units 120, a first broadcast bus 132 connected to a first set of four primary memory units 120 (memory units 0, 1, 2, 3) and a second broadcast bus 132 connected to a second set of four primary memory units 120 (memory units 4, 5, 6, 7), the broadcast scheduler 136 may access the write requests targeted to memory units 0, 5, select the first broadcast bus 132 and the second broadcast bus 132, and select the unicast transfer mode for both broadcast buses 132. In this example, in another case, broadcast scheduler 136 may access a write request targeted to memory units 0, 1, 2, and 3, select the first interconnect, and select the multicast transfer mode for the first broadcast bus 132. In yet another case, as shown in Figure 5, broadcast scheduler 136 may access a write request targeted to all memory units, select both broadcast buses 132, and select the multicast transfer mode for both broadcast buses 132. In yet another case, as shown in Figure 6, broadcast scheduler 136 may access a write request targeted to memory units 0 and 1, detect that the write does not fit into the structure of broadcast bus 132, and serially convert the requests into a first write request to memory unit 1 and a second write request to memory unit 2.

[0035] Thus, the broadcast scheduler 136 can utilize the structure of the primary memory unit queues 134 and the contents of the write requests in the primary memory unit queues 134 to select a set of broadcast buses 132 and select a data transfer mode for the selected set of broadcast buses 132, thereby coordinating the broadcast operation. More specifically, the processor system 100 can include a broadcast scheduler 136 configured to select a combination of broadcast buses 132 from a set of combinations of broadcast buses 132 including a first broadcast bus 132, a second broadcast bus 132, and both the first broadcast bus 132 and the second broadcast bus 132 based on the target subset of the primary memory units 120, and to transfer source data to the target subset of the primary memory units 120 via the selected combination of broadcast buses 132. Furthermore, the broadcast scheduler 136 can select a multicast operation or a unicast operation for each broadcast bus 132 in the selected combination of broadcast buses 132. More specifically, the processor system 100 may include a broadcast scheduler 136 configured to, for each broadcast bus 132 in the subset of broadcast buses 132, select a selected data transfer mode (i.e., unicast or multicast mode) in the set of data transfer modes based on the first-in data transfer requests in the primary memory unit queues 134, and transfer source data to the target subset of the primary memory units 120 according to the selected data transfer mode.

[0036] As described above, in aspects where the primary memory unit queue 134 includes a set of primary memory unit 120 subqueues, the broadcast scheduler 136 may identify a set of similar data transfer requests in a first-in position or set of first-in positions within each primary memory unit 120 subqueue to identify an efficient broadcast operation for accomplishing the set of similar transfer requests. More specifically, the processing system may include a broadcast scheduler 136 configured to identify a set of related data transfer requests within a set of data transfer operations across the set of primary memory unit 120 subqueues that are characterized by common source data, identify a target subset of the primary memory units 120 based on the set of related data transfer requests, and select a subset of the broadcast buses 132 within the set of broadcast buses 132 for transferring the common source data to the target subset of the primary memory units 120 based on the target subset of the primary memory units 120, and transfer the common source data to the target subset of the primary memory units 120 via the subset of the broadcast buses 132. This allows the broadcast scheduler 136 to aggregate multiple data transfer requests across subqueues into a single broadcast or multicast operation.

[0037] In aspects in which broadcast bus 132 is implemented as a network-on-chip, broadcast scheduler 136 can route write requests directly from DMA core 110 to broadcast bus 132. Broadcast bus 132 can then broadcast data corresponding to those write requests through an internal router included in the network-on-chip.

[0038] 4.4 Flow Control Generally, the processor system 100 may include a flow control subsystem to reduce buffer allocation in the primary memory units 120. However, because the processor system 100 includes the broadcast subsystem 130, a bottleneck in a single primary memory unit 120 may prevent efficient data transfer to other primary memory units 120 via the broadcast subsystem 130. For this reason, the processor may include a flow control system specifically designed to prevent those bottlenecks in individual primary memory units 120 and allocate data evenly among the set of primary memory units 120 and corresponding processing units 150.

[0039] In one aspect, the processor system 100 includes a credit-based flow control subsystem. In this aspect, the broadcast subsystem 130 can maintain a number of credits corresponding to each primary memory unit 120 in the set of primary memory units 120. The credit-based flow control system can update the number of credits corresponding to each primary memory unit 120 based on the amount of space available in the primary memory units 120. In this aspect, the broadcast scheduler 136 can then perform a unicast write operation to the primary memory unit 120 with the most credits available. Given an output-fixed scheduling scheme for scheduled parallel processes described further below and assuming homogeneous processing units 150 in the set of processing units 150, the broadcast scheduler 136 can assign a unicast operation to any primary memory unit 120. If the broadcast scheduler 136 assigns multiple partitions in a tier to a single primary memory unit 120, the processing unit 150 corresponding to the primary memory unit 120 can generate an output partition for each input or weight partition consecutively assigned to the primary memory unit 120. Furthermore, in an aspect in which the broadcast scheduler 136 is implemented as a microprocessor, the broadcast scheduler 136 can follow the primary memory unit 120 to which each input or weight partition is assigned and issue a read request to the same primary memory unit 120 to obtain the resulting output partition. Thus, the credit-based flow control system can dynamically reallocate unicast operations within a tier or stage of a scheduled parallel process, thereby ensuring that each primary memory unit 120 in the set of primary memory units 120 contains available space for subsequent broadcast or multicast operations of the scheduled parallel process.Additionally or alternatively, the credit-based flow control system may include a set of reserved credits allocated to broadcast operations over the bus, thereby enabling the broadcast scheduler 136 to reliably schedule broadcast operations during and / or intermittently between unicast operations.

[0040] In another aspect, the processor system 100 can include a handshake-based flow control subsystem. In this aspect, the handshake-based flow control subsystem can issue a binary status indicator indicating whether each primary memory unit 120 in the set of primary memory units 120 is able to receive additional data. In response to the handshake-based flow control subsystem indicating that a subset of the primary memory units 120 is unable to receive data, the broadcast scheduler 136 can halt broadcasting until each of the primary memory units 120 is again able to receive data (as indicated by the handshake-based flow control subsystem). Additionally or alternatively, in response to one or more primary memory units 120 being unable to receive data due to exceeding a threshold number of cycles as indicated by the handshake-based flow control subsystem, the broadcast system can instead split the multicast or broadcast operations in the primary memory unit queue 134 into a set of unicast operations.

[0041] 4.5 Shared Memory Unit Generally, the processor system 100 includes a shared memory unit 140 that is divided into a set of memory modules, allowing for multi-channel reads, multi-channel writes, and independent power mode control of the memory modules. More specifically, the shared memory unit 140 is implemented as multiple RAM memory modules accessible via a multi-channel interface. Additionally, the shared memory unit 140 includes a set of data transfer ports, including a set of read ports, a set of write ports, and / or a set of read / write ports. Thus, the shared memory unit 140 can execute multiple simultaneous read / write requests issued by the DMA core 110 while maintaining a low level of power consumption.

[0042] In one embodiment, the processor system 100 may include a 4 megabyte shared memory unit 140 divided into 64 memory modules. Each memory module is individually addressable via a simultaneous read / write port. Each memory module is also connected to a power management unit, allowing power mode control of the individual memory modules via the power management unit. Additionally, each memory module may report read and write activity to the power management unit, allowing the power management unit to track the idle factor of each memory module of the shared memory unit 140.

[0043] In another aspect, the processor system 100 may include a multi-channel shared memory unit 140 interface that allows for parallel reads and writes to the shared memory unit 140. In one example, the shared memory unit 140 may include a three-channel read / write interface, allowing the processor system 100 to read from and write to three independent memory modules simultaneously. Thus, the processor system 100 may read data from and write data to up to six memory modules simultaneously.

[0044] In yet another aspect, each channel in the shared memory unit 140 interface is assigned to a DMA engine in DMA core 110 dedicated to transferring data between shared memory unit 140 and the set of primary memory units 120, or to a DMA engine dedicated to transferring data between shared memory unit 140 and main memory. In one embodiment, shared memory unit 140 includes more channels dedicated to transferring data between shared memory unit 140 and the set of primary memory units 120 (e.g., two channels dedicated to the set of primary memory units 120 and one channel dedicated to main memory).

[0045] 4.6 Memory Management Subsystem Generally, the memory management subsystem includes a set of hardware components in an interface between the DMA core 110 and the shared memory unit 140. More specifically, the memory management subsystem includes a shared memory unit queue configured to store a second set of data transfer requests associated with the shared memory unit 140, a conflict resolution scheduler, and a power management unit configured, for each memory module in the set of memory modules, to switch the memory module to a sleep mode in response to detecting an idle factor of the memory module greater than a threshold idle factor. Thus, the memory management subsystem increases the data transfer bandwidth between the shared memory unit 140 and the DMA core 110, and thus increases the rate of data transfer between the shared memory unit 140 and the set of primary memory units 120 (e.g., via the DMA core 110) without increasing the power consumption of the shared memory unit 140. Additionally, the memory management subsystem can also prevent read / write collisions (e.g., due to simultaneous reads or writes to the same memory module) despite an increase in read / write requests processed by the shared memory unit 140.

[0046] 4.6.1 Shared Memory Unit Queues Generally, the shared memory unit queue is a hardware-implemented queue that tracks read and write requests made by DMA core 110 to shared memory unit 140. More specifically, the shared memory unit queue is a queue electronically coupled to DMA core 110 that contains a subqueue for each request channel in the shared memory unit 140 interface. Thus, the shared memory unit queue serves as a request buffer for shared memory unit 140.

[0047] In one embodiment, the shared memory unit queue is a FIFO queue that includes a subqueue for each interface channel of the shared memory unit 140. For example, in an embodiment in which the shared queue includes three read / write interface channels, the shared memory unit queue can include six subqueues, each corresponding to a read or write of one of the interface channels.

[0048] In another aspect, the shared memory unit queue may be implemented as a reorder buffer and reordered by a conflict resolution scheduler to prevent read or write conflicts with the shared memory unit 140 from occurring.

[0049] 4.6.2 Conflict Resolution Scheduler Generally, the conflict resolution scheduler is a multiple-read multiple-write scheduler that can access a set of the earliest entries in the shared memory unit queue (running in parallel over the shared memory unit 140 interface channel) to detect potential conflicts between the entries before issuing a corresponding request to the shared memory unit 140. More specifically, the conflict resolution scheduler can be a hardware-implemented finite state machine or microprocessor configured to detect conflicting read or write requests in the shared memory unit queue. In particular, the conflict resolution scheduler can be configured to prevent simultaneous access to memory modules in a set of memory modules through a set of data transfer ports.

[0050] In this manner, the conflict resolution scheduler can prevent fatal read or write errors in the shared memory unit 140, even as the increased bandwidth enabled by the broadcast subsystem 130 increases the rate at which parallel read and write requests are issued by the processor system 100.

[0051] The conflict resolution scheduler detects conflicts between read or write requests in the shared memory unit queue by detecting whether two or more requests stored at the same level of the shared memory unit queue are reading from or writing to the same memory module in the shared memory unit 140 based on the source and destination addresses of the respective request types.

[0052] In one aspect in which the memory modules of the shared memory unit 140 include simultaneous read / write ports, the memory management subsystem may include two conflict resolution schedulers, including a first conflict resolution scheduler for a subqueue of the shared memory unit queue corresponding to read requests and a second conflict resolution scheduler for a subqueue of the shared memory unit queue corresponding to write requests.

[0053] In one aspect where the shared memory unit queue is implemented as a FIFO queue, the contention resolution scheduler can access the set of earliest entries in the shared memory unit queue, detect those entries reading or writing to the same memory module, and block those entries from being issued from all subqueues of the shared memory unit queue that read or write to that memory module. Thus, the contention resolution scheduler can prevent contention between requests issued to the shared memory unit 140.

[0054] In another aspect, where the shared memory unit queue is implemented as a reorder buffer, the contention resolution scheduler can access all entries in the shared memory unit queue and issue them out of order (as opposed to simply stalling subqueues so that conflicting requests are not issued simultaneously). Thus, by including a shared memory unit queue implemented as a reorder buffer, the memory management subsystem can increase utilization of the multi-channel interface of the shared memory unit 140.

[0055] 4.6.3 Power Management Unit In general, the power management unit monitors the activity of the set of memory modules of the shared memory unit 140 individually and selectively activates and deactivates them to reduce the power consumption of the shared memory unit 140. More specifically, the power management unit monitors the reads and writes to each memory module of the shared memory unit 140 and can set the power settings to a sleep mode or an active mode in response to the reads and writes occurring to each memory module. In particular, the power management unit is configured to calculate an idle factor for the memory module for each cycle of the processor system 100 and for each memory module in the set of memory modules based on the number of idle cycles of the memory module since the most recent data transfer request to the memory module. Thus, the power management unit dynamically reduces the number of memory modules currently operating at full power based on the current demands of the scheduled parallel processes being executed by the processor system 100.

[0056] The memory management subsystem may include a power management unit configured to track an idle factor of each memory module in the shared memory unit 140 and set the power mode of the memory module to a sleep mode in response to detecting the idle factor of the memory module exceeding a threshold idle factor. In one aspect, the power management unit calculates the idle factor of each memory module based on the number of clock cycles since the most recent read or write to the memory module. To this end, the power management unit may maintain a counter for each memory module in the shared memory unit 140 to increment a value indicative of the idle factor of the corresponding memory module.

[0057] The power management unit can be configured during initialization of a scheduled parallel process to have a preset threshold idle factor that corresponds to a particular parallel process executed by the processor system 100. In one aspect, the scheduling application calculates the idle factor based on a simulated distribution of access frequencies across the set of memory modules when executing the scheduled parallel process. Thus, the scheduling application can adjust the threshold idle factor based on the particular scheduled parallel task.

[0058] In another aspect, the power management unit is directly connected to DMA core 110 and can access source and destination addresses of requests made to shared memory unit 140 prior to issuance of the requests by DMA core 110. Thus, upon accessing the requests, the power management unit can identify whether the memory module corresponding to the request is currently in sleep mode, and can place the memory module in active mode in response to detecting that the memory module is in sleep mode. Thus, the power management unit can preemptively wake up the memory module from sleep mode in response to accessing read and write requests directly from the DMA, thereby reducing the idle time of processor system 100 caused by waking up the memory module from sleep mode, which can take up to 30 cycles.

[0059] In yet another aspect, the power control unit may be directly connected to a control processor, which may issue instructions to the power management unit to specifically set the power modes of individual memory modules based on a schedule. For example, in anticipation of additional accesses to the shared memory unit 140 (e.g., across additional memory modules currently in sleep mode), the scheduling application may include specific instructions for waking up those additional memory modules, and the control processor may issue those instructions directly to the power management unit.

[0060] 5. Processor System Operation 2, the processor system 100, through the set of components described above, can execute any parallel process and specifically scheduled parallel processes according to the blocks of method S100 described further below. Furthermore, the processor system 100, in cooperation with a scheduling application (e.g., the scheduler described in U.S. patent application Ser. No. 17 / 127,904), can fully utilize the capabilities of the processor system 100 to increase the data transfer bandwidth between the shared memory unit 140 and the set of primary memory units 120, while reducing the power consumption of the shared memory unit 140 and the data transfer operations.

[0061] 5.1 CNN Scheduling Generally, the above-described processor system 100 is configured to execute scheduled parallel processing that utilizes the broadcast capability of the processor system 100. According to the method S100, the processor system 100 executes a statically scheduled CNN by computing a first subset of layers of the CNN according to an input broadcast-output fixed data flow in blocks S110, S120, S130, and S140, and computing a second subset of layers of the CNN according to a weight broadcast-output fixed data flow in blocks S150, S160, S170, and S180. Prior to execution of the method S100 by the processor system 100, the scheduling application can calculate, for each layer of the CNN, whether an input broadcast data flow or a weight broadcast data flow is more efficient for the layer, and schedule data transfer operations corresponding to the calculated data flow based on the characteristics of the layer. Thus, the processor system 100 exploits the scheduling process of the scheduling application and the broadcast capability of its hardware to execute the CNN with reduced inference time and power consumption.

[0062] In one aspect, a scheduling application can evaluate each layer of a CNN based on heuristics such as the relative sizes of the layer's input tensors and the layer's weight tensors. For example, the scheduling application can calculate the difference between the size of the input tensor and the size of the weight tensor for each layer and classify the layer as either an input broadcast layer or a weight broadcast layer based on the difference. In another example, the scheduling application can classify a first layer as an input broadcast layer in response to detecting that the size of the layer's input tensor exceeds the size of the layer's weight tensor, and classify a second layer as a weight broadcast layer in response to detecting that the size of the layer's weight tensor exceeds the size of the layer's input tensor.

[0063] In another aspect, for each layer, the scheduling application can simulate the execution of the layer according to the input broadcast data flow and the weight broadcast data flow (e.g., via a virtualized version of the processor system 100) and select the data flow of the layer based on a preset objective, such as minimizing the inference time or minimizing the power consumption. Thus, the scheduling application can estimate the exact processing time and power consumption for each type of data flow based on the characteristics of the processor system 100 and the particular CNN executed on the processor system 100, and select the data flow for each layer of the CNN according to the preset objective and the results of the simulation.

[0064] 6. Layer Storage Generally, the processor system 100 can store layers of an artificial neural network (e.g., CNN, etc.) including input tensors and weight tensors in response to a set of scheduled transfer operations between the main memory of the processor system 100 and the shared memory unit 140 of the processor. The processor system 100 can store a set of input tensors and weight tensor partitions for an input broadcast-output fixed data flow, or the processor system 100 can store a set of weight tensors and input tensor partitions for a weight broadcast-output fixed data flow. More specifically, the processor system 100 can store a first weight tensor including a set of weight tensor partitions at a first source address and a first input tensor larger than the first weight tensor at a second source address in the shared memory unit 140 in block S110. Further, in block S150, the processor system 100 may store the second weight tensor at a third source address and the second input tensor at a fourth source address in the shared memory unit 140, where the second input tensor includes the set of input tensor partitions and is smaller than the second weight tensor, so that the processor system 100 can access the inputs of the scheduled layer from the shared memory unit 140.

[0065] 7. Running the broadcast Generally, processor system 100 may perform broadcast operations (e.g., blocks S120, S160 of method S100) via broadcast subsystem 130 by retrieving a set of instructions from a scheduled parallel process into a buffer or set of buffers in DMA core 110. DMA core 110 may then issue those instructions to associated resources in processor system 100 to perform the broadcast operation on source data in shared memory unit 140. More specifically, processor system 100 may issue a read request for source data to the memory management subsystem, transfer the source data from shared memory unit 140 to internal data buffer 112 of DMA core 110, transfer a write request (including the source data) indicating a relative destination address (at each primary memory unit 120 in the set of target primary memory units 120) to broadcast subsystem 130 (e.g., primary memory unit queue 134), and simultaneously transfer the source data from the shared memory unit queue to the relative destination address at each primary memory unit 120 in the set of target primary memory units 120 over a set of broadcast buses 132 of processor system 100. Thus, the system may transfer source data from shared memory unit 140 to a set of primary memory units 120 in parallel via a single read request and a single write request.

[0066] While simultaneously transferring source data from the data buffer 112 of the DMA access core to the set of target primary memory units 120, the processor system 100 can include the source data for the broadcast operation in a write request sent from the DMA core 110 to the broadcast subsystem 130. To this end, the processor system 100 can transfer the source data to an intermediate buffer in the broadcast subsystem 130 before broadcasting the source data over the set of broadcast buses 132. Alternatively, the processor system 100 can store the source data of each data transfer request in a primary memory unit queue 134, where the source data is combined with a corresponding relative destination address and any other instructions, such as a data transfer mode (e.g., unicast, multicast, or broadcast) of the data transfer operation, the target primary memory unit 120 of the data transfer request, or the target broadcast bus 132 of the data transfer operation.

[0067] In one aspect, the processor system 100 may issue a data transfer request to the broadcast subsystem 130 indicating a set of target primary memory units 120, and for each broadcast bus 132 in the set of broadcast buses 132 of the broadcast subsystem 130, select a data transfer mode for the broadcast bus 132 based on the set of target primary memory units 120 indicated in the data transfer request. Thus, the processor system 100 may issue a data transfer request indicating a set of target primary memory units 120, and select a set of broadcast buses 132 and a corresponding data transfer mode to satisfy the data transfer request.

[0068] In another aspect, the processor system 100 may issue data transfer requests to the broadcast subsystem 130 indicating a set of target broadcast buses 132 and a target data transfer mode for each target broadcast bus 132, and for each broadcast bus 132 in the set of target broadcast buses 132, select a data transfer mode for the broadcast bus 132 based on the indicated data transfer mode. Thus, the processor system 100 may reduce overhead in the broadcast subsystem 130 by increasing the specificity of the data transfer requests issued to the broadcast subsystem 130.

[0069] Typically, method S100 includes executing an input broadcast layer and a weight broadcast layer scheduled by a scheduling application for the artificial neural network. Blocks of method S100 corresponding to the input broadcast layer and blocks of method S100 corresponding to the weight broadcast layer are respectively described below.

[0070] 8. Executing the Input Broadcast Layer Generally, the processor system 100 may broadcast a first input tensor from a source address to a relative destination address within the set of primary memory units 120 in block S120. More specifically, the processor system 100 may issue a first read request via the direct memory access core 110 for the first input tensor at a second source address, load the first input tensor into the data buffer 112 in response to the first read request, issue a first write request via the direct memory access core 110 to the broadcast subsystem 130 specifying the first relative destination address, and transfer the first input tensor from the data buffer 112 to the first relative destination address in each primary memory unit 120 in the set of primary memory units 120 via the set of broadcast buses 132 of the processor system 100 simultaneously. Thus, in block S120, the processor system 100 broadcasts the input tensor to each of the set of primary memory units 120 of the processor system 100.

[0071] To compute the output for the input broadcast layer of the artificial neural network, the processor system 100 also distributes a set of weight partitions, each partition including a subsection of the weight tensor for the input broadcast layer. More specifically, the processor system 100 can transfer a weight tensor partition in the set of weight tensor partitions from a first source address to a first destination address in the primary memory unit 120 of the processing unit 150 for each processing unit 150 in the set of processing units 150 in block S130. In this manner, the processor system 100 can broadcast the input tensors of the layer of the artificial neural network and unicast the set of weight partitions serially, thereby making the input data and weight partition data available to each processing unit in the set of processing units.

[0072] Upon receiving both the input data and the weight partition data at each primary memory unit 120 in the set of target primary memory units 120, the processor system 100 can calculate, via a set of processor units corresponding to the target set of primary memory units 120, an output partition based on the input tensor and the weight partition for each primary memory unit 120 in the set of target primary memory units 120. More specifically, the processor system 100 can generate, in block S140, an output tensor partition of the first output tensor based on the first input tensor and the weight tensor partition in the processing unit 150. Thus, by repeatedly executing blocks S110, S120, S130, S140 over successive layers of the artificial neural network, the processor system 100 can continuously execute the scheduled parallel processes.

[0073] In one aspect, the processor system 100 may generate an output tensor partition of an output tensor based on the input tensor and the weight tensor partition by performing a convolution operation based on a first input tensor and the weight tensor partition via a set of processor units and a corresponding set of target primary memory units 120. The processor system 100 may generate the output tensor partition by performing a one-dimensional convolution or a two-dimensional convolution via the set of processor units. Additionally or alternatively, the processor system 100 may perform any other tensor operation based on one or more tensors stored in the set of target primary memory units 120.

[0074] 9. Running the weight broadcast layer Generally, the processor system 100 may broadcast the weight tensor from a source address to relative destination addresses within the set of primary memory units 120 in block S160. More specifically, the processor system 100 may issue a second read request via the direct memory access core 110 for the second weight tensor at a third source address, load the second weight tensor into the data buffer 112 in response to the second read request, issue a second write request via the direct memory access core 110 to the broadcast subsystem 130 specifying the second relative destination address, and transfer the second weight tensor from the data buffer 112 to the second relative destination address in each primary memory unit 120 in the set of primary memory units 120 simultaneously via the set of broadcast buses 132 of the processor system 100. Thus, in block S160, the processor system 100 broadcasts the weight tensor to each of the set of primary memory units 120 in the processor system 100.

[0075] To compute the output for the weight broadcast layer of the artificial neural network, the processor system 100 also distributes a set of input partitions, each input partition including a subsection of the input tensor for the weight broadcast layer. More specifically, the processor system 100 can transfer, for each processing unit 150 in the set of processing units 150, an input tensor partition in the set of input tensor partitions from a fourth source address to a second destination address in the primary memory unit 120 of the processing unit 150 in block S170. In this manner, the processor system 100 can broadcast the weight tensors of the layer of the artificial neural network and unicast the set of input tensor partitions serially, thereby making the input data and weight partition data available to each processing unit in the set of processing units for the weight broadcast layer.

[0076] Upon receiving both the input partition data and the weight data at each primary memory unit 120 in the set of target primary memory units 120, the processor system 100 can calculate an output partition based on the input tensor partition and the weight tensor for each primary memory unit 120 in the set of target primary memory units 120 via a set of processor units corresponding to the target set of primary memory units 120. More specifically, the processor system 100 can generate, in block S180, an output tensor partition of the first output tensor based on the second weight tensor and the input tensor partition at the processing unit 150. Thus, by repeatedly executing blocks S150, S160, S170, S180 over successive layers of the artificial neural network, the processor system 100 can continuously execute the scheduled parallel processing.

[0077] As described above with respect to the execution of the input broadcast layer, the processor system 100 may generate an output tensor partition of the output tensor based on the weight tensor and the input tensor partition by performing a convolution operation based on the second weight tensor and the input tensor partition via the set of processor units and the corresponding set of target primary memory units 120. The processor system 100 may generate the output tensor partition by performing a one-dimensional convolution or a two-dimensional convolution via the set of processor units. Additionally or alternatively, the processor system 100 may perform any other tensor operation based on one or more tensors stored in the set of target primary memory units 120.

[0078] The systems and methods described herein may be embodied and / or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions. The instructions may be executed by a computer-executable component integrated with an application, an applet, a host, a server, a network, a website, a communication service, a communication interface, a hardware / firmware / software element of a user computer or mobile device, a wristband, a smartphone, or any suitable combination thereof. Other systems and methods of embodiments may be embodied and / or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions. The instructions may be executed by a computer-executable component integrated with the above-mentioned types of devices and networks. The computer-readable medium may be stored in any suitable computer-readable medium, such as a RAM, a ROM, a flash memory, an EEPROM, an optical device (CD or DVD), a hard drive, a floppy drive, or any suitable device. The computer-executable component may be a processor, although any suitable dedicated hardware device may (alternatively or additionally) execute the instructions.

[0079] As will be appreciated from the above detailed description and drawings, and the appended claims, those skilled in the art can make modifications and variations to the embodiments of the invention without departing from the scope of the invention, which is defined in the following claims.

Claims

1. 1. A processor system comprising: - a direct memory access core including a data buffer; - a set of primary memory units; A broadcast subsystem, a set of broadcast buses, each broadcast bus in the set of broadcast buses electrically coupled to a subset of primary memory units in the set of primary memory units; o A primary memory unit queue, configured to store a first set of data transfer requests associated with the set of primary memory units; and a primary memory unit queue electrically coupled to the data buffer; o a broadcast scheduler, electrically coupled to the primary memory unit queue; electrically coupled to the set of broadcast buses; and a broadcast scheduler configured to transfer source data from the data buffer via the set of broadcast buses to a target subset of primary memory units in the set of primary memory units based on the set of data transfer requests stored in the primary memory unit queues; and A broadcast subsystem that includes A processor system comprising:

2. 2. The processor system according to claim 1, The processor system further comprising a shared memory unit electrically coupled to the data buffer.

3. 3. The processor system according to claim 2, Further comprising a memory management subsystem, the memory management subsystem comprising: a shared memory unit queue configured to store a second set of data transfer requests associated with the shared memory unit; and o for each memory module in the set of memory modules, a power management unit configured to switch the memory module to a sleep mode in response to detecting an idle factor of the memory module greater than a threshold idle factor; said shared memory unit comprising: o a set of said memory modules; o a set of data transfer ports.

4. 4. The processor system according to claim 3, The processor system, wherein the power management unit is further configured to calculate, in each cycle of the processor system, for each memory module in the set of memory modules, an idle coefficient for the memory module based on the number of idle cycles of the memory module since the most recent data transfer request to the memory module.

5. 4. The processor system according to claim 3, 4. A processor system according to claim 3, wherein the shared memory unit queue includes a reorder buffer.

6. 4. The processor system according to claim 3, 11. The processor system of claim 1, wherein the memory management subsystem further comprises a conflict resolution scheduler configured to prevent simultaneous access to memory modules in the set of memory modules via the set of data transfer ports.

7. 4. The processor system according to claim 3, and a set of direct memory access cores, the set of direct memory access cores comprising: a first subset of direct memory access cores configured to issue a first set of data transfer requests to the primary memory unit queue, the first subset of direct memory access cores including the direct memory access core; a second subset of direct memory access cores configured to issue a second set of data transfer requests to the shared memory unit queue.

8. 2. The processor system according to claim 1, 11. A processor system further comprising a set of processing units, each processing unit in the set of processing units corresponding to a primary memory unit in the set of primary memory units.

9. 2. The processor system according to claim 1, 11. A processor system comprising: a direct memory access core including a tensor traversal engine configured to issue data transfer requests specifying multi-dimensional data transfer operations.

10. 1. A broadcast subsystem in a processor system, comprising: a set of broadcast buses, each broadcast bus in the set of broadcast buses being electrically coupled to a subset of primary memory units in the set of primary memory units of the processor system; a primary memory unit queue configured to store a set of data transfer requests associated with the set of primary memory units, the primary memory unit queue including a set of primary memory unit subqueues, each primary memory unit subqueue in the set of primary memory unit subqueues corresponding to a primary memory unit in the set of primary memory units; A broadcast scheduler, o electrically coupled to the primary memory unit queue; o electrically coupled to the set of broadcast buses; and a broadcast scheduler based on the set of data transfer requests stored in the primary memory unit queue; A broadcast subsystem comprising:

11. 11. The broadcast subsystem of claim 10, the broadcast scheduler is further configured to select a subset of broadcast buses in the set of broadcast buses for first-in data transfer requests in the set of data transfer requests stored in the primary memory unit queues, and to transfer source data to the target subset of the primary memory units in accordance with the first-in data transfer requests.

12. 12. The broadcast subsystem of claim 11, each broadcast bus in the set of broadcast buses is configured to operate according to one of a set of data transfer modes, the set of data transfer modes being o a unicast mode for transferring source data to one primary memory unit in the subset of primary memory units; a multicast mode for transferring source data to each primary memory unit in the subset of primary memory units; the broadcast scheduler, for each broadcast bus in the subset of broadcast buses and for a first-come data transfer request in a queue of the primary memory unit, selecting a selected data transfer mode within the set of data transfer modes based on a first-in data transfer request in the primary memory unit queue; and o a broadcast subsystem further configured to transfer source data to a target subset of said primary memory units in accordance with said selected data transfer mode.

13. 11. The broadcast subsystem of claim 10, the set of broadcast buses a first broadcast bus electrically coupled to a first subset of primary memory units in the set of primary memory units; a second broadcast bus electrically coupled to a second subset of primary memory units in the set of primary memory units.

14. 14. The broadcast subsystem of claim 13, the first subset of primary memory units includes a first set of four primary memory units; a broadcast subsystem, wherein the second subset of primary memory units includes a second set of four primary memory units different from the first set of four primary memory units.

15. 14. The broadcast subsystem of claim 13, The broadcast scheduler, configured to select a broadcast bus combination from a set of broadcast bus combinations based on the target subset of primary memory units, the set of broadcast bus combinations comprising: o a first broadcast bus; and a second broadcast bus; and a first broadcast bus and a second broadcast bus; The broadcast subsystem, wherein the broadcast scheduler is further configured to forward source data to a target subset of the primary memory units via the combination of broadcast buses.

16. 11. The broadcast subsystem of claim 10, The broadcast scheduler, identifying a set of related data transfer requests in a set of data transfer operations spanning the set of primary memory unit subqueues, the set of related data transfer requests being characterized by a common source data; - identifying a target subset of primary memory units based on the set of related data transfer requests; selecting a subset of broadcast buses in the set of broadcast buses to transfer the common source data to the target subset of primary memory units based on the target subset of primary memory units; a broadcast subsystem configured to transfer the common source data to a target subset of the primary memory units over a subset of the broadcast bus.

17. 11. The broadcast subsystem of claim 10, a primary memory unit queue including a set of broadcast bus subqueues, each broadcast bus subqueue in the set of broadcast bus subqueues corresponding to a broadcast bus in the set of broadcast buses.

18. 11. The broadcast subsystem of claim 10, The broadcast subsystem further comprising a credit-based flow control subsystem.

19. 11. The broadcast subsystem of claim 10, The broadcast subsystem further comprising a handshake-based flow control subsystem.

Images