Circuits and methods for direct memory access using network-on-chip

Abstract

Circuits and methods for direct memory access using a network-on-chip are provided. A configurable integrated circuit includes a network-on-chip and a response buffer circuit coupled to the network-on-chip. The response buffer circuit includes a direct memory access circuit and a controller circuit. Direct memory access circuitry generates read and write requests to access memory circuitry. The controller circuit provides a read request and a write request to the memory circuit over the network-on-chip. The controller circuit exchanges data for read requests and write requests with the memory circuit.

Description

Technical Field

[0001] This disclosure relates to circuitry and methods for direct memory access using on-chip networks. Background Technology

[0002] Configurable integrated circuits (ICs) can be configured by users to implement desired custom logic functions. In a typical scenario, logic designers use computer-aided design (CAD) tools to design the custom circuit. When the design process is complete, the CAD tool generates an image containing configuration data bits. These configuration data bits are then loaded into configuration memory elements, which configure the configurable logic circuitry within the IC to perform the functions of the custom circuit design. Summary of the Invention

[0003] In one aspect, embodiments of this application provide a configurable integrated circuit, including: a first on-chip network; and a response buffer circuit coupled to the first on-chip network, wherein the response buffer circuit includes a direct memory access circuit and a controller circuit, wherein the first on-chip network is embedded in the configurable integrated circuit, wherein the direct memory access circuit generates a first read request and a first write request to access the first memory circuit received from a host circuit, wherein the controller circuit provides the first read request and the first write request to the first memory circuit through the first on-chip network, and wherein the controller circuit exchanges first data for the first read request and the first write request with the first memory circuit.

[0004] In another aspect, embodiments of this application provide a method for performing read and write transactions in a configurable integrated circuit, the method comprising: generating read requests for read transactions and write requests for write transactions of accessing a memory circuit using a direct memory access circuit in the configurable integrated circuit; providing the read requests and write requests from the direct memory access circuit to the memory circuit via a first on-chip network in the configurable integrated circuit using a scheduler circuit in the configurable integrated circuit; and exchanging data for the read requests and write requests with the memory circuit using the scheduler circuit.

[0005] In another aspect, embodiments of this application provide a non-transitory computer-readable storage medium including instructions stored thereon, which, when executed by a configurable integrated circuit, cause the configurable integrated circuit to: generate read and write requests for accessing memory circuits using direct memory access circuitry; provide the read and write requests from the direct memory access circuitry to the memory circuitry via an on-chip network using controller circuitry; and exchange data for the read and write requests with the memory circuitry using the controller circuitry. Attached Figure Description

[0006] Figure 1 This is a diagram illustrating a portion of the microarchitecture of an integrated circuit (IC), which includes a main on-chip network (MNOC), a response buffer circuit including direct memory access (DMA) circuitry, and a micro on-chip network (uNOC).

[0007] Figure 2A This illustrates an example of a host performing a status query by sending a read request along with an initial write request and the written data to a response buffer circuit.

[0008] Figure 2B This is an example of a host using push / push write requests to a local mailbox to perform a status query.

[0009] Figure 3 This is a diagram illustrating an example of a system in which the host sends... Figure 1 The DMA circuit sends a descriptor, then sends a status request for the transaction, and finally receives a status response.

[0010] Figure 4 It is shown Figure 1 A diagram illustrating the states of a finite state machine (FSM) circuit and examples of transitions between states.

[0011] Figure 5 This is a diagram showing the components in a structural sector of an integrated circuit (IC) that enable read and write transactions to the memory.

[0012] Figure 6 This is an illustration showing an example of a system that can be used for two-dimensional (2D) read or write applications.

[0013] Figure 7 This is an illustration showing an example of a system that can be used for three-dimensional (3D) reading or writing applications.

[0014] Figure 8 This is an illustration of a configurable integrated circuit (IC).

[0015] Figure 9A A block diagram is shown of a system that can be used to implement circuit designs programmed into programmable logic devices using design software.

[0016] Figure 9B This is an illustration depicting an example of a programmable logic device, which includes a structural die and a base die coupled to each other via microbumps.

[0017] Figure 10 This is a block diagram illustrating a computing system configured to implement one or more aspects of the embodiments described herein. Detailed Implementation

[0018] In some types of previously known configurable integrated circuits (ICs), such as field-programmable gate arrays (FPGAs), direct and remote proxy transactions to memory circuitry within the IC are initiated by an external host. This technique does not allow for step-by-step memory access or indirection, is not optimized for use cases utilizing a central scheduler, and is not optimized for three-dimensional (3D) use cases.

[0019] Based on some examples disclosed herein, DMA (Direct Memory Access) circuitry within response buffer (RB) circuitry in an integrated circuit (IC) efficiently performs access to memory circuit blocks by emphasizing transaction indirectness to control transactions across the entire IC (e.g., within a structural region of the IC). The response buffer circuitry is coupled to a micro-NOC (micro-NOC). DMA circuitry improves overall system efficiency and throughput by enabling a central scheduler (or multiple schedulers) to control transactions across the entire IC. DMA enables artificial intelligence (AI) use cases, more direct communication with three-dimensional (3D) memory, and communication with embedded data in block random access memory (BRAM) within the IC.

[0020] According to some examples, the DMA circuit may also include: a finite state machine (FSM) that expands received descriptors including read or write requests to be issued; a control status register (CSR) circuit that includes content-addressable memory (CAM) to track incomplete transactions; and an error monitoring circuit block that continuously polls returned error signals. Each transaction can be a read transaction or a write transaction. As disclosed below, each read transaction may include one or more read requests and / or each write transaction may include one or more write requests.

[0021] The DMA circuitry also includes critical stepping and transaction control capabilities. Furthermore, the DMA circuitry can update the mailbox (e.g., a host-local mailbox) upon transaction completion. The DMA circuitry offers the advantage of indirection, allowing users to efficiently access the IC and all embedded BRAMs within the external 3D memory IC that communicates with the IC along the microchip network-on-a-chip (uNOC). Because the mailbox is host-local, it minimizes the initial latency required for host polling completion, thereby improving overall transaction efficiency. The DMA circuitry is also highly configurable, allowing users maximum flexibility in transfer completion synchronization, mailbox status location and configuration, and advanced interface operations.

[0022] One or more specific examples are described below. To provide a concise description of these examples, not all characteristics of the actual implementation are described in this specification. It should be understood that, as in any engineering or design project, the development of any such actual implementation requires numerous implementation-specific decisions to achieve the developer's specific goals, such as adhering to system-related and business-related constraints, which may vary depending on the implementation. Furthermore, it should be understood that such development work may be complex and time-consuming, but it remains a routine task of design, fabrication, and production for those of ordinary skill in the art who benefit from this disclosure.

[0023] Throughout the specification and claims, the terms "connected" and "connected" refer to a direct electrical connection between connected circuits without any intermediate devices. The terms "coupled" and "coupled" refer to a direct electrical connection between circuits or an indirect electrical connection via one or more passive or active intermediate devices (which allow information transfer between circuits). The term "circuit" can refer to one or more passive and / or active electrical components arranged to cooperate with each other to provide a desired function.

[0024] This disclosure discusses integrated circuit devices, including configurable (programmable) integrated circuits such as field-programmable gate arrays (FPGAs) and programmable logic devices. As discussed herein, integrated circuits (ICs) can include hard logic and / or soft logic. The circuitry in an integrated circuit device (e.g., in a configurable IC) that can be configured by an end user is referred to as “soft logic.” “Hard logic” generally refers to circuitry in an integrated circuit device that has far fewer configurable features than soft logic, or no configurable features at all.

[0025] Data transfers between Block Random Access Memory (BRAM) and peripheral memory occur frequently in configurable ICs. To alleviate the burden on the host / initiator to manually move data within the system, Direct Memory Access (DMA) circuitry is provided in the response buffer circuitry to offer efficient indirection and conserve soft logic resources for computational purposes rather than routing. DMA circuitry can use descriptors to perform data movement operations across and between BRAM, 3D memory, and peripheral memory. Descriptors can have different transfer sizes and different source and destination addresses. Within each descriptor, the transaction request specified by the host is fully configurable. Examples of host placement include structural regions (including configurable logic) within a configurable integrated circuit (IC), hard or soft processors within the configurable IC, or IC dies connected to the configurable IC.

[0026] The DMA circuitry works in conjunction with the Processing Elements (PEs) and the host scheduler. The host scheduler handles data transfers between off-chip or remote memory and on-chip memory coupled to the microNOC by sending descriptors to the DMA. Descriptors can be generated at compile time and written to the interface from the host scheduler. The host scheduler has the ability to send descriptors to any DMA circuit in the IC. The IC can have one or more DMA circuits in each Response Buffer (RB) circuit in the IC. The PE is involved in data operations performed in the memory coupled to the microNOC. Each Response Buffer (RB) circuit has at least one DMA circuit that issues commands to one or more microNOC columns coupled to that RB circuit according to the unfolding of the corresponding descriptor. Rules determine when the host scheduler can send a descriptor to avoid corrupted / invalid data on the memory coupled to each microNOC.

[0027] The DMA circuit receives descriptors from channels embedded in the interface of the RB circuit. Both the RB circuit and the DMA circuit within it are end-to-end backpressured. Due to the ring structure of the NOC, the end-to-end backpressure of the microNOC flow, and the structure of the DMA circuit, deadlock will not occur on the microNOC. Because of these characteristics, virtual channels on the microNOC are not necessary. If a new descriptor arrives and the DMA circuit stalls, the descriptor is backed up to the NOC without causing a deadlock.

[0028] Figure 1 This is a diagram illustrating a portion of the microarchitecture of an integrated circuit (IC), including a main on-chip network (MNOC) 101, a response buffer (RB) circuit 102, a micro on-chip network (micro NOC or uNOC) 103, and a block random access memory (BRAM) 125. The response buffer (RB) circuit 102 includes a DMA (Direct Memory Access) circuit 104 and a response buffer (CB) controller / scheduler circuit 110. The DMA circuit 104 includes a command first-in-first-out (FIFO) circuit 105, a control status register (CSR) circuit 107, an error monitor circuit 108, a finite state machine (FSM) circuit 109, multiplexer circuits 111-112, and a barrel shifter (BS) / FIFO circuit 121-122. The CSR circuit 107 includes a register 113 for pause, a register 114 for reset, and a content-addressable memory (CAM) 115 for status and error storage. Error monitor circuit 108 includes error polling circuit 116. FSM circuit 109 processes read transactions 117 and write transactions 118. Figure 1 Dashed lines are shown around some blocks to improve the visibility of signal lines entering and exiting these blocks. The DMA circuit 104 can be optimized to have a small IC area footprint.

[0029] RB 102 is coupled to MNOC 101 and micro NOC 103 via conductors. According to some examples, the IC can have, for instance... Figure 1 The multiple RB circuits 102 shown are coupled to MNOC 101 and micro NOC 103. Figure 1 The IC can be any type of integrated circuit (IC), such as a configurable IC (e.g., a field-programmable gate array (FPGA) or a programmable logic device (PLD)), a microprocessor IC, a graphics processing unit IC, a memory IC, an application-specific IC, a transceiver IC, etc. In the examples disclosed herein, for illustrative purposes, circuits, methods, and systems are described in the context of configurable ICs such as FPGAs or PLDs.

[0030] Figure 1 The response buffer (RB) circuit 102 can be used to perform two-dimensional (2D) read and write transactions as well as three-dimensional (3D) read and write transactions. 2D transactions are those going to and from... Figure 1 The memory circuitry within the IC and along the length and / or width of the IC Figure 1 The IC is coupled to the external peripheral memory IC in the same plane for read and write transactions of the memory circuit. 3D transactions are to and from the IC. Figure 1 This refers to read and write transactions of external memory ICs coupled to and stacked vertically. Each read transaction may include one or more read requests and / or each write transaction may include one or more write requests. Read and write transactions are also referred to as transactions in this document.

[0031] In the following discussion and figures, various terms used in the Advanced Extensible Interface 4 (AXI4) interface protocol are referenced as examples, and these examples are not intended to be limiting. As used below, the initiator can be the administrator, and the target can be the subordinate. It should be understood that the techniques disclosed herein can be used with any interface protocol. The AXI4 channels, requests, and responses used herein are listed below.

[0032] AR-I is a read request originating from the initiator (management interface).

[0033] AR-T is a read request (or status request) to enter the target (subordinate interface).

[0034] RI is the read response returned to the initiator.

[0035] RT is the read response (or status response) leaving the target.

[0036] AW-I is a write request originating from the initiator.

[0037] AW-T is a write request to the target.

[0038] WI is the write data channel originating from the initiator.

[0039] WT is the write data channel to the target.

[0040] BI is the write response returned to the initiator.

[0041] BT is the write response that leaves the target.

[0042] The following discussion also references some specific AXI4 signals. For example, ARID is the read address ID signal from the AR channel, which is an identifier label for a set of read address signals. ARID matches the RID generated for the transaction. RID is the read ID tag signal from the R channel, which is an identifier label for a set of read data signals generated by the target. RID matches the ARID given for the transaction. AWID is the write address ID signal from the W channel, which is an identifier label for a set of write address signals. AxID refers to both AWID and ARID.

[0043] RRESP is a signal from the R channel that indicates the status of a transaction. RRESP indicates whether the transaction was executed, or whether the transaction hit a fail-safe error or a decode error. BRESP is a signal from the B channel that indicates the status of a transaction. BRESP indicates whether the transaction was executed, or whether the transaction hit a fail-safe error or a decode error. xRESP refers to both RRESP and BRESP.

[0044] RLAST is the last read signal from the R channel, indicating the last transfer in a read transaction. WLAST is the last write signal from the W channel, indicating the last transfer in a write transaction. xLAST refers to both RLAST and WLAST. WDATA-T is the write data field in the W channel that enters the target interface. In the context of DMA circuit 104, WDATA-T is the incoming signal containing the descriptor.

[0045] SOP (Start of Group) is a signal sent to the memories coupled to the micro NOC 103 to ensure that these memories are ready to read data from and write data to the micro NOC 103. SID (Stream Identifier) ​​is an identifier that logically groups the block random access memory (BRAM) 125 along the columns of the micro NOC 103.

[0046] From the perspective of the target RB circuit 102, a read transaction occurs when the DMA circuit 104 initiates a read request (AR-I) from the RB circuit 102. The RB controller / scheduler circuit 110 receives the read request (AR-I) from the DMA circuit 104 and then sends the read request (AR-I) to the memory via the MNOC 101. The read response (RI) data generated in response to the read request (AR-I) is returned from the memory to the RB controller / scheduler 110 via the MNOC 101, and then the RB controller / scheduler 110 writes the read response (RI) data to the block random access memory (BRAM) 125 via the micro NOC 103.

[0047] In a write transaction, DMA circuit 104 generates a read request (AR-T), and RB controller / schedule 110 sends the read request (AR-T) via microNOC 103 to read write data from BRAM 125. Then, a write request (AW-I) is generated from an expanded descriptor specifying each configurable AW field. The microNOC provides a read response (RT), which becomes write data (WI) issued from RB circuit 102 to MNOC 101. DMA circuit 104 generates a write request (AW-I), and RB controller / schedule 110 provides the write request (AW-I) and write data (WI) to the memory via MNOC 101. AW-I and WI commands do not necessarily need to be issued simultaneously. A write response (BI) from the memory (e.g., peripheral memory) is then returned to DMA circuit 104 for processing. In some examples, the AR / AW request is not sent directly, but rather information about the address range to be read or written is transmitted by RB circuit 102. In other examples, the complete AR / AW request is sent by RB circuit 102. As used herein, the transfer follows the typical DMA definition of moving an entire descriptor block. DMA circuit 104 can serialize the requests and return them in the order they were received from the host.

[0048] The high-level flow of a 2D read transaction is now described. Initially, after a handshake between RB circuit 102 and the host, a descriptor embedded in the W channel is transferred to DMA circuit 104 in the IC. DMA circuit 104 acts as the target when receiving descriptor commands. FSM 109 in DMA circuit 104 expands the descriptor into a read request (AR-I) and a write request (AW-T). BS / FIFO 121 serializes these commands such that AW-T is sent first to RB controller / scheduler 110, followed by AR-I. In some embodiments, RB controller / scheduler 110 uses address information embedded in AW-T to send the start of packet (SOP) to ensure BRAM 125 is ready to receive incoming read response (RI) data. RB controller / scheduler 110 then issues a read request (AR-I) via MNOC 101 to read from memory (e.g., peripheral memory). DMA circuit 104 acts as the initiator when issuing a new transaction. Finally, the read response (RI) data is returned from memory to the RB controller / scheduler 110 via MNOC 101. The RB controller / scheduler 110 then maps the incoming read response (RI) data to be written as write data (WT) to the correct BRAM 125 via the columns of micro NOC 103. The connection to the host is then complete.

[0049] The high-level flow of a 2D write transaction is now described. Initially, after a handshake between RB circuit 102 and the host, a descriptor is transferred to DMA circuit 104. DMA circuit 104 acts as the target when receiving a descriptor command. DMA circuit 104 expands the descriptor into a write request (AW-I). To obtain the write data (WI), DMA circuit 104 generates a read request (AR-T). In some embodiments, RB controller / scheduler 110 then sends an SOP to the target BRAM 125 via micro NOC 103, followed by the read request (AR-T). The read response data (RT / WI) is then sent from the BRAM back to RB controller / scheduler 110 in RB circuit 102 via micro NOC 103. RB controller / scheduler 110 then sends the write request AW-I and write data WI received from BRAM 125 via MNOC 101 to write to memory (e.g., peripheral memory). The AW-I and WI commands do not necessarily need to be issued simultaneously. DMA circuit 104 acts as the initiator when issuing a new transaction. The RB controller / scheduler 110 receives the write response BI returned from the memory. Then, the connection with the host is completed.

[0050] As a specific example not intended to be limiting, the command first-in-first-out (FIFO) circuit 105 can be an 8-depth, 256-bit wide FIFO buffer that holds all DMA descriptors awaiting expansion. In some implementations, once a descriptor is loaded into the command FIFO circuit 105, the write response BT is returned from the command FIFO circuit 105 to the host via multiplexer circuit 111.

[0051] In other implementations, the write response BT is returned after the complete transaction is completed. If the previous descriptor ended with a bus error and the descriptor came from the same host, the automatic BRESP signal is overridden by the BRESP signal that identifies the error and is generated by the error monitor circuit 108. The multiplexer circuit 111 is then configured using the select signal TRANS_ERR to provide the BRESP signal from the error monitor circuit 108 as the write response BT to the host.

[0052] As an example not intended to be limiting, the descriptor resides in the WDATA field of the write data channel WT, while the write request channel AW-T is used to route the descriptor to the target DMA circuit 104 in the IC. As another concrete example not intended to be limiting, two or more descriptors can be packaged into a single WDATA field. Now describe the descriptor concept and the various fields.

[0053] For DMA transaction interleaving, the transaction identifier (AxID) is used for unwrapped transactions. The default mode is to set all transaction identifiers (AxID) to 0 to maintain the order of all transactions issued by all hosts, both within and between them, especially on the NOC. Users can optimize transaction throughput and latency by configuring the transaction identifier field to allow DMA transaction interleaving. Additionally, the tag field is used by hosts as the transaction identifier for status requests. As a concrete example not intended to be restrictive, the tag field can have 6 bits to match NOCs that can have up to 64 incomplete transactions in order to maintain bandwidth.

[0054] For transaction completion synchronization, a signal indicating whether a transaction has completed early is used to configure when the CAM circuit 115 and mailbox are updated with a completion status. If the signal indicates that the transaction has not completed early, the DMA circuit 104 will not assert that the transfer is complete until all responses to the transfer have been returned. If the signal indicates that the transaction has completed early, the DMA circuit 104 asserts completion when the transaction is published. Using this signal is helpful for user synchronization because it is used to ensure that the buffer is fully updated before the user reads from it. This signal can be used for the last descriptor in a transaction. For example, if a user has 10 DMA transfers and needs to synchronize the completion of the 10th transfer, the user can set this signal to 1 for the first 9 descriptors and then set the signal to 0 for the 10th descriptor to ensure that the accessed buffer has the latest data.

[0055] Step-through access is often crucial for artificial intelligence (AI) workloads, image and video processing, and very long Fast Fourier Transforms (FFTs), where intermediate stages overflow into Dynamic Random Access Memory (DRAM). For step-through memory access in RB circuit 102, various fields can be used. For example, the first field can indicate the number of bytes incremented by the current target memory address to complete the memory access each time a transaction is completed. The second field can indicate the number of bytes to skip to reach the next memory access point after the first field is completed during the step-through memory access. The third field can be the total number of transactions to be performed in the step-through access mode.

[0056] In some embodiments, DMA circuit 104 is multi-host, meaning it can handle multiple hosts simultaneously sending descriptors to the same DMA circuit 104. In other embodiments, DMA circuit 104 is single-host, meaning it can handshake with a host, complete a transaction for that host, and close the connection to another host before establishing a connection with that host. Once a descriptor is stored in command FIFO circuit 105, command FIFO circuit 105 handshakes with CSR 107 to ensure sufficient space in CAM 115, and FIFO circuit 105 handshakes with FSM 109 to ensure no descriptor is currently being expanded. Once both handshakes have occurred, the descriptor is popped and pushed forward to FSM circuit 109 for expansion. If a transaction's expanded descriptor is stalled and backpressured, the transaction is backpressured into a buffer in NOC so that future DMA descriptors are not blocked from being pushed forward. The entire process is end-to-end backpressured. Each descriptor is expanded one at a time, but reentrancy is allowed (e.g., there can be multiple expanded descriptors and ongoing transactions). These transactions are all tracked by identifiers and tags in CAM circuit 115.

[0057] The CSR circuit 107 controls the maintenance of the status and error information of each incomplete transaction of the DMA circuit 104 in the CAM circuit 115, and holds the pause and reset signals in register circuits 113-114 respectively. To track this information for each read and write transaction, the CSR circuit 107 uses the CAM circuit 115 to store a unique host-given identifier (ID) and status information for each transaction. The status information includes the percentage of completion of the step transaction, whether the transaction has been completed, and the error type and address in case of an error. This information is very useful for user debugging and scheduler planning.

[0058] More technical description of the transaction flow of DMA circuit 104 is now provided. If a read or write transaction is new, it is identified as new because the unique AxID and Tag concatenation vector of the transaction are not yet the keys in CAM circuit 115. CSR circuit 107 loads the AxID into an empty slot in CAM circuit 115. The field indicating the percentage of transaction completion is initially set by the newly expanded descriptor. After this descriptor is pushed out by FSM circuit 109, this field is updated as xLAST is sent back to RB circuit 102 and listened to by DMA circuit 104 until the transaction is complete. At this point, the transaction completion signal goes high. Depending on the mailbox configuration, the mailbox is updated with the AxID of the completed transaction, and the transaction is pushed out of CAM circuit 115.

[0059] If a transaction hits an error, the error polling circuit 116 sends an error message (xRESP) to the CAM circuit 115, and the address page with the error is stored in the CAM circuit 115. When the host sends a ping command to the DMA circuit 104 for a status update, the error message is sent to the host and pushed out of the CAM circuit 115.

[0060] DMA circuit 104 uses a single reset, cold power-on reset (POR), and FIFO refresh signal. The FIFO refresh signal can be accessed by the user for debugging purposes or used solely to reset registers 113-114 and FSM circuit 109. Only trusted sources can access the reset and FIFO refresh signals and send them to DMA circuit 104. If RB circuit 102 detects an invalid host address, the transaction raises an error.

[0061] The target DMA circuit 104 uses a Read Identifier (RID) tracking mechanism for read transactions. The RID tracking mechanism is central to DMA transaction interleaving because it specifically enables read transaction interleaving, giving users more freedom in determining how to execute a set of transactions. The RID tracking mechanism also allows for finer-grained control over memory access, as it enables two transactions to access different portions of the same BRAM memory.

[0062] For non-DMA-initiated transactions, the SID memory group initiates an AR-I, and therefore expects to receive an RI returned to the memory group. However, when the DMA circuit 104 issues a read request (AR-I) and receives read response (RI) data returned for that read request, the AR-I / RI is mapped to the AW-T / WT channel. Therefore, the RID from the returned RI is tracked and mapped to the previously sent AR-I, instead of flowing directly to the memory group. This ensures that the incoming read response RI data is written to the correct address within the correct memory group along the micro NOC 103. Therefore, in embodiments with finer-grained control over BRAM access, RB 102 may include RID tracking circuitry that a user can use to write to any addressable memory within the memory group.

[0063] According to one example, CSR circuit 107 may include RID tracking circuitry with three elements: a linked list, a CAM, and a free-pool first-in-first-out (FIFO) buffer, all with depths matching the maximum number of stream identifiers (SIDs). The CAM uses the incoming read identifier (RID) and SID as keys and returns a head pointer to the linked list for the corresponding transaction. The linked list points to BRAM addresses. The free-pool FIFO buffer allows indexing of the linked list. This structure allows for read interleaving complexity and also allows users to read from the same memory group.

[0064] A more flexible implementation of the DMA circuit 104 uses this mechanism to ensure that different read address identifiers (ARIDs) and read identifiers (RIDs) arrive at the correct address within the memory group. In another implementation of the DMA circuit 104, it restricts the user to a single AxID per memory group at a time, the returned data is serialized, and no trace block is required.

[0065] Typically, when the host sends a descriptor for a transaction to the DMA circuit 104, the host needs to be able to check the completion status of the transaction and be able to receive any messages in the event of a bus error. The DMA circuit 104 can perform various transaction status query mechanisms. Each of these transaction status query mechanisms differs in terms of host latency and the necessary configuration of the micro NOC 103. Depending on the implementation, the host can perform status queries by: (1) sending a read request AR-I command along with an initial write request AW / WI, (2) performing AR-I polling, or (3) performing a push / push write request AW / WI write to the local mailbox.

[0066] Figure 2A This is an illustration of an example of a host performing a status query by sending a read request AR-I along with an initial write request and write data AW-I / WI to a response buffer (RB) circuit 202 in an integrated circuit (IC). Along with the initial write request / write data AW-I / WI containing a descriptor, the host simultaneously sends a read request AR-I / AR-T to a DMA (Direct Memory Access) circuit 201 (e.g., DMA circuit 104). The write request / write data corresponds to a SID (Stream Identifier) ​​of a set of block random access memories (BRAM) along the micro NOC. Upon arrival, the read request AR-I / AR-T is stored in the DMA circuit 201 as the descriptor is expanded and execution begins. When the transaction completes, the DMA circuit 201 pushes a delayed read response RT / RI back to the host, which contains information detailing the transaction ID, transaction completion, error messages, and the fill status of the first-in-first-out (FIFO) buffer in the command FIFO circuit 105. Because the read request AR-T has already been stored in the DMA circuit 201, the delay of the returned read response RT is half that of the AR-I polling, which requires a complete round trip to receive the status.

[0067] When RB circuit 202 receives a read request AR-I / AR-T, DMA circuit 201 uses the lower bits of the address field to associate the read request AR-I / AR-T with the initial write request AW / WT / WI. The lower bits of the address are identical, representing the host initiating the transaction and a tag identifying the host. Therefore, the incoming AR-T status query can access the expanded AW / WT transaction information so that it can be sent back to the host in the delayed read response RT.

[0068] According to an alternative implementation, the host can also perform state queries by executing read request AR-I polling. (See also...) Figure 1If the host wants to check the status of a transaction after the descriptor has been sent to the RB circuit 102, the host can send a polling read request AR-T to the target DMA circuit 104. Upon receiving the polling read request AR-T, the DMA circuit 104 immediately responds with a read response RT, which includes the error register in the CSR circuit 107, the current status of the FSM circuit 109, the fill level of the command FIFO circuit 105, and a tag to confirm that the status is for a correct transaction.

[0069] As an example, a host can send a read request AR-I, wait for an immediate RT read response, and only send another polling read request AR-I after receiving the response. This implementation minimizes traffic on the MNOC 101 while still receiving relevant status, completion, and error information.

[0070] In the event of an error, DMA circuit 104 returns a write response BT containing the error to the correct host to notify it. To obtain more detailed error information, the host then sends a read request AR-I to DMA circuit 104. The target DMA circuit 104 then responds with the address where the error occurred and any other relevant information.

[0071] If (1) the host receives an error BRESP message, (2) the host needs to determine the fill level of the command FIFO circuit 105 to determine how many more descriptors can be sent, or (3) the host needs to check the completion status of a transaction, then the host sends a polling read request AR-I instead of using a set of BRAMs in another micro NOC 103 column, as might happen with a local BRAM mailbox, as described below. Figure 2A In the publicly disclosed example, when RB circuit 102 receives a read request AR-T, DMA circuit 104 uses the host address stored in the transaction and the tag given by the host to associate the read request with the correct write request AW / WT.

[0072] According to another alternative implementation, the host can also perform status queries by executing push / push write requests (AW / WI) to a local mailbox (e.g., in BRAM). Figure 2B This is an illustration of an example of a host using push / push write requests to a local mailbox to perform a status query. The purpose of the mailbox is to check the completion status of transactions faster than the AR-I technique described above for read requests. The host (or processing element (PE)) enables the mailbox to quickly check which transactions have completed so that the host / PE can then process this data.

[0073] In some implementations, the mailbox can be located on the initiating microNOC 103 column to minimize the distance that host state requests must travel. In other implementations, to maximize ease of use, the mailbox can be located on any other microNOC 103 column or close to the main processing element. Enabling this mailbox for a microNOC 103 column uses one SID / memory group.

[0074] exist Figure 2B In the example, the host sends a write request AW-I and write data WI to RB circuit 204 (e.g., RB circuit 102). These requests and data are pushed to DMA circuit 203 (e.g., DMA circuit 104) to be expanded and issued. Upon completion of the transaction, DMA circuit 203 pushes a write response BI back to the host and pushes the write request AW-T and write data WT to a mailbox. The mailbox location is user-selectable and configurable. The mailbox address is located in the descriptor of the initial transaction sent from the host.

[0075] The mailbox only includes completed transactions for the host specified for that particular mailbox, which can be determined before runtime. When a transaction completes, the target DMA circuit 204 sends the transaction tag and AxID to the mailbox so that it can be loaded into the correct FIFO, which is organized by SID to avoid head-of-line blocking. The host can then pop the completed transaction tag that matches the host's SID from the FIFO.

[0076] Refer again Figure 1 The purpose of the error monitor circuit 108 in the DMA circuit 104 is to poll the xRESP signal to look for any errors and forward these errors to the CSR circuit 107 for storage. The error polling circuit 116 is a sub-block of the error monitor circuit 108, which controls the monitoring of the incoming RRESP and BRESP signals from the MNOC 101. If any value read by xRESP is not 0, an error has occurred, and the error flag signal in the error polling circuit 116 goes high. The error polling circuit 116 then sends the xRESP value to the error register in the CAM circuit 115 in the CSR circuit 107 for storage, and the error polling circuit 116 configures the multiplexer circuit 111 to select the BRESP signal from the error monitor circuit 108 by using the selection signal TRANS_ERR, thereby sending the value in the BRESP signal (e.g., BRESP=2) to the host via the multiplexer circuit 111.

[0077] Figure 3 This is an illustration of an example system in which the host sends a descriptor to DMA circuit 104, then sends a status request for a transaction, and finally receives a status response. Figure 3The circuitry shown includes an on-chip network (NOC) 301, response buffer (RB) circuits 302-303 (e.g., two RB circuits 102), and micro NOCs 321-324 within an integrated circuit (IC). Micro NOCs 321-322 are coupled to RB circuit 302, and micro NOCs 323-324 are coupled to RB circuit 303. Along with an initial write request AW-I / write data WI containing a descriptor, host 311 sends a read request AR-T to DMA circuit 304 (e.g., DMA circuit 104) in RB circuit 302 via micro NOCs 323 and NOC 301, as indicated by arrow 306. Upon arrival of the read request AR-T, it is stored in memory as the descriptor is expanded and execution begins. When the transaction is complete, the DMA circuit 304 pushes a delayed read response RT back to the host 311 via the NOC 301. This response contains detailed information including the transaction ID, error messages, transaction completion status, and the filling status of the command FIFO circuit 105. Because the read request is already stored in the DMA circuit 304, the delay of the returned read response RT is half that of using regular polling, which requires a full round trip to receive the status.

[0078] exist Figure 3 In this process, the delayed read response RT is stored in DMA circuit 304 until the transaction is complete. Then, in response to a status query AR, DMA circuit 304 sends the delayed status response RT back to host 311 via NOC 301 and micro NOC 323, as indicated by arrow 307. The delayed read response RT status query is a global option that can be enabled by the user (e.g., using a user interface). Delayed read response RI / RT may be desirable if the use case is based on a state machine or processor core without a timeout mechanism. For example, if the circuit design consists of microcontrollers receiving data from multiple receive streams, delayed read response RI / RT status queries improve host efficiency by reducing round-trip latency.

[0079] For the polling read response RT, if host 311 wants to check the status of a transaction after the descriptor has been sent, host 311 sends a read request AR-T to the target DMA circuit 304 via micro NOC 323 and NOC 301, as shown by arrow 306. Upon receiving the polling read request AR-T, DMA circuit 304 immediately responds with a read response RT, as shown by arrow 307. The read response RT contains the error register in CSR circuit 107, the current status of FSM circuit 109, the fill level of command FIFO circuit 105, and a tag to confirm that the status is for a correct transaction. As described above, polling can be sent from host 311 at any time after a write request / write data with a descriptor has been pushed to DMA circuit 304. For example, if the circuit design is processor-centric or has very high performance, an exclusive polling RT response status query mechanism can be used.

[0080] Figure 4 It is shown Figure 1 This diagram illustrates the states of the finite state machine (FSM) in the FSM circuit 109 of the DMA circuit 104 and examples of the transitions between states. The finite state machine (FSM) implemented by the FSM circuit 109 has a reset state 401, an idle state 402, a read transaction state 403, a write transaction state 404, an error state 405, a user pause state 406, and an end transaction state 407. The FSM implemented by the FSM circuit 109 transitions between states 401-407, as follows: Figure 4 As indicated by the arrows in the diagram. The FSM enters the reset state 401 in the aforementioned cold POR. The FSM enters the error state 405 in response to any of the aforementioned errors. The FSM is idle in the idle state 402, paused in the user paused state 406, and terminates the transaction in the transaction termination state 407.

[0081] The FSM circuit 109 in DMA circuit 104 controls the descriptor expansion process. The main purpose of FSM circuit 109 is to expand the incoming descriptors received from command FIFO circuit 105 via multiplexer circuit 112. When FSM circuit 109 is available, the FSM ready signal is asserted. If valid data exists in command FIFO circuit 105, and if there is storage space in CAM circuit 115 in CSR circuit 107, a new descriptor is sent to FSM circuit 109. Once a new descriptor is received, the direction bit determines whether the new descriptor is a read transaction or a write transaction, and then FSM enters the corresponding state 403 or 404. If necessary, the step logic of each of these states 403-404 expands each step transfer and sends it out. When a transaction is pushed out, if the transaction is a read transaction, FSM circuit 109 issues a read request AR-I and a write request AW-T, or if the transaction is a write transaction, it issues a write request AW-I and a read request AR-T. When both transactions are pushed out of FSM circuit 109, they are sent to FIFO blocks 121-122, which serialize AW-T / AR-I and AR-T / AW-I respectively to ensure that micro NOC 103 is ready before data return.

[0082] To maintain the transaction protocol, the FSM circuit 109 only pushes out complete transactions. For each of these two types of transactions, the FSM circuit 109 maintains only one port to help correctly serialize the transactions. (See again...) Figure 1 The FSM circuit 109 receives read and write transactions from the command FIFO circuit 105 via a multiplexer circuit 112 configured by the selection signal R_W_TRANS. When a transaction is pushed out, the FSM circuit 109 issues a read request AR-I and a write request AW-T for read transaction 117, or a write request AW-I and a read request AR-T for write transaction 118. When read and write transactions 117-118 are pushed out, these transactions are sent to the barrel shifter (BS) / FIFO circuits 121-122, respectively. The BS / FIFO circuit 121 ensures that if the transaction is a read transaction, the RB controller / scheduler 110 first issues a write request AW-T (i.e., ensures that the micro NOC 103 bus is ready to receive data), and then issues a read request AR-I (i.e., acquires data) via the MNOC 101. The BS / FIFO circuit 122 ensures that if the transaction is a write transaction, the RB controller / scheduler 110 first issues a read request AR-T through the micro NOC 103, and then issues a write request AW-I / write data WI through the MNOC 101.

[0083] Figure 5This is a diagram illustrating the components in a structural sector 500 of a configurable integrated circuit (IC) that enable read and write transactions to memory. Figure 5 The components shown include six response buffer (RB) circuits 501-506, a processing element (PE) 507, a scheduler 508, micro NOCs 511-512 coupled to the RB circuit 502, a ping buffer 509, and a pong buffer 510. Each of the RB circuits 501-506 includes as described herein. Figure 1 The disclosed DMA circuit 104.

[0084] An example of an operation that can be performed to implement a transaction is now described using the components shown in structure sector 500. Initially, scheduler 508 issues a write request AW to write a DMA descriptor to a target response buffer circuit (e.g., RB circuit 502). Next, target response buffer circuit 502 receives the write request and extracts write data WDATA with DMA descriptor information. Then, DMA circuit 104 in RB circuit 502 issues a read / write request targeting the appropriate SID mapped from the memory address field of the descriptor. Then, the corresponding SID in the BRAM (e.g., RB circuit 502) in structure sector 500 is retrieved via micro NOC 511. Figure 5 (As shown in ping buffer 509) Reads data / writes data to the corresponding SID BRAM. Scheduler 508 coordinates the necessary handshake signals to ensure that micro NOC 511 is ready to receive transactions.

[0085] Processing element 507 also monitors the micro NOC enable signal to ensure that the BRAM can be accessed with valid data and to determine when a DMA transaction for the corresponding SID buffer is complete. The transaction can then begin on the buffer (such as...). Figure 5 The processing element 507 processes the ping buffer 509 (shown in the diagram). After processing the ping buffer 509, the processing element 507 signals the scheduler 508 via the micro NOC 512 that the scheduler 508 can continue processing new DMA descriptors targeting that SID buffer. The processing element 507 sends a packet to the scheduler 508 including the SID, ready bit, and BRAM address or processing element block tag. Then, the processing element (PE) 507 completes the processing of the pong buffer 510. Data in the pong buffer 510 can be processed by the PE 507 while a transaction is being executed for a specific ping buffer 509. Similarly, data in the ping buffer 509 can be processed by the PE 507 while a transaction is being executed for a specific pong buffer 510.

[0086] The scheduler 508 can be implemented in soft logic using one of several different configurations for pushing descriptors to the DMA circuit. Figure 5 The example implements a double-buffering scheme using ping buffer 509 and pong buffer 510, which largely hides the wait time between the completion of a data transaction and the start of processing element 507. The software can track ping buffer 509 and pong buffer 510.

[0087] Figure 5 The example maintains a stack of descriptors for each SID buffer. When a signal indicating that processing element 507 has finished processing the data on the SID buffer is received from processing element 507, a descriptor is popped from the stack one at a time. A ping-pong buffer between multiple SID buffers (e.g., ping buffer 509 and pong buffer 510) and processing element 507 can be used. Figure 5 This is achieved through components. If a buffer spanning multiple SIDs is used, the processing element 507 can listen for micro NOC enable signals from multiple SIDs to determine when to begin processing data.

[0088] As mentioned above, for 2D applications, the DMA descriptor targets the response buffer circuit, and thus the response buffer circuit is the initiator of the DMA transaction. Figure 6 This is an illustration showing an example of a system that can be used for two-dimensional (2D) read or write applications. Figure 6 The system includes a 2D read or write application 601, a main NOC (MNOC) 602, a response buffer circuit 603 including a DMA circuit 104, a micro NOC 604, and a BRAM 605.

[0089] The 2D read application 601 initiates a write request AW-T with write data WT to the RB / DMA 603 via MNOC 602. The start address / SOP is sent from the RB controller / scheduler 110 in the RB / DMA circuit 603 to the BRAM 605 via a column of microNOC 604. RB / DMA 603 sends a read request AR-I to the 2D read application 601 via MNOC 602. The 2D read application 601 then sends a read response RI (remapped to write data WT) to write to the BRAM 605 via MNOC 602, RB / DMA circuit 603, and microNOC 604. The SOP address is sent from the microNOC controller in the response buffer circuit, and the specific microNOC address is specified by the descriptor memory address field.

[0090] The 2D write application 601 can initiate a write request AW-T with write data WT to the RB / DMA 603 via MNOC 602. The write data WI / read response RT start address is sent from the RB controller / scheduler 110 in the RB / DMA circuit 603 to the BRAM 605 via the column of micro NOC 604. The RB / DMA 603 then sends a write request AW-I to the 2D write application 601 via MNOC 602. The BRAM 605 then sends the write data WI (remapped to read response RT) read from the BRAM 605 to the RB / DMA circuit 603 via the column of micro NOC 604. The RB / DMA circuit 603 then sends the write data WI to the 2D write application 601 via MNOC 602. The 2D write application 601 then sends a read response BI to the RB / DMA circuit 603 via MNOC 602.

[0091] Figure 7 This is an illustration of an example system that can be used for 3D read or write applications to perform three-dimensional (3D) transactions. A three-dimensional (3D) transaction is going to / from... Figure 1 The main IC is a stack of memory IC dies in the vertical direction. Figure 7 The system includes a 3D read or write application 611, a main NOC (MNOC) 612, a response buffer circuit 613 including DMA circuitry 104, a micro NOC 614, and a 3D input / output (3DIO) interface 615 (e.g., coupled to memory IC dies stacked vertically). Unlike 2D transactions (in which AR-T / AW-T commands are only used to wake up the memory coupled to the micro NOC along the column via SOP), 3D transactions use full AR-T and AW-T commands sent down to the 3D memory. 3D transactions are full, valid Advanced Extensible Interface (AXI) transactions, and the 3DIO interface 615 sends the BT back to the initiating RB circuitry 102.

[0092] A write request AW-T with write data WT is initiated from MNOC 612 to RB / DMA 613, targeting 3D read application 611. The write request AW-T is sent from RB / DMA circuit 613 to the vertically stacked ICs via the column of micro NOC 614 and 3DIO interface 615. RB / DMA 613 then sends a read request AR-I to 3D read application 611 via MNOC 612. 3D read application 611 then sends a read response RI (remapped to write data WT) to write to the vertically stacked ICs via MNOC 612, RB / DMA circuit 613, column of micro NOC 614, and 3DIO interface 615. The write response BT is then sent back to RB / DMA circuit 613 via 3DIO interface 615 and column of micro NOC 614.

[0093] A write request AW-T with write data WT is initiated from MNOC 612 to RB / DMA 613, targeting 3D writing application 611. A read request AR-T requesting the establishment of the start address is sent from RB / DMA circuit 613 through the column of micro NOC 614 and 3DIO interface 615 to the vertically stacked ICs. RB / DMA 613 then sends a write request AW-I to 3D writing application 611 via MNOC 612. Then, a read response RT (remapped to write data WI) is sent from the vertically stacked ICs to 3D writing application 611 through 3DIO interface 615, column of micro NOC 614, RB / DMA circuit 613, and MNOC 612. Then, 3D writing application 611 sends a read response BI to RB / DMA circuit 613 via MNOC 612.

[0094] Based on some exemplary applications of the DMA circuit 104, artificial intelligence (AI) workloads can use the techniques disclosed herein to transfer activation tensors from external memory to on-chip memory in small blocks (this is referred to as dicing). Because the data is typically populated to facilitate memory and interface access patterns, the DMA circuit 104 can be used to provide step-by-step memory access to external memory for AI workloads.

[0095] Most of the DMA transactions discussed earlier in this paper allow for read transactions (e.g., reading from peripheral memory and writing to BRAM coupled to the micro NOC) or write transactions (e.g., reading from BRAM coupled to the micro NOC and writing to peripheral memory). Another common use of DMA circuit 104 is to perform read-modify-write operations on a set of BRAMs. Therefore, another application of DMA circuit 104 is to create a shared memory mode for DMA transactions, namely the Dynamic Read Response / Write Data (RI / WI) mode.

[0096] The high-level flow of this shared memory mode DMA transaction is now described. First, a specific target DMA circuit 104 initiates a read transaction for SID A. Along the target micro NOC column, the read response RI from the peripheral memory arrives and is written to a set of BRAMs. Next, the structure logic processes the data of the read transaction. Then, the target DMA circuit 104 initiates a write transaction for SID A, using aliasing to maintain the same SID. The processed data (still in the same set of BRAMs for SID A) is then sent to the DMA circuit 104 in the read response RT (WI) channel for writing back to the peripheral memory.

[0097] Figure 8 This is an illustration of a configurable integrated circuit (IC) 800. The configurable IC 800 can include, as described herein, [the following is a description of, a specific IC]. Figure 1-7 Examples of publicly disclosed circuits and NOC ICs. (e.g.) Figure 8 As shown, the configurable integrated circuit 800 includes a two-dimensional array of configurable logic blocks, including, for example, a logic array block (LAB) 810 and other configurable logic blocks such as a random access memory (RAM) block 830 (e.g., BRAM) and a digital signal processing (DSP) block 820. The configurable logic blocks (such as LAB 810) may include smaller configurable regions (e.g., configurable logic elements, configurable logic blocks, or adaptive logic modules (ALMs)) that receive input signals and perform custom functions on the input signals to generate output signals.

[0098] The configurable integrated circuit 800 also includes programmable interconnect circuitry in the form of vertical routing channels 840 (i.e., interconnects formed along the vertical axis of the configurable integrated circuit 800) and horizontal routing channels 850 (i.e., interconnects formed along the horizontal axis of the configurable integrated circuit 800), each routing channel including at least one rail for routing at least one wire. One or more of the routing channels 840 and / or 850 may be part of a network on-chip (NOC) with router circuitry.

[0099] In addition, the configurable integrated circuit 800 has input / output elements (IOEs) 802 (e.g., including I / O circuit blocks) for driving the configurable integrated circuit 800 to emit signals and receive signals from other devices. The input / output elements 802 may include parallel input / output circuitry, serial data transceiver circuitry, differential receiver and transmitter circuitry, or other circuitry for connecting one integrated circuit to another. The input / output elements 802 may include general purpose input / output (GPIO) circuitry (e.g., on the top and bottom edges of IC 800), high-speed input / output (HSIO) circuitry (e.g., on the left edge of IC 800), and on-package input / output (OPIO) circuitry (e.g., on the right edge of IC 800).

[0100] As shown in the figure, the input / output elements 802 can be located on the periphery of the IC. The configurable integrated circuit 800 can have input / output elements 802 arranged in different ways if desired. For example, the input / output elements 802 can form one or more columns of input / output elements, which can be located anywhere on the configurable integrated circuit 800 (e.g., uniformly distributed across the width of the configurable integrated circuit). If desired, the input / output elements 802 can form one or more rows of input / output elements (e.g., distributed across the height of the configurable integrated circuit). Alternatively, the input / output elements 802 can form islands of input / output elements, which can be distributed or clustered in selected areas on the surface of the configurable integrated circuit 800.

[0101] Note that, in addition to Figure 8 In addition to the interconnect topology shown, other routing topologies can be used. For example, in the case of a three-dimensional integrated circuit, the routing topology may include wires traveling diagonally or horizontally and vertically along different portions of its extent, as well as wires perpendicular to the device plane, and the drivers of the wires may be located at locations different from one end of the wire. The routing topology may include global wires spanning substantially the entire configurable integrated circuit 800, partially global wires (such as wires spanning a portion of the configurable integrated circuit 800), interlaced wires of specific lengths, smaller local wires, or any other suitable interconnect resource arrangement.

[0102] Furthermore, it should be understood that the examples disclosed herein can be implemented in any type of integrated circuit. If desired, the functional blocks of such an integrated circuit can be arranged in more levels or layers, with multiple functional blocks interconnected to form a larger block. Other device arrangements can use functional blocks that are not arranged in rows and columns.

[0103] The configurable integrated circuit 800 may also include programmable memory elements. Configuration data (also known as programming data) can be loaded into the memory elements using input / output elements (IOE) 802. Once loaded, each memory element provides a corresponding static control signal that controls the operation of the associated functional block (e.g., LAB 810, DSP 820, RAM 830, or input / output element 802).

[0104] In a typical scenario, the output of a loaded memory element is applied to the gate of a field-effect transistor in a functional block to turn certain transistors on or off, thereby configuring the logic (including routing paths) in the functional block. Programmable logic circuit elements controlled in this way include portions of multiplexers (e.g., multiplexers for forming routing paths in interconnect circuits), lookup tables, logic arrays, AND logic gates, OR logic gates, NAND logic gates, NOR logic gates, transmission gates, etc.

[0105] Memory elements can use any suitable volatile and / or non-volatile memory structure, such as random access memory (RAM) cells, fuses, antifuse, programmable read-only memory cells, mask programming and laser programming structures, combinations of these structures, and so on. Because configuration data is loaded into the memory element during programming, the memory element is sometimes referred to as configuration memory or programmable memory element.

[0106] Programmable memory elements can be organized in a configuration memory array with rows and columns. Data registers spanning all columns and address registers spanning all rows can receive configuration data. Configuration data can be transferred to the data registers. When the appropriate address register is asserted, the data registers write the configuration data to the configuration memory element in the row specified by the address register.

[0107] The configurable integrated circuit 800 may include a configuration memory organized by sectors, whereby each sector may include configuration bits that specify the function and / or interconnection of sub-components and wires in or across the sector. Each sector may include separate data and address registers.

[0108] Figure 8The configurable IC 800 is merely one example of an IC that can be used in conjunction with the embodiments disclosed herein. The embodiments disclosed herein can be used in conjunction with any suitable electronic integrated circuit or system. For example, the embodiments disclosed herein can be used in conjunction with a variety of electronic devices, such as processor integrated circuits, central processing units, memory integrated circuits, graphics processing unit integrated circuits, application-specific standard products (ASSPs), application-specific integrated circuits (ASICs), and configurable logic integrated circuits. Examples of configurable logic integrated circuits include programmable array logic (PAL), programmable logic arrays (PLA), field-programmable logic arrays (FPLA), electrically programmable logic devices (EPLDs), electrically erasable programmable logic devices (EEPLDs), logic cell arrays (LCA), complex programmable logic devices (CPLDs), and field-programmable gate arrays (FPGAs), to name just a few.

[0109] The integrated circuits disclosed in one or more embodiments herein may be part of a data processing system that includes one or more of the following components: a processor; memory; input / output circuitry; and peripheral devices. The data processing system can be used in a wide variety of applications, such as computer networking, data networking, instrument configuration, video processing, digital signal processing, or any other suitable application. The integrated circuits can be used to perform a variety of different logical functions.

[0110] Generally, software and data used to perform any of the functions disclosed herein may be stored in a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium is a tangible computer-readable storage medium that stores data and software for later access, rather than a medium that merely transmits electrical signals (e.g., wires). Software code may sometimes be referred to as software, data, program instructions, commands, or code. Non-transitory computer-readable storage media may include, for example, computer memory chips, non-volatile memory (such as non-volatile random access memory (NVRAM)), one or more hard drives (e.g., magnetic drives or solid-state drives), one or more removable flash drives or other removable media, compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs (BDs), other optical media, and floppy disks, magnetic tapes, or any other suitable memory or storage device(s).

[0111] Figure 9AA block diagram of system 10 is shown, which can be used to implement circuit designs to be programmed into programmable logic device 19 using design software. Designers can implement circuit design functions on integrated circuits such as reconfigurable programmable logic device 19 (e.g., field-programmable gate array (FPGA)). Designers can use design software 14 to implement the circuit designs to be programmed into programmable logic device 19. Design software 14 can use compiler 16 to generate low-level circuit design programs (bitstreams) 18, sometimes referred to as program object files and / or configuration programs, for programming programmable logic device 19. Therefore, compiler 16 can provide programmable logic device 19 with machine-readable instructions representing the circuit design. For example, programmable logic device 19 can receive one or more programs (bitstreams) 18 that describe the hardware implementation to be stored in programmable logic device 19. Programs (bitstreams) 18 can be programmed into programmable logic device 19 as configuration program 20. In some cases, configuration program 20 can represent accelerator functions to be performed for machine learning, video processing, speech recognition, image recognition, or other highly specialized tasks.

[0112] In some implementations, the programmable logic device can be any integrated circuit device, including programmable logic devices having two separate integrated circuit dies (where at least some programmable logic structures are separate from at least some structural support circuitry that operates the programmable logic structures). Figure 9B An example of such a programmable logic device is shown, but many other examples may also be used, and it should be understood that this disclosure is intended to cover any suitable programmable logic device in which the programmable logic structure and the structure-supporting circuitry are at least partially separated on different integrated circuit dies.

[0113] Figure 9B This is an illustration depicting an example of a programmable logic device 19, which includes three structural dies 22 and two base dies 24 interconnected by microbumps 26. Figure 9B In the example, at least some programmable logic structures of the programmable logic device 19 are located in three structure dies 22, and at least some structure support circuitry for operating the programmable logic structures is located in two base dies 24. For example, Figure 8 Some circuitry of the configurable IC 800 shown (e.g., LAB810, DSP 820, and RAM 830) may be located in the structure die 22, and some circuitry of the IC 800 (e.g., input / output element 802) may be located in the base die 24. As another example, the base die 24 may include 3D memory circuitry accessible via micro NOC 103 from the RB circuitry 102 in the structure die 22.

[0114] Despite Figure 9B The intermediate die 22 and the base die 24 may appear in a one-to-one or two-to-one relationship, but other relationships may also be used. For example, a single base die 24 may be attached to several intermediate dies 22, or several base dies 24 may be attached to a single intermediate die 22, or several base dies 24 may be attached to several intermediate dies 22 (e.g., in an interleaved pattern). Peripheral circuitry 28 may be attached to, embedded in, and / or disposed on top of the base die 24, and a heat sink 30 may be used to reduce heat buildup on the programmable logic device 19. The heat sink 30 may appear above (as shown) and / or below the package (e.g., as a double-sided heat sink). The base die 24 may be attached to the package substrate 32 via conductive bumps 34. Figure 9B In the example, two pairs of structural dies 22 and base dies 24 are shown to be communicatively connected to each other via interconnect bridge 36 (e.g., Embedded Multi-Die Interconnect Bridge (EMIB)) and microbumps 38 at bridge interfaces 39 in base die 24.

[0115] Combined, the structure die 22 and the base die 24 can be used as a programmable logic device 19 (such as a field-programmable gate array (FPGA)). It should be understood that, for example, when the structure die 22 and the base die 24 are operated in combination, the FPGA can represent the circuit type and / or logic arrangement of the programmable logic device. Furthermore, for the purposes of this example, an FPGA has been discussed herein, but it should be understood that any suitable type of programmable logic device can be used.

[0116] Figure 10 This is a block diagram illustrating a computing system 1000 configured to implement one or more aspects of the embodiments described herein. The computing system 1000 includes a processing subsystem 70 having one or more processors 74, system memory 72, and programmable logic devices 19 communicating via interconnect paths, which may include a memory hub 71. The memory hub 71 may be a separate component within a chipset assembly or integrated within one or more processors 74. The memory hub 71 is coupled to an input / output (I / O) subsystem 50 via a communication link 76. The I / O subsystem 50 includes the input / output (I / O) hub 51, which enables the computing system 1000 to receive input from one or more input devices 62. Furthermore, the I / O hub 51 enables a display controller, which may be included in one or more processors 74, to provide output to one or more display devices 61. In one embodiment, the one or more display devices 61 coupled to the I / O hub 51 may include a local display device, an internal display device, or an embedded display device.

[0117] In one embodiment, the processing subsystem 70 includes one or more parallel processors 75 coupled to the memory hub 71 via a bus or other communication link 73. The communication link 73 may use any number of standards-based communication link technologies or protocols, such as, but not limited to, PCI Express, or may be a vendor-specific communication interface or communication architecture. In one embodiment, the one or more parallel processors 75 form a computation-centric parallel or vector processing system that may include a large number of processing cores and / or processing clusters, such as multi-integrated core (MIC) processors. In one embodiment, the one or more parallel processors 75 form a graphics processing subsystem that can output pixels to one or more display devices 61 coupled via I / O hub 51. The one or more parallel processors 75 may also include a display controller and a display interface (not shown) for direct connection to the one or more display devices 63.

[0118] Within the I / O subsystem 50, system storage unit 56 can be connected to I / O hub 51 to provide storage for computing system 1000. I / O switch 52 can be used to provide an interface mechanism to enable connectivity between I / O hub 51 and other components, such as network adapter 54 and / or wireless network adapter 53, which can be integrated into the platform, and various other devices that can be added via add-on devices 55 (one or more). Network adapter 54 can be an Ethernet adapter or another wired network adapter. Wireless network adapter 53 can include one or more of Wi-Fi, Bluetooth, Near Field Communication (NFC), or other network devices including one or more wireless radios.

[0119] The computing system 1000 may include Figure 10 Other components not shown include other port connections that can also be connected to I / O hub 51, optical storage drivers, video capture devices, etc. Figure 10 The communication paths for interconnecting various components can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect) based protocols (e.g., PCI Express), or any other bus or point-to-point communication interface and / or (one or more) protocols, such as NV Link High-Speed ​​Interconnect, or interconnect protocols known in the art.

[0120] In one embodiment, one or more parallel processors 75 include circuitry optimized for graphics and video processing, including, for example, video output circuitry, and constitute a graphics processing unit (GPU). In another embodiment, one or more parallel processors 75 include circuitry optimized for general-purpose processing while preserving the underlying computing architecture. In yet another embodiment, components of the computing system 1000 may be integrated with one or more other system elements on a single integrated circuit. For example, one or more parallel processors 75, memory hub 71, one or more processors 74, and I / O hub 51 may be integrated into a system-on-a-chip (SoC) integrated circuit. Alternatively, components of the computing system 1000 may be integrated into a single package to form a system-in-package (SIP) configuration. In one embodiment, at least a portion of the components of the computing system 1000 may be integrated into a multi-chip module (MCM) that can interconnect with other MCMs to a modular computing system.

[0121] The computing system 1000 shown herein is illustrative. Other variations and modifications are also possible. The connectivity topology, including the number and arrangement of bridges, the number of processors(one or more) 74, and the number of parallel processors(one or more) 75, can be modified as needed. For example, in some embodiments, system memory 72 is connected directly to processors(one or more) 74 instead of via bridges, while other devices communicate with system memory 72 via memory hub 71 and processors(one or more) 74. In other alternative topologies, parallel processors(one or more) 75 are connected to I / O hub 51 or directly to one of processors(one or more) 74, instead of to memory hub 71. In other embodiments, I / O hub 51 and memory hub 71 may be integrated into a single chip. Some embodiments may include two or more sets of processors(one or more) 74 attached via multiple sockets that may couple to two or more instances of parallel processors(one or more) 75.

[0122] Some specific components shown in this article are optional and may not be included in all implementations of the computing system 1000. For example, any number of additional cards or peripherals can be supported, or certain components can be eliminated. Furthermore, some architectures can be adapted to... Figure 10 The components shown are similar to those in the diagram and use different terminology. For example, in some architectures, memory hub 71 may be referred to as the Northbridge, while I / O hub 51 may be referred to as the Southbridge.

[0123] Now, additional examples are described. Example 1 is a configurable integrated circuit including: a first on-chip network; and a response buffer circuit coupled to the first on-chip network, wherein the response buffer circuit includes direct memory access circuitry and controller circuitry, wherein the first on-chip network is embedded in the configurable integrated circuit, wherein the direct memory access circuitry generates a first read request and a first write request to access the first memory circuitry received from a host circuit, wherein the controller circuitry provides the first read request and the first write request to the first memory circuitry via the first on-chip network, and wherein the controller circuitry exchanges first data with the first memory circuitry for the first read request and the first write request.

[0124] In Example 2, the configurable integrated circuit of Example 1 may optionally include, wherein the first memory circuitry includes block random access memory in the configurable integrated circuit.

[0125] In Example 3, the configurable integrated circuit of any one of Examples 1 to 2 may optionally include a first memory circuitry comprising a memory external to the configurable integrated circuit, the memory being located in at least one die stacked vertically with the configurable integrated circuit.

[0126] In Example 4, the configurable integrated circuit of any of Examples 1 to 3 may optionally include, wherein the direct memory access circuitry includes a command first-in-first-out (FIFO) circuitry that stores descriptors for read transactions and write transactions, and wherein the direct memory access circuitry also includes a finite state machine that uses the descriptors for read transactions and write transactions to generate a first read request for a read transaction and a first write request for a write transaction.

[0127] In Example 5, the configurable integrated circuit of any of Examples 1 to 4 may optionally include a direct memory access circuitry comprising a control status register circuitry that stores status, error, pause, and reset information for a read transaction corresponding to a first read request and a write transaction corresponding to a first write request.

[0128] In Example 6, the configurable integrated circuit of Example 5 may optionally include a control status register circuitry that includes a content-addressable memory that stores the address of errors in each of the read and write transactions, the error type of any errors, the completion status, and a unique identifier.

[0129] In Example 7, the configurable integrated circuit of any of Examples 1 to 6 may optionally include, wherein the direct memory access circuitry includes an error monitor circuitry that polls incoming signals to look for errors in transactions including a first read request and a first write request, and forwards the errors to the storage circuitry for storage.

[0130] In Example 8, the configurable integrated circuit of any of Examples 1 to 7 further includes: a second on-chip network coupled to a response buffer circuit, wherein the controller circuit provides a second read request and a second write request to the second memory circuit via the second on-chip network, and wherein the controller circuit exchanges second data for the second read request and the second write request with the second memory circuit.

[0131] In Example 9, the configurable integrated circuit of Example 8 may optionally include a second memory circuitry comprising a memory located outside the configurable integrated circuit, the memory being located in at least one die surrounding the configurable integrated circuit.

[0132] Example 10 is a method for performing read and write transactions in a configurable integrated circuit, the method comprising: generating read requests for read transactions and write requests for write transactions of accessing memory circuitry using direct memory access circuitry in the configurable integrated circuit; providing the read and write requests from the direct memory access circuitry to the memory circuitry via a first on-chip network in the configurable integrated circuitry using a scheduler circuitry in the configurable integrated circuitry; and exchanging data for the read and write requests with the memory circuitry using the scheduler circuitry.

[0133] In Example 11, the method of Example 10 further includes: sending a response from the direct memory access circuit via a second on-chip network in a configurable integrated circuit to a status query for one of a read transaction or a write transaction, the response including: an error value, a tag for confirming that the response is for one of a read transaction or a write transaction from the direct memory access circuit, a fill level of a command first-in-first-out circuit storing descriptors of the read transaction and the write transaction in the direct memory access circuit, or a completion status of one of the read transaction or the write transaction.

[0134] In Example 12, the method of any one of Examples 10 to 11 may optionally include, wherein the memory circuitry is located in dies stacked vertically with the configurable integrated circuit, and wherein read transactions and write transactions are three-dimensional transactions to and from these dies.

[0135] In Example 13, the method of any of Examples 10 to 12 further includes: using a read identifier tracking mechanism to track one of the read transactions by tracking the returned read responses and mapping the read responses to one of the read requests, so as to allow the user to write to any addressable memory within the memory group.

[0136] In Example 14, the method of any one of Examples 10 to 13 may optionally include, wherein generating the read request for the read transaction and the write request for the write transaction further includes: storing the descriptors of the read transaction and the write transaction in a command first-in-first-out circuit in the direct memory access circuitry, wherein the descriptors may be configured by the user to manipulate transaction synchronization, interleaving, and memory striding.

[0137] In Example 15, the method of any one of Examples 10 to 14 further includes: storing descriptors for read transactions and write transactions in a first-in-first-out circuit in a direct memory access circuit; and using a finite state machine in the direct memory access circuit to process the descriptors to generate read requests for read transactions and write requests for write transactions.

[0138] Example 16 is a non-transitory computer-readable storage medium including instructions stored thereon that, when executed by a configurable integrated circuit, cause the configurable integrated circuit to: generate read and write requests for accessing memory circuitry using direct memory access circuitry; provide the read and write requests from the direct memory access circuitry to the memory circuitry via an on-chip network using controller circuitry; and exchange data for the read and write requests with the memory circuitry using the controller circuitry.

[0139] In Example 17, the non-transitory computer-readable storage medium of Example 16 may optionally include, wherein the instructions further enable a configurable integrated circuit to store status, error, pause, and reset information of read transactions corresponding to read requests and write transactions corresponding to write requests in a control status register circuit in a direct memory access circuit.

[0140] In Example 18, the non-transitory computer-readable storage medium of any of Examples 16-17 may optionally include, wherein the instructions further enable the configurable integrated circuit to: store descriptors for read transactions and write transactions in a first-in-first-out circuit in a direct memory access circuit; provide the descriptors to a finite state machine in the direct memory access circuit; and use the finite state machine to process the descriptors for read transactions and write transactions to generate read requests for read transactions and write requests for write transactions.

[0141] In Example 19, the non-transitory computer-readable storage medium of any of Examples 16 to 18 may optionally include, wherein the instructions further enable the configurable integrated circuit to: poll incoming signals using an error monitor circuit in a direct memory access circuit to look for errors in transactions including read requests and write requests; and forward the errors to storage circuitry for storage.

[0142] In Example 20, the non-transitory computer-readable storage medium of any of Examples 16 to 19 may optionally include, wherein the instructions further enable the configurable integrated circuit to: use controller circuitry to exchange data for read and write requests with memory circuitry via an on-chip network, wherein the on-chip network is located within the configurable integrated circuit.

[0143] For illustrative purposes, the above description of exemplary embodiments has been provided. The above description is not intended to be exhaustive or to limit the examples disclosed herein. The foregoing is merely for illustrating the principles of this disclosure, and various modifications can be made by those skilled in the art. The above embodiments can be implemented individually or in any combination.

Claims

1. A configurable integrated circuit, comprising: The first one was uploaded online; and A response buffer circuit is coupled to the first on-chip network, wherein the response buffer circuit includes a direct memory access circuit and a controller circuit, wherein the first on-chip network is embedded in the configurable integrated circuit, wherein the direct memory access circuit generates a first read request and a first write request to access the first memory circuit received from the host circuit, wherein the controller circuit provides the first read request and the first write request to the first memory circuit through the first on-chip network, and wherein the controller circuit exchanges first data with the first memory circuit for the first read request and the first write request.

2. The configurable integrated circuit of claim 1, wherein the first memory circuitry comprises a block random access memory in the configurable integrated circuit.

3. The configurable integrated circuit according to any one of claims 1 to 2, wherein the first memory circuitry includes a memory external to the configurable integrated circuit, the memory being located in at least one die stacked vertically with the configurable integrated circuit.

4. The configurable integrated circuit according to any one of claims 1 to 2, wherein the direct memory access circuit includes a command first-in-first-out (FIFO) circuit that stores descriptors for read transactions and write transactions, and wherein the direct memory access circuit further includes a finite state machine that uses the descriptors of the read transactions and the write transactions to generate a first read request for the read transaction and a first write request for the write transaction.

5. The configurable integrated circuit according to any one of claims 1 to 2, wherein the direct memory access circuit includes a control status register circuit, the control status register circuit storing status, error, pause, and reset information of the read transaction corresponding to the first read request and the write transaction corresponding to the first write request.

6. The configurable integrated circuit of claim 5, wherein the control status register circuitry includes a content-addressable memory that stores the address of errors, error type of any errors, completion status, and unique identifier in each of the read and write transactions.

7. The configurable integrated circuit according to any one of claims 1 to 2, wherein the direct memory access circuitry includes an error monitor circuitry that polls incoming signals to look for errors in a transaction including the first read request and the first write request, and forwards the errors to the storage circuitry for storage.

8. The configurable integrated circuit according to any one of claims 1 to 2, further comprising: A second on-chip network is coupled to the response buffer circuit, wherein the controller circuit provides a second read request and a second write request to the second memory circuit via the second on-chip network, and wherein the controller circuit exchanges second data with the second memory circuit for the second read request and the second write request.

9. The configurable integrated circuit of claim 8, wherein the second memory circuit includes a memory external to the configurable integrated circuit, the memory being located in at least one die surrounding the configurable integrated circuit.

10. A method for performing read and write transactions in a configurable integrated circuit, the method comprising: The direct memory access circuitry in the configurable integrated circuit is used to generate read requests for read transactions and write requests for write transactions that access the memory circuitry. The scheduler circuit in the configurable integrated circuit provides the read request and the write request from the direct memory access circuit to the memory circuit via a first on-chip network in the configurable integrated circuit. as well as The scheduler circuit is used to exchange data with the memory circuit for the read request and the write request.

11. The method of claim 10, further comprising: The direct memory access circuit sends a response to a status query for one of the read transactions or the write transactions via a second on-chip network in the configurable integrated circuit. The response includes: an error value, a tag confirming that the response is for one of the read transactions or the write transactions from the direct memory access circuit, the fill level of the command FIFO circuit storing descriptors of the read transactions and the write transactions in the direct memory access circuit, or the completion status of one of the read transactions or the write transactions.

12. The method of any one of claims 10 to 11, wherein the memory circuit is located in a die stacked vertically with the configurable integrated circuit, and wherein the read transaction and the write transaction are three-dimensional transactions to and from the die.

13. The method according to any one of claims 10 to 11, further comprising: The read identifier tracking mechanism is used to track one of the read transactions by tracking the returned read responses and mapping the read responses to one of the read requests, so as to allow the user to write to any addressable memory within the memory group.

14. The method according to any one of claims 10 to 11, wherein generating the read request for the read transaction and the write request for the write transaction further comprises: The descriptors for the read and write transactions are stored in the command FIFO circuitry of the direct memory access circuitry, wherein the descriptors can be configured by the user to manipulate transaction synchronization, interleaving, and memory strafing.

15. The method according to any one of claims 10 to 11, further comprising: The descriptors of the read transaction and the write transaction are stored in the first-in-first-out circuit of the direct memory access circuit; and The descriptor is processed using a finite state machine in the direct memory access circuitry to generate the read request for the read transaction and the write request for the write transaction.

16. A non-transitory computer-readable storage medium comprising instructions stored thereon, the instructions causing the configurable integrated circuit to: Use direct memory access circuitry to generate read and write requests for accessing memory circuitry; The controller circuitry provides the read and write requests from the direct memory access circuitry to the memory circuitry via an on-chip network. as well as The controller circuit is used to exchange data with the memory circuit for the read request and the write request.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions further cause the configurable integrated circuit to: The status, error, pause, and reset information of the read transaction corresponding to the read request and the write transaction corresponding to the write request are stored in the control status register circuit in the direct memory access circuit.

18. The non-transitory computer-readable storage medium according to any one of claims 16 to 17, wherein the instructions further cause the configurable integrated circuit to: The descriptors of the read transaction and the write transaction are stored in the first-in-first-out circuit of the direct memory access circuit; The descriptor is provided to the finite state machine in the direct memory access circuitry; as well as The finite state machine is used to process the descriptors of the read transaction and the write transaction to generate the read request of the read transaction and the write request of the write transaction.

19. The non-transitory computer-readable storage medium according to any one of claims 16 to 17, wherein the instructions further cause the configurable integrated circuit to: The error monitor circuitry in the direct memory access circuitry is used to poll the incoming signals to look for errors in transactions including the read request and the write request; and The error is forwarded to the storage circuit for storage.

20. The non-transitory computer-readable storage medium according to any one of claims 16 to 17, wherein the instructions further cause the configurable integrated circuit to: The controller circuitry uses the on-chip network to exchange data for the read and write requests with the memory circuitry, wherein the on-chip network is located within the configurable integrated circuit.

Images