Interconnect circuit for multi-channel and multi-requester memory systems

Abstract

An integrated circuit device includes interconnect circuitry. The interconnect circuit includes an interleaving switch circuit, a network switch circuit, and a crossbar switch circuit. The interleaved switch circuits are coupled to a requester device. The first interleaved switch circuit includes a first port. The first interleaving switch circuit receives a first memory command and outputs the first memory command via a first communication path connected to the first port based on a memory address of the first memory command. The network switching circuits are connected to the interleaving switching circuits. A first network switch circuit is connected to the first communication paths and routes first memory commands along the first communication paths based on memory addresses. A first crossbar circuit of the crossbar circuits receives a first memory command from the first communication paths and outputs the first memory command to a first memory device of the memory devices associated with the memory.

Description

Technical Field

[0001] Examples of this disclosure generally relate to interconnect circuitry for integrated circuit devices that connect requester circuitry elements to memory channels of memory devices in a manner that is at least partially non-blocking. Background Technology

[0002] Integrated circuit (IC) devices include a memory subsystem circuitry that controls access to one or more memory devices. An IC device includes multiple requester elements. A requester element is a circuitry element within the IC device that outputs memory commands (e.g., memory read commands and memory write commands) to the memory device to read and write data to the memory device. The memory subsystem circuitry includes interconnect circuitry and memory channels that connect the circuitry elements of the IC device to the memory device, thereby enabling the circuitry elements to access the memory device. The interconnect circuitry is configured such that access by one circuitry element does not block access by another circuitry element. To provide non-blocking access, the interconnect circuitry includes multiple crossbar switch circuits. Each crossbar switch circuit has multiple input ports and multiple output ports. The number of input ports and output ports increases based on the number of requesters and the number of memory channels. However, to support high-speed memory devices with an increased number of channels, the number and / or size of the crossbar switch circuits increase, thereby increasing the circuit area of ​​the corresponding IC device and increasing the power requirements of the corresponding IC device, which increases the manufacturing cost of the IC device. Therefore, there is a need for an improved interconnect circuit that can support high-speed memory devices while having reduced circuit area and / or lower power requirements. Summary of the Invention

[0003] In one example, an interconnect circuit includes interleaved switching circuits, network switching circuits, and cross switching circuits. These interleaved switching circuits are coupled to a requester device. A first interleaved switching circuit among these interleaved switching circuits includes a first port. The first interleaved switching circuit receives a first memory command and outputs the first memory command via a first communication path connected to the first port, based on a memory address of the first memory command. These network switching circuits are connected to these interleaved switching circuits. A first network switching circuit among these network switching circuits is connected to the first communication path and routes the first memory command along the first communication path based on the memory address. These cross switching circuits are coupled to a memory device. A first cross switching circuit among these cross switching circuits receives the first memory command from the first communication path and outputs the first memory command to a first memory device associated with the memory address.

[0004] In one example, an integrated circuit (IC) device includes a requester circuit element and interconnect circuitry. The requester circuit element outputs a first memory command. The interconnect circuitry is connected to the requester circuit element. The interconnect circuitry includes interleaved switch circuitry, network switch circuitry, and cross switch circuitry. A first interleaved switch circuitry of these interleaved switch circuitry receives the first memory command and outputs the first memory command via a first communication path connected to a first port based on a memory address of the first memory command. The network switch circuitry is connected to these interleaved switch circuitry. A first network switch circuitry of these network switch circuitry is connected to these first communication paths and routes the first memory command along these first communication paths based on the memory address. The cross switch circuitry is coupled to a memory device. A first cross switch circuitry of these cross switch circuitry receives the first memory command from these first communication paths and outputs the first memory command to a first memory device associated with the memory address.

[0005] In one example, a method includes receiving a first memory command at a first interleaving switch circuit in an interleaving switch circuit of an interconnect circuit. Furthermore, the method includes outputting the first memory command, based on a memory address of the first memory command, via a first port of the first interleaving switch circuit, via a first communication path of the interconnect circuit. The method further includes routing the first memory command, based on the memory address, along the first communication paths via a first network switch circuit in a network switch circuit of the interconnect circuit. The first network switch circuit is connected to the first communication paths. Additionally, the method includes outputting the first memory command to a first memory device associated with the memory address in a memory device via a first crossbar switch circuit in a crossbar switch circuit of the interconnect circuit. The first crossbar switch circuit is connected to the first communication paths.

[0006] These and other aspects can be understood by referring to the following detailed description. Attached Figure Description

[0007] To gain a more detailed understanding of the features described above, a more specific description of the brief summary can be obtained by referring to the exemplary embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only typical exemplary embodiments and should not be considered as limiting the scope of the description.

[0008] Figure 1 A block diagram of an integrated circuit device is shown.

[0009] Figure 2 A block diagram of the interconnect circuit is shown.

[0010] Figure 3A block diagram of the interconnect circuit is shown.

[0011] Figure 4 A block diagram of the interconnect circuit is shown.

[0012] Figure 5 A flowchart illustrating a method for routing memory commands within an interconnect circuit is provided.

[0013] For ease of understanding, the same reference numerals are used where possible to denote common elements in the accompanying figures. Consider that elements of one example can be advantageously incorporated into other examples. Detailed Implementation

[0014] Various features are described below with reference to the accompanying drawings. It should be noted that the drawings may be drawn to scale or not, and elements with similar structures or functions are indicated by similar reference numerals in all drawings. It should be noted that the drawings are intended only to facilitate the description of features. They are not intended as an exhaustive description of the features, nor as a limitation on the scope of the claims. Furthermore, the illustrated examples do not necessarily possess all the aspects or advantages shown. Aspects or advantages described in connection with a particular example are not necessarily limited to that example and may be practiced in any other example even if not so illustrated or so explicitly described.

[0015] Integrated circuit (IC) devices use memory commands to interact with memory devices. The memory devices can be external to and connected to the corresponding IC device. In one or more examples, the IC device includes a memory subsystem circuitry that connects circuitry elements within the IC device to the memory device via memory channels. The memory subsystem circuitry includes interface circuitry that includes cross-connected circuitry vertically and horizontally to the circuitry elements. The vertical and horizontal connections include network switch circuitry that allows each circuitry element to communicate a memory command to each memory device. The circuitry elements may be referred to as requester circuitry elements. In one example, the number of requester circuitry elements is 48. In another example, the number of requester circuitry elements is greater than or less than 48.

[0016] In one or more examples, the memory device is a high-bandwidth memory (HBM) device. Furthermore, the memory device may include a stack of memory chips. In this example, the memory device includes multiple memory chips connected horizontally and / or vertically to each other. The memory device includes multiple memory channels for connecting the memory device to memory subsystem circuitry of an IC device. The number of memory channels is 32. In other examples, the number of memory channels is greater than or less than 32.

[0017] The memory subsystem provides a communication path from each requester circuit element to each memory channel of each memory device. The memory subsystem circuitry is configured such that each requester circuit element is provided with a concurrent, non-blocking connection from any requester circuit element to any memory channel of the memory device. In a non-blocking connection, a connection from one circuit element to a memory channel does not interfere with a connection from another circuit element to another memory channel. In one example, a crossbar switch circuit with an ingress port number at least equal to the number of requester circuit elements and an egress port number at least equal to the number of memory channels can be used. However, in this example, the circuit area of ​​the crossbar switch circuit is unacceptable in terms of cost and power performance. To reduce the circuit area associated with the crossbar switch circuit, the crossbar switch circuit is divided into multiple smaller crossbar switch circuits. However, due to the large number of requester circuit elements and memory channels, the circuit area of ​​the smaller crossbar switch circuits is unacceptable in terms of cost and power performance, much like the circuit area of ​​a single crossbar switch circuit. In one example, the interconnect circuitry of the memory system is a network-on-chip (NoC) with multiple communication paths (NoC channels) that enable traffic (e.g., memory command communication) to be transmitted between each requester circuit element and each memory channel. The communication paths include horizontal and vertical communication paths to connect the requester circuit elements to the memory channels. In one example, each crossbar switch circuit is connected to eight vertical communication paths. However, the performance of this interconnect circuitry is limited by the bandwidth of the horizontal communication paths.

[0018] As described herein, an IC device includes a memory system circuit with interconnect circuitry, which includes interleaved switching circuitry, network switches, and cross switches. The interleaved switching circuitry, network switches, and cross switches divide the interconnect circuitry into multiple communication paths (e.g., vertical and horizontal communication paths) that allow traffic (e.g., memory command communication) to be transmitted between each requester circuit element and each memory channel. The interleaved switching circuitry corresponds to interleaving within a corresponding memory device. The interleaved switching circuitry directs received memory commands to the network switch circuitry and cross switches based on the relationship between the corresponding memory address and the interleaving of the memory device. Furthermore, data can be routed from the memory device back to the interleaved switching circuitry via the cross switches and network switches. Compared to interconnect circuitry with other configurations, the interconnect circuitry described herein offers reduced circuit area and improved performance. Therefore, IC devices including interconnect circuitry as described herein offer reduced semiconductor manufacturing costs and improved performance.

[0019] Figure 1A block diagram of an IC device 100 according to one or more examples is illustrated. In one example, IC device 100 includes an IC chip. In another example, IC device 100 includes two or more IC chips interconnected with each other. In one or more examples, IC device 100 is a field-programmable gate array (FPGA). In other examples, IC device 100 may be implemented using any of a variety of different types of IC architectures. IC device 100 may be a system-on-a-chip (SoC). IC device 100 includes multiple subsystems including a digital processing engine (DPE) array 110, a processing system (PS) 118, programmable logic (PL) circuitry 120, hard block (HB) circuitry 122, input / output (I / O) circuitry 124, and interconnect circuitry 116. In some examples, each subsystem includes at least some programmable components or circuitry, as described herein. In some examples, some of these subsystems may include non-programmable special-purpose circuitry. Other circuitry, such as other IP blocks, such as system monitors or other devices, may be included in IC device 100.

[0020] DPE array 110 includes multiple interconnected DPEs 112. Each DPE in DPE 112 is a hardwired circuit block and can be programmable. Each DPE in DPE 112 includes a core, memory module, interconnect circuitry, and other circuit elements. In other examples, DPE 112 may have different configurations. Figure 1 In the example, the DPE array 110 includes a two-dimensional array of DPEs 112 and array interface circuitry 114. The DPE array 110 includes one or more DPEs 112. The DPEs 112 are arranged in aligned rows and aligned columns. However, other arrangements of the DPEs 112 are possible. In some examples, the DPEs 112 may be arranged such that the DPEs 112 in selected rows and / or columns are horizontally reversed or flipped relative to the DPEs 112 in adjacent rows and / or columns. In other examples, the rows and / or columns of the DPEs 112 may be offset relative to adjacent rows and / or columns.

[0021] DPE 112 can handle various data within DPE array 110 via different mechanisms. DPE 112 are connected to form a DPE interconnect network. To form the DPE interconnect network, each DPE 112 is connected to vertically adjacent DPE 112 and horizontally adjacent DPE 112 or other adjacent DPEs. The DPE interconnect network includes a streaming network and a memory-mapped network. The streaming network includes interconnected streaming switches, and application data and direct memory access (DMA) can be communicated between DPE 112 via this streaming network. The memory-mapped network includes interconnected memory-mapped switches, and configuration and control data can be communicated between DPE 112 via this memory-mapped network. Adjacent DPE 112 can also communicate via shared memory. Independent cascading streaming can be implemented between DPE 112.

[0022] The DPE array 110 also includes array interface circuitry 114, which comprises tiles. Each tile of the array interface circuitry 114 can be hardwired and programmable. Each tile may include, for example... Figure 3 The examples and the following text are relative to Figure 3 The architecture described. Array interface circuit 114 provides an interface between DPE 112 of DPE array 110 and other subsystems of IC device 100, such as interconnect circuit 116 and PL circuit 120.

[0023] In some examples, the array interface circuitry 114 is coupled to the adjacent DPE 112. For example, as... Figure 1 As illustrated, array interface circuitry 114 can be connected to each DPE 112 in the bottom row (e.g., the row closest to array interface circuitry 114) of DPE 112 in DPE array 110. Array interface circuitry 114 provides communication between DPE array 110 and interconnect circuitry 116.

[0024] Processing system 118 may be or include any of a variety of different processor types and numbers of processor cores. For example, processing system 118 may be implemented as a single processor, such as a single core capable of executing program instruction code. Alternatively, processing system 118 may be implemented as a multi-core processor. Processing system 118 may be implemented using any of a variety of different architectures. Example architectures that may be used to implement processing system 118 may include ARM processor architecture, x86 processor architecture, graphics processing unit (GPU) architecture, mobile processor architecture, digital signal processor (DSP) architecture, or other suitable architectures capable of executing computer-readable program instruction code.

[0025] Programmable logic circuit 120 is a logic circuit that can be programmed to perform a specified function. For example, programmable logic circuit 120 may be implemented as a braid of an FPGA. Programmable logic circuit 120 may include programmable logic elements, including configurable logic blocks (CLBs), lookup tables (LUTs), blocks of random access memory (BRAM), ultra-RAM (URAM), input / output blocks (IOBs), digital signal processing blocks (DSPs), clock managers, and / or delay-locked loops (DLLs). In some architectures, programmable logic circuit 120 includes columns of programmable logic elements, where each column includes a single type of programmable logic element (e.g., a column of CLBs, a column of BRAMs, etc.). Programmable logic elements may have one or more associated programmable interconnect elements. For example, in some architectures, programmable logic circuit 120 includes a column of programmable interconnect elements associated with and adjacent to each column of programmable logic elements. In such examples, each programmable interconnect element is connected to an associated programmable logic element in an adjacent column and to an adjacent programmable interconnect element within the same column. The interconnected columns of programmable interconnect elements can form a global routing network within programmable logic circuit 120.

[0026] Programmable logic circuit 120 has associated configuration frame interconnects (CFs). Configuration frame nodes residing on the platform management controller (PMC) of processing system 118 are connected to the CF. The PMC transmits configuration data to the configuration frame nodes, and the configuration frame nodes format the configuration data into frames and send these frames via the CF to programmable logic elements and programmable interconnect elements. The configuration data can then be loaded into the internal configuration memory cells of the programmable logic elements and programmable interconnect elements, which define how the programmable elements are configured and operate. Any number of different segments or regions of programmable logic circuit 120 can be implemented in IC device 100.

[0027] Hard-block circuitry 122 may be or include one or more memory controllers (such as a Double Data Rate (DDR) memory controller, one or more High Bandwidth Memory (HBM) memory controllers, etc.), peripheral component fast interconnect (PCIe) blocks, Ethernet cores (such as a 100Gbps (C=100) media address controller (CMAC), multi-rate MAC (MRMAC), etc.), forward error correction (FEC) blocks, analog-to-digital converters (ADCs), digital-to-analog converters (DACs), and / or any other hard-block circuitry. I / O 124 may be implemented as an Extremely High Performance Input / Output (XPIO), a Multi-Gigabit Transceiver (MGT), or any other input / output block. Either hard-block circuitry 122 and / or I / O 124 may be programmable.

[0028] Interconnect circuitry 116 includes a programmable network and peripheral interconnects. In one example, interconnect circuitry 116 is a network on-chip (NoC). The programmable network communicatively couples subsystems of IC device 100 and any other circuitry together. The programmable network includes packet switches and interconnects connecting these packet switches. Each packet switch performs packet switching within the programmable network. The programmable network has interface circuitry at its edges.

[0029] In an example where interconnect circuit 116 is a NoC, the interface circuitry includes a NoC master unit (NMU) and a NoC slave unit (NSU). Each NMU is an entry circuit connecting the master circuitry to the programmable network, and each NSU is an exit circuit connecting the programmable network to the slave endpoint circuitry. The NMU is communicatively coupled to the NSU via the interconnect lines of the NoC packet switches and the programmable network. The NoC packet switches are interconnected with each other and connected to the NMUs and NSUs to implement multiple physical channels in the programmable network. The NoC packet switches, NMUs, and NSUs include register blocks that determine the operation of the respective NoC packet switch, NMU, or NSU.

[0030] Interconnect circuit 116 includes one or more physical channels. Physical channels may also have one or more virtual channels. Virtual channels enable weighting for prioritizing various communications along any physical channel. NoC packet switches also support multiple virtual channels per physical channel. The programmable network includes end-to-end Quality of Service (QoS) features for controlling data flows within it. In the example, the programmable network first segments data flows into specified service classes. Data flows within the same service class may share virtual or physical transmission paths or have independent virtual or physical transmission paths. The QoS scheme applies multiple priority levels across service classes. Within and across service classes, the programmable network applies a weighted arbitration scheme to shape service flows and provide bandwidth and latency that meet user requirements.

[0031] Peripheral interconnects include circuitry for writing to register blocks that determine the functionality of NMU, NSU, and NoC packet switches. Peripheral interconnects include peripheral interconnects coupled to register blocks for programming to configure functionality. Register blocks in the NMU, NSU, and NoC packet switches of the programmable network support interrupts, QoS, error handling and reporting, transaction control, power management, and address mapping control. Peripheral interconnects include a peripheral interconnect root node residing on processing system 118, interconnecting peripheral interconnect switches connected to the peripheral interconnect root node, and protocol blocks connected to the interconnecting peripheral interconnect switches and their corresponding register blocks.

[0032] To write to a register block, a main circuit (such as processing system 118) transmits configuration data to a peripheral interconnect root node, which packages the configuration data into a memory-mapped write request in a format implemented by the peripheral interconnect. The peripheral interconnect sends the memory-mapped write request to interconnect peripheral interconnect switches, which route the request to a protocol block connected to the register block to which the request is directed. The protocol block then translates the memory-mapped write request into a format implemented by the register block and sends the translated request to the register block to write the configuration data to that register block. Peripheral interconnects can be used to program any programmable boundary circuitry of IC device 100. For example, peripheral interconnects can be used to program any programmable hard-block circuitry 122 and / or I / O 124.

[0033] The various subsystems and circuits of IC device 100 are communicatively coupled through various communication mechanisms. Some subsystems or circuits can be directly connected to other subsystems or circuits. As illustrated, I / O 124 is directly connected to hard circuit 122 and programmable logic circuit 120, and hard circuit 122 is further directly connected to programmable logic circuit 120 and processing system 118. Programmable logic circuit 120 is directly connected to DPE array 110. DPE array 110, processing system 118, programmable logic circuit 120, hard circuit 122, and I / O 124 are communicatively coupled together via a programmable network of interconnect circuit 116.

[0034] The DPE 112 and array interface circuitry 114 can be programmed by: loading configuration data into a configuration register that defines the operation of the DPE 112 and array interface circuitry 114; loading program instruction code into program memory for execution by the DPE 112; and / or loading application data into the memory bank of the DPE 112. The processing system 118 can transmit configuration data, program instruction code, and / or application data from interconnect circuitry 116 to array interface circuitry 114 via a programmable network. At each circuit element of array interface circuitry 114 that receives configuration data, program instruction code, and / or application data, the configuration data, program instruction code, and / or application data received from the programmable network are converted into memory-mapped transactions, which are routed via the memory-mapped network to the configuration register, program memory, and / or memory bank addressed by the memory-mapped transaction (and thus to the target DPE 112). The configuration data, program instruction code, and / or application data are written to the configuration register, program memory, and / or memory bank in the form of memory-mapped transactions.

[0035] The use of a DPE array as described herein, in conjunction with one or more other subsystems, provides heterogeneous processing capabilities for IC device 100. Thus, IC device 100 can have increased processing power while maintaining low area usage and power consumption. For example, DPE array 110 can be used to hardware accelerate specific operations and / or perform functions offloaded from one or more subsystems of IC device 100. For example, when used with processing system 118, DPE array 110 can be used as a hardware accelerator. Processing system 118 can offload operations to be performed by DPE array 110 or a portion thereof. In other examples, DPE array 110 can be used to perform computationally intensive operations, such as generating digital predistortion to be provided to analog / mixed-signal circuitry.

[0036] In some examples, IC device 100 is communicatively coupled to other components. As illustrated, IC device 100 is communicatively coupled to a memory device (e.g., double data rate dynamic RAM (DDRDRAM)). Memory device 126 includes one or more memory chips, which may be located on the same board (e.g., an evaluation board) as IC device 100 or on a different board. In one example, memory device 126 includes a memory controller that receives data from I / O 124. Furthermore, memory device 126 is a high-bandwidth memory (HBM). Memory device 126 is communicatively coupled to I / O 124, which is connected to hardware circuitry 122 (e.g., one or more memory controllers). Hardware circuitry 122 is connected to a processing system 118 that is capable of reading data from memory device 126 and writing read data to memory device 126 via hardware circuitry 122 and I / O 124.

[0037] In one example, memory device 126 includes memory channels connecting memory device 126 to IC device 100. In one example, memory device 126 includes 32 memory channels. In other examples, memory device 126 includes more or fewer than 32 channels. As described in more detail below, interconnect circuitry 116 includes interleaved switching circuitry, network switching circuitry, and cross-switching circuitry that routes memory commands from requester circuit elements (e.g., DPE 112) to memory device 126. Because the use of interleaved switching circuitry, network switching circuitry, and cross-switching circuitry reduces the circuit area and power usage of interconnect circuitry 116 compared to other configurations, the semiconductor manufacturing cost of IC device 100 is reduced, while the performance of IC device 100 is improved.

[0038] Figure 2 A block diagram of interconnection circuit 116 according to one or more examples is shown. Figure 2As illustrated, interconnect circuit 116 includes interleaved switching circuit 210, network switching circuit 220, and crossover switching circuit 230. In one example, interconnect circuit 116 is connected to memory controller circuit 240. Memory controller circuit 240 is in Figure 1 It is implemented within the hard block circuit 122.

[0039] The interleaving switch circuit 210 includes one or more interleaving switch circuits 210 that receive memory commands and route the memory commands based on addresses within those memory commands. For example, a memory device (e.g., Figure 1 The memory devices 126 can be interleaved. In this example, for a granularity of 256 bytes and an address modulus range of 1024, the memory devices are interleaved such that addresses 0 to 255 are associated with the first memory device, addresses 256 to 511 with the second memory device, addresses 512 to 767 with the third memory device, and addresses 768 to 1023 with the fourth memory device. This pattern is repeated for addresses greater than 1024. In other examples, the granularity is greater than or less than 256 bytes. In one particular example, the granularity is 64 bytes. In the interleaved memory devices, memory commands can span multiple different memory devices. For example, a memory command associated with addresses 245 to 275 is associated with both the first and second memory devices.

[0040] Each interleaved memory device is associated with a different memory address based on its granularity and the number of interleaved memory devices. The address associated with each memory address is associated with a communication path within interconnect circuitry 116. Each interleaved switching circuit in interleaving circuitry 210 receives a memory command and outputs the memory command to a communication path based on the memory address of the memory command. Each interleaved switching circuit 210 includes one or more input ports and one or more output ports. The number of output ports corresponds to the interleaving order. In one example, the interleaving order is four. In other examples, the interleaving order is greater than or less than four. In one example, the interleaving order corresponds to the interleaving order of one or more memory devices. The number of input ports corresponds to the number of requester circuit elements connected to each interleaved switching circuit in interleaving switching circuitry 210.

[0041] Network switch circuit 220 includes two or more network switch circuits connected to communication paths and transmitting memory commands horizontally and / or vertically along these communication paths. In one example, each communication path is associated with one or more network switch circuits 220. Each network switch circuit 220 includes two or more ports. In one or more examples, each network switch circuit in network switch circuit 220 includes three or four ports. The ports of network switch circuit 220 are bidirectional.

[0042] The cross switch circuit 230 includes two or more cross switch circuits. Each cross switch circuit 230 is connected to a communication path associated with a corresponding memory device. For example, a communication path associated with a first memory device is connected to the first cross switch circuit 230, a communication path associated with a second memory device is connected to the second cross switch circuit 230, a communication path associated with a third memory device is connected to the third cross switch circuit 230, and a communication path associated with a fourth memory device is connected to the fourth cross switch circuit 230. In one example, each memory device is connected to a communication path associated with multiple cross switch circuits 230, and each cross switch circuit 230 is connected to a communication path associated with a single memory device. In this example, a communication path associated with the first memory device is connected to the first and second cross switch circuits 230, a communication path associated with the second memory device is connected to the third and fourth cross switch circuits 230, a communication path associated with the third memory device is connected to the fifth and sixth cross switch circuits 230, and a communication path associated with the seventh and eighth memory devices is connected to the fourth cross switch circuit 230. Figure 3 and Figure 4 An example of the connection between the communication path and the cross switch circuit 230 is illustrated.

[0043] Crossbar switch circuit 230 is connected to memory controller circuit 240. Each crossbar switch circuit in crossbar switch circuit 230 is connected to memory controller circuit 240 and outputs memory commands to memory controller circuit 240. Memory controller circuit 240 outputs commands to memory devices (e.g., memory device 126).

[0044] In one example, each interleaved switch circuit 210 is connected to the NMU associated with the requester circuit element, and each cross switch circuit 230 is connected to the NSU associated with the memory controller circuit 240 and the corresponding memory channel.

[0045] Figure 3A block diagram illustrating an interconnect circuit 300 according to one or more examples is shown. The interconnect circuit 300 includes an interleaving switch circuit 310, a network switch circuit 320, and a cross switch circuit 330. The interleaving switch circuit 310 is configured to... Figure 2 The network switch circuit 320 is configured in a similar manner to the interleaved switch circuit 210. Figure 2 The network switch circuit 220 is configured in a similar manner, and the cross switch circuit 330 is configured in a manner similar to that of the network switch circuit 220. Figure 2 The cross switch circuit 230 is configured in a similar manner. In one example, each cross switch circuit 310 includes three input ports and four output ports. In other examples, the number of input ports and output ports may be different. Furthermore, each cross switch circuit 330 has two input ports and two output ports. In other examples, the number of input ports and output ports may be different. The network switch circuit 320 includes one or more input ports and one or more output ports. In one example, different combinations of network switch circuits 320 with different numbers of input ports and output ports can be used.

[0046] Each interleaved switch circuit in the interleaved switch circuit 310 is connected to two or more vertical communication paths 324. For example... Figure 3 As illustrated in the example, each interleaved switch circuit in the interleaved switch circuit 310 is connected to four vertical communication paths 324. Each vertical communication path 324 may include one or more wires. In one example, the vertical communication path 324 is a channel.

[0047] Interleaved switch circuit 310 is connected to cross switch circuit 330 via vertical communication path 324, horizontal communication path 322, and network switch circuit 320. Horizontal communication path 322 may include one or more wires. In one example, vertical communication path 322 is a channel.

[0048] Network switch circuit 320 interconnects vertical communication path 324 and horizontal communication path 322. Each horizontal communication path 322 is connected to a plurality of cross switch circuits 330 via network switch circuit 320 and vertical communication path 324. The interconnection between each horizontal communication path and vertical communication path 324 in the horizontal communication path 322, implemented using network switch circuit 320, forms the connection between interleaved switch circuit 310 and cross switch circuit 330. For example, each interleaved switch circuit 310 is connected to each cross switch circuit in cross switch circuit 330 via different combinations of horizontal communication path 322 and vertical communication path 324 using network switch circuit 320.

[0049] For example, the output port of the interleaved switch circuit 3101 is connected to a vertical communication path 3241, which is connected to a horizontal communication path 3221 via a network switch circuit 3202. Furthermore, the vertical communication path 3241 is connected to a vertical communication path 3245 via network switch circuits 3201 and 3202 and the horizontal communication path 3221. The vertical communication path 3245 is connected to the input port of the cross switch circuit 3301. Therefore, the interleaved switch circuit 3101 is connected to the cross switch circuit 3301 via vertical communication paths 3241 and 3245, network switch circuits 3201 and 3202, and the horizontal communication path 3221.

[0050] The output port of the interleaved switch circuit 3101 is connected to the vertical communication path 3242, which is connected to the horizontal communication path 3222 via network switch circuits 3203 and 3204. Furthermore, the vertical communication path 3242 is connected to the vertical communication path 3246 via network switch circuits 3203 and 3204 and the horizontal communication path 3222. The vertical communication path 3246 is connected to the input port of the cross switch circuit 3302. Therefore, the interleaved switch circuit 3101 is connected to the cross switch circuit 3302 via the vertical communication paths 3242 and 3246, the network switch circuits 3203 and 3204, and the horizontal communication path 3222.

[0051] The output port of the interleaved switch circuit 3101 is connected to the vertical communication path 3243, which is connected to the horizontal communication path 3223 via the port of the network switch circuit 3205. Furthermore, the vertical communication path 3243 is connected to the vertical communication path 3247 via the ports of the network switch circuits 3205 and 3206 and the horizontal communication path 3223. The vertical communication path 3247 is connected to the input port of the cross switch circuit 3303. Therefore, the interleaved switch circuit 3101 is connected to the cross switch circuit 3303 via the vertical communication paths 3243 and 3247, the network switch circuits 3205 and 3206, and the horizontal communication path 3223.

[0052] The output port of the interleaved switch circuit 3104 is connected to the vertical communication path 3244, which is connected to the horizontal communication path 3224 via the port of the network switch circuit 3207. Furthermore, the vertical communication path 3244 is connected to the vertical communication path 3248 via the ports of the network switch circuits 3207 and 3208 and the horizontal communication path 3224. The vertical communication path 3248 is connected to the input port of the cross switch circuit 3304. Therefore, the interleaved switch circuit 3101 is connected to the cross switch circuit 3304 via the vertical communication paths 3244 and 3248, the network switch circuits 3207 and 3208, and the horizontal communication path 3224.

[0053] Interleaved switch circuit 3101 is connected to other cross switch circuits in cross switch circuit 330 via other combinations of vertical communication path 324, horizontal communication path 322, and network switch circuit 320. Furthermore, interleaved switch circuits 3102 to 3108 are connected to cross switch circuit 330 via combinations of vertical communication path 324, horizontal communication path 322, and network switch circuit 320. In one or more examples, each interleaved switch circuit among interleaved switch circuits 3102 to 3108 is connected to a corresponding cross switch circuit in cross switch circuit 330 via a unique combination of vertical communication path 324, horizontal communication path 322, and network switch circuit 320.

[0054] Each crossbar switch circuit 330 is associated with a different memory address range. Therefore, each horizontal communication path 322, vertical communication path 324, and network switch circuit 320 connected to the crossbar switch circuit 330 is associated with a memory address range of the crossbar switch circuit 330. In one example, one or more horizontal communication paths 322, one or more vertical communication paths 324, and / or one or more network switch circuits 320 are connected to multiple crossbar switch circuits 330. In this example, such horizontal communication paths 322, vertical communication paths 324, and / or network switch circuits 320 are associated with multiple memory address ranges.

[0055] In one example, the interleaving switch circuit 310 routes a memory command to one of the vertical communication paths 324 based on the memory address of the received memory command. The interleaving switch circuit 310 determines which vertical communication path 324 is associated with the memory address and routes the memory command to that vertical communication path 324. The memory command is received by a port of a network switch circuit 320 connected to the vertical communication path 324, and this network switch circuit routes the memory command to another network switch circuit 320 along the corresponding horizontal communication path 322 based on the memory address. The network switch circuit 320 routes the memory command to either the vertical communication path 324 or another network switch circuit 320 connected to the same horizontal communication path 322 based on the memory address. This routing process is repeated until the memory command is output to a target interleaving switch circuit 330. The target interleaving switch circuit 330 outputs the memory command to the memory device 340 via the corresponding channel. Data from the memory device 340 is routed back to the requester device by reversing the above routing process.

[0056] Figure 4 A block diagram illustrating an interconnect circuit 400 according to one or more examples is shown. The interconnect circuit 400 includes an interleaving switch circuit 410, a network switch circuit 420, and a cross switch circuit 430. The interleaving switch circuit 410 is configured to... Figure 2The network switch circuit 420 is configured in a similar manner to the interleaved switch circuit 210. Figure 2 The network switch circuit 220 is configured in a similar manner, and the cross switch circuit 430 is configured in a manner similar to that of the network switch circuit 220. Figure 2 The cross switch circuit 230 is configured in a similar manner. In one example, each cross switch circuit 410 includes four input ports and four output ports. In other examples, the number of input ports and output ports may be different. Furthermore, each cross switch circuit 430 has three input ports and three output ports. In other examples, the number of input ports and output ports may be different.

[0057] Interconnect circuit 400 is configured in a similar manner to interconnect circuit 300, i.e., interleaved switch circuit 410 is connected to cross switch circuit 430 via vertical communication path 424, horizontal communication path 422 and network switch circuit 420, just as interleaved switch circuit 310 is connected to cross switch circuit 330 via vertical communication path 324, horizontal communication path 322 and network switch circuit 320.

[0058] As mentioned above Figure 3 As described, the network switch circuit 420 interconnects the vertical communication path 424 and the horizontal communication path 422. Each horizontal communication path 422 is connected to a plurality of cross switch circuits 430 via the network switch circuit 420 and the vertical communication path 424. The interconnection between each horizontal communication path 422 and the vertical communication path 424 implemented using the network switch circuit 420 forms the connection between the interleaved switch circuit 310 and the cross switch circuit 430. For example, each interleaved switch circuit 410 is connected to each cross switch circuit in the cross switch circuit 430 via different combinations of the network switch circuit 420 via the horizontal communication path 422 and the vertical communication path 424.

[0059] In addition, as mentioned above... Figure 3 As described, each crossbar switch circuit 430 is associated with a different memory address range. Therefore, each horizontal communication path 422, vertical communication path 424, and network switch circuit 420 connected to the crossbar switch circuit 430 is associated with a memory address range of the crossbar switch circuit 430. In one example, one or more horizontal communication paths 422, one or more vertical communication paths 424, and / or one or more network switch circuits 420 are connected to multiple crossbar switch circuits 430. In this example, such horizontal communication paths 422, vertical communication paths 424, and / or network switch circuits 420 are associated with multiple memory address ranges.

[0060] Figure 3 and Figure 4Two example configurations of interconnecting circuits (e.g., interconnecting circuits 300 and 400) are illustrated. In other examples, interconnecting circuits may be configured in other ways to include more or fewer interleaved switching circuits, network switching circuits, vertical communication paths, horizontal communication paths, and / or cross-switching circuits.

[0061] Figure 5 A flowchart illustrating a method 500 for routing memory commands in interconnect circuitry, according to one or more examples, is shown. Method 500 is comprised of interconnect circuitry (e.g., Figure 2 Interconnection circuit 116 Figure 3 Interconnection circuit 300 or Figure 4 The interconnect circuit 400) is executed. See below for reference. Figure 3 Description method 500.

[0062] At 510 of method 500, a memory command is received. The memory command is received by the interleaved switching circuit of the interconnect circuit. In one example, the memory command is received by the interleaved switching circuit 310 of the interconnect circuit 300.

[0063] At 520 of method 500, the memory command is routed on a vertical communication channel based on its memory address. In one example, interleaving switch circuit 3101 outputs the memory command on one of the vertical communication paths 3241 to 3244 based on its memory address. For example, interleaving switch circuit 3101 determines the memory address from the memory command or identifies the memory address based on an identifier associated with the memory command. Interleaving switch circuit 3101 selects one of the vertical communication paths 3241 to 3244 for outputting the memory command based on the memory address. For example, each of the vertical communication paths 3241 to 3244 is associated with one or more different memory address ranges. The memory address ranges correspond to interleaving applied to memory devices (e.g., memory device 340) connected to interconnect circuit 300. In one example, the interleaved switching circuit 3101 selects the vertical communication path 3241 based on a comparison between the memory address and the memory address range, and outputs a memory command on the vertical communication path 3241 to route the memory address along the vertical communication path 3241.

[0064] At 530 of method 500, the memory command is routed to a crossbar switch circuit via one or more vertical communication paths, one or more horizontal communication paths, and one or more network switch circuits based on the memory address of the memory command. For example, the memory command is routed to crossbar switch circuit 3301 via vertical communication paths 3241 and 3245, horizontal communication path 3221, and network switch circuits 3201 and 3202. Network switch circuit 3202 receives the memory command via vertical communication path 3241. Network switch circuit 3202 determines that the memory command will be routed to network switch circuit 3201 via horizontal communication path 3221. Network switch circuit 3202 determines how to route the memory command based on the address of the memory command and / or an identifier associated with the memory command. Network switch circuit 3202 determines that network switch circuit 3201 is the target network switch circuit 3201 because network switch circuit 3201 is associated with a memory address range associated with the crossbar switch circuit 3301, which corresponds to the memory address of the memory command. In one example, each network switch circuit in network switch circuit 320 is associated with one or more memory addresses. In this example, network switch circuit 3202 compares the memory address or identifier of a memory command with the range associated with network switch circuit 3201 and the next network switch circuit 320 connected to the horizontal communication path 3221. Network switch circuit 3202 determines that network switch circuit 3201 is the target network switch circuit 3201 based on the memory address or identifier comparison.

[0065] The network switch circuit 3201 receives a memory command (and an identifier, when provided) and outputs the memory command to the cross switch circuit 3301.

[0066] At 540 of method 500, a memory command is output to the memory device by a crossbar switch circuit. For example, crossbar switch circuit 3301 outputs the memory command to memory device 340 via a memory channel. Crossbar switch circuit 3301 determines (or identifies) which memory channel is associated with a memory device that is associated with the memory address range that includes the memory address of the memory command, and outputs the memory command on the determined memory channel.

[0067] In one example, two or more interleaved switching circuits in interleaved switching circuit 310 receive and / or route memory commands during overlapping periods. Additionally, two or more network switching circuits 320 receive and / or route memory commands during overlapping periods. Each interleaved switching circuit in interleaved switching circuit 310 routes memory commands as described above with respect to method 520. Furthermore, each network switching circuit in network switching circuit 320 routes memory commands as described above with respect to method 530. Each cross-switching circuit in cross-switching circuit 330 routes memory commands as described above with respect to method 540.

[0068] The techniques described above can also be expressed in the following non-limiting embodiments.

[0069] Example 1. An interconnect circuit comprising: an interleaving switch circuit coupled to a requester device, a first interleaving switch circuit including a first port and configured to: receive a first memory command; and output the first memory command via a first communication path connected to the first port based on a memory address of the first memory command; a network switch circuit connected to the interleaving switch circuit, a first network switch circuit connected to the first communication path and configured to route the first memory command along the first communication path based on the memory address; and a cross switch circuit coupled to a memory device, a first cross switch circuit configured to: receive the first memory command from the first communication path and output the first memory command to a first memory device associated with the memory address.

[0070] Example 2. The interconnect circuit according to Example 1, wherein the first communication path is associated with a first memory address range among a plurality of memory address ranges, and wherein the first interleaving switch circuit is further configured to: output the first memory command via the first communication path based on a comparison of the memory address with the plurality of memory address ranges.

[0071] Example 3. The interconnection circuit according to Example 1, wherein the first communication path is interconnected via one or more network switch circuits in the network switch circuit.

[0072] Example 4. The interconnect circuit according to Example 1, wherein the first network switch circuit is configured to output the first memory command to a second network switch circuit in the network switch circuit based on the memory address, wherein the second network switch circuit is associated with a memory address range including the memory address.

[0073] Example 5. According to the interconnection circuit of Example 4, wherein the first network switch circuit is connected to the first interleaved switch circuit via a first communication path in the first communication path, the first network switch circuit is connected to the second network switch circuit via a second communication path in the first communication path, and the second network switch circuit is connected to the first cross switch circuit via a third communication path in the first communication path.

[0074] Example 6. The interconnect circuit according to Example 1, wherein the memory devices are interleaved, and each cross switch circuit in the cross switch circuit is associated with a corresponding memory address range of the interleaved memory devices.

[0075] Example 7. According to the interconnect circuit of Example 1, the second interleaving switch circuit in the interleaving switch circuit is configured to route the second memory command to the first interleaving switch circuit via a second communication path having the second network switch circuit in the network switch circuit based on the memory address of the second memory command, wherein the memory address of the first memory command and the memory address of the second memory command are within a first memory address range for interleaving the memory devices.

[0076] Example 8. An integrated circuit (IC) device, the IC device comprising: a requester circuit element configured to output a first memory command; and interconnection circuitry connected to the requester circuit element and comprising: a first interleaving switch circuit in an interleaving switch circuit, the first interleaving switch circuit being configured to: receive the first memory command; and output the first memory command via a first communication path connected to a first port based on a memory address of the first memory command; a network switch circuit connected to the interleaving switch circuit, the first network switch circuit in the network switch circuit being connected to the first communication path and configured to route the first memory command along the first communication path based on the memory address; and a cross switch circuit coupled to a memory device, the first cross switch circuit in the cross switch circuit being configured to: receive the first memory command from the first communication path and output the first memory command to a first memory device associated with the memory address in the memory device.

[0077] Example 9. According to the IC device of Example 8, wherein the first communication path is associated with a first memory address range among a plurality of memory address ranges, and wherein the first interleaving switch circuit is further configured to: output the first memory command via the first communication path based on a comparison of the memory address with the plurality of memory address ranges.

[0078] Example 10. The IC device according to Example 8, wherein the first communication path is interconnected via one or more network switch circuits in the network switch circuit.

[0079] Example 11. The IC device according to Example 8, wherein the first network switch circuit is configured to output the first memory command to a second network switch circuit in the network switch circuit based on the memory address, wherein the second network switch circuit is associated with a memory address range including the memory address.

[0080] Example 12. According to the IC device of Example 11, wherein the first network switch circuit is connected to the first interleaved switch circuit via a first communication path in the first communication path, the first network switch circuit is connected to the second network switch circuit via a second communication path in the first communication path, and the second network switch circuit is connected to the first cross switch circuit via a third communication path in the first communication path.

[0081] Example 13. The IC device according to Example 8, wherein the memory devices are interleaved, and each cross switch circuit in the cross switch circuit is associated with a corresponding memory address range of the interleaved memory devices.

[0082] Example 14. According to the IC device of Example 8, the second interleaving switch circuit in the interleaving switch circuit is configured to route the second memory command to the first interleaving switch circuit via a second communication path having the second network switch circuit in the network switch circuit based on the memory address of the second memory command, wherein the memory address of the first memory command and the memory address of the second memory command are within a first memory address range for interleaving the memory devices.

[0083] Example 15. A method comprising: receiving a first memory command at a first interleaving switch circuit in an interleaving switch circuit of an interconnect circuit; outputting the first memory command via a first communication path of the interconnect circuit via a first port of the first interleaving switch circuit based on a memory address of the first memory command; routing the first memory command along the first communication path via a first network switch circuit in a network switch circuit of the interconnect circuit based on the memory address, the first network switch circuit being connected to the first communication path; and outputting the first memory command to a first memory device associated with the memory address in a memory device via a first cross switch circuit in a cross switch circuit of the interconnect circuit, the first cross switch circuit being connected to the first communication path.

[0084] Example 16. According to the method of Example 15, the method further includes: determining, via the first communication path, the output of the first memory command based on a comparison between the memory address and a first memory address range among a plurality of memory address ranges by the first interleaving switch circuit, wherein the first memory address range is associated with the first communication path.

[0085] Example 17. The method according to Example 15, wherein the first communication path is interconnected via one or more network switch circuits in the network switch circuit.

[0086] Example 18. The method according to Example 15, the method further comprising: outputting the first memory command to a second network switch circuit in the network switch circuit based on the memory address via the first network switch circuit, wherein the second network switch circuit is associated with a memory address range including the memory address.

[0087] Example 19. According to the method of Example 18, wherein the first network switch circuit is connected to the first interleaved switch circuit via a first communication path in the first communication path, the first network switch circuit is connected to the second network switch circuit via a second communication path in the first communication path, and the second network switch circuit is connected to the first cross switch circuit via a third communication path in the first communication path.

[0088] Example 20. The method according to Example 15, wherein the memory devices are interleaved, and each cross switch circuit in the cross switch circuit is associated with a corresponding memory address range of the interleaved memory devices.

[0089] While the foregoing describes specific embodiments, other and additional examples may be devised without departing from the basic scope of the invention, the scope of which is defined by the appended claims.

Claims

1. An interconnect circuit comprising: interleaved switch circuits coupled to a requester device, a first interleaved switch circuit of the interleaved switch circuits comprising a first port and configured to: receive a first memory command; and output the first memory command via a first communication lane connected to a first one of the first ports based on a memory address of the first memory command; network switch circuits connected to the interleaved switch circuits, a first network switch circuit of the network switch circuits connected to the first communication lane and configured to route the first memory command along the first communication lane based on the memory address; and crossbar switch circuits coupled to memory devices, a first crossbar switch circuit of the crossbar switch circuits configured to receive the first memory command from the first communication lane and output the first memory command to a first one of the memory devices associated with the memory address.

2. The interconnect circuit of claim 1, wherein the first communication lane is associated with a first memory address range of a plurality of memory address ranges, and wherein the first interleaved switch circuit is further configured to output the first memory command via the first communication lane based on a comparison of the memory address to the plurality of memory address ranges.

3. The interconnect circuit of claim 1, wherein the first communication lane is interconnected via one or more of the network switch circuits.

4. The interconnect circuit of claim 1, wherein the first network switch circuit is configured to output the first memory command to a second one of the network switch circuits based on the memory address, wherein the second network switch circuit is associated with a memory address range that includes the memory address.

5. The interconnect circuit of claim 4, wherein the first network switch circuit is connected to the first interleaved switch circuit via a first one of the first communication lanes, the first network switch circuit is connected to the second network switch circuit via a second one of the first communication lanes, and the second network switch circuit is connected to the first crossbar switch circuit via a third one of the first communication lanes.

6. The interconnect circuit of claim 1, wherein the memory devices are interleaved with one another, and each of the crossbar switch circuits is associated with a respective memory address range of the interleaved memory devices. ​ 7. The interconnect circuit of claim 1, wherein a second one of the interleave switch circuits is configured to route a second memory command to the first crossbar switch circuit via a second communication lane having a second one of the network switch circuits based on a memory address of the second memory command, wherein the memory address of the first memory command and the memory address of the second memory command are within a first memory address range for interleaving the memory device.

8. An integrated circuit (IC) device, the integrated circuit (IC) device comprising: a requestor circuit element configured to output a first memory command; and an interconnect circuit connected to the requestor circuit element and comprising: a first one of interleave switch circuits configured to: receive the first memory command; and output the first memory command via a first communication lane connected to a first port based on a memory address of the first memory command; a network switch circuit connected to the interleave switch circuits, a first one of the network switch circuits connected to the first communication lane and configured to route the first memory command along the first communication lane based on the memory address; and a crossbar switch circuit coupled to a memory device, a first one of the crossbar switch circuits configured to receive the first memory command from the first communication lane and output the first memory command to a first one of the memory devices associated with the memory address.

9. The IC device of claim 8, wherein the first communication lane is associated with a first one of a plurality of memory address ranges, and wherein the first interleave switch circuit is further configured to output the first memory command via the first communication lane based on a comparison of the memory address to the plurality of memory address ranges.

10. The IC device of claim 8, wherein the first communication lane is interconnected via one or more of the network switch circuits.

11. The IC device of claim 8, wherein the first network switch circuit is configured to output the first memory command to a second one of the network switch circuits based on the memory address, wherein the second network switch circuit is associated with a memory address range that includes the memory address.

12. The IC device of claim 11, wherein the first network switch circuit is connected to the first interleave switch circuit via a first one of the first communication lanes, the first network switch circuit is connected to the second network switch circuit via a second one of the first communication lanes, and the second network switch circuit is connected to the first crossbar switch circuit via a third one of the first communication lanes.

13. The IC device of claim 8, wherein the memory devices are interleaved with each other and each crossbar circuit in the crossbar circuit is associated with a respective memory address range of the interleaved memory devices.

14. The IC device of claim 8, wherein a second crossbar circuit in the crossbar circuit is configured to route a second memory command to the first crossbar circuit via a second communication lane having a second network switch circuit in the network switch circuit based on a memory address of the second memory command, wherein the memory address of the first memory command and the memory address of the second memory command are within a first memory address range used to interleave the memory devices.

15. A method comprising: receiving a first memory command at a first crossbar circuit in a crossbar circuit of an interconnect circuit; outputting the first memory command via a first port of the first crossbar circuit via a first communication lane of the interconnect circuit based on a memory address of the first memory command; routing the first memory command along the first communication lane via a first network switch circuit in a network switch circuit of the interconnect circuit based on the memory address, the first network switch circuit connected to the first communication lane; and outputting the first memory command to a first memory device of memory devices associated with the memory address via a first crossbar circuit in a crossbar circuit of the interconnect circuit, the first crossbar circuit connected to the first communication lane. ​

Images