Methods for improving data throughput in open-core protocol-based interconnect networks using dynamically selectable redundant physical paths with a shared link

Abstract

Integrated circuit, including: a first Fabric (102) with which a large number of initiators (106) are communicatively coupled; a second fabric with which a multitude of goals are communicatively coupled; first and second fabric-to-fabric links (300, 302), each supporting communication between the first (102) and second fabrics; and a routing circuit and associated logic that are operationally coupled with the first and second fabric-to-fabric links (300, 302), wherein the routing circuit and logic are configured to dynamically route data originating from at least one initiator (106) and destined for a destination (108) by selectively using one of the first or second fabric-to-fabric links (300, 302) to route the data from the at least one initiator (106) to the destination (108); where routing of data includes determining whether an address used by the initiator (106) for the target (108) is part of a first or second address range of the target (108) in an address map of the first fabric (102), wherein the first address range is associated with the first fabric-to-fabric link (300, 302) and the second address range is associated with the second fabric-to-fabric link, and routing the data via the fabric-to-fabric link (300, 302) associated with the determined address range.

Description

FIELD OF THE INVENTION

[0001] The field of the invention relates generally to computer system interfaces and, more particularly, but not exclusively, to techniques for improving the data throughput performance of interconnects on systems-on-chips (SoCs) and the like. BACKGROUND INFORMATION

[0002] Computer systems typically use one or more interconnects to facilitate communication between system components, such as processors and memory. Interconnects and / or expansion interfaces may also be used to support built-in and additional devices, such as input / output (IO) devices and expansion cards, and the like. For many years after the introduction of the personal computer, the primary form of interconnect was a parallel bus. Parallel bus structures were used for both internal data transfers and expansion buses, such as ISA (Industry Standard Architecture), MCA (Micro Channel Architecture), EISA (Extended Industry Standard Architecture), and VESA Local Bus. In the early 1990s, Intel Corporation introduced the PCI (Peripheral Component Interconnect) computer bus.PCI improved on previous bus technologies by not only increasing bus speed but also introducing automatic configuration and transaction-based data transfers using shared address and data lines.

[0003] Over time, computer processor clock speeds increased at a faster pace than parallel bus clock speeds. As a result, computer workloads were often limited by interconnect bottlenecks rather than processor speed. Although parallel buses support the transfer of a large amount of data (for example, 32 or even 64 bits under PCI-X) with each cycle, their clock speeds are limited by skew considerations, resulting in a practical limit on the maximum bus speed. To overcome this problem, high-speed serial interconnects were developed. Examples of early serial connections include Serial ATA, USB (Universal Serial Bus), FireWire, and RapidIO.

[0004] Another widely used serial connection standard is PCI Express, also called PCIe, which was introduced in 2004 under the PCIe 1.0 standard. PCIe was designed to replace older PCI and PCI-X standards and provides legacy support. PCIe uses point-to-point serial links instead of a shared parallel bus architecture. Each link supports a point-to-point communication channel between two PCIe ports using one or more lines, with each line comprising a bidirectional serial connection (link). The lines are physically routed using a crossbar switch architecture, which supports communication between multiple devices at the same time. As a result of its inherent advantages, PCIe has replaced PCI as the most common connection in today's PCs. PCIe is an industry standard maintained by the PCI-SIG (Special Interest Group).As such, PCIe pads are available from many ASIC and silicon manufacturers.

[0005] Recently, Intel launched the QuickPath Interconnect ® (QPI). QPI was originally implemented as a point-to-point processor interconnect that replaces the front side bus on platforms that use high-performance processors, such as Intel ® Xeon ® - and Itanium ® processors. QPI is scalable and particularly advantageous in systems with multiple processors that share memory resources. QPI transactions use packet-based transfers using a multi-layer protocol architecture. Its features include support for coherent transactions (e.g., memory coherence).

[0006] Recently, the Open Core Protocol was also introduced. It is an openly licensed, core-centric protocol designed to meet current system-level integration requirements. OCP defines a bus-independent, configurable, and scalable interface for on-chip subsystem communications. The current version of the OCP specification is the OCP 3.0 specification (updated from the previous version, OCP 2.2), both of which are available for download at ocpip.org.

[0007] Other recent advances include multi-core processors, multi-function SoCs, and higher-density cores and chips. At the same time, contributions are being made to reducing energy consumption, particularly for mobile platforms. To take advantage of the scalability offered by these advances, the numerous and sometimes conflicting constraints must be addressed. For example, when crossbar interconnects (aka fabrics) are implemented in an SoC, latency and energy consumption increase as a function of the number of IP blocks connected to the fabric. At the same time, virtual point-to-point links facilitated by such crossbar interconnects can provide significant inter-IP block communication throughput. Accordingly, it would be advantageous to implement scalable architectures that support improved throughput without a corresponding increase in energy consumption.

[0008] WO 2010 / 117524 A2 discloses, in one embodiment, an integrated endpoint with a virtual port coupled between an upstream fabric and an integrated device fabric, and comprising multifunction logic for handling various functions for one or more intellectual property (IP) blocks coupled to the integrated device fabric. The integrated device fabric has a primary channel for communicating data and command information between the IP block and the upstream fabric, and a sideband channel for communicating sideband information between the IP block and the multifunction logic. Other embodiments and claims.

[0009] US 2003 / 0208644 A1 discloses that, in a network connecting a plurality of devices, the image processing function of each device, information on the specification of each device, and the like are collected and controlled, and functions that can be realized by combining two or more devices are determined. Profiles indicating respective combinations are transmitted to each device. Each device that has received the profiles displays executable functions based on the profiles and performs controls such as device operation, data transmission, and the like to realize a function selected from the displayed functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals refer to like parts throughout the several views unless otherwise indicated: Fig. Figure 1a shows an SoC architecture containing a set of fabrics and corresponding fabric-to-fabric links; Fig. 1b shows an implementation of the SoC architecture of Fig. 1a, further showing a plurality of shared fabric-to-fabric links; Fig. 1c shows an implementation of the SoC architecture of Fig. 1a, further showing a plurality of redundant fabric-to-fabric links supporting dynamic routing of data between fabrics; Fig.Figure 2 shows a block diagram illustrating the use of basic objects from the Open Core protocol; Fig. Figure 3a shows various unidirectional links in a pair of OCP fabrics, and a pair of fabric-to-fabric links between the OCP fabrics, and further shows address mappings for each of the OCP fabrics; Fig. 3b shows the fabric configuration of Fig. 3a, further comprising a redundant fabric-to-fabric link between the OCP fabrics; Fig. Figure 3c shows a variation of the configuration of Fig. 3b, further including selection logic for dynamically routing data from initiators to targets, and further showing corresponding modifications to the fabric address mappings; Fig. 4 is a diagram showing details of an implementation of the selectors of Fig. 3c, according to one embodiment; and Fig.Figure 5 is a block diagram of an exemplary SoC in which a bridge hierarchy is implemented in a south complex. DETAILED DESCRIPTION

[0011] Described herein are embodiments of methods and apparatus for facilitating data throughput improvements in interconnect fabrics using dynamically selectable redundant shared links. In the following description, numerous specific details are set forth, such as implementations utilizing OCP links, in order to provide a thorough understanding of embodiments of the invention. However, one skilled in the art will recognize that the invention may be practiced without one or more of the specific details, or with different methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.

[0012] Reference throughout the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" in various places throughout the specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0013] For clarity, individual components in the figures may be referred to herein by their designations or labels rather than by their specific reference numbers. For example, the labeling of nodes or blocks in different figures provides information for identifying the node / block and / or its function; such information cannot be conveyed solely with separate reference numbers. Additionally, reference numbers referring to a particular type of component (as opposed to a specific component) may be represented by a reference number followed by "(type)", meaning "typical". It is understood that the configuration of these components will typically be composed of similar components that exist but are not shown in the drawings for simplicity and clarity.

[0014] As discussed above, computer architectures are moving from architectures with discrete components to SoC-based architectures. Modularity is also an important aspect of SoC architectures. Typically, the system designer will integrate numerous functional blocks, including functional blocks commonly referred to in the industry as intellectual property (IP) cores, IP blocks, or simply IP. For purposes herein, these functional blocks will be referred to generically as IP blocks or simply "IP"; it is understood that the terminology IP blocks or IP also covers IP cores and any other component or block, commonly known as IP, as would be understood by those in the SoC design and manufacturing industry. These IP blocks generally serve one or more dedicated functions and often include existing circuit design blocks licensed from numerous vendors or developed in-house.To integrate these IP blocks, numerous interfaces are designed in the SoC.

[0015] Fig.1a shows an exemplary SoC architecture 100 employing multiple interconnect fabrics 102 configured in a hierarchical manner. The term "hierarchical" means that the interconnect network comprises an interconnected group of subnetworks, commonly referred to as interconnect fabrics or simply fabrics. For simplicity, the fabrics are generally referred to herein by their identifiers (e.g., Fabric 1, Fabric 2, etc.) rather than a separate reference number for each fabric. Accordingly, the SoC architecture includes Fabrics 1-5, each operating as an independent subnetwork interconnected to the other fabrics to facilitate communication across multiple subnetworks, as explained below.Numerous IP blocks 104 (also referred to as IP cores by the OCP specification) are shown connected to corresponding fabrics in SoC architecture 100; these IP blocks are labeled AV and, for convenience, are referred to herein by their letter designation. The numerous IP blocks are representative of components or functional units (i.e., functional blocks) typically used in SoC designs, including, but not limited to, processor cores, memory cache components and agents, memory controllers, I / O controllers and interfaces, peripherals and peripheral interfaces, video and audio components and interfaces, platform management components, etc.

[0016] Interconnect Fabrics, such as Fabrics 1-5 in Fig.1a, support communication between the IP blocks using corresponding wiring and protocols. In general, the structure of a particular interconnect fabric may include a full crossbar mesh, a sparsely populated fabric comprising a plurality of point-to-point links, a shared bus-type architecture, or a ring topology. In one embodiment, the SoC architecture 100 is generalized such that each of the fabrics 1-5 may be configured in any of these topologies. Furthermore, the particular interconnect structure and protocols are also generalized in the SoC architecture 100 such that the various fabrics may use the same or different interconnect structures and protocols.For example, it may be desirable to use an interconnect supporting a cache coherence protocol (e.g., QPI) for communication between processor cores and memory cache-related IP blocks, while other fabrics and protocols, such as OCP, can be used for other fabrics in the architecture. Optionally, a single protocol can be used for the entire architecture, or the protocols defined in the . Fig. 1a may represent a portion of an SoC. For example, in one embodiment, the SoC architecture 100 corresponds to a South complex in an SoC architecture as shown in Fig. 5 and described below.

[0017] In one embodiment, each of fabrics 1-5 comprises an OCP fabric. Under the Open Core Protocol, communications are initiated by an initiator (I) and directed across the fabric to a destination (T), as described in detail below. Accordingly, initiator blocks 106 (I) and destination blocks 108 (T) are located within respective IP blocks AV in Fig. 1. In general, an initiator block I or target block T represents an interface circuit to facilitate communication between an IP block and the fabric to which it is coupled, and accordingly, initiator and target blocks may also refer to initiator and target interfaces in the IP blocks shown here.

[0018] Also depicted in the SoC architecture 100 are a plurality of fabric-to-fabric links 110 used to couple pairs of fabrics for communication. These are denoted as FF nm, where n denotes one of the coupled fabrics and m represents the other. For example, the fabric-to-fabric link FF 1-2 facilitates communication between Fabric 1 and Fabric 2. Although referred to as fabric-to-fabric links, these links can also function as fabric-to-fabric bridges (bridges) in cases where the connected fabrics use different structures and protocols (for example, a QPI-to-OCP bridge), or in cases where the clock speeds of a pair of connected fabrics using the same protocol are different.

[0019] In Fig.1a, numerous IP blocks 104 are depicted as including an initiator block I and / or a target block T, indicating that the interfacing capability with the OCP fabric may vary for the numerous IP blocks. For example, peripheral devices such as audio inputs (microphones) and outputs (speakers) may only require unidirectional communication with other IP blocks, noting that corresponding audio interface components may support either bidirectional or unidirectional communication. Similarly, each of the fabric-to-fabric links 110 is depicted as a double-headed arrow to indicate support for bidirectional communication across the links. However, reference here to bidirectional communication does not require bidirectional physical links (i.e., a physical connection that supports bidirectional communication across its wires).Rather, as explained in more detail below, bidirectional communication between IP blocks or fabrics can be facilitated by using unidirectional links operating in opposite directions. Alternatively, bidirectional data transfers can occur over individual unidirectional links (i.e., corresponding sets of wires are used to support bidirectional data transfers), for example, when an initiator performs read and write operations with a target.

[0020] Aspects of the embodiments disclosed herein may be advantageously employed in mobile platforms, such as smartphones and tablets. Power consumption plays a significant role in these types of platforms, and thus, any reduction in power consumption is beneficial. In accordance with the teachings herein, power consumption may be reduced through the use of a hierarchical interconnect fabric architecture and through the use of dynamically selectable routing across redundant shared physical links.

[0021] Compared to a common bus or fully populated crossbar interconnects, hierarchical networks generally have an advantage [1] in energy efficiency, since selected fabrics can be power gated when not needed, and [2] in silicon area, since the complexity of a fabric (implemented as a crossbar) grows with the square of the number of connected units. In general, the use of any active transistor operation or any operation that places an electrical load on a SoC circuit results in the consumption of energy. Furthermore, the higher the operating frequency (e.g., clock cycle or frequency) used for a circuit, such as interconnects, the higher the energy consumption. In this regard, energy can be saved by using less physical interconnect structure (i.e.,, fewer "wires" and corresponding interface circuitry, such as buffers and switching logic) and / or operation at lower frequencies. In both cases, however, there is a trade-off between power consumption and throughput.

[0022] One way to address this trade-off is to use lower frequency clock rates for parts of the SoC architecture that do not require the higher rates generally associated with processor and memory operations or video operations. For example, audio components, peripherals, and I / O components generally classified as slow can be configured to connect to corresponding interconnect fabric blocks operating at low clock rates. Because the hierarchical nature of the SoC architecture 100 supports the implementation of separate fabrics, individual fabrics can operate at lower frequencies or be placed in a standby or powered-down state.For example, if audio IP blocks are paired with a separate fabric, the fabric can be powered off or placed in a standby state when audio functions are not required, thus preserving battery life.

[0023] Another inventive aspect of the present invention is the use of redundant physical links that can be dynamically configured to support higher transfer rates under certain operating conditions, while simultaneously reducing power consumption under operating conditions requiring lower transfer rates. In the following embodiments, Open Core Protocol-compliant fabrics are used to demonstrate aspects of implementing these features. However, it should be noted that fabric implementations using other protocols may also be used to achieve similar improvements in data throughput and power reduction.

[0024] The Open Core Protocol defines a point-to-point interface between two communicating entities, such as IP cores and bus interface modules (bus wrappers), also referred to here as agents. One entity acts as the master of the OCP instance and the other as the slave. Only the master can present commands and is the controlling entity. The slave responds to commands presented to it by either accepting data from the master or presenting data to the master. For the two entities to communicate in a peer-to-peer manner, there must be two instances of the OCP connecting them: one where the first instance is a master and one where the first instance is a slave.

[0025] Fig.Figure 2 shows a simple system containing a wrapped bus and three IP core units: one that is a system target, one that is a system initiator, and one that is both an initiator and a target. It should be understood that, in general, "master" and "initiator" are synonymous and, as such, these terms can be used interchangeably here. Similarly, "slave" and "target" are synonymous and can be used interchangeably.

[0026] The characteristics of the IP core determine whether the core requires a master, slave, or both sides of the OCP; the wrapper interface modules must act as the complementary side of the OCP for each connected unit. A transfer across this system proceeds as follows: A system initiator (such as the OCP master) presents command, control, and possibly data to its connected slave (a bus wrapper interface module). The interface module plays the request across the on-chip bus system. The OCP does not specify the embedded bus functionality. Instead, the interface designer converts the OCP request into an embedded bus transfer. The receiving bus wrapper interface module (as the OCP master) converts the embedded bus operation into a legal OCP command. The system target (OCP slave) receives the command and performs the requested action.

[0027] Each OCP instance is configured based on the requirements of the connected units (by selecting signals or bit widths of a specific signal) and is independent of the others. For example, system initiators may require more address bits in their OCP instances than the system targets; the additional address bits could be used by the embedded bus to select which bus target is addressed by the system initiator.

[0028] The OCP is flexible. There are several useful models for how existing IP cores communicate with each other. Some use pipelining to improve bandwidth and latency characteristics. Others use multicycle access models, where signals are kept static across multiple clock cycles to simplify timing analysis and reduce implementation area. Support for this wide range of behavior is possible through the use of synchronous handshaking signals, which allow both the master and slave to control when signals are allowed to change.

[0029] Table 1 below lists the basic OCP signals. Generally, further details regarding many aspects of OCP are set forth in the Open Core Protocol Specification 2.2 (or 3.0). In addition to this basic OCP, there are numerous optional signals that can be implemented, as defined by the OCP specifications. TABLE 1 name Width driver function Clk 1 varies Clock input EnableClk 1 varies activate OCP clock MAddr configurable master Transfer address MCmd 3 master Transfer order MData configurable master Write data MDataValid 1 master Write data valid MRespAccept 1 master Master accepts answer SCmdAccept 1 Slave Slave accepts transfer SData configurable Slave Reading data SDataAccept 1 Slave Slave accepts write data SResp 2 Slave Transfer response Clk

[0030] Input clock signal for the OCP clock. The rising edge of the OCP clock is defined as a rising edge of Clk that samples the asserted EnableClk. Falling edges of Clk and any rising edge of Clk that do not sample the asserted EnableClk do not constitute rising edges of the OCP clock. EnableClkEnableClk

[0031] Specifies which rising edges of Clk are the rising edges of the OCP clock, that is, which rising edges of Clk sample and advance the interface state. Use the enableClk parameter to configure this signal. EnableClk is driven by a third unit and serves as an input to both the master and slave.

[0032] If enableclk is set to 0 (default), the signal is not present and the OCP behaves as if EnableClk is assumed to be constant. In this case, all rising edges of Clk are rising edges of the OCP clock. MAddr

[0033] The transfer address, MAddr, specifies the slave-dependent address of the resource addressed by the current transfer. Use the address parameter to configure this field in the OCP. Use the address width (addr_wdth) parameter to configure the width of this field. MAddr is a byte address based on the OCP word length (data_wdth). data_wdth defines a minimum addr_wdth value based on the data bus byte width and is defined as: min_addr_wdth=max(1,floor(log2(data_wdth))−2).

[0034] If the OCP word length is greater than a single byte, the aggregate is addressed at the OCP word-oriented address and the least significant address bits are wired to 0. If the OCP word length is not a power of two, the address is the same as it would be for an OCP interface with a word length equal to the next larger power of two. MCmd

[0035] Transfer command. This signal indicates the type of OCP transfer requested by the master. Each non-idle command is either a read or write request, depending on the direction of data flow. MData

[0036] Write data. This field contains the write data from the master to the slave. The field is configured in the OCP using the mdata parameter, and its width is configured using the data_wdth parameter. The width is not limited to multiples of 8. MDataValid

[0037] Write data valid. When set to 1, this bit indicates that the data in the MData field is valid. Use the DataHandshake parameter to configure this field in the OCP. MRespAccept

[0038] Accept master response. The master indicates that it accepts the current response from the slave with a value of 1 in the MRespAccept signal. Use the respaccept parameter to enable this field in the OCP. SCmdAccept

[0039] Slave accepts transfer. A value of 1 in the SCmdAccept signal indicates that the slave accepts the master's transfer request. Use the cmdaccept parameter to configure this field in the OCP. SData

[0040] This field contains the requested read data from the slave to the master. The field is configured in the OCP using the sdata parameter, and its width is configured using the data_wdth parameter. The width is not limited to multiples of 8. SDataAccept

[0041] Slave accepts write data. The slave indicates that it is accepting pipelined write data from the master with a value of 1 in SDataAccept. This signal is only meaningful when a data handshake is used. Use the dataaccept parameter to configure this field in the OCP. SResp

[0042] Response field from the slave to a transfer request from the master. This field is configured in the OCP using the resp parameter.

[0043] As can be seen from the above, a given OCP link can support bidirectional traffic (for example, writing from a master to a slave and reading from a slave by a master). However, for clarity, the direction of a given link is from its initiator (i.e., master under OCP) to its target (i.e., slave under OCP) and is referred to as a unidirectional link.

[0044] Fig.Figure 1b shows an example of the use of shared physical OCP links between OCP fabrics. Each of these links comprises a set of OCP wires with data transfers implemented according to applicable OCP protocol signaling. Each fabric-to-fabric link is labeled FF ms, where m corresponds to the fabric operating as the master and s corresponds to the fabric operating as the slave. For example, the pair of links between Fabric 1 and Fabric 2 is labeled FF 1-2 and FF 2-1.

[0045] Fig.Figure 1b also illustrates two data transfers initiated by IP blocks IP-B and IP-D connected to Fabric 1, respectively targeting IP block IP-J connected to Fabric 2 and IP block IP-O connected to Fabric 4. As can be seen, these two data transfers use the same physical data path defined by the fabric-to-fabric link FF 1-2 and are thus represented as shared links. While this is beneficial for facilitating communication between IP blocks coupled to numerous fabrics, it results in traffic congestion requiring arbitration of the shared links, resulting in bandwidth bottlenecks and thus reducing overall throughput.

[0046] In accordance with the teachings herein, one or more redundant fabric-to-fabric links are implemented to reduce traffic congestion and improve overall throughput. Details of example implementations of such redundant fabric-to-fabric links are provided in the Fig. 1c, Fig. 3b and Fig. 3c. For example, in the SoC architecture 100C of Fig. 1c a redundant set of fabric-to-fabric OCP links (one in each direction) to architecture 100b of Fig. 1b. As before, each fabric-to-fabric link is labeled as FF ms, with the addition of an "R" to indicate that the link is redundant. It is noted that the inclusion of adding a pair of unidirectional links between the fabrics in Fig.1c is merely exemplary, as it is not necessary to add redundant fabric-to-fabric links between any particular pair of fabrics, and the number of links added in a particular direction does not need to match the number of links added (if any) in an opposite direction. Furthermore, the general concept can be extended to further add additional fabric-to-fabric links between a particular pair of fabrics, if desired.

[0047] The Fig. 3a and Fig. 3b show various OCP links within fabrics 1 and 2 and fabric-to-fabric links 200 and 202 between fabrics 1 and 2. For simplicity and clarity, only selected initiators and targets are shown in the Fig. 1a-c shown in the Fig. 3a-c. The initiators in the Fig.3a-c are labeled Ix-y, where x identifies the fabric and y identifies the specific initiator of the fabric. Similarly, targets with Tx-y are labeled Fig. 3a-c. To avoid clutter, the IP blocks corresponding to these initiators and targets are also included in the Fig. 3a-c are not shown; however, it is understood that these IP blocks would be coupled with the applicable fabrics in an actual implementation.

[0048] Both the Fig. 3a as well as the Fig. 3b shows common initiators, targets and corresponding OCP point-to-point links. Fig.Figure 3b further illustrates the addition of a redundant fabric-to-fabric OCP link 304 between the target agent (TA) 306 and the initiator agent (IA) 308 of fabric 1 and 2, respectively. The redundant link is realized by implementing a second instance of the link target agent (box labeled TA) logic in fabric 1 and a second instance of the link initiator agent (box labeled IA) logic in fabric 2, and connecting these agents with (physical) wires according to the specific OCP link width to be implemented. The properties of the OCP bus of the original and redundant links are thus identical if the configuration of both the fabric-to-fabric link 302 and the fabric-to-fabric link 304 is the same. Furthermore, additional OCP links are shown between each of the initiators I1-1, I1-2 and I1-3 and target agent 306 and between initiator agent 308 and targets T2-4 and T2-5.As illustrated, the addition of a redundant fabric-to-fabric OCP link 304 provides a parallel data path to the original shared fabric-to-fabric OCP link 302.

[0049] A modification has also been made to the address mapping of both fabrics. This modification was made to enable a fair distribution of data paths between those using the original shared link and those using the redundant link. Fig. 3b shows the example partitioning such that paths to destinations T2-1, T2-2, and T2-3 use the original shared link, while paths to destinations T2-4 and T2-5 use the redundant shared link. (Note that with this partitioning, the two Fig. 3a would now use different shared links and would not impose any delays on each other.)

[0050] The lower part of each of the Fig. Figures 3a-c show destination address mappings for the destinations in Fabrics 1 and 2. Under OCP, unidirectional point-to-point links are routed between initiators and destinations, thus predefining the destination address ranges of the destinations for each fabric. Each destination is assigned a fixed address range, and since a given destination can be accessed by multiple initiators, the address ranges for some destinations can be larger and / or occupy multiple segments. This is especially true for destination addresses corresponding to shared fabric-to-fabric OCP links. For example, note that the address ranges for destinations T1-4 in Fabric 1 contain four segmented ranges, including three segments with a length longer than comparable segments for destinations T1-1, T1-2, and T1-3.

[0051] The Fig.The addressing scheme shown in Figure 3a corresponds to the use of conventional common fabric-to-fabric OCP links, while the addressing scheme in Fig. 3b corresponds to the addition of the added fabric-to-fabric link 204. To accommodate the new destination T1-5, the highest two address ranges for T1-4 are Fig. 3a T1-5 in Fig. 3b, while the rest of the address range mappings remain the same. Note that this simultaneously creates a partitioning of paths between transfers over the original shared fabric-to-fabric link 202 and the added redundant fabric-to-fabric link 204. For example, transfers from initiators in Fabric 1 corresponding to destinations T2-4 and T2-5 in Fabric 2 are now routed to fabric-to-fabric link 204.

[0052] Fig.Figure 3c shows an optional extension that can be implemented to further improve data throughput. The extension attempts to opportunistically reroute a path that would normally use the original shared link to use the redundant link instead in the case where the original link is busy executing previous instructions while the redundant link is idle (thereby avoiding the latency associated with flushing the previous instructions). The details of the extension are as follows: A destination that the original link (e.g., fabric-to-fabric link 302) uses is selected. In this case, destination T2-3. The address mapping region associated with the destination is examined. The starting address for the region is determined. In this case, the address is T2_3_StartAddr. The size of the region is determined, and from this, the effective number of address bits is derived as N.(N means that the size of the region is less than or equal to 2. N bytes.) The address map is searched for a suitable alias region. The alias region must be equal in size to the T2-3 region and be located in (previously) unused space (gray area in the address map). In Fig.2c, such a region exists as a T2-3 alias with start address T2_3_AliasStartAddr. The address mappings for the two fabrics are then updated so that when an address is presented in the alias region (by a Fabric 1 initiator), the path is routed through the redundant link (Fabric-to-Fabric link 304) to destination T2-3 (using the dynamic path marked with a dashed line). Finally, a (full combination) logic block, labeled selector 310, is added (for each Fabric 1 initiator). The selector's role is to multiplex, if necessary, in the alias version of the T2-3 address (originally coming from the initiator).

[0053] In general, the use of the alias addressing scheme (and associated dynamic routing) is transparent to both initiators and targets. Accordingly, the destination addresses used by an initiator interface to transfer data between an initiator and a target remain unchanged. Furthermore, rerouting is not limited to routes from an original link to a redundant link. Rather, rerouting can also be applied to a command that would default to the redundant link but could dynamically switch to the original link instead.

[0054] Fig.Figure 4 shows exemplary implementation details of selector 310 according to one embodiment. In general, most of the logic performed by selector 310 is implemented via a logic block 400, which shows the logic used to determine whether or not to use an alias address or an original address. Fig.The example implementation shown in Figure 3 assumes a 32-bit address range for the fabrics and generally uses standard OCP signals for the majority of inputs, as illustrated above. However, it should be noted that T1_4_active and T1_5_active are not OCP-standardized signals, but are signals generally available in typical fabric implementations. They indicate when the respective target agent (within Fabric 1) is busy fulfilling a previous request (i.e., still waiting for a previous command to complete). In cases where the _active signal is unavailable, but the OCP signal SThreadBusy (and / or SDataThreadBusy) is available, this busy signal can be used instead. Alternatively, the OCP signal SCmdAccept (inverted) could be used, if available and if the sleep / idle state for the signal is high.The logic shown in logic block 400 may typically be implemented using standard embedded logic design techniques, such as the use of ASIC or FPGA logic design tools, programmed logic tools, etc.

[0055] As discussed above, the fabric architectures discussed here may be representative of all or part of the fabrics used in an SoC. An example of the latter is shown in Fig.5, which shows an SoC architecture 500 with a north complex 502 and a south complex 100d. The north complex may typically include processor cores coupled to cache and memory components via a coherent fabric. Accordingly, north complex 502 is depicted as including a central processing unit (CPU) 505 containing a plurality of processor cores 506, each of which is coupled to a coherent fabric 508, such as a QPI fabric. Also shown is a memory block 510 coupled to the coherent fabric 508—this memory block is generally intended to represent numerous memory-related components that may be present in the architecture, such as caches, caching agents, memory controllers, etc.In general, an SoC architecture will include one or more levels of on-chip caches and may have some on-chip mass storage-level memory or one or more interfaces to off-chip memory; memory block 510 is intended to be representative of all of these configurations.

[0056] The North complex also includes another fabric, such as an INTEL On-Chip Scalable Fabric (IOSF) or OCP fabric 512, which is operatively coupled to the coherent fabric 508 via a fabric-to-fabric bridge 514. A plurality of IP blocks 516 are communicatively coupled to the fabric 512. In addition to the Fig. 5, the North complex 502 may include additional components and fabrics as will be appreciated by those skilled in the art.

[0057] South complex 100d generally illustrates SoC architecture 100 and 100c, as discussed above. Compared to SoC architecture 100c, south complex 100d contains only single unidirectional link pairs between Fabric 2 and Fabric 3 and between Fabric 4 and Fabric 5. As discussed above, this is merely exemplary of various link configurations in a hierarchical fabric that may be implemented according to the teachings herein.

[0058] Also shown in architecture 500 is an IOSF / OCP-to-OCP bridge 518. This serves to generally illustrate a bridge that could be implemented between fabric 512 of the North complex and fabric 1 of the South complex, which in this configuration would comprise an OCP fabric (where fabric 512 comprises either an IOSF or OCP fabric). In cases where the fabric protocols are different, a fabric protocol bridge will be implemented. If both fabric 512 and fabric 1 are OCP fabrics, then either an OCP fabric-to-fabric bridge or an OCP fabric-to-fabric link may be used, depending on applicable design parameters. For example, if the clock rate of Fabric 512 is significantly different from the clock rate of Fabric 1, then the bridge would support a clock domain crossing function while still implementing OCP signals on both interfaces to the bridge.

[0059] Embodiments of the present invention discussed above can generally be implemented in an integrated circuit having a semiconductor chip using known design and manufacturing techniques. In one embodiment, fabric creation tools from Sonics, Inc., can be implemented to facilitate design implementations. Although illustrated as being implemented on an SoC, the use of redundant fabric-to-fabric links with optional dynamic routing can also be implemented on other types of components, including I / O chips, peripheral chips, controllers, and other types of integrated circuits.

[0060] Additionally, embodiments of the present description may be implemented not only in a semiconductor chip, but also in a machine-readable medium. For example, the designs described above may be stored on and / or embedded in machine-readable media connected to a design tool used to design semiconductor devices. Examples include a netlist formatted in the VHSIC Hardware Description Language (VHDL), Verilog, or SPICE language. Some netlist examples include: a behavior-level netlist, a Register Transfer Level (RTL) netlist, a gate-level netlist, and a transistor-level netlist. Machine-readable media also includes media with layout information, such as a GDS-II file.Furthermore, netlist files or other machine-readable semiconductor chip design media may be used in a simulation environment to perform the methods of the teachings described above.

[0061] The above description of illustrated embodiments of the invention, including what is described in the abstract, is not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. While specific embodiments of the invention and examples thereof are described herein for illustrative purposes, numerous equivalent modifications are possible within the scope of the invention, as will be appreciated by those skilled in the relevant art.

[0062] These modifications may be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the description and drawings. Rather, the scope of the invention is to be determined entirely by the appended claims, which are to be interpreted in accordance with established teachings of claim interpretation.

Claims

[1] Integrated circuit comprising: a first fabric (102) to which a plurality of initiators (106) are communicatively coupled; a second fabric with which a multitude of targets are communicatively coupled; first and second fabric-to-fabric links (300, 302), each supporting communication between the first (102) and second fabrics; and a routing circuit and associated logic operatively coupled to the first and second fabric-to-fabric links (300, 302), wherein the routing circuitry and logic are configured to dynamically route data originating from at least one initiator (106) and destined for a destination (108) by selectively using one of the first or second fabric-to-fabric links (300, 302) to route the data from the at least one initiator (106) to the destination (108); wherein routing data comprises determining whether an address used by the initiator (106) for the target (108) is part of a first or second address range of the target (108) in an address map of the first fabric (102), the first address range being associated with the first fabric-to-fabric link (300, 302) and the second address range being associated with the second fabric-to-fabric link, and routing the data over the fabric-to-fabric link (300, 302) associated with the determined address range. [2] The integrated circuit of claim 1, wherein the routing circuit is configured to route data between initiators (106) communicatively coupled to the first fabric (102) to targets (108) communicatively coupled to the second fabric (102) using the first fabric-to-fabric link (300) by default, and further wherein the routing circuit is configured to detect use of the first fabric-to-fabric link (300), and upon detection of use of the first fabric-to-fabric link (300) concurrently with an initiator (106) initiating a data transfer to a target (108) communicatively coupled to the second fabric (102), the data transfer between the first and second fabrics (102) is routed using the second fabric-to-fabric link (302). becomes. [3] The integrated circuit of claim 1, wherein the routing circuit is operatively coupled to at least one initiator fabric interface to which a corresponding initiator (106) is communicatively coupled, and wherein dynamic routing using the first or second fabric-to-fabric links (300,302) is effected by selecting an address that causes the data to be routed over one of the first and second fabric-to-fabric links (300, 302). [4] The integrated circuit of claim 3, wherein the address corresponds to an alias address of a target agent interface of the first fabric (102). [5] The integrated circuit of claim 1, wherein each of the first and second fabrics (102) comprises a plurality of unidirectional point-to-point interconnects, and wherein each of the first and second fabric-to-fabric links (300, 302) comprises unidirectional point-to-point interconnects. [6] The integrated circuit of claim 5, wherein the first and second fabrics (102) comprise Open Core Protocol (OCP) fabrics and each of the first and second fabric-to-fabric links comprises OCP links. [7] The integrated circuit of claim 1, further comprising: a third fabric (102) with which at least one target (108) is communicatively coupled; and a third fabric-to-fabric link that supports communication between the second and third fabrics (102), wherein the routing circuit is configured to selectively route data originating from an initiator (106) communicatively coupled to the first fabric (102) to a target (108) communicatively coupled to the third fabric (102) over one of the first and second fabric-to-fabric links (300, 302). [8] Method comprising: Initiating a data transfer at an initiator (106) communicatively coupled to a first fabric (102), the data transfer being intended for a target (108) communicatively coupled to a second fabric (102), the first and second fabrics (102) being communicatively coupled via first and second fabric-to-fabric links (300, 302); and selectively routing data corresponding to the data transfer from the initiator (106) to the destination (108) via one of the first and second fabric-to-fabric links (300, 302); wherein routing data comprises determining whether an address used by the initiator (106) for the target (108) is part of a first or second address range of the target (108) in an address map of the first fabric (102), the first address range being associated with the first fabric-to-fabric link (300, 302) and the second address range being associated with the second fabric-to-fabric link (300, 302), and routing the data over the fabric-to-fabric link (300, 302) associated with the determined address range. [9] The method of claim 8, wherein the fabrics (102) comprise Open Core Protocol (OCP) fabrics (512) and the fabric-to-fabric links (300, 302) comprise OCP links. [10] The method of claim 8, wherein the fabrics (102) use unidirectional point-to-point links and each of the fabric-to-fabric links (300, 302) comprises a unidirectional point-to-point link. [11] The method of claim 8, wherein the selective routing operations used to transfer data from the initiator (106) to the target (108) are transparent to the initiator (106). [12] The method of claim 8, further comprising implementing the selective routing using embedded logic having a plurality of logic gates. [13] The method of claim 8, further comprising: Detecting whether the first fabric-to-fabric link (300) is busy; and if the first fabric-to-fabric link (300) is busy, routing the data transfer via the second fabric-to-fabric link (302), otherwise routing the data transfer via the first fabric-to-fabric link (300). [14] System-on-a-Chip (SoC) (500), comprising: a plurality of Open Core Protocol (OCP) fabrics (512) having interfaces compliant with the Open Core Protocol, including at least a first and second OCP fabric (512); a plurality of intellectual property (IP) blocks, each communicatively coupled to one of the plurality of OCP fabrics (512), at least a portion of the IP blocks including an interface for communicating with other IP blocks via the OCP fabric (512) to which it is communicatively coupled, the interface including, for each of these IP blocks, an initiator interface or a target interface or both an initiator and a target interface; first and second fabric-to-fabric OCP links, each supporting communication between the first and second OCP fabrics (512); Routing circuitry and associated logic operatively coupled to the first and second fabric-to-fabric OCP links, wherein the routing circuitry and logic are configured to route data originating from an initiator of a first IP block communicatively coupled to the first OCP fabric (512) and destined for a destination interface of a second IP block communicatively coupled to the second OCP fabric (512) by selectively using one of the first and second fabric-to-fabric OCP links to route the data from the first IP block to the second IP block; wherein routing data comprises determining whether an address used by the initiator (106) for the target (108) is part of a first or second address range of the target (108) in an address map of the first fabric (102), the first address range being associated with the first fabric-to-fabric link (300, 302) and the second address range being associated with the second fabric-to-fabric link (300, 302), and routing the data over the fabric-to-fabric link (300, 302) associated with the determined address range. [15] The SoC (500) of claim 14, wherein the plurality of fabrics are interconnected using a plurality of links to form a hierarchical fabric structure. [16] The SoC (500) of claim 14, wherein each of the first and second OCP fabrics comprises a plurality of unidirectional point-to-point interconnects, and wherein each of the first and second fabric-to-fabric OCP links comprises unidirectional point-to-point interconnects. [17] The SoC (500) of claim 14, wherein the SoC (500) includes a North complex (502) and a South complex (100d), and the plurality of OCP fabrics (512) are implemented in the South complex (100d). [18] The SoC (500) of claim 14, wherein the routing circuitry is configured to route data between IP blocks including an initiator interface communicatively coupled to the first fabric (102) and IP blocks including a target interface communicatively coupled to the second fabric using the first fabric-to-fabric OCP link by default, and further wherein the routing circuitry is configured to route the data using the second fabric-to-fabric OCP link when the first fabric-to-fabric OCP link is determined to be busy. [19] The SoC (500) of claim 14, wherein the routing circuit is operatively coupled to at least one initiator fabric interface to which a corresponding initiator interface of an IP block is communicatively coupled, and wherein dynamic routing using the first or second fabric-to-fabric OCP link is effected by selecting an address that causes the data to be routed over one of the first and second fabric-to-fabric OCP links.

Images