Bridge circuitry

Abstract

Bridge circuitry 12 configured to bridge between a memory system interconnect 8 and an external port controller 14; the bridge circuitry comprising: an internal communication link interface configure

Description

BACKGROUND Technical Field The present technique relates to the field of integrated circuits. Technical Background An integrated circuit may communicate with one or more external link partners (e.g. input / output devices, other chiplets in a distributed compute system, etc.) via an external communication link. An external link protocol may define the communication format for data exchanged via the external communication link, so that an integrated circuit and a link partner provided by different vendors can be inter-operable with each other. SUMMARY At least some examples of the present technique provide an apparatus comprising: an external port controller configured to control communication, via an external communication link for communicating with a link partner, of external link protocol packets defined according to an external link protocol; and bridge circuitry configured to map between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by a memory system interconnect; in which: the bridge circuitry and the external port controller are coupled via an internal communication link and are configured to use an internal link protocol to transport the external link protocol packets between the bridge circuitry and the external port controller; and the bridge circuitry comprises transaction ordering circuitry configured to enforce external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link. At least some examples of the present technique provide bridge circuitry configured to bridge between a memory system interconnect and an external port controller; the bridge circuitry comprising: an internal communication link interface configured to communicate with the external port controller via an internal communication link, wherein the internal communication link interface is configured to use an internal link protocol to transport external link protocol packets between the bridge circuitry and the external port controller, the external link protocol packets defined according to an external link protocol for an external communication link between the external port controller and a link partner; protocol mapping circuitry configured to map between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by the memory system interconnect; and transaction ordering circuitry configured to enforce external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link. At least some examples of the present technique provide a system comprising: the apparatus or the bridge circuitry described above, implemented in at least one packaged chip; at least one system component; and a board, wherein the at least one packaged chip and the at least one system component are assembled on the board. At least some examples of the present technique provide a chip-containing product comprising the system described above, wherein the system is assembled on a further board with at least one other product component. At least some examples of the present technique provide computer-readable code for fabrication of the apparatus or the bridge circuitry described above. At least some examples of the present technique provide a method for bridge circuitry configured to bridge between a memory system interconnect and an external port controller configured to control communication, via an external communication link for communicating with a link partner, of external link protocol packets defined according to an external link protocol; the method comprising: the bridge circuitry communicating with the external port controller via an internal communication link according to an internal link protocol used to transport the external link protocol packets between the bridge circuitry and the external port controller; the bridge circuitry mapping between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by the memory system interconnect; and using transaction ordering circuitry comprised by the bridge circuitry, enforcing external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link. Further aspects, features and advantages of the present technique will be apparent from the following description of examples, which is to be read in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates an example of a data processing system; Figure 2 schematically illustrates use of various communication protocols within the data processing system; Figure 3 illustrates an example of a layered protocol (in particular, PCIe) for communication on an external communication link; Figure 4 illustrates an example packet format for the PCIe standard; Figure 5 illustrates a comparative approach in which an external port controller is responsible for managing external link protocol ordering rules, interrupt handling and requests for address translation; Figure 6 illustrates a further comparative approach in which translation and interrupt servicing is performed in line on a path to a memory system interconnect bridge and responsibility for enforcing external link protocol ordering rules remains with the external port controller; Figure 7 illustrates an example in which the bridge circuitry comprises transaction ordering circuitry; Figure 8 illustrates an example of use of an internal link protocol for conveying transaction layer packets of an external link protocol between the bridge circuitry and the external port controller; Figure 9 illustrates various communication layers in use within the system; Figure 10 illustrates an example division of responsibility between the bridge circuitry 12 and the external port controller; Figure 11 illustrates an example of the bridge circuitry; Figure 12 illustrates a method; and Figure 13 illustrates a system and a chip-containing product. DESCRIPTION OF EXAMPLES An external port controller is provided in an apparatus (e.g. an integrated circuit, chiplet or system-on-chip), to manage communications with a link partner via an external communication link according to an external link protocol. The apparatus may also have a memory system interconnect used to connect internal units of the apparatus, such as compute units and memory system storage. Hence, bridge circuitry may be provided to map between external link protocol packets defined according to the external link protocol used for the external communication link and memory system interconnect transactions defined according to a memory system interconnect protocol used by a memory system interconnect. The bridge circuitry and the external port controller are coupled via an internal communication link and use an internal link protocol to transport the external link protocol packets between the bridge circuitry and the external port controller. The external link protocol may impose certain transaction ordering rules which restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link. For example, the external link protocol may define various transaction classes (e.g. non-posted requests requiring a completion response, posted requests not requiring a completion response, and completion responses), and may impose class-based ordering rules which define, depending on which class of transactions a given earlier transaction and a given later transaction belong to, whether the given later transaction is allowed to bypass the given earlier transaction. The external link protocol may also allow external link protocol packets to specify attributes enabling more relaxed ordering rules to be applied. In the absence of an attribute allowing more relaxed ordering, the default ordering rules may in some cases be stricter than ordering requirements imposed by the memory system interconnect protocol, so some additional ordering enforcement may be applied that would not be applied if the only ordering requirements were those enforced by the memory system interconnect protocol. In a typical apparatus, the responsibility for enforcing the external link protocol transaction ordering rules would lie with the external port controller. In contrast, the inventors have proposed an approach where the bridge circuitry comprises transaction ordering circuitry configured to enforce external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link. This is counter-intuitive since one would think that the external port controller acting as the gateway to the apparatus from the external communication link would be the place to enforce the requirements imposed by the external link protocol. However, the inventors have recognised that the external port controller circuitry may be designed by an independent entity to other portions of the apparatus, with different parties supplying circuit designs for different portions of the apparatus, and hence the external port controller often does not have detailed knowledge of the memory system layout used in other parts of the apparatus. This means that if the responsibility for enforcement of the external link protocol transaction ordering rules lies with the external port controller, then in the absence of any knowledge of memory-system-specific layout information which could assist with identifying opportunities for more relaxed ordering requirements, the external port controller may have to assume conservatively that the worst case scenario applies whereby respective transactions could risk targeting the same region of memory so that they have to be strongly ordered relative to each other. This tends to reduce processing performance in the system as a whole because the conservative ordering model enforced by the external port controller would tend to increase the likelihood that transaction flows corresponding to respective external link protocol transactions have to be serialized when mapped to corresponding memory system transactions in the memory system interconnect protocol. In contrast, the bridge circuitry which bridges to the memory system interconnect protocol is typically developed by the same entity that handles the corresponding memory system interconnect and so can more feasibly consider features available in the memory system interconnect protocol and / or additional information on specific properties of the particular memory system within the apparatus, so that opportunities for more relaxed ordering approaches can be feasible, increasing the likelihood that some transaction flows corresponding to ordered transactions required to be ordered in the external link protocol can nevertheless be at least partially parallelized when mapped to corresponding transactions on the memory system interconnect. Therefore, by assigning responsibility for enforcing the external link protocol transaction ordering rules to the bridge circuitry, instead of the external port controller, less conservative ordering models become feasible, which can help to boost throughput of memory system transactions in the memory system interconnect, and hence processing performance. The transaction ordering circuitry may control the ordering between the respective data access transactions in dependence on memory map information indicative of properties of a memory system accessible via the memory system interconnect. The memory map information may for example indicate mapping between regions of memory address space and target nodes within a memory system, which can be useful for identifying cases when two requests may be processed at least partially in parallel even if required to be ordered relative to each other by the external link protocol transaction ordering rules. In some examples, the transaction ordering circuitry is configured to permit inbound ordered write transactions, which are received from the link partner to request write access to data in a memory system accessible via the memory system interconnect and are required by the external link protocol transaction ordering rules to be serialized, to be processed at least partially in parallel when the inbound ordered write transactions target different target nodes of the memory system. For example, the bridge circuitry may be able to invoke memory system interconnect protocol specific mechanisms enabling requests for the inbound ordered write transactions to be issued on the memory system interconnect so as to be in flight on the memory system interconnect in parallel, to enable time consuming processing steps such as awaiting snoop responses from caches which could hold a copy of the targeted data to be parallelized for the respective inbound ordered write transactions. The bridge circuitry may serialize commitment of the inbound ordered write transactions, e.g. by deferring a completion response to a given inbound ordered write transaction until completion responses for one or more earlier inbound ordered write transactions have been issued. This approach would typically not be feasible if the external link protocol transaction ordering rules were handled by the external port controller, because the external port controller would often be developed in isolation of any detailed knowledge of features supported by the memory system interconnect protocol to support more efficient ordered write transaction handling, and because the external port controller would not typically have knowledge of the memory map information enabling determination of whether ordered write transactions are targeting different target nodes of the memory system. The transaction ordering circuitry may not be the only functionality that, while traditionally carried out by the external port controller, could be reassigned to be carried out by the bridge circuitry. For example, the bridge circuitry may comprises address translation interface circuitry configured to request address translation of a target address specified by an inbound transaction received from the link partner. The requested address translation could be an on-demand address translation requested at a time that a memory system transaction is required for the translated address, or could be an “address translation service” request which is requested in advance of actually needing read / write access to a memory system location corresponding to the translated address (the address translation service request may cause a translated address to be returned to the requester link partner which can cache the translated address locally so that when it actually needs to access that memory address, the translated address is already available and so the latency of the address translation process can be reduced at the time the memory access is needed). The address translation interface circuitry may, if included in the external port controller, require the circuitry of the external port controller to be developed with awareness of the specific protocols used to interface with address translation circuitry of a host system, which may make it harder for an external port controller vendor to keep up with changes in host system protocols. By moving the address translation interface, by which translations are requested for transactions originating from a link partner, to bridge circuitry lying closer to the memory system interconnect fabric of the host apparatus, it becomes more feasible to use more complex address translation interface protocols specific to an address translation scheme used in the host system, while maintaining acceptable throughput rates for handling of external protocol transactions. In some examples, the address translation interface circuitry of the bridge circuitry is configured to trigger a lookup of at least one translation lookaside buffer (TLB) configured to cache address translation information. Interfaces for looking up a translation lookaside buffer may require more specific protocols that are specific to the particular translation scheme used in a host system, so including such an interface in the external port controller will increase design complexity for an external port controller vendor who may have to develop additional host-specific versions depending on the particular host system for which a given external port controller design is developed. This problem can be reduced by moving responsibility for interfacing with TLBs to the bridge circuitry. In one particular example, for instance, the address translation interface circuitry of the bridge circuitry may communicate with address translation circuitry (such as a system memory management unit and / or one or more TLBs) according to a local translation interface protocol or a distributed translation interface protocol. A local translation interface protocol comprises a protocol by which a component can request translations from address translation circuitry, with any requests to TLBs being managed by the address translation circuitry rather than the component requesting the translations. In contrast, a distributed translation interface protocol comprises a protocol by which the requesting component can directly request translations from TLBs, which can be helpful for implementing distributed TLBs so that a number of distributed components each have associated TLBs, with a central address translation component separate from the requesting component managing page table walks for filling those TLBs when there is a TLB miss. An example of the local translation interface protocol is the AM BA® LTI protocol and an example of the distributed translation interface protocol is the AMBA® DTI protocol. By removing the need for the external port controller to include support for such translation interface protocols, the design complexity of the external port controller can be greatly reduced. Another function that could be migrated from the external port controller to the bridge circuitry could be handling of message signalled interrupts. A message signalled interrupt (MSI) may be a memory transaction that specifies as its target address an address mapped for representing interrupt requests. This allows the datapath on the external communication link to be shared between read / write memory accesses and interrupt signalling, to avoid needing dedicated interrupt signal paths to be implemented in hardware on the external communication link. The MSI enables an endpoint or other link partner to cause an interrupt to be directed to compute circuitry within the host system, but the identification of which transactions relate to MSIs, and the corresponding interface to cause an interrupt controller to generate an interrupt signal when a MSI is detected, may introduce additional complexity to the external port controller design. For example, the interrupt controller may use a further communication protocol not otherwise required for the external port controller. Hence, in some examples, the bridge circuitry comprises message signalled interrupt handling circuitry configured to detect message signalled interrupts (MSIs) represented by external link protocol packets received from the link partner, and to signal the message signalled interrupts to an interrupt controller. By moving responsibility for signalling MSIs to the interrupt controller from the external port controller to the bridge circuitry which is responsible for mapping to a memory system interconnect protocol, the additional interface for interacting with the interrupt controller (which may use a protocol that is specific to a particular host system) may be eliminated from the external port controller, making it more feasible to develop a host-agnostic design for the external port controller that is less dependent on features of a particular host system. This can reduce design and developments costs (e.g. less redesign work to keep up with new iterations of communication protocols released over time), and make it more feasible for the external port controller to be developed as part of a separate chiplet which is usable with a variety of different host implementations. In one particular example, for instance, the message signalled interrupt handling circuitry of the bridge circuitry may communicate with the interrupt controller according to a streaming protocol, such as the AMBA® 4 AXI4-Stream protocol. The streaming protocol may be a unidirectional protocol designed for high-speed streaming of data, with less need for addressing or complex handshaking than other protocols. The streaming protocol used for the communication with the interrupt controller may be a different protocol to the internal link protocol used on the internal communication link. Again, this streaming protocol would typically not otherwise be needed for the external port controller. By moving the responsibility for MSI processing to the bridge circuitry, the designer of the external port controller does not need to spend development resource supporting the additional protocol (in contrast, the designer of the bridge circuitry may often be the same party that provides the interrupt controller and so may already be aware of requirements for this protocol). In some examples, the bridge circuitry is coupled, via the internal communication link, to a plurality of external port controllers configured to manage bifurcated streams of external link protocol packets. For example, the bifurcated streams may comprises streams of external link protocol packets to be routed over respective subsets of lanes within a physical interface slot for the external communication link. Bifurcation may be a technique supported by certain external link protocols to enable a single external connector slot to be partitioned to be shared by multiple link partners. Each bifurcated stream may typically have a respective external port controller. The bridge circuitry may be implemented at a point in processing flow where the bifurcated streams have converged, so the bridge circuitry may be shared between multiple external port controllers corresponding to the bifurcated streams. This provides an another reason why it can be useful to move functionality such as enforcement of external link protocol transaction ordering rules, the address translation interface circuitry, the MSI handling circuitry and / or other shared transaction handling circuitry from the external port controller up to the bridge circuitry, as this helps reduce duplication of the same functionality in multiple external port controllers, enabling a more efficient circuit design from a circuit area and power perspective. In some examples, the internal link protocol provides a format for transporting one or more external link protocol packets as payload data associated with control information defined according to the internal link protocol. The internal link protocol may view the external link protocol packets simply as data to be conveyed, without any specific features of the internal link protocol reflecting corresponding features of the external link protocol. Hence, the internal link protocol packets may be seen as “empty vessels” which merely convey the external link protocol packets blindly without any attempt to understand their meaning or remap them to corresponding features of the internal link protocol. For example, the external link protocol packets received at the external port controller may be transmitted to the bridge circuitry unchanged on a data channel of the internal communication link (with accompanying control information, defined according to the internal link protocol, being sent on a control channel of the internal communication link, e.g. to indicate internal link packet start / end points). The internal link protocol may be a read / write-agnostic protocol, which does not distinguish read transactions for providing read access to memory from write transactions for providing write access to memory. In contrast, the memory system interconnect protocol and the external link protocol may be read / write-aware protocols providing separate transaction types for representing read and write transactions respectively. The bridge circuitry may include conversion circuitry to map between the read / write transactions of the external link protocol and the memory system interconnect protocol. On the other hand, control information defined according to the internal link protocol may merely encapsulate, or accompany, the external link protocol transactions on the internal communication link, without providing any read / write-aware mapping of the information carried by those external link protocol transactions. By using an internal link protocol which is agnostic to read / write-specific features, a simpler interface can be provided at both the bridge circuitry and the external port controller for interacting with the internal communication link. In some examples, the internal link protocol supports transmission of a plurality of external link protocol packets in a single flit defined according to the internal link protocol. This can be helpful for improving bandwidth on the internal communication link, which may be important for keeping up with increasing transfer rate demands imposed by the latest versions of the external link protocol. The term “flit” is short for “flow digit” and refers to the smallest indivisible unit of data for which independent control of routing is offered by the internal communication link (hence, while one flit may be routed along a communications path selected separately from the path used for another flit, it is not possible to route respective subsets of bits within a flit via independently selected paths). In some examples, the internal communication link supports transfer of at least 2048 bits of data per flit (the internal communication link can also support narrower widths less than 2048 bits as well, such as 512 bits or 1024 bits). However, it can be useful to support 2048 bits as this may be a communication rate which is higher than supported by many typical transfer interface protocols. In one particular example, the internal link protocol comprises CXS (the AMBA® CXS, or Credited extensible Stream, streaming interface protocol provided by Arm® Limited). CXS is a protocol-agnostic transport interface that enables multiple external link protocol packets to be transferred per internal link protocol flit over shared wires (e.g. shared between read and write transactions), so can be particularly suited to enabling a reduction in the hardware cost of implementing internal link wiring, while still supporting the high transfer bandwidths required by the latest versions of external link protocols. However, it will be appreciated that other internal link protocols could also be used. For example, another internal link protocol that could be used may be the Streaming Fabric Interface (SFI) provided by Intel. The external link protocol may comprise an input / output (I / O) interface protocol. For example, the external link protocol may be an expansion bus interface which enables connection between a given chip within a host compute system and link partners such as peripheral (I / O) devices or other chiplets of a distributed multi-chip compute system. For example, the external link protocol may be a PCIe-based protocol, which is derived from the PCIe (Peripheral Component Interconnect Express) standard. For example, the PCIe-based protocol may be PCIe itself, or other related protocols such as CXL (Compute Express Link) which is derived from PCIe. The external link protocol may comprise a layered protocol, which is based on multiple layers of packet encoding schemes, with one layer encapsulating (e.g. with additional packet headers / footers) a packet defined according to a preceding layer of the protocol. Examples of layered protocols include the PCIe-based protocols mentioned above as well as other protocols such as the AMBA® CHI Chip-to-Chip (C2C) protocol provided by Arm® Limited, which is used for chip-to-chip communication in a multi-chip compute system. In one particular example, the external link protocol packets may comprise PCIe transaction layer packets. The PCIe specification may also define a data link layer and physical layer, but any framing information for transaction layer packets encoded according to the data link layer or physical layer may be removed prior to the PCIe transaction layer packets being routed over the internal communication link to the bridge circuitry. Hence, the external link protocol packets may comprise the transaction layer packets as defined by PCIe. It is not necessary for the bridge circuitry to consider encoding / decoding of other PCIe layers such as the data link layer and physical layer. The external port controller may comprise data link layer encoding / decoding circuitry configured to encode / decode PCIe data link layer information for transporting on the external communication link. For example, the PCIe data link layer information could include data link layer packets (DLLPs) and / or data link layer framing information encoded into framing bits around a transaction layer packet (TLP). Hence, the external port controller may be the entity that is responsible for encoding and decoding according to the data link layer defined in the PCIe standard. The external port controller does not need to be responsible for encoding or decoding according to the transaction layer of PCIe (since it may be the bridge circuitry and the link partner that are respectively responsible for encoding and decoding transaction layer packets). Also, the external port controller does not need to be responsible for encoding or decoding a physical layer of the PCIe specification, as this may be done by a separate PHY controller. The memory system interconnect protocol could comprise any memory system fabric protocol that can be used to connect between memory system requesters (e.g. compute units, or root ports interfacing with external communication links) which request read / write access to a memory system and memory system completers (e.g. memory storage units, or root ports interfacing with external communication links) which service the read / write requests issued by the memory system requesters. For example, the memory system interconnect protocol could be a coherent interconnect protocol, which has features for guaranteeing cache coherency of data from the memory system cached in respective private caches associated with memory system requesters. For example, the coherent interconnect protocol could be the AMBA® CHI protocol provided by Arm® Limited. Alternatively, the memory system interconnect protocol may comprise a non-coherent interconnect protocol, such as AMBA® AXI protocol provided by Arm® Limited. In some examples, the apparatus may additionally comprise the memory system interconnect; compute circuitry coupled to the memory system interconnect; and memory storage circuitry coupled to the memory system interconnect. However, in other examples, these further components may be licensed for manufacture separate from the bridge circuitry and external port controller, so may be considered external to the apparatus comprising the bridge circuitry and the external port controller. Also, in some examples, the bridge circuitry may be a separately licensed circuit component, separate from the external port controller which may be provided by a different circuit design vendor from the bridge circuitry. Hence, the bridge circuitry itself may in some cases be viewed as a standalone component which may be marketed independently for inclusion in a larger system designed by a downstream party in a chain of design / manufacture. The bridge circuitry may comprise an internal communication link interface configured to communicate with the external port controller via an internal communication link. The internal communication link interface is configured to use an internal link protocol to transport external link protocol packets between the bridge circuitry and the external port controller. The external link protocol packets are defined according to an external link protocol for an external communication link between the external port controller and a link partner. The bridge circuitry also comprises protocol mapping circuitry configured to map between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by the memory system interconnect; and transaction ordering circuitry configured to enforce external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link. For similar reasons to the apparatus above, such bridge circuitry can help to reduce the amount of conservatism required for ordering transactions according to external link protocol ordering rules, compared to an approach which applies such order enforcement at the external port controller. Specific examples are now set out with reference to the drawings. Figure 1 illustrates an example of a data processing system comprising one or more integrated circuits 2, including at least one integrated circuit 2 acting as an example of the apparatus described above. While Figure 1 for sake example shows a system with two interconnected integrated circuits (chiplets) 2 connected by a chip-to-chip link 22, other examples of the apparatus 2 may be a system-on-chip implemented on a single integrated circuit 2. Also, while in this particular example, both integrated circuits 2 comprise bridge circuitry and an external port controller as described earlier, it is not essential for every integrated circuit 2 in the system to comprise this circuitry, and some examples could include at least one integrated circuit 2 which does not have such bridge circuitry or external port controller at all, or which has bridge circuitry or an external port controller that operates in a different manner to that described above. A given integrated circuit (apparatus) 2 comprises a number of compute circuit units 4, 6, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs). While Figure 1 shows an example having two CPUs 4 and one GPU 6 per apparatus 2, other numbers and types of compute units may be provided, and it is not essential for each integrated circuit 2 in a multi-chip compute system to have the same number of compute units. The compute circuit units 4, 6 share access to a memory system comprising memory storage circuitry 10, which is accessible via a memory system interconnect 8 which may implement a coherent memory system interconnect protocol, such as AMBA® CHI, or a noncoherent memory system interconnect protocol, such as AMBA® AXI. If the memory system interconnect 8 implements a coherent memory system interconnect protocol, then the memory system interconnect may have at least one instance of home node circuitry 9 for implementing coherency enforcement rules to maintain coherency for data cached in the private caches of the compute circuit units 4, 6. For example, the home node circuitry 9 may be responsible for generating, in response to a read / write access to a given address initiated by one requester, snoop requests for snooping nodes which could hold cached data for that address. Any known home node I coherency protocol technique may be used to maintain cache coherency in the system. The apparatus 2 may have at least one root port 14 acting as an externally facing interface for communication with one or more link partners (generically labelled 18 in subsequent drawings), such as endpoint devices 19 or switches 20. For a given root port 14, the corresponding link partner 19, 20 is located off-chip on a separate integrated circuit from the integrated circuit 2 comprising that root port 14. Communications with the link partner are via an external communication link 16 based on an external link protocol, which may be an I / O protocol such as PCIe, CXL, AMBA® AXI C2C, etc. The link partner may be any externally located device separate from the integrated circuit 2. For example, examples of link partners may include endpoint devices 19 such as peripherals such as user interface controllers, network interface controllers, controllers for interacting with external memory storage devices, etc. A link partner could also be another system-on-chip similar to the apparatus 2 itself, within a distributed compute system comprising multiple such apparatuses 2 (similar to the relationship between the chiplets 2 connected via the chip-to-chip link 22 as shown in Figure 1). Some root ports 14 may be coupled via the external communication link 16 to multiple endpoints 19 accessible via a switch 20 (the switch and endpoints not being part of the apparatus 2 itself). Hence, in some cases, the switch 20 acts as the link partner of the root port 14, since the root port 14 has no knowledge of the details of the communication partner device at the other end of the communications link 16. While Figure 1 shows an example where the root port circuitry 14 is on the same integrated circuit 2 as other parts of the host apparatus 2 for which it acts as an interface to the external endpoint 18, it is also possible that the root port circuitry 14 could be implemented on a separate chiplet from other parts of its associated host apparatus 2, with a chip-to-chip link 22 between the root port 14 and rest of the host apparatus 2. The external link protocol used on the external communication link 16 may define read / write transactions in a different manner to the protocol used by the memory system interconnect 8 that links compute circuitry 4, 6 to memory storage 10 within the apparatus 2. Therefore, bridge circuitry 12 may be provided between a root port 14 and the memory system interconnect 8, to map between read / write memory access transactions defined in external link protocol packets on the external communication link 16 and memory system interconnect transactions according to the protocol used on the memory system interconnect 8. It will be appreciated that Figure 1 shows just one example arrangement for a processing system, giving an example context in which bridge circuitry 12 may be provided. However, other examples may implement a different configuration of the bridge circuitry 12 relative to other units (e.g. with additional intermediate units between the root port 14 and bridge circuitry 12). Figure 2 illustrates an example of different protocols involved in respective communication links in use within the system of integrated circuits 2 shown in Figure 1. An external port controller 15 is provided within the root port 14 for controlling the external port at the boundary of apparatus 2 that interfaces with the external communications link 16. The external port controller 15 communicates with a link partner 18 (e.g. an endpoint 19 or switch 20) according to an external link protocol, e.g. PCIe or another I / O protocol. As shown in Figure 3, the PCIe protocol is a layered protocol which includes a transaction layer, a data link layer and a physical layer. The transaction layer defines a transaction layer packet format which distinguishes various classes of transactions, including posted transactions (write requests which do not require a completion response), non-posted transactions (read requests or write requests which do require a completion response) and completion transactions (completions sent in response to non-posted read or write requests). The transaction layer packet format has an encoding which differentiates read and write transactions. As shown in Figure 4, a transaction layer packet may comprise a packet header defining parameters of the transaction, such as transaction type (posted / non-posted / completion, read / write, etc.), a target memory address of the transaction, data payload length, and other attributes (e.g. a relaxed ordering attribute specifying whether a more relaxed ordering model than the stronger default ordering rules is appropriate for this transaction). Optionally, the transaction layer packet includes payload data (e.g. read data for a read transaction response or write data for a write transaction request). Payload data may not be needed for some transactions such as read requests or write completion responses. The transaction layer packet encoding can also optionally include an error correcting code (e.g. end to end cyclic redundancy check, ECRC, code) to protect against transmission errors affecting the transaction layer encoding. The data link layer is responsible for link management and data integrity, including error detection and error correction, so adds a link layer cyclic redundancy check code (LCRC), in addition to the ECRC if an ECRC is provided by the transaction layer. As shown in Figure 4, the data link encapsulates the transaction layer packet using framing information specifying a sequence number and the LCRC. The data link layer may also provide data link layer packets DLLPs) which are separate from the transaction layer packets and are communicated over the external communication link. The Physical Layer specifies circuitry required for physical interface operation, including driver and input buffers, parallel-to-serial and serial-to-parallel conversion, phase locked loops (PLLs), and impedance matching circuitry. The physical layer circuitry adds framing symbols to the transmitted packets, which enable a receiver to detect the start and end of packets. Hence, there may be a number of layers of encoding / decoding applied at the interfaces to the external communication link. Referring again to Figure 2, responsibility for encoding / decoding the transaction layer packet data may lie with the bridge circuitry 12 and a link partner 18 respectively (although the external port controller 15 could optionally have some circuitry for checking that transaction layer packets are correctly formed). More particularly, within the bridge circuitry 12, protocol mapping circuitry 54 may be provided to map between the transaction layer packets of the external link protocol and the memory system interconnect transactions of the memory system interconnect protocol. Responsibility for encoding / decoding the data link layer packet data lies with the link partner 18 and data link layer encoding / decoding circuitry 40 implemented at the external port controller 15 within the root port 14. Responsibility for encoding / decoding the physical layer packet data lies with the link partner 18 and a PHY controller (not shown in Figure 2) that is implemented within the root port 14. Hence, for inbound transactions received from the link partner 18 requesting access to the host memory system 10 of the apparatus, the protocol mapping steps are as follows: • the link partner 18 (or an endpoint device that is in communication with the link partner 18) generates the transaction / data link / physical layer packet encodings according to the external link protocol, and the link partner 18 transmits the encoded packets across the external communication link 16; • a PHY controller (not shown in Figure 2) decodes the physical layer of the transmitted packets; • data link layer decoding circuitry 40 within the external port controller 15 decodes data link layer packets of the transmitted packets, and decodes / removes any data link layer framing information from transaction layer packets defined according to the external link protocol; • the external port controller 15 transmits the transaction layer packets to the bridge circuitry 12 as payload data transmitted within an internal link protocol used for an internal communication link 70 between the bridge 12 and the external port controller 15; • the bridge circuitry 12 decodes the transaction layer packets (defined according to the external link protocol but transported as payload data in at least one flit transmitted according to the internal link protocol), and uses its protocol mapping circuitry 54 to map any read / write requests or other transactions defined in those packets to corresponding transactions defined according to the memory system interconnect protocol, which are forwarded to the memory system interconnect 8 for servicing by the host’s memory system. On the other hand, for outbound transactions to be transmitted to a link partner 18, e.g. generated based on memory system interconnect transactions to addresses mapped to the root port 14, the protocol mapping steps are as follows: • in response to receipt of a transaction from the memory system interconnect 8 (defined according to the memory system interconnect protocol), the protocol mapping circuitry 54 at the bridge circuitry 12 generates corresponding transaction layer packets according to the external link protocol used on the external communication link 16 (e.g. with read / write transactions mapped to corresponding encodings of read / write transactions in the external link protocol). • the bridge circuitry 12 transmits the external link protocol transaction layer packets on the internal communication link 70, as payload data accompanied by control information defined according to the internal link protocol. • the external port controller 15 extracts the transaction layer packets from the payload data conveyed on the internal communication link, and uses its data link layer encoding / decoding circuitry 40 to append data link layer encoded framing information (e.g. the sequence number and LCRC as shown in Figure 4) to the transaction layer packets. The external port controller 15 may also be responsible for generation of any data link layer packets (DLLPs). • the PHY controller of the root port 14 encodes the physical layer of the external link protocol packets (as a wrapper around the framed transaction layer packet generated by data link layer encoding / decoding circuitry 40), and transmits the physical layer packets on the external communication link 16 to the link partner 18. Figure 5 illustrates, for comparative purposes only, an approach for implementing functionality for processing transaction layer packets of the external link protocol using the external port controller 15. Figure 5 shows an example which uses bifurcation of external link streams, where a single 16-lane physical external link port is controlled by a given 16-lane (X16) PHY controller 42, but its 16 lanes are capable of being sub-divided for use in communicating bifurcated streams of packets managed by respective external port controllers 15. For instance, the example of Figure 5 includes a 16-lane external port controller 15 to be used when all 16 lanes of the physical communication link are used in a non-bifurcated manner, and 8-lane, 4-lane and 4-lane controllers 15 which can be used in a bifurcated mode of operation to control communications on 8 lanes, 4 lanes and 4 lanes of the 16-lane physical interface respectively. Each external port controller 15 communicates, via a non-coherent interconnect 38, with the coherent bridge circuitry 12 which handles conversion to the coherent interconnect protocol used by the memory system interconnect 8 of the host apparatus 2. In this example, the noncoherent interconnect 38 operates according to the AM BA® AXI non-coherent interconnect protocol, so the external port controller 15 includes AXI bridge circuitry 40 for mapping the read / write transactions of the external link protocol onto the read / write transactions of the AXI protocol. The AXI communication paths on the non-coherent interconnect 38 include separate channels for read and write transactions respectively, so the AXI bridge circuitry 40 of the external port controller 15 has to be aware of specific read / write protocols used in another protocol. The AXI bridge circuitry 40, external port controller 15 and PHY controller 42 may be provided within root port circuitry 14 provided by a different circuit design vendor to the non-coherent interconnect 38, coherent bridge circuitry 12 and other components shown as part of the host system 2 in Figure 5. Also, the external port controller 15 in this example has interfaces to an interrupt unit 30 for interrupting the host system based on message signalled interrupts (MSIs) detected in the external link protocol transactions received over the external communication link 16, and to one or more address translation units 32, 34, such as system memory management unit (input / output memory management unit) and / or translation lookaside buffer. As there are multiple external port controllers 15 for managing bifurcated streams of transactions, each of those controllers 15 has a respective AXI bridge 40 and interfaces to the interrupt unit 30 and translation units 32, so a relatively complex network of AXI interface connections is needed between each AXI bridge 40 and the non-coherent interconnect 38, interrupt unit 30 and translation units 32, 34. In the comparative example of Figure 5, the external port controller 15 is responsible for many instances of functionality required to be implemented for compliance with the external link protocol. For example, if the external link protocol is PCIe, the following PCIe-related functionality can be implemented at the external port controller 15: • PCIe ordering point to enforce PCIe ordering rules. Note: PCIe ordering rules are enforced across outbound and inbound traffic. The ordering rules are defined to avoid deadlocks and problems between producers and consumers interacting with data at a given address. The PCIe ordering rules specify, for various classes of requests such as posted requests, non-posted requests and completion requests, whether a later request is allowed to pass an earlier request. The PCIe standard also implements a relaxed ordering attribute which if set for a given transaction allows a more relaxed ordering scheme to be used for that transaction. • PCIe address translation services (ATS) and / or Translation as a Service for inbound untranslated requests, which cause a translation to be performed by a translation unit 32, 34 for the target address of the request. • PCIe interrupt handling (MSI) - detection of whether a memory transaction specifies an address designated for representing MSIs, and for detected MSIs, signalling of the interrupt to the interrupt controller unit 30 which controls delivery of an interrupt to compute circuitry 4 of the host system. • PCIe error handling including PCIe compliant error detection, reporting, logging, and signalling (e.g. Advanced Error Reporting (AER)). • Bridging PCIe traffic to a system fabric bus protocol (e.g. AXI in the example of Figure 5); • PCIe Transaction Layer verification which includes: o Sanitizing inbound PCIe traffic for PCIe rules o Guaranteeing PCIe compliance for outbound PCIe traffic; • PCIe flow control (e.g. based on a credit-based flow control system to ensure that one of the root port 14 and link partner 18 acting as transmitter does not transmit more packets over the external link 16 than can be accepted by buffer circuitry at the recipient). • PCIe defined Power Management (e.g. managing power down of circuitry associated with the external communication link 16 when there is no traffic to be communicated over the external communication link 16). • PCIe configuration space and its handling mechanism (ECAM, Enhanced Configuration Access Mechanism). These features enable software on the host system access to access configuration registers of an endpoint 19 or other link partner 18. • PCIe security protocol including TDISP (TEE Device Interface Security Protocol), Key management, IDE (Integrity and Data Encryption). These techniques enable an endpoint device to be used in conjunction with a trusted execution environment (TEE) or secure enclave operating on the host system 2, with encryption and integrity protection applied to the traffic on the external communication link 16 to prevent unauthorized access to, and tampering with, data communicated in the external link traffic. • Transaction Layer reliability which includes end to end CRC (ECRC) and Data Link Layer reliability which includes LCRC, FEC (Forward Error Correction) and Replay (the replay feature provides a buffer holding transmitting packets until they are acknowledged by the recipient, to allow packets lost in the communication link to be resent if necessary). • PCIe defined advanced capabilities including: o Downstream Port Containment (DPC) - a feature to prevent propagation of PCIe traffic beyond a port at which an uncorrectable error is detected, to prevent corruption of data at locations beyond that port. o Tag management for non-posted requests - use of tags to match returning completion responses with pending non-posted requests. This can cause the design of circuitry for the external port controller 15 to be extremely complex, especially as few of the above-mentioned capabilities are well contained and implemented within the external port controller 15. Many of these functions (e.g. bridging to a system fabric bus protocol, MSI handling, translation services, etc.) have to be implemented in coordination with rest of the system components. This introduces various protocols and interfaces at the controller 15 to communicate / coordinate with the rest of the system. This is problematic because typically the external port controller 15 may be developed independently of the rest of the system 2 by a different party from the system designer. Another disadvantage with the approach shown in Figure 5 is that it introduces multiple ordering points within the system and requires ordering guarantees to be passed between each of these ordering points (Inbound: PCIe -> AXI -> coherent interconnect protocol (e.g. CHI) and Outbound: CHI -> AXI -> PCIe). Moreover, implementing the ordering point at the PCIe controller, which is far from the system fabric, restricts its ability to parallelize the processing of incoming PCIe traffic, forcing the controller to implement a conservative ordering model because the controller 15 does not have access to any memory map information related to the memory layout of the host memory system 10, which could otherwise be used to identify cases when a more relaxed ordering would not change the results required by the PCIe ordering rules. Another issue with the approach shown in Figure 5 is that later generations (e.g. generation 7) of PCIe require a higher translation rate, requiring multiple address translations per cycle. Keeping with the required translation rate may require multiple distributed translation units 32, 34 and micro / nano TLB caches of address translation data closer to the external port controller 15(i.e. closer to the source of the translation request). Including interfaces to multiple distributed address translation units 30 in the external port controller introduces more complex additional protocols into the external port controller 15 design, such as considering specific protocols associated with a system memory management unit design provided by the host system designer. The use of the AXI protocol on the non-coherent interconnect 38 linking the external port controller 15 and coherent bridge 12 means a that read / write-aware protocol is necessary on the intervening interconnect 38, so that specific PCIe to AXI bridge functionality 40 is needed at the controller 15. This means that the controller 15 design will need to be continually revised to take account of subsequent changes to the non-coherent interconnect protocol used on the noncoherent interconnect 38, as well as being updated to changes in the external link (PCIe) protocol. Handling incoming MSIs at the PCIe controller 15 also introduces an additional type of interface (since the interrupt controller 30 for the system may use yet another interface protocol to receive communications relating to MSIs), introducing further complexity in the external port controller 15. Also, if MSIs have to be handled from the external port controller 15, delays in detection of MSIs may cause head of line blocking on the PCIe traffic, where non-MSI packets are delayed because they are held up in a queue behind a MSI awaiting servicing by the interrupt unit 30. As shown in Figure 6, an alternative approach shown for comparative purposes only is to implement circuitry 30, 32, 34 for translation and interrupt servicing as an inline unit on the path from the non-coherent interconnect 38 to the coherent bridge 12. However, while this enables some complexity to be removed from the external port controller 15, it introduces performance issues because now all inbound transactions being routed from external port controller 15 to coherent bridge 12 pass via the translation and interrupt servicing circuitry 30, 32, 34, so if a queue of transactions being processed at the translation and interrupt servicing circuitry 30, 32, 34 is stalled, this causes increased risk of head of line blocking of other transactions compared to the interrupt handling / translation as a service approach shown in Figure 5 where there are separate interfaces to the interrupt unit 30 and translation units 32, 34 from each external port controller. In any case, Figure 6 still suffers from the disadvantage that the external port controller 15 is responsible for enforcing PCIe ordering rules and so is forced to use an extremely conservative ordering model (e.g. simply serializing processing of affected requests to delay issuing a later request until an acknowledgement of an earlier request is received). Figure 7 shows an example of the present technique in which these problems can be addressed. In the example of Figure 7, communications between the external port controllers 15 and the bridge circuitry 12 are via an internal communication link 70. A new boundary is defined in the hardware protocol stack for implementing the functionality required by the external link protocol, with various functions (such as external link transaction ordering rules enforcement, MSI handling and address translation) moved to be performed at the bridge circuitry 12 which includes the protocol mapping circuitry 54 for mapping between transaction layer packets of the external link protocol and the memory system interconnect transactions according to the protocol used on the memory system interconnect 8. By changing the delineation of roles and responsibilities on each side of the cut point (the internal communication link 70), this better partitions the functionality between system fabric (bridge circuitry 12 and an associated interconnect 8 such as a coherent mesh network, CMN) and the root port circuitry 14 comprising the external port controller 15, which is significant because often the root port circuitry 14 is designed by a different party to the system fabric components including the bridge circuitry 12. Hence, the defined boundary segregates PCIe protocol layers (Physical Layer, Data Link Layer and majority of the Transaction Layer) on one side and the system fabric protocols (for e.g., AMBA® protocols like CHI, LTI, DTI) on another side. This allows the two sides to be developed independently where the external link (PCIe) controller circuitry 15 can be designed / updated for the latest PCIe specification and the system fabric dependent components 12, 8 can be designed / updated for proprietary system interconnect specifications such as the various AMBA® protocols. This new defined boundary moves system side capabilities like ordering point (final point of serialization - i.e. the transaction ordering circuitry 58 which enforces the PCIe ordering rules), address translation services (implementing in band, PCIe Address Translation Service and / or Translation as a Service, TaaS) and interrupt handling (MSI) etc. into the bridge circuitry 12 closer to the system fabric so that these can be efficiently handled. Meanwhile, other PCIe-specific capabilities which require less knowledge of specific system fabric protocols, including implementation of ECAM (PCIe configuration space and handling of this configuration space), Power Management, Error detection / reporting, TDISP and IDE (selective and link IDE), Tag management of non-posted requests (NPR), are implemented at the external port controller 15. Based on this newly defined boundary, this allows use of a lightweight, credited interface protocol, such as CXS, as the standard internal communication link 70 between the bridge circuitry 12 and the external port controller 15, with the internal communication link 70 being used to transport transaction layer packets of the PCIe or another external link protocol between the bridge 12 and the external link protocol 15. CXS is a protocol-agnostic interface protocol that enables the transfer of multiple packets over shared wires. The internal link protocol (e.g. CXS) is read / write agnostic - it does not differentiate read and write transactions, so a shared communication channel is used to convey PCIe transaction layer packets as payload data accompanied by CXS control information, regardless of whether those transaction layer packets are intended to represent read transactions, write transactions or other transactions (e.g. completions). By avoiding the need for specific read / write channel transactions to be encoded on the internal communication link 70, this means there is no need for the external port controller 15 to include any specific protocol remapping circuitry to map between read / write transactions according to PCIe or another external link protocol and read / write transactions according to AXI, CHI or another memory interconnect protocol. Also, CXS supports transmission of multiple PCIe transactions in one flit, so can be helpful in achieving greater transfer bandwidths than would be possible using alternative internal link protocols such as AXI. For example, CXS may support 2048-bit wide flits which are not possible in AXI. Hence, using CXS helps reduce the number of wires needed on the internal communication link 70 between the external port controller 15 and bridge 12 while still achieving full PCIe wire bandwidths (equivalent to x16 PCIe generation 7 bandwidth) and enables processing of PCIe traffic (including smaller packets) at line rate. CXS also facilitates efficient SoC partitioning, which could further help placing the PCIe controller / PHY of a root port 14 on a separate chiplet (IO chiplet) from the rest of the System IP components like system fabric 8, bridge 12, SMMLI 13, interrupt controller 30, which are implemented on another chiplet. As shown in Figure 7, the bridge circuitry 12 comprises transaction ordering circuitry 58 which is responsible for enforcing the transaction ordering rules required by the external link protocol. For example, the transaction ordering circuitry 58 may control timing of issuing of requests on the memory system interconnect 8 and / or transmission of responses back to the external port controller 15, to ensure that a younger inbound request from the link partner 18 bypassing an older inbound request does not yield a result that violates the ordering rules required by the external link protocol. Moving the ordering point closer to the system fabric (by locating the transaction ordering circuitry 58 in the bridge 12 rather than at the controller 15 as in the comparative examples of Figures 5 and 6) allows more efficient implementation of PCIe ordering rules. Using the memory system fabric’s system address map structures, the bridge circuitry 12 can get additional information of the target memory range, e.g. identifying whether an address for a given inbound transaction targets on-chip memory storage 10 on the same chip as the bridge 12, remote memory on a different die 2 or socket accessed via a chip-to-chip link 22, or a link partner 18 accessed via a root port 14 according to the external link protocol, and based on this information we can implement more efficient handling of PCIe traffic (e.g. strongly ordered and bulk data transfers defined according to PCIe). For example, the ordering circuitry 58 can exploit the “Ordered Write Observation” write streaming flow supported by the AMBA® CHI memory system interconnect protocol, which allows ordered write requests (write requests which architecturally are required to be serialised) to be at least partially parallelised, by issuing corresponding requests to the memory system interconnect 8 without waiting for each older ordered request to complete, but serializing the return of corresponding completion messages to the requester, so that a completion for a younger ordered request is not sent until completions have been sent for all older requests required to be correctly ordered relative to that younger ordered request. This approach can significantly improve processing performance because it enables snooping of private caches to be initiated earlier by the memory system interconnect 8 so that delays in receiving snoop responses for each ordered write request can be at least partially overlapped to reduce the total latency of processing the set of ordered write requests. The ordered write observation flow is possible when the ordered write requests are detected as targeting different nodes of the memory system (where the target node could be any of a given memory storage unit 10, a home node 9, an interface for a chip-to-chip link 22 to another chiplet, or a root port 14 for communicating with link partners 18 over an external communication link, for example). It is not typically practical to expose to the external port controller 15 the memory map information required to identify the target node for a given write request, and in any case supporting ordered write observation at the external port controller 15 would require detailed knowledge of a specific memory system interconnect protocol 8, which is undesirable for root port circuitry 14 designed separately from the memory system interconnect. Hence, by moving the ordering point enforcement to be carried out by transaction ordering circuitry 58 at the bridge circuitry 52, this improves performance by being able to process multiple strongly ordered requests in parallel. The ordered write flow is just one example of a system-fabric-dependent technique which becomes available by moving the ordering point 58 to the bridge circuitry 12. It will be appreciated that other protocol dependent features may also become available by changing the responsibility for ordering PCIe-based transactions received from a link partner. Similarly, by moving the address translation service to the bridge circuitry closer to the system fabric (so that the bridge circuitry 12 includes an interface for interacting with a system memory management unit (SMMLI) 13 or translation lookaside buffer), this eliminates more complex host-system-dependent translation interface protocols from the external port controller circuitry 15 (e.g. protocols such as a distributed translation interface protocol, e.g. AMBA® DTI, or local translation interface protocol, e.g. AMBA® LTI). This also enables more efficient implementation of micro / nano TLB caches, which helps to meet required translation rate to keep with the bandwidth required by later generations of PCIe. This approach also reduces the complexity of implementing a network of interfaces into the address translation units from each bifurcated external port controller 15 as in the approach shown in Figure 5. Also, the interrupt handling is moved to the bridge circuitry 12 closer to the system fabric, eliminating the need for an interrupt-controller-specific protocol (e.g. a further streaming protocol such as AMBA® 4 AXI4-Stream protocol) to be supported in an interface from the external port controller 15 to the interrupt controller 30. This approach can also alleviates head of blocking on MSI processing. An advantage of implementing more PCIe functionality at the bridge circuitry 12 can be that the transaction ordering enforcement circuitry 58, interfaces to the SMMLI 13 and interrupt controller 30, and other PCIe transaction handling circuitry, can be shared by multiple bifurcated streams, rather than being duplicated per external port controller 15. Figure 8 illustrates in more detail an example implementation of channels of communication on the internal communication link 70. The CXS protocol is a unidirectional protocol, so for bidirectional communication, at least two separate CXS channels would be used: an inbound channel 71 for inbound transactions from external port controller 15 to bridge 12, and an outbound channel 72 for outbound transactions from bridge 12 to external port controller 15. The main inbound / outbound CXS channels 71, 72 can have variable data width depending on the desired bandwidth to be supported. For example, Figure 7 shows an implementation where the width of a communication link to a given controller 15 scales with the bandwidth expected to be supported by that controller, e.g. with the X16 controller having a 2048-bit CXS datapath for both inbound and outbound channels 71, 72 while the X8 and X4 controllers have 1024-bit and 512-bit CXS datapaths respectively. Here, the data width of a given channel 71, 72 refers to the width of the data payload passed on the channel, excluding any accompanying control information. Although not essential, in this example some additional inbound and outbound CXS channels 73, 74 are provided, denoted as “CXS-Lite” as they may have narrower datapaths than the main channels 71, 72. The CXS Lite channels 73, 74 are used for sideband messages such as shadow configuration writes, error messages, connection management and flow control messages. By providing a separate sideband channel 73, 74 from the main channel 71, 72 used for routing the external link protocol packets themselves, this prevents control messages delaying the transmission of the external link protocol packets, to improve throughput of the external link transactions. As shown in Figure 8, for each CXS channel 71-74 (including the main band channels 71, 72 or the sideband channels 73, 74), that channel comprises a control channel 80 and a data channel 85, with the data channel 85 transporting the transaction layer packets of the PCIe or other external link protocol as data payload, accompanied by control data on the control channel 80 which is defined according to the CXL protocol, and is encoded to denote information such as packet size or packet start / end of packets encoded on the data channel 85. As shown in Figure 9, with the use of CXS as the internal link protocol on the internal communication link 70, a number of layers of encoding / decoding are involved in a system comprising the integrated circuit (apparatus) 2 and link partner 18 (e.g. endpoint 19). The layers include: • a system fabric protocol encoding / decoding layer 90, implemented by the protocol mapping circuitry 54 at the bridge circuitry 52 (although not shown in Figure 9, the memory system interconnect 8 and / or other system components 4, 6, 10 of the host system may have similar system fabric protocol encoding / decoding layer for encoding / decoding according to the memory system interconnect protocol); • an application layer 91, implemented using encoding / decoding circuitry at the bridge circuitry 91 and the link partner 18 (or at an endpoint 19 which communicates with the link partner 18). The application layer 91 is outside the scope of what is defined in the PCIe specification, but may define information to be included as payload data within a transaction layer packet. For example, the application layer may implement additional networking protocols. Although the application layer encoding and decoding may be applied at the link partner / endpoint end for inbound and outbound transactions respectively, the external port controller 15 could also include application layer circuitry for updating or verifying application layer encoded payload data within a PCIe transaction layer packet. • the transaction layer 92 defined according to PCIe (external link protocol), which is encoded / decoded using the protocol mapping circuitry 54 at the bridge circuitry 12 and corresponding encoding / decoding circuitry at the link partner 18 (or at an endpoint 19 which communicates with the link partner 18). Although the link partner 18 or endpoint 19 encodes inbound transaction layer packets and decodes outbound transaction layer packets, the external port controller 15 may also include transaction layer circuitry for verifying that transaction layer packets are correctly formed, when conveying transaction layer packets between the bridge 12 and link partner 18. • a network layer 93, which in this example is according to the CXS protocol, but could also use other internal link protocols such as SFI, and for which the internal link control information is encoded / decoded by internal link interfaces at the bridge circuitry 12, internal communication link router 70 and external port controller 95. • the data link layer 94 defined according to PCIe (external link protocol), which is encoded / decoded by the link partner 18 and the data link layer encoding / decoding circuitry 40 of the external port controller 15. • the physical layer 95 defined according to PCIe (external link protocol), which is encoded / decoded by a PHY controller 42 associated with the root port 14 and by the link partner 18. Hence, while the network layer 93 introduces another layer of encoding / decoding between the external port controller 15 and bridge 12, the use of CXS as the internal link protocol greatly simplifies the interface circuitry for encoding communications on the internal communication link 70, in comparison to other protocols such as AXI which require more specific mapping of external link protocol packets to a read / write specific format. Figure 10 illustrates an example of the revised partitioning of functionality between the bridge circuitry 12 and the external port controller 15. For this particular example, the external port controller 15 remains responsible for the following functionality: • PCIe defined advanced capabilities such as DPC and tag management as discussed above; • TDISP and IDE as discussed above; • Transaction layer protocol checking (verification that transaction layer packets are correctly formed) and transaction layer cyclic redundancy checks (ECRCs); • PCIe error handling as mentioned earlier; • PCIe receiver buffers for buffering packets received from the link partner. • power management for the PCIe external communication link 16. On the other hand, the bridge circuitry 12 includes circuitry implementing the following functions: • address translation, based on an interface to a System Memory Management Unit 13 or TLBs; • MSI processing; • mapping between the transaction layer packets of PCIe and the (coherent) memory system interconnect protocol used by the main system interconnect fabric 8; • ECAM; • transaction ordering according to PCIe ordering rules; • shadow configuration space; • forward-programming of root port routing tables, enabling specific tunneling of traffic from one root port to another when an inbound transaction from one root port 14 targets another root port 14; • maintenance of system address maps identifying mappings of address space onto respective memory system nodes of the apparatus 2, which can be used to manage transaction flows more efficiently. With this division of responsibility, functions that require more detailed knowledge of host system specific protocols or memory map information may be performed more efficiently at the bridge circuitry 12 than would be possible at the external port controller 15 which may be designed as part of root port circuitry 14 which is relatively agnostic to the host system-specific protocols and may in some instances be designed as a separate chiplet for use with a range of different host systems. Figure 11 illustrates an example of the bridge circuitry 12 in more detail. The bridge circuitry 12 includes a memory system interconnect interface 50 for transmitting and receiving memory system interconnect protocol messages to / from the memory system interconnect 8, and the protocol mapping circuitry 54 described earlier for mapping between the memory system interconnect protocol messages and the transaction layer packets according to the external link protocol used by the external communication link 16. The bridge circuitry also includes an internal link interface 52 for encoding / decoding the control channel information to be transmitted on the control channel 80 for messages transmitted / received on the internal communication link 70, according to the internal link protocol (e.g. CXS) used on that link 70. The bridge circuitry 12 has control circuitry 56 which detects properties of transaction layer packets processed by the bridge circuitry 12, to check which additional services / functionality are required for processing those transaction layer packets. For example, transaction layer packets corresponding to ordered requests which are required to follow external link protocol ordering rules are subject to order control performed by the transaction ordering circuitry 58 as described earlier. The transaction ordering circuitry 58 may consider memory system map information identifying mapping of addresses to nodes of the host memory address system, when controlling the ordering. For ordered write requests inbound from the link partner 18 towards the host memory system, the transaction ordering circuitry 58 may use an ordered write optimized transaction flow, which enables the requests corresponding to the ordered write requests to be issued in a non-serialized manner (if they target different memory system nodes of the host memory system), and the transaction ordering circuitry 58 tracks in-order commitment of those writes to ensure that the completion responses to the ordered write requests are signalled to the link partner 18 in an order matching the original request order. Also, the bridge circuitry 12 comprises address translation interface circuitry 60 which provides an interface to address translation circuitry 32, 34 such as a system memory management unit and / or TLBs. For example, communications via the address translation interface circuitry may be according to the AMBA® DTI or LTI protocols, which define an interface by which address translations can be requested. Hence, requests requiring address translation can invoke the translation service provided by the address translation interface circuitry 60. Also, the bridge circuitry 12 comprises message signalled interrupt (MSI) handling circuitry 62 which provides an interface to an interrupt controller 30 which generates interrupts to be routed across the host system 2 to a compute unit such as a CPU 4. When the control circuitry 56 detects a transaction layer packet representing a read / write transaction specifying as its target address an address mapped for MSIs, the control circuitry 56 causes the MSI to be signalled to the interrupt controller 30 via the MSI handling circuitry 62. The interface between the MSI handling circuitry 62 and interrupt controller 30 can be via a host system specific protocol such as AMBA® 4 AXI4-Stream or another streaming protocol. Hence, this approach enables address translation and interrupt handling to remain as an “on demand service” function, rather than as in-line functionality causing all requests to be delayed by requests awaiting address translation of MSI handling, but without needing the externally developed external port controller 15 circuitry to have the dedicated interfaces 60, 62 for interacting with the address translation units 32, 34 or interrupt controller 30. Figure 12 illustrates a method according to the present technique. At step 100, the bridge circuitry 12 communicates with the external port controller 15 via an internal communication link 70 according to an internal link protocol used to transport the external link protocol packets between the bridge circuitry and the external port controller. At step 102, the bridge circuitry maps between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by the memory system interconnect 8. At step 104, transaction ordering circuitry 54 of the bridge circuitry 12 enforces external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner 18 on the external communication link 16. Concepts described herein may be embodied in a system comprising at least one packaged chip. The apparatus 2, or in some cases just the bridge circuitry 12, as described earlier is implemented in the at least one packaged chip (either being implemented in one specific chip of the system, or distributed over more than one packaged chip). The at least one packaged chip is assembled on a board with at least one system component. A chip-containing product may comprise the system assembled on a further board with at least one other product component. The system or the chip-containing product may be assembled into a housing or onto a structural support (such as a frame or blade). As shown in Figure 13 one or more packaged chips 400, with the apparatus 2 or bridge circuitry 12 described above implemented on one chip or distributed over two or more of the chips, are manufactured by a semiconductor chip manufacturer. In some examples, the chip product 400 made by the semiconductor chip manufacturer may be provided as a semiconductor package which comprises a protective casing (e.g. made of metal, plastic, glass or ceramic) containing the semiconductor devices implementing the apparatus 2 or bridge circuitry 12 described above and connectors, such as lands, balls or pins, for connecting the semiconductor devices to an external environment. Where more than one chip 400 is provided, these could be provided as separate integrated circuits (provided as separate packages), or could be packaged by the semiconductor provider into a multi-chip semiconductor package (e.g. using an interposer, or by using three-dimensional integration to provide a multi-layer chip product comprising two or more vertically stacked integrated circuit layers). In some examples, a collection of chiplets (i.e. small modular chips with particular functionality) may itself be referred to as a chip. A chiplet may be packaged individually in a semiconductor package and / or together with other chiplets into a multi-chiplet semiconductor package (e.g. using an interposer, or by using three-dimensional integration to provide a multilayer chiplet product comprising two or more vertically stacked integrated circuit layers). The one or more packaged chips 400 are assembled on a board 402 together with at least one system component 404 to provide a system 406. For example, the board may comprise a printed circuit board. The board substrate may be made of any of a variety of materials, e.g. plastic, glass, ceramic, or a flexible substrate material such as paper, plastic or textile material. The at least one system component 404 comprise one or more external components which are not part of the one or more packaged chip(s) 400. For example, the at least one system component 404 could include, for example, any one or more of the following: another packaged chip (e.g. provided by a different manufacturer or produced on a different process node), an interface module, a resistor, a capacitor, an inductor, a transformer, a diode, a transistor and / or a sensor. A chip-containing product 416 is manufactured comprising the system 406 (including the board 402, the one or more chips 400 and the at least one system component 404) and one or more product components 412. The product components 412 comprise one or more further components which are not part of the system 406. As a non-exhaustive list of examples, the one or more product components 412 could include a user input / output device such as a keypad, touch screen, microphone, loudspeaker, display screen, haptic device, etc.; a wireless communication transmitter / receiver; a sensor; an actuator for actuating mechanical motion; a thermal control device; a further packaged chip; an interface module; a resistor; a capacitor; an inductor; a transformer; a diode; and / or a transistor. The system 406 and one or more product components 412 may be assembled on to a further board 414. The board 402 or the further board 414 may be provided on or within a device housing or other structural support (e.g. a frame or blade) to provide a product which can be handled by a user and / or is intended for operational use by a person or company. The system 406 or the chip-containing product 416 may be at least one of: an end-user product, a machine, a medical device, a computing or telecommunications infrastructure product, or an automation control system. For example, as a non-exhaustive list of examples, the chipcontaining product could be any of the following: a telecommunications device, a mobile phone, a tablet, a laptop, a computer, a server (e.g. a rack server or blade server), an infrastructure device, networking equipment, a vehicle or other automotive product, industrial machinery, consumer device, smart card, credit card, smart glasses, avionics device, robotics device, camera, television, smart television, DVD players, set top box, wearable device, domestic appliance, smart meter, medical device, heating / lighting control device, sensor, and / or a control system for controlling public infrastructure equipment such as smart motorway or traffic lights. Concepts described herein may be embodied in computer-readable code for fabrication of an apparatus that embodies the described concepts. For example, the computer-readable code can be used at one or more stages of a semiconductor design and fabrication process, including an electronic design automation (EDA) stage, to fabricate an integrated circuit comprising the apparatus embodying the concepts. The above computer-readable code may additionally or alternatively enable the definition, modelling, simulation, verification and / or testing of an apparatus embodying the concepts described herein. For example, the computer-readable code for fabrication of an apparatus embodying the concepts described herein can be embodied in code defining a hardware description language (HDL) representation of the concepts. For example, the code may define a register-transfer-level (RTL) abstraction of one or more logic circuits for defining an apparatus embodying the concepts. The code may define a HDL representation of the one or more logic circuits embodying the apparatus in Verilog, SystemVerilog, Chisel, or VHDL (Very High-Speed Integrated Circuit Hardware Description Language) as well as intermediate representations such as FIRRTL. Computer-readable code may provide definitions embodying the concept using system-level modelling languages such as SystemC and SystemVerilog or other behavioural representations of the concepts that can be interpreted by a computer to enable simulation, functional and / or formal verification, and testing of the concepts. Additionally or alternatively, the computer-readable code may define a low-level description of integrated circuit components that embody concepts described herein, such as one or more netlists or integrated circuit layout definitions, including representations such as GDSII. The one or more netlists or other computer-readable representation of integrated circuit components may be generated by applying one or more logic synthesis processes to an RTL representation to generate definitions for use in fabrication of an apparatus embodying the invention. Alternatively or additionally, the one or more logic synthesis processes can generate from the computer-readable code a bitstream to be loaded into a field programmable gate array (FPGA) to configure the FPGA to embody the described concepts. The FPGA may be deployed for the purposes of verification and test of the concepts prior to fabrication in an integrated circuit or the FPGA may be deployed in a product directly. The computer-readable code may comprise a mix of code representations for fabrication of an apparatus, for example including a mix of one or more of an RTL representation, a netlist representation, or another computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus embodying the invention. Alternatively or additionally, the concept may be defined in a combination of a computer-readable definition to be used in a semiconductor design and fabrication process to fabricate an apparatus and computer-readable code defining instructions which are to be executed by the defined apparatus once fabricated. Such computer-readable code can be disposed in any known transitory computer-readable medium (such as wired or wireless transmission of code over a network) or non-transitory computer-readable medium such as semiconductor, magnetic disk, or optical disc. An integrated circuit fabricated using the computer-readable code may comprise components such as one or more of a central processing unit, graphics processing unit, neural processing unit, digital signal processor or other components that individually or collectively embody the concept. In the present application, the words “configured to...” are used to mean that an element of an apparatus has a configuration able to carry out the defined operation. In this context, a “configuration” means an arrangement or manner of interconnection of hardware or software. For example, the apparatus may have dedicated hardware which provides the defined operation, or a processor or other processing device may be programmed to perform the function. “Configured to” does not imply that the apparatus element needs to be changed in any way in order to provide the defined operation. In the present application, lists of features preceded with the phrase “at least one of” mean that any one or more of those features can be provided either individually or in combination. For example, “at least one of: A, B and C” encompasses any of the following options: A alone (without B or C), B alone (without A or C), C alone (without A or B), A and B in combination (without C), A and C in combination (without B), B and C in combination (without A), or A, B and C in combination. Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one skilled in the art without departing from the scope of the invention as defined by the appended claims.

Claims

1. An apparatus comprising:an external port controller configured to control communication, via an external communication link for communicating with a link partner, of external link protocol packets defined according to an external link protocol; andbridge circuitry configured to map between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by a memory system interconnect; in which:the bridge circuitry and the external port controller are coupled via an internal communication link and are configured to use an internal link protocol to transport the external link protocol packets between the bridge circuitry and the external port controller; andthe bridge circuitry comprises transaction ordering circuitry configured to enforce external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link.

2. The apparatus according to claim 1, in which the transaction ordering circuitry is configured to control the ordering between the respective data access transactions in dependence on memory map information indicative of properties of a memory system accessible via the memory system interconnect.

3. The apparatus according to any of claims 1 and 2, in which the transaction ordering circuitry is configured to permit inbound ordered write transactions, which are received from the link partner to request write access to data in a memory system accessible via the memory system interconnect and are required by the external link protocol transaction ordering rules to be serialized, to be processed at least partially in parallel when the inbound ordered write transactions target different target nodes of the memory system.

4. The apparatus according to any preceding claim, in which the bridge circuitry comprises address translation interface circuitry configured to request address translation of a target address specified by an inbound transaction received from the link partner.

5. The apparatus according to claim 4, in which the address translation interface circuitry of the bridge circuitry is configured to trigger a lookup of at least one translation lookaside buffer configured to cache address translation information.

6. The apparatus according to any of claims 4 and 5, in which the address translation interface circuitry of the bridge circuitry is configured to communicate with address translation circuitry according to a local translation interface protocol or a distributed translation interface protocol.

7. The apparatus according to any preceding claim, in which the bridge circuitry comprises message signalled interrupt handling circuitry configured to detect message signalled interrupts represented by external link protocol packets received from the link partner, and to signal the message signalled interrupts to an interrupt controller.

8. The apparatus according to claim 7, in which the message signalled interrupt handling circuitry of the bridge circuitry is configured to communicate with the interrupt controller according to a streaming protocol.

9. The apparatus according to any preceding claim, in which the bridge circuitry is coupled, via the internal communication link, to a plurality of external port controllers configured to manage bifurcated streams of external link protocol packets.

10. The apparatus according to any preceding claim, in which the internal link protocol provides a format for transporting one or more external link protocol packets as payload data associated with control information defined according to the internal link protocol.

11. The apparatus according to any preceding claim, in which the internal link protocol supports transmission of a plurality of external link protocol packets in a single flit defined according to the internal link protocol.

12. The apparatus according to any preceding claim, in which the internal link protocol comprises a read / write-agnostic protocol.

13. The apparatus according to any preceding claim, in which the internal communication link supports transfer of at least 2048 bits of data per flit.

14. The apparatus according to any preceding claim, in which the internal link protocol comprises CXS.

15. The apparatus according to any preceding claim, in which the external link protocol comprises an input / output interface protocol.

16. The apparatus according to any preceding claim, in which the external link protocol comprises a layered protocol.

17. The apparatus according to any preceding claim, in which the external link protocol packets comprise PCIe transaction layer packets.

18. The apparatus according to claim 17, in which the external port controller comprises data link layer encoding / decoding circuitry configured to encode / decode PCIe data link layer information for transporting on the external communication link.

19. The apparatus according to any preceding claim, in which the memory system interconnect protocol comprises a coherent interconnect protocol.

20. The apparatus according to any of claims 1 to 18, in which the memory system interconnect protocol comprises a non-coherent interconnect protocol.

21. The apparatus according to any preceding claim, comprising:the memory system interconnect;compute circuitry coupled to the memory system interconnect; andmemory storage circuitry coupled to the memory system interconnect.

22. Bridge circuitry configured to bridge between a memory system interconnect and an external port controller; the bridge circuitry comprising:an internal communication link interface configured to communicate with the external port controller via an internal communication link, wherein the internal communication link interface is configured to use an internal link protocol to transport external link protocol packets between the bridge circuitry and the external port controller, the external link protocol packets defined according to an external link protocol for an external communication link between the external port controller and a link partner;protocol mapping circuitry configured to map between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by the memory system interconnect; andtransaction ordering circuitry configured to enforce external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link.

23. A system comprising:the apparatus according to any of claims 1 to 21 or the bridge circuitry according to claim 22, implemented in at least one packaged chip;at least one system component; anda board, wherein the at least one packaged chip and the at least one system component are assembled on the board.

24. A chip-containing product comprising the system of claim 23, wherein the system is assembled on a further board with at least one other product component.

25. Computer-readable code for fabrication of the apparatus according to any of claims 1 to 21 or the bridge circuitry according to claim 22.

26. A method for bridge circuitry configured to bridge between a memory system interconnect and an external port controller configured to control communication, via an external communication link for communicating with a link partner, of external link protocol packets defined according to an external link protocol;the method comprising:the bridge circuitry communicating with the external port controller via an internal communication link according to an internal link protocol used to transport the external link protocol packets between the bridge circuitry and the external port controller;the bridge circuitry mapping between the external link protocol packets and memory system interconnect transactions defined according to a memory system interconnect protocol used by the memory system interconnect; andusing transaction ordering circuitry comprised by the bridge circuitry, enforcing external link protocol transaction ordering rules imposed by the external link protocol to restrict ordering between respective data access transactions corresponding to external link protocol packets communicated with the link partner on the external communication link.

Images