Accessing fabric attached memory

Abstract

A memory access system comprising a fabric switch which is connectable to a computer device that generates memory access requests. The switch is also connectable to downstream fabric-attached memory d

Description

FIELD

[0001] The present application relates to accessing memory in computing systems, particularly accessing large memories in fabric-attached systems. BACKGROUND

[0002] As the demand for artificial intelligence (AI), machine learning (ML), and other computationally expensive applications increases, so too does the need for highly reliable computer memory systems.

[0003] Processors configured to implement computational tasks are becoming more powerful, and are able to execute instructions at increasing rates. However, optimal performance of these processors cannot be achieved without comparable performance of memory devices in the computer system. As memory capacity increases, problems with reliability become more prevalent. It is important that improvements in reliability of memory keep up with the trend of increasing processor performance.

[0004] Recent developments in interconnect technologies are enabling low-latency and high-bandwidth memory to be attached over a switching fabric.

[0005] One recent development in the field includes Compute Express Link (CXL), an open standard for high-speed and high-bandwidth connections between devices in high performance computing systems. CXL enables lower latency memory devices, such as DRAMS, to be accessible to host systems via a switching fabric.

[0006] CXL is designed to leverage the CXL protocols and the PCI Express 5.0 physical layer infrastructure to provide high-bandwidth, low latency connectivity' between a host processor and other CXL-enabled devices.

[0007] Increasingly, computing systems, such as high-performance systems, are moving towards heterogeneous computing architectures, with application-specific accelerators being used to offload work otherwise performed by a CPU. CXL is well-suited to these environments because it supports memory cache coherency.

[0008] However, as the capacity of memory in a system scales, so too does the likelihood of memory errors, due to issues such as signal interference. For computer systems to remain scalable and keep up with the increasing demand for complex computing tasks, memory systems therein must also scale without becoming too slow, or too error prone, so that the processing power of the system is not severely limited by memory latency or unreliability. SUMMARY

[0009] Summary

[0010] According to a first aspect of the invention there is provided a memory access system comprising: a fabric switch connectable between a first computer device which is configured to generate memory access requests and a plurality of downstream fabric-attached memory devices which are configured to respond to memory access requests, the fabric switch comprising: an upstream port connectable to the first computer device; a plurality of downstream ports, each of which is connectable to a respective downstream fabric-attached memory device; and routing logic configured to route data of each memory access request received at the upstream port to the plurality of downstream ports based on respective port identifiers, the mcmoiy access system comprising computer logic configured to segment a data item of a memory access request received at the upstream port into a plurality of segments, wherein the routing logic is configured to allocate each of the segments of the plurality of segments to a downstream port and to assign a corresponding port identifier to each segment for use by the routing logic.

[0011] In some embodiments, the computer logic is further configured to generate from the data item a redundant data segment. The routing logic may be configured to allocate to the redundant data segment a port identifier for a downstream port connected to a redundant fabric-attached memory device.

[0012] In some embodiments, the system comprises access logic configured, in response to a read memory access request issued by the host device, to access the plurality of segments from the plurality of downstream fabric-attached memory devices and to reassemble the data item.

[0013] In some embodiments, the routing logic comprises sequencing instructions configured to iteratively route each of the plurality of segments over a crossbar of the fabric switch to respective ones of the downstream ports based on the port identifier.

[0014] In some embodiments, the memory access system further comprises a write multiplexor configured to iteratively route each of the plurality of segments over the crossbar. The write multiplexor may comprise a plurality of input lines including a respective input line for each segment and the redundant data segment, and may further comprise a control line configured to iteratively select an input line from which to forward data over the crossbar based on the sequencing instructions.

[0015] In some embodiments, the fabric switch comprises a plurality of registers, the plurality of registers including a respective member register configured to store port identifiers corresponding to each of the plurality of downstream ports.

[0016] In some embodiments, wherein the plurality of registers further includes a segmentation-enable register configured to indicate whether to execute the computer logic.

[0017] In some embodiments, the computer logic is further configured to, based on detection of an error in a segment accessed from a downstream fabric-attached memory device: load each of the plurality of segments into a first buffer of the upstream port; using at least the redundant data segment and one or more segments in which no error is detected, reconstruct a corrected segment corresponding to the segment in which an error is detected; for each segment in which no error is detected, forward the segment in the first buffer to a second buffer of the upstream port; and for the segment in which an error is detected, forward the corrected segment to the second buffer. wherein the computer logic is configured to reassemble the data item by applying reconstruction logic to the segments in the second buffer.

[0018] In some embodiments, the fabric switch comprises a plurality of correction multiplexors, each correction multiplexor comprising a first input line from the first buffer and a second input line for receiving a corrected segment. The computer logic may be configured to forward segments to the second buffer based on a control signal provided to each multiplexor by sequencing logic of the upstream port, the control signal indicating an input line from which data is to be forwarded to the second buffer.

[0019] In some embodiments, the fabric switch is a first fabric switch, and a first downstream port of the first fabric switch is connectable to a fabric-attached memory’ device via a second fabric switch, wherein the first downstream port of the first fabric switch is connectable to a second port of the second fabric switch, and wherein the second fabric switch is configured to forward memory access requests received at the upstream port of the first fabric switch to a third port of the second fabric switch, the third port connectable to a fabric-attached memory device configured to respond to the memory access requests.

[0020] In some embodiments, the redundant data segment comprises one or more of error correction codes, syndrome codes and erasure codes.

[0021] In some embodiments, the system operates in accordance with a Compute Express Link interconnect protocol.

[0022] In some embodiments, the fabric switch is connectable to the downstream fabric-attached memory devices such that, on receipt of an indication that access requests from a first downstream fabric-attached memory' device result in an error with a frequency exceeding an acceptable error frequency threshold, the first downstream fabric-attached memory device is removable and replaceable with a second downstream fabric-attached memory device while the system remains in operation.

[0023] According to a second aspect of the invention, there is provided a method comprising: receiving a memory access request at an upstream port of a fabric switch, the fabric switch connectable between a first computer device configured to generate the memory access request and a plurality of downstream fabric-attached memory devices configured to respond to the memory access request; segmenting, by computer logic, a data item of the memory access request received at the upstream port into a plurality of segments; allocating each of the segments of the plurality of segments to a downstream port by assigning a corresponding port identifier to each segment; and routing, by the upstream port of the fabric switch, each segment to a respective one of the plurality-- of downstream ports based on the allocated port identifier, each downstream port connectable to a respective downstream fabric-attached memory' device.

[0024] In some embodiments, the method further comprises generating, by the computer logic, a redundant data segment from the data item, and allocating to the redundant data segment a port identifier for a downstream port connected to a redundant fabric-attached memory device.

[0025] In some embodiments, the method further comprises, in response to a read memory access request issued by the host device, accessing the plurality of segments from the plurality of downstream fabric-attached memory' devices and reassembling the data item.

[0026] In some embodiments, the method further comprises iteratively routing, by sequencing logic of the computer logic, each of the plurality of segments over a crossbar of the fabric syvitch to respective ones of the downstream ports based on the port identifier.

[0027] In some embodiments, the method further comprises iteratively routing, by a write multiplexor, each of the plurality of segments over the crossbar, wherein the write multiplexor comprises a plurality of input lines including a respective input line for each segment and the redundant data segment, and further comprises a control line configured to iteratively select an input line from yvhich to forward data over the crossbar based on the sequencing instructions.

[0028] In some embodiments, the method further comprises storing port identifiers corresponding to each of the plurality of downstream ports in a plurality of registers. The plurality of registers may include a respective member register for each of the downstream ports.

[0029] In some embodiments, the method further comprises storing, in a segmentationenable register, an indication of whether to execute the computer logic.

[0030] In some embodiments, the method further comprises, based on detection of an error in a segment accessed from a downstream fabric-attached memory device: loading each of the plurality of segments into a first buffer of the upstream port; using at least the redundant data segment and one or more segments in which an error is not detected, reconstruct a corrected segment corresponding to the segment in which an error is detected that does not comprise an error with respect to a corresponding segment allocated to the downstream port; for each segment in which no error is detected, forwarding the segment in the first buffer to a second buffer of the upstream port; for the segment in which an error is detected, forwarding the corrected segment to the second buffer; and reassembling the data item by applying reconstruction logic to the segments in the second buffer.

[0031] In some embodiments, the method further comprises: forwarding the segments to the second buffer based on a control signal provided to each of a plurality of correction multiplexors by sequencing logic of the upstream port. The control signal may indicate an input line from which data is to be forwarded to the second buffer, each correction multiplexor comprising a first input line from the first buffer and a second input line for receiving a corrected segment.

[0032] In some embodiments, generating the redundant data segment comprises generating one or more of error correction codes, syndrome codes and erasure codes.

[0033] In some embodiments, the method further comprises the use of a Compute Express Link interconnect protocol.

[0034] According to a third aspect of the invention there is provided a transitory or non-transitory computer readable media comprising instructions which, when executed by a processor of a computer device, cause the processor to implement a method according to any of the embodiments provided above. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] To assist understanding of the present disclosure and to show how embodiments may be put into effect, reference is made by way of example to the accompanying drawings in which:

[0036] Figure 1 shows a highly schematic diagram of a switched fabric computing system.

[0037] Figure 2 shows a highly schematic diagram of an exemplary switched fabric topology configurable under CXL 2.0.

[0038] Figure 3 shows a highly schematic representation of a store operation according to some examples.

[0039] Figure 4 shows a highly schematic representation of a load operation according to some examples.

[0040] Figure 5 shows a highly schematic diagram of a set of upstream port registers for implementing a fabric-attached memory system in accordance with some embodiments.

[0041] Figure 6 shows a highly schematic diagram of a store operation at a hardware level, according to some embodiments.

[0042] Figure 7 shows a highly schematic diagram of a load operation at a hardware level, according to some embodiments.

[0043] Figure 8 shows an exemplary switched fabric topology anticipated to be configurable under CXL 3.0.

[0044] Figure 9 shows a highly schematic diagram fabric attached-memory system according to some embodiments.

[0045] Figure 10 shows a highly schematic diagram of a process for regenerating data in a replacement memory device, according to some embodiments.

[0046] In the drawings, corresponding reference characters indicate corresponding components. The skilled person will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. Also, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various example embodiments.

[0047] It will be appreciated that where reference numerals of the form xxx-a, xxx-b etc., are used to denote a particular instance of a feature of a drawing, the same reference numeral without a specified letter suffix (e.g., a, b, etc.) may denote a generic instance of the same feature. DETAILED DESCRIPTION

[0048] The present invention relates to fabric attached memory’ techniques in the field of high-performance computing. In this context the term ’high -performance ‘computing does not imply any particular architecture, but denotes computing where access to high capacity memory is warranted. It is desirable if the memory’ capacity can be utilized with high reliability and low latency.

[0049] Recent developments in interconnect technologies are enabling low-latency and high-bandwidth memory' to be attached over a switching fabric. It is primarily with respect to these fabric-attached memory' solutions that the present disclosure is directed. Fabric attached memory devices may be DRAMs or DIMMs for example - these devices allow direct memory access with relatively low latencies.

[0050] In particular, the present disclosure is concerned with allocation of data in fabric attached memory, and utilises data striping and memory redundancy techniques in fabric-attached memory devices to improve reliability and protect against memory errors.

[0051] Figure 1 shows a highly schematic diagram of a computing system 100 according to some embodiments of the present invention.

[0052] The system 100 comprises a host 110, which may comprise a central processing unit (CPU). The host is connected to a fabric switch 120 via an upstream port 121 of the switch 120. Typically, a switch may be connected to a plurality of devices in a network, and may facilitate communication therebetween. For example, switches may comprise a plurality of multiplexing circuits configured to route data between a first device in the network and a second device in the network. CXL switches may be configured to provide this functionality, with data being transmitted according to CXL protocols. A fabric consists of one or more switches which are connected to end point devices, upstream and / or downstream, such as a host and memory devices.

[0053] The present disclosure provides additional functionality in a fabric attached memory access computing system in the form of RAIM logic. A method of the present disclosure utilises data striping in memory', a technique which is referred to as ’Redundant Array of Independent Memory’, herein RAIM'. By providing a RAIM-enabled system which splits up a memory' access into separate chunks for allocation across multiple memory-' devices it is possible to protect against failure of any particular module in a fabric attached system. The RAIM logic may be provided in a modified switch, or elsewhere in the system, for example at the device attached to the upstream port.

[0054] A modified switch 120 according to one embodiment of the present disclosure comprises RAIM logic 124 on the upstream port 121.

[0055] Tire RAIM logic 124 is implemented in a state machine in the switch 120, for example in the form of a simple pipeline. As described in more detail later herein, functions such as parity generation and parity deployment, when storing and loading respectively, may be implemented by the RAIM logic state machine 124 on the upstream port 121 of the modified switch 120. The upstream port of the switch 120 further comprises registers 122 for holding information concerning data allocation. Exemplary register structures are shown in Figure 5, which is described later herein. At a hardware level, the RAIM logic 124 may comprise buffers and sequencers.

[0056] Port management, and performance of general switching and routing operations, may be a responsibility of the RAIM logic 124, in cooperation with the registers 122.

[0057] The exemplary switch 121 of Figure 1 further comprises a plurality of downstream ports 126, e.g., ports 12a-126e. Each of the downstream port 126a-126e is communicatively coupled to the upstream port 122 via respective crossbar links 128aM28e. It will be appreciated therefore that the upstream port 122 and logic unit 124 may comprise a plurality of multiplexors which are configured to send data packets along selected links 128 to appropriate doyvnstream ports 126, for example based on software instructions executed by the routing logic. The routing logic of the switch uses port identifiers held in the registers 122 for routing data.

[0058] RAIM logic 124 at the upstream port may be configured to implement re-packetisation of data received at the upstream port. That is, the switch 120 may be configured to operate on data in a particular format. Incoming data received at the upstream port (e.g., in a write memory access request from the host) may be received in a first packet format, but operations in the switch may cause re-packetisation to implement the data segmentation, parity generation, and transmission of data across a switch crossbar to the downstream ports 126. With reference briefly to Figure 3, which is described in more detail later, reference numeral 320 denotes write data'. The write data 320 may be considered a pipeline state of the RAIM logic state machine 124, with write data 320 being packetized in accordance with the format of the switch 120.

[0059] Data manipulation and transmission across the switch and downstream memory devices are described in more detail later herein. Particularly with reference to Figures 4, 5, 8, and 9.

[0060] In the example of Figure 1, each downstream port is connected to a respective memory device 130. The downstream ports 126a-126e are respectively connected to memory devices 130aM30e. As discussed briefly above, CXL technology inherently supports switched fabric topologies in which a CXL switch provides one or more downstream connections to one or more additional CXL device, such as the memory devices 130.

[0061] In the present disclosure, the function of the switch is enhanced by enabling memory redundancy. In the example of Figure 1, memory devices 130a-130e are respectively labelled ‘A’, ‘B’, ‘C’, ‘D’, and ‘P'. As described later herein, the memory devices 130 labelled A-D may be designated for data striping. Memory device 130c, labelled ‘P’, may be designated for redundancy purposes for data security in the event of memory errors. Redundancy and its use in embodiments of the present invention are described in more detail later herein. In general terms, the RAIM logic 124 may be configured to generate parity data (or other error correction data) at the upstream port for redundancy purposes.

[0062] The system 100 of Figure 1 represents a switched fabric network, wherein nodes in the system (e.g., the host 110 and the memory devices 130) are interconnected via a switch, e.g., switch 120. Hie memory devices 130 may therefore be considered fabric-attached memory. The physical links and software protocols by which nodes in the system communicate may enable low-latency memory access by the host.

[0063] CXL is an exemplary open standard interconnect technology which enables the above fabric-attached memory capability. Whilst the description herein refers to features of the CXL standard to exemplify embodiments of the invention, it will be appreciated that other interconnect and communication protocols may be used.

[0064] The techniques described herein may leverage data striping techniques, redundancy-based error management, and fabric attached memory devices to provide reliable and scalable memory systems that may operate in a wide range of topologies.

[0065] Before the exemplary embodiments of the invention are described in more detail, an overview of the CXL standard and a discussion of data striping and redundancy error mitigation is provided.

[0066] CXL | Brief Overview

[0067] CXL is an interconnect standard, which enables high-speed memory access and low-latency interconnection. Specification details for a first three iterations of CXL (CXL 1.0, CXL 2.0, and CXL 3.0) are known, but only CXL 1.0 features have yet been practically implemented. .

[0068] By way of exampleCXL attached memory devices may have a latency of between 170 250 ns. For context, main memory- attached to a CPU may have a latency of, for example, 50 140 ns.

[0069] CXL 1. 0 further supports cache coherency, which is important in parallel and distributed computing applications. All processors in such a system have a consistent view of memory, and if one processor modifies a memory location, all other processors accessing the same location will observe the updated value.

[0070] CXL 1.0 introduces three CXL protocols: CXL.io, CXL.cache, and CXL.mem. Particular reference is made herein to the CXL.mem protocol, which provides a host processor with access to device-attached memory using load / store commands. For example, the CXL.mem protocol may be used to issue load / store requests to fabric attached memory over a switch. A load request is a memory read access request to load a data item from memory. A store request is a memory' write access request to store a data item into memory.

[0071] Memory pooling between devices may also be enabled with CXL 1.0. That is, memory resources are consolidated in a central system, and components in the system such as hosts and accelerators etc. may request allocations in the pool, as and when needed. The available memory is therefore shared among different hosts, which improves efficiency of memory usage and improves scalability.

[0072] As discussed previously, CXL is an emerging technology, with many features being envisioned but not yet practically implemented. CXL 2.0 and 3.0 may, in future, enable configuration of systems that include accelerators and other types of device. By way of example, horizontal scaling of switches — i.e., switches linked to one or more further downstream switch — is also envisaged with CXL 3.0, enabling configuration of more complex topologies. CXL systems that provide memory' sharing functionality are also envisaged with CXL 3.0, allowing a plurality of hosts to access a memory location at the same time and still see up-to-date information at that location.

[0073] Devices suitable for use in a CXL system may be categorised into one of three classes. Class 1 devices include accelerators which typically lack a local memory. Class 2 devices include devices which have memory oftheir own (e.g., GPUs etc.). Class 3 devices include memory devices that are independent of the host’s main memory, and which can be attached via CXL to improve bandwidth and capacity. The host may access and manages device memory using the CXL.mem protocol. The present disclosure primarily relates to the use of Class 3 devices for use in providing a reliable fabric-attached memory system.

[0074] References herein to 'CXL devices’ may be understood as being CXL-enabled; i.e., devices that are equipped to operate under the CXL protocols. A CXL-enabled device may, for example, comprise CXL logic implemented on a port, such that communications from other devices under CXL protocols in a CXL-based system may be interpretable by the receiving device.

[0075] The above devices may already be connected in a system via PCIe protocol. However, the CXL protocols extend the capabilities of devices configured to operate under PCIe. .

[0076] Figure 2 shows a highly schematic diagram of a computer system operating under the CXL protocols. Figure 2 does not represent a system configured to implement the fabric-attached memory techniques of the present disclosure, but rather shows a computer topology that is configurable under CXL 1.0. Figure 2 shows a host device 210 connected to a plurality of switch devices 220. CXL allows cascaded switching, and it will be appreciated that a single host may be connected to more than the two switches shown in Figure 2.

[0077] Each switch 220 in Figure 2 is shown to be connected to a plurality of fabric-attached devices 230, which may comprise memory devices, network interface cards, host devices, accelerators, or any other CXL-compatible device falling under classes 1-3 described above.

[0078] Data striping and Redundancy

[0079] The present disclosure uses data striping for improving reliability of fabric-attached memory devices connected to processors by a switching fabric. Data striping techniques involve segmentation of data and subsequent storage of the segments across multiple memory devices. Accessing striped data therefore includes access requests to the multiple memory devices. These access requests may be made simultaneously across the memory devices, such that the throughput of data to the requesting device scales with the number of memory devices that are accessed. That is, striped data segments on different memory devices can be accessed in parallel, providing a net throughput equal to a cumulative throughput of the memory devices. The read rate scales with the number of memory devices that are simultaneously sending their stored segment(s).

[0080] It will be appreciated, as discussed further with reference to Figure 5, that balancing of processor speed and memory throughput to achieve optimal performance with minimal buffering may be done by careful selection of a particular number of fabric-attached memory devices to participate in a data striping scheme. That is, a quantity of memory devices may be selected such that the throughput of the memory devices in the striped data system is optimised for the particular processing speed of the host. Nevertheless, the techniques described herein are primarily directed to improving reliability of fabric-attached memory systems.

[0081] Whilst data striping techniques may improve data access throughput, memory error rate also scales with the number of memory devices.

[0082] Error handling is important in memory systems. One source of memory errors is background radiation, in the form of alpha particles, interacting with the system and causing individual bit flips. However, such bit flips may also occur due to, for example, electrical interference (e.g., channel noise), and other external factors, and can lead to unreliable and unstable performance if there is no provision for mitigating the impact of such errors. Furthermore, memory' errors may occur due to memory device degradation, as is explained in more detail later herein.

[0083] Error detection and correction mechanisms, such as parity checks, erasure codes, and error correction codes (ECCs) may be implemented to preserve data in the event of an error. However, different error handling techniques provide varying levels of protection against error. ECC is used in high-end systems, but further mitigation techniques such as redundancy may be required in larger systems, such as the aggregated fabric-attached memory systems described herein.

[0084] Redundancy techniques may be implemented in conjunction with data striping techniques to protect against the risk of data loss or corruption. The present inventors have recognized that data striping may be advantageously applied to fabric attached memory as described to address the technical problem of improving reliability in fabric attached memory.

[0085] Redundancy is the generation of overhead data to ensure data may be reconstructed in the event of an error.

[0086] Redundancy may advantageously be implemented in memory systems that use a striping algorithm. This is because memory systems that use a striping algorithm comprise multiple memory devices, and the likelihood of a memory error scales with the number of memory' devices.

[0087] Generation of overhead data improves the ability of a system to reconstruct erroneous data, at the expense of providing additional memory devices. However, such a compromise may be acceptable when a memory system already comprises multiple devices, such as in systems described herein.

[0088] In some examples of the present disclosure, five memory devices may be logically bundled for data striping. In such an example, data may be striped across four of the memory devices, and the fifth device may represent a redundant memory device which holds parity data for error correction purposes.

[0089] Described below are examples of data striping and redundancy techniques being implemented by an upstream port of a switch in respect of fabric-attached memory devices.

[0090] Upstream RAIM Embodiments

[0091] Reference is now made to Figures 3 and 4, which demonstrate writing and reading operations in a RAIM system, which may be implemented in some embodiments of the invention.

[0092] Figure 3 shows a store operation 300 issued, for example by a host such as a CPU, in respect of a write data item 320. The write data 320 is split into portions A - D (respectively denoted 322 328). each portion being a segment of the write data 320. The write data 320 may represent a pipeline state of a RAIM logic state machine configured to segment a data packet received from the upstream host for storing. That is, the striping instruction may be executed in a pipeline state machine of an upstream port in a switch. The instruction may cause the logic unit, on execution, to segment the write data into the portions 322-328, and to allocate respective port identifiers to the segments based on configuration data held in registers of the upstream port. The port identifiers are used to address ports which are attached to respective memory devices where the segments of the striped data are to be stored. In practice, the allocation of port identifiers to the segments may comprise a re-packetisation of each segment of data into a format supported for transmission through the switch. The port identifiers are used by routing logic of the switch to route the segments through the switching fabric to the correct downstream ports and hence the correct fabric attached memory' device. The ports (and their memory devices) may be configured as a subset of all the ports attached to the switch depending on the degree of striping that is to be utilised.

[0093] In the example of Figure 3, segments 322-238 of the write data 320 are respectively routed to and stored in memory devices 130a-130d.

[0094] Figure 3 further represents a parity generation operation 332, which generates redundant data from the write data 320 and subsequent storage of the redundant data in a redundant data memory device 130e. A port identifier is allocated to the redundant data for routing by the routing logic of the switch to a port attached to the redundant memory-- device 130e.

[0095] By way of example, erasure coding may be implemented for subsequent reconstruction of data in which an error is detected. Those skilled in the art will understand that erasure coding involves the transforming of a bit string into a longer ‘word’ such that the original message may be reconstructed from a subset of the longer word. Erasure coding therefore involves the generation of redundant data, as the longer word comprises a data overhead compared to the original bit string.

[0096] In general, the redundant data may comprise a parity bit and one or more additional bit string configured for error correction .Error checking and subsequent reconstruction may be implemented on reading the data, as described with reference to Figure 4.

[0097] Note that memory devices 130a-e in Figure 3 correspond to memory' devices 130a-e as represented in Figure 1.

[0098] Figure 4 represents a load operation in which data stored in a RAIM system is accessed to retrieve separately stored segments of the write data, checked for errors, optionally reconstructed if an error is detected, combined, and sent upstream to the requesting device.

[0099] Following from the store operation represented in Figure 3, a segmented data item is distributed across four exemplary memory devices 130a-130d, and an additional memory device 130e stores redundant data for error correction and data reconstruction in the event of a memory error. For example, erasure codes or the overhead data thereof may be stored in the redundant memory device 130e.

[00100] In response to a load operation issued, for example, by a host such as a CPU, each memory device 130a-130d may be accessed and the data segment held in each memory device may be transmitted upstream to a combine buffer 420, which may be provided in the switch.

[00101] As described above, the data segments are re-packaged as they are transmitted over the switch during a store operation, and the repackaging may include provision of one or more parity / error bit.

[00102] When a load operation is instigated, responsive to a memory read request, a memory device which holds one of the data segments to be read may indicate an error to the switch. For example the device may have failed such that the data it holds is no longer reliable. An error indicator is set against the relevant data segment.

[00103] In the example of Figure 4, data held in mcmon device 130d comprises an error, and the segment X 422 provided by memory device 130d to the memory buffer is corrupt.

[00104] In the example of Figure 4, wherein an error indicator is set on segment D, reconstruction logic may use data from the additional redundant data memory device 130e to reconstruct a corrected version 422 of segment D.

[00105] When correct versions of the each segment 322-326, 422, are available, the segments may be combined at a combine buffer 420 to assemble a read data item 410 corresponding to the data item 320 that was previously stored. Assembling the original, combined data item may further comprise re-packaging the data before transmission to the host, such that the data is in a packet format that the host is configured to operate on.

[00106] As discussed previously, combine logic may be implemented by a RAIM logic state machine, and the segments 322 326. and 422, and the read data item 410 may be considered pipeline states therein, having a distinct packet format compared to the data packet sent upstream to the host.

[00107] Reference is now made to Figure 5, which shows a highly schematic diagram representing registers for configuring a fabric attached memory system according to embodiments of the present disclosure.

[00108] Figure 5 represents a register for holding a RAIM-enable bit 510, and a plurality of member registers 520, each comprising a port ID, wherein each port is connected to a corresponding fabric attached member device. For example, member registers 521527 respectively corresponding to ports connected to fabric-attached members 1-4, and member register 529 corresponds to a port connected to fabric-attached member ‘P’.

[00109] Members 1-4 may represent respective fabric-attached memory devices, such as devices 130a-130d as shown in Figure 1. Member P may represent an additional redundancy memory device, such as device 130e of Figure 1.

[00110] The RAIM enable register 510 comprises a single bit to indicated whether or not RAIM logic is to be implemented. Only a single bit is required to indicate this binary state. The RAIM enable register 510 may be a configuration register, configured at boot and not updated during runtime.

[00111] The size, in bits, of a member register 520 must offer at least as many states as there are downstream connections on the switch. That is, if a switch comprises 8 downstream ports — for connecting to 8 respective downstream devices — each register (of which there must be 8, one for each downstream device), must comprise at least 3 bits, because 23 = 8. This allows a unique identifier to be provided in each register, for each of the 8 downstream devices. This can be seen in Figure 5, wherein the member registers comprise three bits.

[00112] In some examples, other devices than the member devices 1-4 and P may be connected downstream of the switch. In such an example, additional member registers 520 may be provided in the upstream port to accommodate the additional devices. Again, as the number of downstream connections increases, so too does the minimum size of the member registers, as each register needs to hold a value that can uniquely identify a port connected to a downstream device.

[00113]

[00114] Figure 5 represents a memory system comprising four memory devices across which a data item is striped, and a single additional memory device to which redundancy data is stored. Such a system may be referred to as a ‘4+1’ system.

[00115] It will be appreciated by those skilled in the art that the registers represented in Figure 5 may be altered to accommodate more, or fewer, fabric attached memory devices, with varying numbers of redundant memory devices. For example, a 3+1, 4+2, or 5+1 system may be configurable.

[00116] Reference is now made to Figure 6, which shows a highly schematic hardware diagram of the write process represented in Figure 3.

[00117] Figure 6 shows a write request 610 comprising a payload, the request 610 issued from the upstream port 121. The request 610 may be loaded into a request payload buffer 620, which may comprise segmentation of the payload by, for example, RAIM logic 124 implemented by a state machine. For clarity, the RAIM logic 124 is not shown in Figure 6.

[00118] Figure 6 shows data segments 322-328 loaded into the payload buffer 620. Each segment 322-328 may be repackaged by the RAIM logic 124 such that the segments are packetized in a format suitable for transmission across a switch crossbar 660. Each segment 322-328 may be considered a pipeline state.

[00119] The RAIM logic may further operate on the payload to generate parity 332, constructing a redundancy data packet 630 for storage in a redundant memory device (e.g., device 130e of Figure 1).

[00120] Each segment 322328 of the payload buffer 620, and a buffer or other structure comprising the redundant data packet 630, comprises a connection to a respective input of a write multiplexor (MUX) 640. The write MUX 640 is configured to output a single write data packet 650 along an output bus of the write MUX, the write data packet 650 being a data packed provided at one of the respective inputs to the write MUX; i.e., one of the segments 322-328 or the redundant data packet 630.

[00121] The write MUX 640 forwards a data packed provided to a particular one of the input lines, the particular input line selected based on a write sequence instruction 670 that acts as a selector signal to the write MUX 640. The sequencing instruction 670 may be issued by sequencing logic of the upstream port 121, such as by the RAIM logic 124.

[00122] The write packet 650 is transmitted across the switch crossbar 660 to a downstream port, and then to a connected downstream memory' device.

[00123] The sequencing instructions 670 may iteratively change the selector signal, causing the write MUX to forward a different input segment along the output bus across the switch crossbar to the correct downstream port. That is, each of the segments 322-328 and 630 may sequentially by forwarded by the write MUX 640 to the correct downstream port via the switch crossbar. It will be understood that the exact sequence order in which segments are forwarded is trivial.

[00124] Reference is now made to Figure 7, which shows a highly schematic hardware diagram of the read process represented in Figure 4.

[00125] Figure 7 shows the upstream port 121 issuing load sequence instructions 710 to the switch crossbar 660 to iteratively access data segments and redundancy data striped across the member memory modules. The load instructions may be received at the upstream port as a single read request, and split on the upstream port into separate requests, for accessing the different memory' devices across which the data to be read is striped. Address manipulation is conducted by a sequencer on the upstream port.

[00126] Based on the load sequencing instructions 710, a response data packet comprising a segment of the overall data item is transmitted across the switch crossbar 660 along an output line connected to a combine buffer 740 and a redundancy buffer or other structure 742 for holding a redundancy data packet. The response data retrieved from the crossbar 660 may be transmitted to all locations on both buffers 740, 742. However, write enable signals (controlled by member registers e.g., 520) ensure that only one corresponding location in the buffer 740, 742 is set, and therefore the read data is only loaded into the correct memory location.

[00127] The sequence instructions 710 may cause sequential accesses to be implemented at the downstream memory devices, until all of the segments and the redundancy data is loaded into the combine buffer 740 and redundancy buffer 742.

[00128] Error detection may be performed at the memory device, such that each packet output across the crossbar 660 is identified as comprising or not comprising an error. Reconstruction logic 750, controlled by error correction sequencing instructions 760 issued from the upstream port, may iteratively reconstruct data segments that are identified as comprising an error. In practice, the logic may generate multiple commands.

[00129] Corrected segments, and / or segments that did not comprise an error, may be forwarded from the combine buffer to a corrected data buffer 780 via a correction MUX array 770. That is, for each portion of the combine buffer 740, corresponding to a respective data segment accessed over the crossbar 660, a correction MUX may be provided.

[00130] Each correction MUX in the array 770 may comprise two input lines. A first input line may extend from a location in the combine buffer 740, and a second input line may extend from a memory location associated with the reconstruct logic 750, at which a corrected segment is compiled. The error correction sequencing instructions 760 may be configured to provide a selector signal to each MUX of the correction MUX array 770, selecting whether the data to be forwarded to the corrected data buffer 780 is to be provided by the first input line from the combine buffer location, or the second input line providing a reconstructed segment.

[00131] That is, the correction MUX array 770 may be provided such that segment data packets comprising no error may be forwarded direct from the combine buffer 740, as accessed across the crossbar 660. However, segment data packets that are loaded into the combine buffer with an error are not provided to the corrected data buffer 780 via the combine buffer input line. Instead, an input line providing a reconstructed, i.e., corrected, segment may instead by forwarded by the corresponding correction MUX into the corrected data buffer.

[00132] A data item 790 for forwarding to the upstream port 121 may be constructed based on the data in the corrected data buffer, using combine logic implemented by the RAIM logic state machine. That is, a re-packaging of the data may be implemented to configure the read data for transmission upstream to the host.

[00133] Reference is now made to Figures 8 and 9, which show implementation options which may be configured to operate in accordance with aspects of the present disclosure.

[00134] Figure 8 shows a computer architecture for implementing the above-described fabric-attached memory' techniques, in which a horizontal scaling for switches is implemented. That is, in Figure 8 a host is connected to a first switch 220a, and the first switch 220a is connected downstream to a second switch 220b. Tire first switch 220a is connected downstream to three memory devices 230, denoted A, B, and C. The second switch 220b is connected downstream to two memory’ devices 230, denoted D and P. The downstream memory devices are coupled via crossbar links 128 through the switches 220a, 220b.

[00135] In a topology such as that of Figure 8, an upstream port of the first switch 220a may be configured to implement the reliable fabric-attached memory techniques described herein. Tire horizontal link 810 between switches may act as part of a switch crossbar, and the second switch 220b may not be responsible for implementing RAIM techniques, for example. The second switch may merely forward requests issued by the upstream port on the first switch 220a.

[00136] Figure 9 shows a larger topology that may be implemented using CXL 3.0 functionalities. Figure 9 shows a plurality of host devices 210, each device connected to a switch 220, the switch having a plurality of downstream ports connected to a respective plurality of downstream devices 230 (e.g., memory devices).

[00137] Each distinct host-switch-device domain in the system of Figure 9 may be referred to as a sub-system.

[00138] Each switch 220 in each subsystem is horizontally connected to each of the other switches 220 in the other subsystems, as is enabled by CXL 3.0. In the example of Figure 9, in which there is a plurality of hosts 210, memory pooling techniques enabled by CXL 2.0 may also be used to allocated different memory locations in different subsystems to different hosts. That is, topological flexibility and horizontal scaling between subsystems such as those of Figure 9 may enable the above techniques to be implemented in a larger computer architecture, further making use of the memory pooling capabilities enabled by CXL 2.0.

[00139] As mentioned previously herein, device degradation may cause memory failure. Device degradation may cause such errors at a rate that is higher than is caused by other external factors such as alpha radiation. In some examples, data packets read from a degraded or faulty memory device may comprise errors unacceptably regularly. Such a memory device may even cause read errors in response to every read request. Tire present system may be suitably configured to enable so-called ‘hot swapping’ of memory devices in a RAIM system.

[00140] Hot-swapping in the present context refers to removal and replacement of a downstream memory device without shutting down or re-booting the system. When it is determined that a downstream memory device needs to be replaced, due to an unacceptable error rate, the following process may be implemented. Note that an unacceptable error rate may be determined using a threshold value as described later. The degraded memory device is unplugged from the system. A replacement memory device is plugged in and prepared as described with reference to Figure 10. In order for a replacement memory device to be effective the data in the replacement memory device needs to be rewritten. The data can be prepared according to one example using the same reconstruct logic which was used to generate corrected data as described above. A reconstruct sequencer 1000 may access all the memory devices in the group to perform a read of the data. The reconstruct sequencer may access these memory' devices via the switch crossbar 660 in the example shown in Figure 10. It will be appreciated that the reconstruct sequencer may be located in other places within the system and may access the memory devices in a different way. When the reconstruct sequencer is used to access the memory devices via the switch crossbar 660, as in the example of Figure 10, a multiplexor 1012 may be inserted to allow selective access to the switch crossbar 660 of either the load sequence instruction 710 from the upstream port (as in Figure 7) or the read request from the reconstruct sequencer 1000 as shown in Figure 10. The multiplexor may be controlled by a control signal 1010 from the reconstruct sequencer itself. The reconstruct control signal 1010 can be set to allow the reconstruct sequencer to have access to the memory devices via the switch crossbar via the multiplexor 1012 when the reconstruct sequencing process is implemented. This could be done as a single process to regenerate the data in the replacement memory device, or it could be set in idle cycles of the switch. In the second case, switch operation can continue, and the reconstruct control signal 1010 may be set in unused cycles of operation.

[00141] Data is read from all of the memory devices, including the new memory device which has been plugged in. Note that it is necessary to set the error bit on the new memory device to indicate that this is the device for which regenerated data is to be provided. This may be done by the reconstruct sequencer or by some other logic in the switch which is implemented during the reconstruction process. With this error bit set, data from the other mcmoiy devices (which are assumed to be non-faulty) are supplied to the reconstruct logic 750 and the reconstructed data is supplied to the switch crossbar as shown by arrow 1014 to be written back into the new memory device.

[00142] It will be understood that the read from the new memory' device in the above implementation is not needed. The data which is read from that memory' device is not useful. In an alternative implementation, the reconstruct sequencer could issue only four read requests, directed to the non-faulty memory devices. That is, it would read only the memory devices from which data was needed to regenerate the data to be written back to the new memory device. In that circumstance, the combined buffer needs to be advised that there is no requirement to wait for a read request to be fulfilled from the new memory' device. This could be done by setting an appropriate flag or using the error bit or in any other appropriate way.

[00143] The reconstruct sequencer may use known data scrubbing techniques to work through the addresses in the replacement memory device. The replacement / regeneration process may be transparent to the host, with the read-generate-write process being implemented without host intervention.

[00144] In practice, the system described herein may provide one or more counter register configured to provide an error counter for each downstream memory device. Each time a read request to a particular downstream memory device returns a data packed in which an error is identified, a value held in a corresponding counter register for the downstream memory device may be incremented.

[00145] When a threshold value is reached in an error counter register, or incrementation occurs at an unacceptable rate, a notification may be sent upstream that notifies the RAIM logic that a memory unit is in need for replacement. The system may be configured to notify a user of the system that such a replacement is required, and may be configured to implement the hot-swap process when the user removes and replaces the faulty memory device.

[00146] In another example, the memory device 230 itself may have error flagging logic. Figure 10 illustrates a memory device 230 with error flagging logic 231. The error flagging logic indicates when it is known that there is a device failure for that memory device 230. In that case, an error flag 232 may be supplied to a controller of the system to indicate that a ’hot swap’ process as described above should be initiated. Furthermore, the memory device may predict a failure ahead of time. That is, memory devices may provide a proactive indication that replacement is required.

[00147] As described above, the hot swap process involves the reconstruction or regeneration of the data in the replacement memory device using the existing reconstruct logic of the system. It will be appreciated, however, that the data could be regenerated in other ways, using the data from the associated memory devices in the group. This could be carried out offline and separate to the RAIM system until the replacement memon device is ready to be plugged in.

[00148] It is noted that while there is a failed memory device plugged in, there is no redundancy ability in the system, but the assumption is made that the remaining memory devices in the system will function without errors until an adequate replacement is made.

Claims

1. A memory access system comprising:a fabnc switch connectable between a first computer device which is configured to generate memory access requests and a plurality of downstream fabric-attached memory devices which are configured to respond to memory access requests, the fabric switch comprising:an upstream port connectable to the first computer device;a plurality of downstream ports, each of which is connectable to a respective downstream fabric-attached memory device; androuting logic configured to route data of each memory access request received at the upstream port to the plurality of downstream ports based on respective port identifiers,the memory access system comprising computer logic configured to segment a data item of a memory access request received at the upstream port into a plurality of segments,wherein the routing logic is configured to allocate each of the segments of the plurality of segments to a downstream port and to assign a corresponding port identifier to each segment for use by the routing logic.

2. The memory access system of claim 1, wherein the computer logic is further configured to generate from the data item a redundant data segment and wherein the routing logic is configured to allocate to the redundant data segment a port identifier for a downstream port connected to a redundant fabric-attached memory device.

3. The memory access system of claim 2, which comprises access logic configured, in responseto a read memory access request issued by the host device, to access the plurality of segments from the plurality of downstream fabric-attached memory devices and to reassemble the data item.

4. The memory' access system of claim 2, wherein the routing logic comprises sequencing instructions configured to iteratively route each of the plurality of segments over a crossbar of the fabric switch to respective ones of the downstream ports based on the port identifier.

5. The memory access system of claim 4, further comprising a write multiplexor configured to iteratively route each of the plurality of segments over the crossbar, wherein the write multiplexor comprises a plurality of input lines including a respective input line for each segment and the redundant data segment, and further comprising a control line configured to iteratively select an input line from which to forward data over the crossbar based on the sequencing instructions.

6. The memory access system of any preceding claim, wherein the fabric switch comprises a plurality of registers, the plurality of registers including a respective member register configured tostore port identifiers corresponding to each of the plurality of downstream ports.

7. The memory access system of claim 6, wherein the plurality of registers further includes a segmentation-enable register configured to indicate whether to execute the computer logic.

8. The memory access system of claim 3, wherein the computer logic is further configured to, based on detection of an error in a segment accessed from a downstream fabric-attached memory device:load each of the plurality of segments into a first buffer of the upstream port;using at least the redundant data segment and one or more segments in which no error is detected, reconstruct a corrected segment corresponding to the segment in which an error is detected;for each segment in which no error is detected, forward the segment in the first buffer to a second buffer of the upstream port; andfor the segment in which an error is detected, forward the corrected segment to the second buffer,wherein the computer logic is configured to reassemble the data item by applying reconstruction logic to the segments in the second buffer.

9. The memory access system of claim 8, wherein the fabric switch comprises a plurality of correction multiplexors, each correction multiplexor comprising a first input line from the first buffer and a second input line for receiving a corrected segment, and wherein the computer logic is configured to forward segments to the second buffer based on a control signal provided to each multiplexor by sequencing logic of the upstream port, the control signal indicating an input line from which data is to be forwarded to the second buffer.

10. The memory access system of any preceding claim, wherein the fabric switch is a first fabric switch, and a first downstream port of the first fabric switch is connectable to a fabric-attached memory device via a second fabric switch,wherein the first downstream port of the first fabric switch is connectable to a second port of the second fabric switch, andwherein the second fabric switch is configured to forward memory access requests received at the upstream port of the first fabric switch to a third port of the second fabric switch, the third port connectable to a fabric-attached memory device configured to respond to the memory7 access requests.

11. The memory access system of claim 2, wherein the redundant data segment comprises oneor more of error correction codes, syndrome codes and erasure codes.

12. The memory access system of any preceding claim, wherein the system operates in accordance with a Compute Express Link interconnect protocol.

13. The memory access system of any preceding claim, wherein the fabric switch is connectable to the downstream fabric-attached memory devices such that, on receipt of an indication that access requests from a first downstream fabric-attached memory device result in an error with a frequency exceeding an acceptable error frequency threshold, the first downstream fabric-attached memory' device is removable and replaceable with a second downstream fabric-attached memory device while the system remains in operation.

14. A method comprising:receiving a memory access request at an upstream port of a fabric switch, the fabric switch connectable between a first computer device configured to generate the memory access request and a plurality of downstream fabric-attached memory devices configured to respond to the memory access request;segmenting, by’ computer logic, a data item of the memory' access request received at the upstream port into a plurality of segments;allocating each of the segments of the plurality of segments to a downstream port by assigning a corresponding port identifier to each segment; androuting, by the upstream port of the fabric switch, each segment to a respective one of the plurality of downstream ports based on the allocated port identifier, each downstream port connectable to a respective downstream fabric-attached memory-’ device.

15. The method of claim 14, further comprising generating, by the computer logic, a redundant data segment from the data item, and allocating to the redundant data segment a port identifier for a downstream port connected to a redundant fabric-attached memory device.

16. The method of claim 15, further comprising, in response to a read memory access request issued by the host device, accessing the plurality of segments from the plurality of downstream fabric-attached memory devices and reassembling the data item.

17. The method of claim 15, further comprising iteratively routing, by sequencing logic of the computer logic, each of the plurality of segments over a crossbar of the fabric switch to respective ones of the downstream ports based on the port identifier.

18. The method of claim 17, further comprising iteratively routing, by a write multiplexor, each of the plurality of segments over the crossbar, wherein the write multiplexor comprises a pluralityof input lines including a respective input line for each segment and the redundant data segment, and further comprises a control line configured to iteratively select an input line from which to forward data over the crossbar based on the sequencing instructions.

19. The method of any of claims 14-18, further comprising storing port identifiers corresponding to each of the plurality of downstream ports in a plurality of registers, the plurality of registers including a respective member register for each of the downstream ports.

20. The method of claim 19, further comprising storing, in a segmentation-enable register, an indication of whether to execute the computer logic.

21. The method of claim 16, further comprising, based on detection of an error in a segment accessed from a downstream fabric-attached memory device:loading each of the plurality of segments into a first buffer of the upstream port;using at least the redundant data segment and one or more segments in which an error is not detected, reconstruct a corrected segment corresponding to the segment in which an error is detected that does not comprise an error with respect to a corresponding segment allocated to the downstream port;for each segment in which no error is detected, forwarding the segment in the first buffer to a second buffer of the upstream port;for the segment in which an error is detected, forwarding the corrected segment to the second buffer; andreassembling the data item by applying reconstruction logic to the segments in the second buffer.

22. The method of claim 21, further comprising:forwarding the segments to the second buffer based on a control signal provided to each of a plurality of correction multiplexors by sequencing logic of the upstream port, the control signal indicating an input line from which data is to be forwarded to the second buffer, each correction multiplexor comprising a first input line from the first buffer and a second input line for receiving a corrected segment.

23. The method of claim 15, wherein generating the redundant data segment comprises generating one or more of error correction codes, syndrome codes and erasure codes.

24. The method of any of claims 14-23, further comprising the use of a Compute Express Link interconnect protocol.

25. A transitory' or non-transitory computer readable media comprising instructions which, when executed by a processor of a computer device, cause the processor to implement a method according to any of claims 14-24.

Images