Processing system and method of operating a processing system

Abstract

A processing system and a method for operating the processing system are disclosed, wherein the processing system increases the total memory space logically available to a processor in a general purpose graphics processing unit (GPGPU) module by utilizing the memory space in a corresponding external memory module and by further utilizing a portion of the memory space in a plurality of other external memory modules to accommodate a terabyte-sized amount of data.

Description

Technical Field

[0001] The present disclosure relates to a processing system, and more particularly, to a processing system and a method of operating the processing system. Background Art

[0002] A graphics processing unit (GPU) is a well-known device for performing the calculations required to fill a frame buffer, which is used to display an image on a screen. The central processing unit (CPU) offloads the task of filling the frame buffer (which may be computationally intensive) to the graphics processing unit (GPU), thereby freeing up the central processing unit to perform other tasks in a timely manner.

[0003] A general-purpose graphics processing unit (GPGPU) is an extension of a graphics processor that can be programmed to perform other non-graphics processing computationally intensive operations. In artificial intelligence (AI) and machine learning applications, a central processing unit is often paired with multiple general-purpose graphics processing units (e.g., 100 GPGPUs) that perform convolution operations in parallel.

[0004] A general-purpose graphics processing unit includes a processor and a memory coupled to the processor. For many artificial intelligence and machine learning applications, a memory with large capacity and fast access speed is necessary. Therefore, when a general-purpose graphics processing unit is used for artificial intelligence and machine learning, a high-bandwidth memory (HBM) known for its large capacity and fast access speed is usually used as the memory of the general-purpose graphics processing unit.

[0005] A typical HBM consists of several dynamic random access memory (DRAM) dies stacked vertically on top of each other to provide large storage capacity in a small form factor, such as 4GB, 24GB, and 64GB. In addition, each DRAM die includes two 128-bit data channels to provide high bandwidth.

[0006] But one problem with current general-purpose GPUs is that when processing terabyte (TB) sized workloads, such as neural networks, recommendation systems, and database machine learning models with a large number of parameters, the performance of general-purpose GPUs is poor because the memory capacity of their HBM is much smaller than a TB byte.

[0007] Furthermore, the maximum capacity of memory (HBM) for general purpose graphics processing units is limited as it becomes increasingly difficult to stack dies vertically on top of each other, effectively limiting the number of dies that can be stacked to form an HBM and the maximum capacity of the HBM. Furthermore, each die in an HBM is typically manufactured with the largest reticle, which in turn limits the maximum size and capacity of the die.

[0008] In summary, since the memory (HBM) of a general-purpose GPU is limited by a limited memory capacity when processing a terabyte-sized workload, it is necessary to increase the memory capacity of the general-purpose GPU. Summary of the invention

[0009] The present disclosure increases the capacity of a memory of a general purpose graphics processing unit.

[0010] The present disclosure includes a processing system having a plurality of external memory units. Each of the plurality of external memory units has an extended memory and an interconnection circuit, the extended memory has a first plurality of memory addresses, the interconnection circuit has a second plurality of memory addresses, and the interconnection circuit has an extended controller and a plurality of ports coupled to the extended controller. Each port of a first external memory in the plurality of external memory units is coupled to another external memory unit in the plurality of external memory units, and a plurality of memory addresses are allocated from the second plurality of memory addresses. Each external memory unit also has a memory control circuit coupled to the extended memory and the interconnection circuit, the memory control circuit receives a memory request having a first memory address, and determines whether the first memory address falls within the first plurality of memory addresses or within the second plurality of memory addresses, when the first memory address falls within the first plurality of memory addresses, the memory control circuit forwards the first memory address to the extended memory, and when the first memory address falls within the second plurality of memory addresses, the memory control circuit forwards the first memory address to the interconnection circuit. When the first memory address falls within the second plurality of memory addresses, the expansion controller of the interconnect circuit receives the first memory address, determines a first port from a plurality of ports, and forwards the first memory address to the first port, wherein the first memory address is included in the plurality of memory addresses assigned to the first port.

[0011] The present disclosure also includes a method for operating a processing system. The method includes: receiving a memory request with a first memory address, and determining whether the first memory address falls within a first plurality of memory addresses associated with an extended memory, or falls within a second plurality of memory addresses associated with an interconnect circuit. The method also includes: when the first memory address falls within the first plurality of memory addresses, forwarding the first memory address to the extended memory, and when the first memory address falls within the second plurality of memory addresses, forwarding the first memory address to the interconnect circuit. The method also includes: when the first memory address falls within the second plurality of memory addresses, receiving the first memory address, determining a first port from a plurality of ports, the first memory address being included in a plurality of memory addresses assigned to the first port, and forwarding the first memory address to the first port.

[0012] The present disclosure also includes a non-transitory computer-readable storage medium having program instructions embedded therein, which, when executed by one or more processors of the device, drives the device to perform a method of operating a processing system. The method includes: receiving a memory request having a first memory address, and determining whether the first memory address falls within a first plurality of memory addresses associated with an extended memory, or falls within a second plurality of memory addresses associated with an interconnect circuit. The method also includes: forwarding the first memory address to the extended memory when the first memory address falls within the first plurality of memory addresses, and forwarding the first memory address to the interconnect circuit when the first memory address falls within the second plurality of memory addresses. The method also includes: receiving the first memory address when the first memory address falls within the second plurality of memory addresses; determining a first port from a plurality of ports, wherein the first memory address is included in a plurality of memory addresses assigned to the first port, and forwarding the first memory address to the first port.

[0013] The disclosed embodiments increase the total memory space logically available to the processor in a general-purpose graphics processing unit (GPGPU) module to accommodate TB-byte-sized data by utilizing the memory space in a corresponding external memory module and by further utilizing a portion of the memory space in multiple other external memory modules.

[0014] A better understanding of the features and advantages of the present disclosure will be obtained by referring to the following detailed description and accompanying drawings, which set forth illustrative embodiments in which the principles of the present disclosure are employed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their description are used to explain the present disclosure and do not constitute a limitation of the present disclosure.

[0016] Figure 1 is a block diagram of an exemplary processing system 100 provided according to an embodiment of the present disclosure.

[0017] Figure 2 is a block diagram of an exemplary general purpose graphics processing unit (GPGPU) module 200 provided according to an embodiment of the present disclosure.

[0018] Figure 3 is a block diagram of an exemplary external memory (SMX) module 300 provided according to an embodiment of the present disclosure.

[0019] Figure 4 is a flowchart of an exemplary method 400 for operating the processing system 100 provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] Figure 1 1 is a block diagram of an exemplary processing system 100 provided according to an embodiment of the present disclosure. As described in more detail below, the processing system 100 greatly increases the available memory space to accommodate TB-sized workloads by utilizing multiple external memory modules that are operatively coupled together, such that the available address space of the first external memory module includes a local address space physically located within the first external memory module and an extended address space physically located within the second and other external memory modules.

[0021] like Figure 1 As shown, the processing system 100 includes a central processing unit (CPU) module 110, a plurality of general purpose graphics processing unit (GPGPU) modules 112, and a plurality of external memory (SMX) modules 114. In this example, for simplicity, one CPU module, three GPGPU modules, and three SMX modules are shown, but other numbers of CPU modules, GPGPU modules, and SMX modules may also be used and configured arbitrarily. In this example, the SMX modules are identical.

[0022] Figure 1 As further shown, the processing system 100 also includes a peripheral component interconnect express (PCIe) bus 116 that couples the CPU module 110, the GPGPU module 112, and the SMX module 114. However, other buses similar to the PCIe bus 116 may also be used as long as the CPU module and the GPGPU module support it.

[0023] The PCIe bus 116 includes a plurality of endpoints, a root complex, and a plurality of PCIe switches 116-S. In this example, the GPGPU module 112 and the SMX module 114 are endpoints when receiving data, and the root complex, which is a separate circuit of the CPU module 110, performs address translation to identify one of the plurality of endpoints based on a memory request. Each PCIe switch 116-s is sequentially located between the CPU module 110 and a plurality of devices.

[0024] In this example, each GPGPU module 112 has a corresponding SMX module 114. The modules are coupled together via a PCIe switch 116-S and allow messages to be forwarded directly between the modules, e.g., from a GPGPU module to a corresponding SMX module and vice versa without having to go through a root complex. Optionally, the PCIe switch 116-S is omitted so that the GPGPU module 112 and the SMX module 114 are directly coupled to the root complex in the CPU module 110.

[0025] in addition, Figure 1 In the example shown, the processing system 100 also includes an interconnect structure 120 for coupling each SMX module 114 to each other SMX module 114. In this example, the interconnect structure 120 is implemented with optical fiber. As shown, the interconnect structure 120 is not connected to the CPU module 110 and the GPGPU module 112. However, although the SMX modules 114 are directly coupled to each other in the figure, other topologies such as daisy chains (1-2, 2-3, 3-4) can also be used instead.

[0026] The memory in the SMX module (e.g., SMX0 module) corresponding to the GPGPU module (e.g., GPGPU0 module) is an extension of the GPGPU memory. Part of the memory in other SMX modules (e.g., SMX1 and SMX2 modules) is an extension of the memory in the SMX0 module. If each SMX module has an additional capacity of 1TB (terabyte), then in the current example of three SMX modules, the processor in the GPGPU will no longer see only one, for example, 24GB of HBM memory, but can see a potential 3TB of memory.

[0027] Figure 2 2 is a block diagram of an exemplary general purpose graphics processing unit (GPGPU) module 200 provided according to an embodiment of the present disclosure. Each GPGPU module 112 may be implemented using a GPGPU module 200. Figure 2As shown, GPGPU module 200 includes core 210 , local cache 212 (having one or more levels) coupled to core 210 , and GPGPU memory 214 , such as high bandwidth memory (HBM), coupled to core 210 and cache 212 .

[0028] In operation, GPGPU memory 214 stores data (instructions and values) accessed by core 210. Cache 212 stores copies of some of the more frequently accessed data in GPGPU memory 214. Core 210 executes instructions and utilizes data stored in cache 212 and GPGPU memory 214.

[0029] GPGPU module 200 also includes a network on chip (NoC) 216 that couples core 210 and GPGPU memory 214 together to provide a high-bandwidth, high-speed communication path between core 210 and GPGPU memory 214. Cache 212 may also be coupled to local memory 214 via the network on chip 216. GPGPU module 200 also includes a PCIe endpoint controller 220 that is coupled to core 210 via the network on chip 216 to provide communication over a PCIe bus, such as PCIe bus 116.

[0030] Figure 3 is a block diagram of an exemplary external memory (SMX) module 300 provided according to an embodiment of the present disclosure. In this example, each SMX module 114 can be implemented with an SMX module 300. Figure 3 As shown, the SMX module 300 includes: a PCIe endpoint controller 310 that receives data from a PCIe bus such as the PCIe bus 116 and sends data to the PCIe bus; and a memory control circuit 312 that is coupled to the PCIe endpoint controller 310 to control data in and out of the memory space of the SMX module 300.

[0031] The SMX module 300 also includes an intra-module memory 314 having a local address space having a first plurality of memory addresses, and an interconnect circuit 316 that controls access to an interconnect structure (e.g., interconnect structure 120) to access an extended memory space having a second plurality of memory addresses. In this example, the interconnect circuit 316 is implemented using memory-over-fabric (MoF) circuitry on a fiber. Memory-over-fabric (MoF) refers to memory access implemented through a fiber.

[0032] The SMX module 300 also includes a conventional Advanced eXtensible Interface (AXI) bus 318 that couples the memory 314 and the circuit 316 to the memory control circuit 312. The AXI bus 318 includes five channels between each AXI node (memory control circuit 312, memory 314, and circuit 316): a read address channel, a read data channel, a write address channel, a write data channel, and a write response channel. The AXI bus protocol supports multiple master devices and multiple slave devices.

[0033] The total memory space of the SMX module 300 includes the local memory space of the memory 314 physically located within the SMX module 300, and the extended memory space physically located on one or more other SMX modules within the processing system. Combining the intra-module memory space and the extra-module memory space allows the SMX module 300 to have a logical memory range larger than its physical memory resources, wherein the extra-module memory space is mapped to the memory space of other SMX modules. In this example, each SMX module 300 stores only instructions and data for the GPGPU.

[0034] The interconnect circuit 316 includes an AXI interface 320, a controller 322 coupled to the AXI interface 320, and a routing table 324 coupled to the controller 322. The AXI interface 320 receives data from and sends data to the AXI bus 318, while the controller 322 controls the transmission of data through the interconnect circuit 316. In addition, the interconnect circuit 316 includes a plurality of transmit / receive ports P1-Pn (optical ports in this example), which are coupled to the controller 322 and other SMX modules in the processing system via an interconnect structure.

[0035] Figure 4 FIG. 4 is a flowchart of an exemplary method 400 for operating a processing system 100 according to an embodiment of the present disclosure. Figure 4 As shown, method 400 begins at 410, where the GPGPU module outputs a memory request with a requested memory address to an assigned SMX module, such as an SMX module corresponding to the GPGPU module, and the SMX module receives the request address. In this example, the GPGPU module outputs the memory request only to the SMX module corresponding to the GPGPU module. When the memory address is physically located on another SMX module, the GPGPU module is unaware.

[0036] Next, method 400 moves to 412, where memory control circuitry of the assigned SMX module receives a memory request from the GPGPU module (via a PCIe switch or root complex) and determines whether the requested address falls within an address associated with a local address space that is physically located on the assigned SMX module or within an address associated with an extended address space that is physically located in one or more remote SMX modules.

[0037] When the requested address falls within the address associated with the local memory space, the method 400 moves to 414, where the memory control circuitry of the assigned SMX module forwards the memory request to the intra-module memory of the assigned SMX module via an internal bus such as an AXI bus. The intra-module memory responds to the memory request by forwarding the read data or write confirmation back to the memory control circuitry of the assigned SMX module via the internal bus, and the memory control circuitry of the assigned SMX module forwards the read data or write confirmation back to the GPGPU module.

[0038] When the requested address falls within the addresses associated with the extended memory space of the remote SMX module, method 400 moves to 416, where the memory control circuit of the assigned SMX module determines the identification of the remote SMX module (which stores multiple addresses associated with the extended address space) based on the requested address and forwards the memory request and the remote identification to the interconnect circuit, such as interconnect circuit 316, via the internal bus.

[0039] Next, method 400 moves to 418, where a controller (e.g., controller 322) of the interconnect circuit of the assigned SMX module checks a routing table (e.g., routing table 324) to identify a port (e.g., port P1 or P2) to output the request, and outputs the memory request to the remote SMX module via the identified port and the interconnect structure (e.g., interconnect structure 120). Thus, the first GPGPU module can access data stored in the SMX module corresponding to the second GPGPU module.

[0040] Next, the method 400 then moves to 420, where a controller of the interconnect circuit of the remote SMX module receives the memory request via a port of the remote SMX module and then outputs the memory request to the in-module memory of the remote SMX module via an internal bus of the remote SMX module.

[0041] The in-module memory of the remote SMX module responds by forwarding the read data or write confirmation to the controller of the interconnection circuit of the remote SMX module via the internal bus of the remote SMX module, and the controller of the interconnection circuit of the remote SMX module forwards the read data or write confirmation back to the controller of the interconnection circuit of the assigned SMX module via the interconnection structure and the port of the assigned SMX module.

[0042] The controller of the interconnect circuit of the assigned SMX module then forwards the read data or write confirmation back to the memory control circuit of the assigned SMX module through the internal bus of the assigned SMX module, and then the memory control circuit forwards the read data or write confirmation back to the GPGPU module.

[0043] Each physical memory resource can be associated with multiple addresses, each address defining a data path. Figure 1 GPGPU0 in the embodiment can send a request for a memory location in the extended memory using the address A+x, where A is the local memory space SMX0 and x is the extended memory space physically located in SMX1. The request is then output to SMX0 via SWITCH0 and then passed to SMX1 via the interconnect structure 120.

[0044] Alternatively, GPGPU0 may send a request for a memory location in extended memory using address B, where B is physically located in the local memory space of SMX1, and then output the request to SMX0 via SWITCH0, which passes the request to the root complex of CPU110 and then sends it to SMX1 via SWITCH1.

[0045] The graph data of the graph neural network is divided and stored in the SMX module. GPGPU can access the graph data in peer mode (by registering the memory extension page table with the GPGPU or using the driver to directly manage the memory resources). Each GPGPU logically sees multiple copies of the graph that are kept in sync, where each device stores a complete graph. Each GPGPU can specify the route by using the most appropriate address (for example, through the local PCI switch and fiber structure for better performance) to access the graph data.

[0046] One of the advantages of the present invention is that the present invention significantly increases the size of the memory space potentially available to the GPGPU, for example, from gigabytes of HBM to multiple terabytes. Another advantage of the present invention is that the present invention obtains a storage space that is larger than physically present through virtualization, thereby saving costs by misleading the GPGPU module into believing that the data is always in the assigned SMX module, so that all modules work in pairs and do not generate too much congestion on the PCIe side. The disclosed embodiment utilizes a custom link connecting the interconnect circuit to obtain additional bandwidth. In addition, the present invention provides cross-GPGPU graph data access, so that the GPGPU can access data in other storage modules based on the knowledge that the interconnect circuit 316 handles request forwarding internally. Although there is a delay associated with expanding the memory space, the delay can be kept to a minimum by utilizing PCIe switches and fiber optic transmission.

[0047] Reference has now been made in detail to various embodiments of the present disclosure, examples of which are shown in the accompanying drawings. Although described in conjunction with various embodiments, it should be understood that these various embodiments are not intended to limit the present disclosure. On the contrary, the present disclosure is intended to cover substitutions, modifications and equivalents included within the scope of the present disclosure as interpreted according to the claims.

[0048] In addition, in the above detailed description of various embodiments of the present disclosure, many specific details are set forth in order to provide a thorough understanding of the present disclosure. However, those of ordinary skill in the art will recognize that the present disclosure can be implemented without these specific details or in the case of their equivalents. In other embodiments, in order to avoid unnecessarily obscuring aspects of the various embodiments of the present disclosure, well-known methods, processes, components and circuits are not described in detail.

[0049] It should be noted that although the method may be described as a sequence of operations for clarity, the described sequence of operations is not necessarily a prescribed order of operations. It should be understood that some operations may be skipped, performed in parallel, or performed without requiring a strict order to be maintained.

[0050] The drawings showing various embodiments of the present disclosure are semi-diagrammatic and not to scale, and in particular, some dimensions are for clarity of presentation and are exaggerated in the drawings. Similarly, although the perspectives of the drawings generally depict similar directions for ease of description, in most cases, such descriptions in the drawings are arbitrary. Generally, various embodiments according to the present disclosure can operate in any direction.

[0051] Some portions of the detailed description may be presented in terms of procedures, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are used by those skilled in the data processing arts to effectively convey the substance of their work to others skilled in the art.

[0052] In the present disclosure, a process, logic block, procedure, etc. is considered to be a self-consistent sequence of operations or instructions leading to a desired result. These operations are those physical operations utilizing physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transmitted, combined, compared, and otherwise manipulated in a computing system. Sometimes, primarily for common reasons, for convenience, these signals are referred to as transactions, bits, values, elements, symbols, characters, samples, pixels, or the like.

[0053] It should be remembered, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise clearly indicated from the following discussion, it will be understood that throughout this disclosure, discussions using terms such as "generate," "determine," "allocate," "aggregate," "utilize," "virtualize," "process," "access," "execute," "store," etc. refer to actions and processes of a computer system or similar electronic computing device or processor.

[0054] A processing system, or similar electronic computing device or processor operates and transforms data represented as physical (electronic) quantities within a computer system memory, registers, other such information storage, and / or other computer-readable media into other data similarly represented as physical quantities within a computer system memory or registers or other such information storage, transmission, or display device.

[0055] With reference to the accompanying drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application have been clearly and completely described in the previous section. It should be noted that in the specification and claims of the present invention and in the above-mentioned drawings, the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or order. It should be understood that these numbers can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in a different order than shown or described here.

[0056] The functions described in the embodiments of the present disclosure, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computing device readable storage medium. Based on such an understanding, a part of the embodiments of the present disclosure that contributes to the prior art or a part of the technical solution can be implemented in the form of a software product stored in a storage medium, including a plurality of instructions for enabling a computing device (which may be a personal computer, server, mobile computing device or network device, etc.) to perform all or part of the steps of the method described in the various embodiments of the present application. The above-mentioned storage medium includes a USB drive, a portable hard disk, a read-only memory (ROM), a random access memory (RAM), a disk, an optical disk, etc. that can store program code.

[0057] The various embodiments in the specification of the present application are described in a progressive manner, and each embodiment focuses on its difference from other embodiments, and the same or similar parts between the various embodiments can refer to another case. The described embodiments are only a part of the embodiments, not all embodiments of the present application. Without departing from the inventive concept of the present invention, all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present application are within the scope of the present application.

[0058] The above embodiments are only used to illustrate rather than limit the technical solutions of the present invention. Although the present disclosure is described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the above embodiments may still be modified or some or all of the technical features therein may be replaced by equivalents. These modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions in the embodiments of the present invention.

[0059] It should be understood that the above description is an example of the present disclosure, and various alternatives of the present disclosure described herein can be used when practicing the present disclosure. For example, various modules can be implemented as cards. Therefore, it is hoped that the above claims define the scope of the present invention and cover structures and methods within the scope of these claims and their equivalents.

Claims

1. A processing system comprising: A plurality of external memory units, each of the plurality of external memory units having: an expansion memory having a first plurality of memory addresses; an interconnect circuit having a second plurality of memory addresses, the interconnect circuit having an expansion controller and a plurality of ports coupled to the expansion controller, each port of a first external memory unit of the plurality of external memory units being coupled to another external memory unit of the plurality of external memory units and being allocated a number of memory addresses from the second plurality of memory addresses; a memory control circuit coupled to the expansion memory and the interconnect circuit, the memory control circuit receiving a memory request having a first memory address, determining whether the first memory address falls within the first plurality of memory addresses or within the second plurality of memory addresses, forwarding the first memory address to the expansion memory when the first memory address falls within the first plurality of memory addresses, and forwarding the first memory address to the interconnect circuit when the first memory address falls within the second plurality of memory addresses, When the first memory address falls within the second plurality of memory addresses, the expansion controller of the interconnection circuit receives the first memory address, determines a first port from the plurality of ports, and forwards the first memory address to the first port, wherein the first memory address is included in the plurality of memory addresses allocated to the first port.

2. The processing system of claim 1, further comprising a plurality of general purpose graphics processing units corresponding to the plurality of external memory units, each general purpose graphics processing unit being coupled to a corresponding external memory unit; a system bus coupled to the plurality of general purpose graphics processing units and the plurality of external memory units, each external memory unit configured to receive the memory request from one of the plurality of general purpose graphics processing units via the system bus; an internal bus coupled to the memory control circuit, the expansion memory, and the interconnection circuit; An interconnect structure is coupled to the plurality of external memory units, the interconnect structure not being connected to any one of the plurality of general purpose graphics processing units.

3. The processing system of claim 2, wherein: A first general purpose graphics processing unit among the plurality of general purpose graphics processing units outputs a memory request only to an external memory unit corresponding thereto, or accesses data stored in an external memory unit corresponding to a second general purpose graphics processing unit among the plurality of general purpose graphics processing units; The plurality of external memory units only store program instructions and data processed by the plurality of general purpose graphics processing units.

4. The processing system of claim 2, wherein: The system bus includes a Peripheral Component Interconnect Express (PCIe) bus, the internal bus includes an Advanced eXtensible Interface (AXI) bus, and the interconnect structure includes an optical fiber.

5. A method of operating a processing system, comprising: receiving a memory request having a first memory address; determining whether the first memory address falls within a first plurality of memory addresses associated with an expansion memory or within a second plurality of memory addresses associated with an interconnect circuit; forwarding the first memory address to the expansion memory when the first memory address falls within the first plurality of memory addresses, and forwarding the first memory address to the interconnect circuit when the first memory address falls within the second plurality of memory addresses; receiving the first memory address when the first memory address falls within the second plurality of memory addresses; Determine a first port from a plurality of ports, wherein the first port is allocated with a plurality of memory addresses including the first memory address; as well as The first memory address is forwarded to the first port.

6. The method of claim 5, wherein: The memory request is received by an external memory unit of a plurality of external memory units from a corresponding one of a plurality of general purpose graphics processing units coupled to the plurality of external memory units.

7. The method of claim 6, wherein: A first general-purpose graphics processing unit of the plurality of general-purpose graphics processing units only outputs a memory request to an external memory unit corresponding thereto or accesses data stored in an external memory unit corresponding to a second general-purpose graphics processing unit of the plurality of general-purpose graphics processing units.

8. The method of claim 6, wherein the plurality of external memory units store only program instructions and data processed by the plurality of general purpose graphics processing units.

9. The method of claim 6, further comprising: receiving the memory request from one of the plurality of general purpose graphics processing units via a system bus coupled to the plurality of general purpose graphics processing units and the plurality of external memory units; coupling memory control circuitry, the expansion memory, and the interconnection circuitry together via an internal bus; as well as coupling the plurality of external memory units together via an interconnect structure that is not connected to any of the plurality of general purpose graphics processing units, The system bus includes a peripheral component interconnect express (PCIe) bus, the internal bus includes an advanced extensible interface (AXI) bus, and the interconnect structure includes an optical fiber.

10. A non-transitory computer-readable storage medium having program instructions embedded therein, which, when executed by one or more processors of a device, cause the device to perform a method of operating a processing system, the method comprising: receiving a memory request having a first memory address; determining whether the first memory address falls within a first plurality of memory addresses associated with an expansion memory or within a second plurality of memory addresses associated with an interconnect circuit; forwarding the first memory address to the expansion memory when the first memory address falls within the first plurality of memory addresses, and forwarding the first memory address to the interconnect circuit when the first memory address falls within the second plurality of memory addresses; receiving the first memory address when the first memory address falls within the second plurality of memory addresses; Determine a first port from a plurality of ports, wherein the first port is allocated with a plurality of memory addresses including the first memory address; as well as The first memory address is forwarded to the first port.

11. The non-transitory computer-readable storage medium of claim 10, wherein: The memory request is received by an external memory unit of a plurality of external memory units from a corresponding one of a plurality of general purpose graphics processing units coupled to the plurality of external memory units.

12. The non-transitory computer-readable storage medium of claim 11, wherein: A first general-purpose graphics processing unit of the plurality of general-purpose graphics processing units outputs only a memory request to an external memory unit corresponding thereto, or accesses data stored in an external memory unit corresponding to a second general-purpose graphics processing unit of the plurality of general-purpose graphics processing units; The plurality of external memory units only store program instructions and data processed by the plurality of general purpose graphics processing units.

13. The non-transitory computer-readable storage medium of claim 11, further comprising: receiving the memory request from one of the plurality of general purpose graphics processing units via a system bus coupled to the plurality of general purpose graphics processing units and the plurality of external memory units; coupling memory control circuitry, the expansion memory, and the interconnection circuitry together via an internal bus; as well as The plurality of external memory units are coupled together via an interconnect structure that is not connected to any of the plurality of general purpose graphics processing units.

Images