Processing system, method of operating a processing system, and computer-readable storage medium

Abstract

A processing system, a method of operating a processing system, and a computer-readable storage medium are disclosed. The system includes at least one external memory and at least one general-purpose graphics processor coupled to the external memory, each general-purpose graphics processor having a core, a cache coupled to the core, and a local memory coupled to the core. The general-purpose graphics processor moves first data in the local memory of the general-purpose graphics processor to the external memory when a number of accesses to the first data in the local memory of the general-purpose graphics processor is less than a threshold value for a period of time, and moves second data in the external memory to the local memory of the general-purpose graphics processor when a number of accesses to the second data in the external memory exceeds the threshold value for the period of time. By moving infrequently accessed data from a faster access memory to a slower access memory, the maximum capacity of the faster access memory can be increased.

Description

Technical Field

[0001] The present invention relates to a processing system, and more particularly, to a processing system, a method of operating the processing system, and a computer-readable storage medium. Background Technology

[0002] A graphics processing unit (GPU) is a well-known device that performs the necessary computations to fill data into a frame buffer, which is then used to display images on the screen. The central processing unit (CPU) delegates the computationally intensive task of filling the frame buffer to the GPU, freeing up the CPU to perform other tasks.

[0003] A general-purpose graphics processing unit (GPGPU) is an extension of a graphics processing unit (GPU). GPGPUs are programmable to perform other non-graphics processing computationally intensive operations. In artificial intelligence (AI) and machine learning applications, a central processing unit (CPU) is typically paired with multiple GPGPUs that perform convolution operations in parallel (e.g., 100 GPGPUs).

[0004] A general-purpose graphics processing unit (GPU) consists of a processor and memory coupled to the processor. For many artificial intelligence and machine learning applications, large-capacity, high-speed memory is essential. Therefore, when a GPU is used for artificial intelligence and machine learning, high-bandwidth memory (HBM), known for its large capacity and high access speed, is typically used as the GPU's memory.

[0005] A typical HBM consists of several dynamic random access memory (DRAM) dies stacked vertically on top of each other to provide large storage capacities in a small form factor, such as 4GB, 24GB, and 64GB. Furthermore, each DRAM die includes two 128-bit data channels to provide high bandwidth.

[0006] However, one problem with current general-purpose graphics processors (GPUs) is that their HBM (Hardware Body Module) effectively has a maximum capacity, which in turn limits the number of operations that a GPU can process in a given time. This maximum capacity exists because it becomes increasingly difficult to vertically stack dies on top of each other, effectively limiting the number of dies that can be stacked to form the HBM and its maximum capacity. Furthermore, each die in the HBM is typically manufactured using a maximum crosshair pattern, which limits the maximum size and capacity of the die.

[0007] Furthermore, aside from the maximum capacity, all or part of other memory (such as a portion of CPU memory) cannot be used as an extension of the general-purpose graphics processor's HBM, thus failing to provide additional fast-access memory capacity. This is because the general-purpose graphics processor is coupled to the extended memory (such as CPU memory) via the Peripheral Component Interconnect Fast (PCIe) bus, and accessing data via the PCIe bus is approximately 100 times slower than accessing data in the HBM, which is too slow for some artificial intelligence and machine learning applications.

[0008] Therefore, since the general-purpose graphics processor memory (HBM) has a maximum capacity, and since all or part of another memory cannot be used as an extension of the general-purpose graphics processor's HBM to provide additional fast-access memory capacity, it is necessary to increase the memory capacity in the general-purpose graphics processor. Summary of the Invention

[0009] This disclosure improves the maximum capacity of memory that can be accessed quickly in a GPGPU module. The processing system provided by this disclosure includes at least one external memory and at least one general-purpose graphics processor coupled to the at least one external memory. Each of the at least one general-purpose graphics processor has a core, a cache coupled to the core, and local memory coupled to the core. Each general-purpose graphics processor requires a first duration to access data stored in its cache, a second duration longer than the first duration to access data stored in its local memory, and a third duration longer than the second duration to access data stored in at least one external memory.

[0010] Within a certain period of time, when some data in the local memory of the general-purpose graphics processor is accessed less than a threshold number of times, the general-purpose graphics processor moves this data from the local memory of the general-purpose graphics processor to at least one external memory, and when some data in at least one external memory is accessed more than the threshold number of times within that period of time, this data is moved from at least one external memory to the local memory of the general-purpose graphics processor. Each external memory is used only to store program instructions and data to be executed by the general-purpose graphics processor.

[0011] This disclosure also includes a method for operating a processing system. The method includes accessing data stored in a cache of a general-purpose graphics processor for a first duration, accessing data stored in local memory of the general-purpose graphics processor for a second duration longer than the first duration, and accessing data stored in at least one external memory for a third duration longer than the second duration.

[0012] The method further includes: moving certain data from the local memory of the general-purpose graphics processor (GPRS) to external memory when the number of accesses to certain data in the GPRS's local memory is less than a threshold within a certain period of time; and moving certain data in the external memory from the external memory to the GPRS's local memory when the number of accesses to certain data in the external memory exceeds the threshold within the same period of time. Each external memory is used only to store program instructions and data to be executed on the GPRS.

[0013] The present invention also includes a non-transitory computer-readable storage medium in which program instructions are embedded, which, when executed by one or more processors of the device, cause the device to perform a method of operating the processing system.

[0014] The method includes accessing data stored in the cache of a general-purpose graphics processor for a first duration, accessing data stored in the local memory of the general-purpose graphics processor for a second duration longer than the first duration, and accessing data stored in at least one external memory for a third duration longer than the second duration.

[0015] The method further includes: moving certain data from the local memory of the general-purpose graphics processor (GPRS) to external memory when the number of accesses to certain data in the GPRS's local memory is less than a threshold within a certain period of time; and moving certain data in the external memory from the external memory to the GPRS's local memory when the number of accesses to certain data in the external memory exceeds the threshold within the same period of time. Each external memory is used only to store program instructions and data to be executed on the GPRS.

[0016] The features and advantages of the present invention will be better understood by referring to the following detailed description and accompanying drawings, which illustrate exemplary embodiments of applying the principles of the invention. Attached Figure Description

[0017] The accompanying drawings of this application are provided to further understand this application and constitute a part of this application. The exemplary embodiments and descriptions of this application are used to explain this application and do not constitute a limitation thereof.

[0018] Figure 1 This is an exemplary block diagram of the processing system 100 provided in this disclosure.

[0019] Figure 2 This is an exemplary block diagram of the general-purpose graphics processing unit (GPGPU) module 200 provided in this disclosure.

[0020] Figure 3 This is an exemplary block diagram of the external memory (SMX) module 300 provided in this disclosure.

[0021] Figure 4 This is an exemplary flowchart provided in this disclosure for illustrating a method 400 for configuring a processing system 100.

[0022] Figure 5 This is an exemplary block diagram provided in this disclosure for illustrating the partitioning and allocation of memory in a processing system 100.

[0023] Figure 6 This is an exemplary flowchart of a method 600 for operating a general-purpose graphics processor provided in this disclosure.

[0024] Figure 7 This is an exemplary flowchart of the method 700 for operating SMX provided in this disclosure. Detailed Implementation

[0025] Figure 1 This is an exemplary block diagram of the processing system 100 provided by the present invention. As will be described in detail below, the processing system 100, which requires a large capacity and fast access speed of memory, increases the maximum capacity of the high-speed memory by adding memory resources that are too slow to meet the demand, and then moves infrequently accessed data from the high-speed memory to the slow memory resources.

[0026] like Figure 1 As shown, the processing system 100 includes a central processing unit (CPU) module 110, multiple general-purpose graphics processing unit (GPGPU) modules 112, and multiple external memory (SMX) modules 114. In this example, for simplicity, one CPU module, three GPGPU modules, and three SMX modules are shown. However, other numbers of CPU modules, GPGPU modules, and SMX modules can be applied and configured arbitrarily.

[0027] Further as Figure 1 As shown, the processing system 100 also includes a peripheral component interconnect fast (PCIe) bus 116 that couples the CPU module 110, the GPGPU module 112, and the SMX module 114 together. The PCIe bus 116 includes multiple endpoints and a root complex, and may include any number of PCIe switches.

[0028] In this example, GPGPU module 112 and SMX module 114 are endpoints when receiving data, while the root complex is a discrete circuit on CPU module 110 used to perform address translation to identify the endpoints. Furthermore, when using a PCIe switch, two or more endpoints can be coupled to the PCIe switch, which in turn is coupled to the root complex. The PCIe switch allows devices coupled to the same PCIe switch to send messages directly without going through the root complex.

[0029] In addition, multiple direct connections 118, such as dedicated data links, can be used alternately instead of the PCIe bus 116. In some cases, each GPGPU module 112 is directly connected to one or more SMX modules 114. Dedicated data links are faster than the PCIe bus, especially when a large number of devices are coupled to and access the PCIe bus, but dedicated data links still add significant latency to memory accesses performed via dedicated data links.

[0030] Therefore, memory access requests via dedicated data links remain too slow to meet many high-speed memory access demands. Thus, while pairing the SMX module 114 with the GPGPU module 112 via link 118 could provide better bandwidth performance, pairing is not mandatory. The GPGPU module 112 and the SMX module 114 can be coupled together in various ways.

[0031] For example, each SMX module 114 can be mapped to multiple GPGPU modules 112. Furthermore, multiple SMX modules 114 can be mapped to a single GPGPU module 112. Additionally, in locations where some GPGPU modules 112 are not paired with SMX modules 114, six GPGPU modules 112 and two SMX modules 114 can be used, but the SMX modules 114 can still be accessed through the upper-layer switch or root complex.

[0032] Figure 2 This is an exemplary block diagram of the general-purpose graphics processing unit (GPGPU) module 200 provided by the present invention. In this example, each GPGPU module 112 can be implemented using the GPGPU module 200. Figure 2 As shown, the GPGPU module 200 includes a core 210, a local cache 212 (level 1 or level 2 cache) coupled to the core 210, and a local memory 214 coupled to the core 210 and the cache 212.

[0033] To meet the demands of artificial intelligence (AI) and machine learning, local memory 214 has a large capacity, such as gigabytes, and is accessed very quickly. Local memory 214 stores data frequently accessed by core 210. Cache 212 stores copies of some more frequently accessed data in local memory 214. Core 210 executes instructions stored in cache 212 and local memory 214 sequentially. Core 210 needs to access data stored in cache 212 for a first duration, and access data stored in local memory 214 for a second duration that is longer than the first duration.

[0034] In one example, local memory 214 is implemented as high-bandwidth memory (HBM), which includes multiple dynamic random access memory (DRAM) dies stacked vertically head-to-head to provide a large storage capacity with a small form factor. For example, the maximum capacity of HBMs could be 4GB, 24GB, and 64GB. Furthermore, each die in the HBM has two 128-bit data channels to provide high bandwidth.

[0035] The GPGPU module 200 also includes an on-chip network (NoC) 216 that couples the core 210 and local memory 214 together to provide a high-bandwidth, high-speed communication path between the core 210 and local memory 214. The cache 212 can also be coupled to local memory 214 via the on-chip network (NoC) 216. The GPGPU module 200 also includes a PCIe endpoint controller 220 coupled to the core 210 via the on-chip network (NoC) 216 to provide communication via a PCIe bus (e.g., PCIe bus 116).

[0036] Core 210 executes instructions stored in cache 212 and memory 214, including application startup instructions, program instructions, and data. Program instructions frequently access some data stored in cache 212 and memory 214, and infrequently access other data stored in memory 214. Furthermore, these program instructions frequently generate data, which, after being stored in memory 214, is frequently accessed in some cases and infrequently in others.

[0037] During operation, if the number of times certain data stored in local memory 214 is accessed is less than a threshold within a certain period of time, the GPGPU module 200 moves this data from local memory 214 to the SMX module, and if the number of times certain data stored in the SMX module is accessed exceeds the threshold within the same period of time, the data is moved from the SMX module to local memory 214.

[0038] Figure 3 This is an exemplary block diagram of the external memory (SMX) module 300 provided in this disclosure. In this example, each SMX module 114 can be implemented using the SMX module 300. Figure 3As shown, the SMX module 300 includes an extended memory 310 for storing data, a memory control circuit 312 coupled to the extended memory 310 to control data entry and exit from the memory 310, and a PCIe endpoint controller 314 coupled to the extended memory 310 and the memory control circuit 312 to control communication via a PCIe bus (e.g., PCIe bus 116). The extended memory 310 stores data that the GPGPU core does not frequently access and only stores the GPGPU's program instructions and data. Furthermore, the SMX module 300 has registers for storing device information such as bandwidth and access time.

[0039] The SMX module 300 is accessed via a PCIe bus or PCIe switch, which significantly increases the memory access time of the extended memory 310. Therefore, even if the extended memory 310 is implemented using very fast memory (e.g., HBM-type memory), remote access to the extended memory 310 is slower than access to local memory (HBM) 214. Consequently, core 210 requires a third time, longer than the second time, to access data stored in the SMX module.

[0040] In operation, CPU module 110 initially loads the GPGPU's program instructions and data into the GPGPU module's local memory 214, and then moves the GPGPU's program instructions and data to the SMX module's extended memory 310 when the instructions and data become less frequently used. Alternatively, CPU module 110 may load a first number of program instructions and data for an application into the GPGPU module's local memory 214 and a second number of program instructions and data for the application into the SMX module's extended memory 310 before executing the application. Neither the GPGPU module's local memory 214 nor the SMX module's extended memory 310 stores the application's program instructions and data until CPU module 110 loads the first and second numbers of program instructions.

[0041] Figure 4 This is an exemplary flowchart provided in this disclosure for illustrating a method 400 for configuring a processing system 100. For example... Figure 4 As shown, method 400 begins at step 410, determining the additional high-speed memory capacity required by each GPGPU module 112 / 200. In some cases, each GPGPU module 112 / 200 requires or is assumed to require additional high-speed memory capacity of the same size; however, in other cases, each GPGPU module 112 / 200 may require additional high-speed memory capacity of a different size.

[0042] Subsequently, method 400 moves to 412 to determine the additional, available memory capacity from SMX modules 114 / 300. SMX module 300 has registers storing device information such as capacity, bandwidth, and access time. To determine availability, method 400 moves to step 412-1 to first divide the extended memory 310 on each SMX module 300 into available and unavailable memory spaces.

[0043] Next, method 400 moves to step 412-2 to identify multiple partitioned memory spaces available in each available memory space. For example, 64GB of available memory space can be divided into two 32GB partitioned memory spaces or four 16GB partitioned memory spaces.

[0044] Following this, method 400 moves to 414 to allocate the available memory space and / or one or more partitioned memory spaces of the extended memory 310 of the SMX modules 114 / 300 to the GPGPU modules with matching needs. For example, if the available memory space of the SMX module matches the additional high-speed memory space required by the GPGPU module, the SMX module is allocated to the GPGPU module.

[0045] If the available memory space of the SMX module is divided into two partitioned memory spaces that match the additional high-speed memory space required by the two GPGPU modules, then one partitioned memory space on the SMX module will be allocated to the first GPGPU module, and the other partitioned memory space on the SMX module will be allocated to the second GPGPU module.

[0046] Configuration input can be obtained through the driver's programmable interface or configuration files. For example, APIs and configuration files can contain information such as the GPGPU's memory expansion capacity, bandwidth, and access time requirements without the user's knowledge. The driver determines how best to partition and allocate the expanded memory based on the physical device registers and link topology. Depending on the user's software / hardware requirements, the driver may fail and provide feedback. Configuration can be static, completed before system use, or dynamic, configured during system use.

[0047] Figure 5 This is an exemplary block diagram provided in this disclosure for illustrating the partitioning and allocation of memory in a processing system 100. For example... Figure 5As shown, the available memory space in the SMX module SMX0 is divided into two parts, with the first part being unavailable (spare) and the second part allocated to the GPGPU module GPGPU0. Therefore, GPGPU0 can quickly access HBM0 to read and write data, while the second part of the available memory space in SMX0 can be accessed slowly to read and write data.

[0048] The available memory space in the SMX module SMX1 is also divided into two parts. The first part is allocated to the GPGPU module GPGPU1, and the second part is allocated to the GPGPGPU module GPGPU2. Therefore, GPGPU1 can quickly access HBM1 to read and write data, and can access the first part of the available memory space in SMX1 slowly, while GPGPU2 can access the second part of the available memory space in SMX1 slowly. Furthermore, the available memory space in the SMX module SMX2 is not partitioned but is also allocated to GPGPU2, which receives additional memory space from both SMX modules.

[0049] Figure 6 This is an exemplary flowchart of a method 600 for operating a general-purpose graphics processor as provided in this disclosure. Figure 6 As shown, method 600, starting at 610, loads program instructions and data as memory blocks and pages into the local memory of the GPGPU module. Additionally, method 600 resets multiple counters and a timer for measuring a predetermined time period.

[0050] Method 600 then moves to step 612 to determine whether a memory cell, such as a memory block or memory page, in local memory has been accessed. Each memory cell has an associated counter, which is reset in step 610. When a memory cell is accessed, method 600 moves to step 614 to increment the count value held by the counter for the number of times the memory cell has been accessed, and then moves to step 616 to determine whether a predetermined time period has elapsed.

[0051] When the predetermined time period has elapsed, method 600 proceeds to step 618 to identify memory cells that were accessed less than a threshold number of times within the predetermined time period. Then, method 600 proceeds to step 620 to move the memory cells accessed less than the threshold number of times from local memory (214) to extended memory (310) in the corresponding SMX module. After the memory cells are moved, method 600 resets the counters and timers and returns to step 612 to determine again whether a memory access has occurred.

[0052] Any copies of the storage cells generated during the movement of data to the extended memory 310 in the corresponding SMX module are removed (e.g., overwritten, pointer deleted), so that after the storage cells are moved to the extended memory 310 in the corresponding SMX module and retained only in the extended memory 310 in the corresponding SMX module, the cache 212 or local memory 214 no longer stores copies of the corresponding data in the extended memory 310.

[0053] Therefore, by moving infrequently accessed data (access count less than a threshold) from the high-speed local memory (HBM) used to meet the system's high-speed access count requirements to the slower extended memory in the corresponding SMX module, which cannot meet the system's remote memory access count requirements, the embodiments of this disclosure effectively increase the memory capacity of the high-speed local memory (HBM).

[0054] Infrequent access to the slow extended memory in the corresponding SMX module can still result in significant time loss. However, occasional data retrieval from the slow extended memory of the corresponding SMX module is tolerable, provided the frequency is low enough.

[0055] Regarding the data loading in step 610, program instructions and data for the GPGPU that the developer knows in advance will not be accessed frequently can be marked and identified. During the same period as loading the data into the high-speed access GPGPU's local memory (HBM), it can be alternately loaded into the slower extended memory in the corresponding SMX module. Furthermore, regardless of whether the data is loaded into high-speed or slow memory, some data can also be marked with a never-moving tag to pin this data to a specific memory location, regardless of its corresponding access frequency.

[0056] Figure 7 This is an exemplary flowchart of the method 700 for operating SMX provided in this disclosure. Figure 7 As shown, method 700 is similar to method 600, with the main difference being steps 718 and 720, in which storage cells that have been accessed more than a threshold are identified and moved from the slow extended memory 310 of the corresponding SMX module back to the high-speed local memory (HBM) 214 of the GPGPU module.

[0057] The accessed data is moved from the high-speed GPGPU's local memory (HBM) to the corresponding SMX module's slower extended memory (or vice versa), and can be returned to the CPU before being moved. Alternatively, the accessed data can be returned to the CPU after being moved.

[0058] Various embodiments of the present disclosure have been described in detail above with reference to the exemplary accompanying drawings. However, although various embodiments have been described above, it should be understood that these embodiments are not intended to limit the present disclosure. Rather, the present disclosure is intended to cover alternatives, modifications, and equivalents that may be included within the scope of protection of the present disclosure as interpreted by the claims.

[0059] Furthermore, numerous specific details have been set forth in the foregoing detailed description of various embodiments of this disclosure to provide a thorough understanding of the disclosure. However, those skilled in the art will recognize that various embodiments of this disclosure may be practiced without these specific details or their equivalents. Moreover, well-known methods, processes, components, and circuits have not been described in detail in these embodiments to avoid unnecessarily obscuring some aspects of the various embodiments of this disclosure.

[0060] Note that although the method is described here as a series of steps for clarity, these steps are not necessarily in a prescribed order. It should be understood that some steps can be skipped, executed in parallel, or performed without needing to maintain a prescribed order.

[0061] The accompanying drawings provided with the various embodiments of this application are semi-illustrative and not drawn to scale; in particular, some dimensions are enlarged in the drawings for clarity. Similarly, although the viewpoints in the drawings generally have similar orientations for ease of description, such orientations are arbitrarily chosen in most cases. Generally, the various embodiments of this disclosure can be implemented in any orientation.

[0062] The specific descriptions above can be presented in the form of programs, logic blocks, processes, and other symbolic representations of operations on data bits in computer memory. Those skilled in the field of data processing can use these descriptions and representations to effectively communicate the essence of their work to others skilled in the field.

[0063] In embodiments of this disclosure, processes, logic blocks, and handling are considered as self-consistent sequences of operations or instructions that lead to desired results. These operations are physical operations utilizing physical quantities. Typically, though not always, these physical quantities exist in the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated in a computing system. It has been found that, for convenience, these signals are often referred to as transactions, bits, values, elements, symbols, characters, samples, pixels, etc., for common usage.

[0064] However, it should be remembered that all these and similar terms are associated with appropriate physical quantities and are merely convenient labels applicable to those physical quantities. Unless otherwise explicitly stated in the following discussion, it should be understood that in this disclosure, discussions using terms such as “generate,” “determine,” “allocate,” “aggregate,” “utilize,” “virtualize,” “process,” “access,” “execute,” and “store” refer to the actions and processes of a computer system or similar electronic computing device or processor.

[0065] A processing system, similar electronic computing device, or processor manipulates and converts first data, characterized as physical (electronic) quantities stored in a memory, register, other such information storage, and / or other computer-readable medium in a computer system, into second data, characterized as similar physical (electronic) quantities stored in a memory, register, other such information storage, transmission, or display device in a computer system.

[0066] The technical solutions of the various embodiments of this disclosure have been clearly and completely described above in conjunction with the accompanying drawings. It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific sequence or order. It should be understood that these numbers can be interchanged where appropriate so that the various embodiments of this disclosure described herein can be implemented in orders other than those shown or described herein.

[0067] The functions described in the various embodiments of this disclosure, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, a portion of the prior art or a portion of the technical solutions in the various embodiments of this disclosure may be embodied in the form of a software product in a storage medium, including a series of instructions for inducing a computing device (which may be a personal computer, server, mobile computing device, or network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes: a USB drive, portable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, optical disk, etc., capable of storing program code.

[0068] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on its differences from other embodiments. Similar or identical parts between embodiments may be referred to in other embodiments. However, the described embodiments are only a portion of the embodiments, and not all embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the various embodiments of this specification without departing from the inventive concept are within the protection scope of this application.

[0069] The above embodiments are for illustrative purposes only and are not intended to limit the technical solutions of this disclosure. Although this disclosure has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be equivalently substituted. These modifications or substitutions will not cause the substance of the corresponding technical solutions to deviate from the scope of the technical solutions provided by the embodiments of this disclosure.

[0070] It should be understood that the above description is exemplary and various alternatives described herein may be used in implementing this application. Various integrations are templated. For example, each GPGPU does not need to be implemented as a module. Therefore, the claims are intended to define the scope of this application and thereby cover the structures and methods within the scope of these claims and their equivalents.

Claims

1. A processing system, comprising: At least one external memory; and At least one general-purpose graphics processor (GPU) coupled to at least one of the external memories, each GPU having a core, a cache coupled to the core, and local memory coupled to the core, wherein the GPU requires a first duration to access data stored in the GPU's cache, requires a second duration longer than the first duration to access data stored in the GPU's local memory, and requires a third duration longer than the second duration to access data stored in the external memory. Within a certain period of time, when the number of accesses to first data in the local memory of the general-purpose graphics processor is less than a threshold, the general-purpose graphics processor moves the first data from the local memory of the general-purpose graphics processor to at least one of the external memories, and when the number of accesses to second data in at least one of the external memories exceeds a threshold within that period of time, the second data is moved from at least one of the external memories to the local memory of the general-purpose graphics processor. Each external memory is used only to store program instructions to be executed by the general-purpose graphics processor and data whose access frequency is less than a threshold within that period of time.

2. The processing system according to claim 1, wherein, One of the external memories is mapped to one of the general-purpose graphics processors.

3. The processing system according to claim 1, wherein, Multiple external memories are mapped to a single general-purpose graphics processor.

4. The processing system of claim 1 further includes multiple direct connections, such that each of the general-purpose graphics processors is directly connected to one or more of at least one of the external memories.

5. The processing system according to claim 1, wherein, The external memory is partitionable, so that one or more of the general-purpose graphics processors are allocated a partition.

6. The processing system of claim 1, wherein the general-purpose graphics processor is used for: Each time a storage cell in the local memory of the general-purpose graphics processor is accessed, the first counter is incremented; Identify storage units that have been accessed less than a threshold number of times within this period; Storage units that are accessed less than a threshold number of times within this period will be moved from the local memory of the general-purpose graphics processor to the external memory. Determine when a memory cell in the external memory is accessed; The second counter is incremented each time a storage cell in the external memory is accessed. Identify storage units that have been accessed more than a threshold number of times within this period; as well as Storage units that have been accessed more than a threshold number of times within that period will be moved from the external memory to the local memory of the general-purpose graphics processor.

7. The processing system of claim 1, further comprising a central processing unit coupled to at least one of the external memories and at least one of the general-purpose graphics processors, the central processing unit being configured to load a first plurality of program instructions of the application into the local memory of the general-purpose graphics processor and load a second plurality of program instructions of the application into the external memory before the application is executed, wherein the local memory and the external memory of the general-purpose graphics processor do not store program instructions of the application before the central processing unit loads the first plurality of program instructions and the second plurality of program instructions.

8. A method for operating a processing system, the method comprising: The system needs to access data stored in the cache of the general-purpose graphics processor for a first duration, access data stored in the local memory of the general-purpose graphics processor for a second duration longer than the first duration, and access data stored in external memory for a third duration longer than the second duration. Within a certain period of time, when the number of accesses to first data in the local memory of the general-purpose graphics processor is less than a threshold, the general-purpose graphics processor moves the first data from the local memory of the general-purpose graphics processor to at least one of the external memories, and when the number of accesses to second data in at least one of the external memories exceeds a threshold within that period of time, the second data is moved from at least one of the external memories to the local memory of the general-purpose graphics processor. Each external memory is used only to store program instructions to be executed by the general-purpose graphics processor and data whose access frequency is less than a threshold within that period of time.

9. The method according to claim 8, wherein, Each of the external memories is mapped to a plurality of the general-purpose graphics processors.

10. The method according to claim 8, wherein, Multiple external memories are mapped to a single general-purpose graphics processor.

11. The method of claim 8, further comprising: Each of the general-purpose graphics processors is directly connected to one or more of the external memories.

12. The method according to claim 8, further comprising: The external memory is partitioned so that one or more of the general-purpose graphics processors are assigned to a partition.

13. The method of claim 8, further comprising: Each time a storage cell in the local memory of the general-purpose graphics processor is accessed, the first counter is incremented; Identify storage units that have been accessed less than a threshold number of times within this period; Storage units that are accessed less than a threshold number of times within this period will be moved from the local memory of the general-purpose graphics processor to the external memory. Determine when a memory cell in the external memory is accessed; The second counter is incremented each time a storage cell in the external memory is accessed. Identify storage units that have been accessed more than a threshold number of times within this period; as well as Storage units that have been accessed more than a threshold number of times within that period will be moved from the external memory to the local memory of the general-purpose graphics processor.

14. The method of claim 8, further comprising: Before the application is executed, a first plurality of program instructions of the application are loaded into the local memory of the general-purpose graphics processor, and a second plurality of program instructions of the application are loaded into the external memory. Before the central processing unit loads the first plurality of program instructions and the second plurality of program instructions, the local memory and the external memory of the general-purpose graphics processor do not store the program instructions of the application.

15. A non-transitory computer-readable storage medium having embedded program instructions that, when executed by one or more processors of a device, cause the device to perform a method of operating a processing system, the method comprising: Access data stored in the cache of the general-purpose graphics processor for a first duration, access data stored in the local memory of the general-purpose graphics processor for a second duration longer than the first duration, and access data stored in external memory for a third duration longer than the second duration. Within a certain period of time, when the number of accesses to first data in the local memory of the general-purpose graphics processor is less than a threshold, the general-purpose graphics processor moves the first data from the local memory of the general-purpose graphics processor to at least one of the external memories, and when the number of accesses to second data in at least one of the external memories exceeds a threshold within that period of time, the second data is moved from at least one of the external memories to the local memory of the general-purpose graphics processor. Each external memory is used only to store program instructions to be executed by the general-purpose graphics processor and data whose access frequency is less than a threshold within that period of time.

16. The non-transitory computer-readable storage medium according to claim 15, wherein, At least one of the external memories is mapped to one of the general-purpose graphics processors.

17. The non-transitory computer-readable storage medium according to claim 15, wherein, Multiple external memories are mapped to a single general-purpose graphics processor.

18. The non-transitory computer-readable storage medium according to claim 15, wherein, The method further includes: directly connecting each of the general-purpose graphics processors to one or more of at least one of the external memories.

19. The non-transitory computer-readable storage medium of claim 15, further comprising: The external memory is partitionable, so that one or more of the general-purpose graphics processors are allocated a partition.

20. The non-transitory computer-readable storage medium of claim 15, wherein the method further comprises: Before the application is executed, a first plurality of program instructions of the application are loaded into the local memory of the general-purpose graphics processor, and a second plurality of program instructions of the application are loaded into the external memory. Before the central processing unit loads the first plurality of program instructions and the second plurality of program instructions, the local memory and the external memory of the general-purpose graphics processor do not store the program instructions of the application.

Images