A multi-card inter-remote memory access page migration method and device, electronic equipment and storage medium

Abstract

This invention discloses a page migration method, apparatus, electronic device, and storage medium for remote memory access between multiple GPUs. It relates to the field of computer technology. The page migration method for remote memory access between multiple GPUs includes: acquiring access data of each computing card to the memory address segment through a hardware statistics unit; determining the access frequency of each memory address segment based on the access data; reporting the page information corresponding to the memory address segment whose access frequency meets preset conditions to the software layer; and the software layer migrating the corresponding page from the current source computing card's video memory to the target computing card's video memory based on the reported page information. This invention is applicable to scenarios where multiple GPUs share data and remote access is frequent.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a page migration method, apparatus, electronic device and storage medium for remote memory access between multiple cards. Background Technology

[0002] General-Purpose Graphics Processing Units (GPGPUs) are widely used in high-performance computing fields such as artificial intelligence training and scientific computing due to their powerful parallel computing capabilities. In multi-GPU collaborative computing scenarios, multiple GPGPUs are interconnected through inter-card interconnect links (such as C2C Link, NVLink, etc.) to jointly complete large-scale computing tasks.

[0003] In a multi-GPU shared memory architecture, a memory page might be stored in the video memory of one GPU but frequently accessed by computation tasks on another GPU. If the location of the memory page is determined solely by the software based on the initial configuration and cannot be dynamically adjusted according to the actual access patterns during runtime, an unreasonable memory allocation may occur. This means that remote access is frequent while local access is infrequent, causing remote computations to consistently obtain data using inefficient remote access methods, severely impacting overall computational efficiency.

[0004] Therefore, the overall computing efficiency of existing multi-card systems is relatively low. Summary of the Invention

[0005] In view of this, this application provides a page migration method, apparatus, electronic device and storage medium for remote memory access between multiple cards, which can improve the overall computing efficiency of multi-card systems.

[0006] Firstly, this application provides a page migration method for remote memory access between multiple cards, including: Access data of each computing card to the memory address range is obtained through a hardware statistics unit; The access frequency of each memory address segment is determined based on the access data; The page information corresponding to the memory address range whose access popularity meets the preset conditions is reported to the software layer. The software layer migrates the corresponding page from the source computing card's memory to the target computing card's memory based on the reported page information.

[0007] Optionally, before obtaining the access data of each computing card to the memory address segment through the hardware statistics unit, the method further includes: configuring each statistical entry of the hardware statistics unit and specifying the memory address segment monitored by each statistical entry.

[0008] Optionally, the statistical entries include the address segment register, the local access counter, the first counter, and the second counter.

[0009] Optionally, obtaining access data of each computing card to the memory address segment through the hardware statistics unit includes: When it is detected that the target address of a memory access request falls into the memory address segment corresponding to the target statistics entry, the counter in the target statistics entry is incremented according to the source of the access request.

[0010] Optionally, the step of incrementing the counter corresponding to the target statistics entry based on the source of the access request includes: If the access request originates from the local computing card, then the local access counter for the target statistical entry is incremented; If the access request originates from the first interconnect port of the remote computing card, then the first counter of the target statistical entry is incremented; If the access request originates from the second interconnect port of the remote computing card, then the second counter of the target statistics entry is incremented.

[0011] Optionally, the access data includes at least: The number of times the remote computing card accesses the memory address range through the inter-card interconnection link and the number of times the local computing card accesses the memory address range.

[0012] Optionally, determining the access frequency of each memory segment based on the access data includes: The hardware statistical unit polls each statistical entry at a preset time interval, reads the count value of each counter, and clears it to zero. For each statistical entry, the remote access count value is subtracted from the local access count value, and the difference is used as the access popularity of the corresponding memory address segment in the current time window; The memory address segments whose access popularity exceeds a preset threshold within the current time window, or whose access popularity ranks among the top N memory address segments in all the statistically analyzed memory address segments, are determined to meet the preset conditions.

[0013] Optionally, the target computing card is a computing card whose remote access popularity to the page meets preset migration conditions.

[0014] Optionally, migrating the corresponding page from the current source compute card memory to the target compute card memory that issued the primary remote access request includes: The software layer selects the computing card whose remote access count value meets the preset migration conditions as the target computing card based on the remote access count value of each remote computing card to the target page. The target page is migrated from the source compute card's video memory to the target compute card's video memory; After the migration is complete, update the global address mapping of the multi-card system and notify all compute cards of the address mapping change.

[0015] Optionally, the method further includes: The hardware statistics unit distinguishes access from different remote computing cards by recording the physical ports through which the access requests pass, and the software layer restores the identity of the computing card from which the access originates based on the mapping relationship between the physical port and the computing card identifier.

[0016] Secondly, this application also provides a page migration device for remote memory access between multiple cards, comprising: The acquisition module is used to acquire access data of each computing card to the memory address segment through the hardware statistics unit; The access popularity determination module is used to determine the access popularity of each memory address segment based on the access data; The reporting module reports the page information corresponding to the memory address range whose access popularity meets the preset conditions to the software layer; The migration control module, in the software layer, migrates the corresponding page from the source computing card's video memory to the target computing card's video memory based on the reported page information.

[0017] Optionally, the acquisition module includes: The counting submodule is used to increment the counter in the target statistics entry according to the source of the access request when the target address of the memory access request falls into the memory address segment corresponding to the target statistics entry.

[0018] Optionally, the heat determination module includes: The numerical statistics submodule is used to poll each statistical entry of the hardware statistics unit at a preset time interval, read the count value of each counter and clear it to zero; The heat calculation submodule is used to subtract the local access count from the remote access count for each statistical entry, and the difference is used as the access heat of the corresponding memory address segment in the current time window. The condition determination submodule is used to determine the memory address segments whose access popularity exceeds a preset threshold within the current time window, or whose access popularity ranks among the top N memory address segments in all the statistically analyzed memory address segments, as meeting the preset condition.

[0019] Optionally, the migration control module includes: The target selection submodule is used by the software layer to select the computing card whose remote access count value meets the preset migration conditions as the target computing card based on the remote access count value of each remote computing card to the target page. The migration execution submodule is used to migrate the target page from the source compute card's video memory to the target compute card's video memory; The address update submodule is used to update the global address mapping of the multi-card system after the migration is completed, and to notify all compute cards of the address mapping change.

[0020] Thirdly, this application also provides an electronic device, which includes: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside the space enclosed by the housing, and the processor and the memory are disposed on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the page migration method for remote memory access between multiple cards as described in any of the first aspects above.

[0021] Fourthly, this application also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the page migration method for remote memory access between multiple cards as described in any of the first aspects.

[0022] The page migration method, apparatus, electronic device, and storage medium for remote memory access between multiple GPUs provided in this application acquire access data of memory address segments from each computing card through a hardware statistics unit, and then determine the access frequency of each memory address segment based on the access data. Next, the page information corresponding to the memory address segments whose access frequency meets preset conditions is reported to the software layer, and the software layer migrates the corresponding pages from the source computing card's video memory to the target computing card's video memory. This overcomes the defect in the prior art that cannot dynamically adjust the page storage location according to the actual access load during the operation of multiple GPUs. It can accurately identify pages that are frequently accessed by remote computing cards but rarely accessed locally, and promptly migrate them to computing cards whose actual usage frequency meets preset migration conditions. This transforms high-latency remote access into low-latency local access, significantly improving the overall computing efficiency of the multi-GPU system. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart of a page migration method for remote memory access between multiple cards provided in this application embodiment; Figure 2 This is a schematic diagram of the multi-card computing system structure provided in an embodiment of this application; Figure 3 A schematic diagram of step S10 of a page migration method for remote memory access between multiple cards provided in an embodiment of this application; Figure 4 A schematic diagram of step S20 of a page migration method for remote memory access between multiple cards provided in an embodiment of this application; Figure 5 A schematic diagram of step S40 of a page migration method for remote memory access between multiple cards provided in an embodiment of this application; Figure 6 A schematic diagram of a page migration device for remote memory access between multiple cards provided in an embodiment of this application; Figure 7 This is a schematic diagram of an electronic device architecture provided in an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0026] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0027] See Figure 1 and Figure 2 This application provides a page migration method for remote memory access between multiple cards, including the following steps: S10. Obtain the access data of each computing card to the memory address segment through the hardware statistics unit.

[0028] Specifically, with Figure 2For example, a multi-card computing system includes multiple computing cards that communicate with each other via a C2C interconnect link. Each computing card includes an on-chip network 14, a hardware statistics unit, video memory 151, and an inter-card interconnect interface, including a first interconnect port 25 and a second interconnect port 26. The hardware statistics unit is located between the on-chip network 14 and the video memory 151 and is used to monitor all memory access requests passing through this path. The hardware statistics unit includes one or more statistical entries, each corresponding to a memory address segment to be monitored.

[0029] Optionally, before step S10, the method further includes: configuring each statistical entry of the hardware statistics unit and specifying the memory address range monitored by each statistical entry.

[0030] Specifically, to achieve precise monitoring of specific memory regions, during system initialization or operation, the software layer writes configuration information to the hardware statistics units of each computing card, specifying the memory address range monitored by each statistical entry. The software layer writes the start and end addresses corresponding to each statistical entry into the address segment register of that statistical entry. The software layer can configure any number of statistical entries as needed, with each statistical entry monitoring an independent memory address range.

[0031] This embodiment, by pre-configuring the memory address ranges monitored by each statistical entry, enables small-scale, high-precision statistics on specific address regions in multi-card memory access scenarios, avoiding interference from irrelevant memory accesses on the statistical results and improving the accuracy and effectiveness of access heat judgment.

[0032] See Figure 2 In some embodiments, each statistics entry includes the following registers and / or counters: Address segment register 21 is used to store the memory address segment information monitored by this statistical entry, such as the start address and end address; Local access counter 22 is used to count the number of times the local computing card accesses the memory address range; The first counter 23 is used to count the number of accesses from the remote computing card through the first interconnect port 25 of the card interconnect interface; The second counter 24 is used to count the number of accesses from the remote computing card via the second interconnect port 26 of the card interconnect interface.

[0033] Specifically, this embodiment sets an address segment register and three types of access counters in each statistical entry, so that each statistical entry can independently count the number of local accesses and accesses from different remote interconnect ports, providing basic data for the accurate calculation of access popularity in the future.

[0034] Optionally, step S10 specifically includes: when the hardware statistics unit detects that the target address of a memory access request falls into the memory address segment corresponding to a certain statistical entry, the hardware statistics unit increments the counter corresponding to the statistical entry according to the source of the access request.

[0035] Specifically, the hardware statistics unit compares the target address of a memory access request with the address range stored in the address segment register of each statistics entry to determine whether the request hits a statistics entry. If it does, it further identifies whether the access request originates from the computing core of the local computing card or from a remote interconnect port and increments the corresponding counter.

[0036] This embodiment achieves zero-overhead real-time filtering of memory access requests by comparing the target address with the address range stored in the address segment register in real time, thus ensuring the real-time nature and accuracy of the statistics.

[0037] See Figure 3 The specific rules for incrementing the counter corresponding to the target statistical entry in step S10 are as follows: S11. If the access request originates from the local computing card, that is, the access initiated by the computing core of this card, then the local access counter 22 of the target statistical entry is incremented. S12. If the access request originates from the first interconnect port 25 of the remote computing card, then the first counter 23 of the target statistical entry is incremented. S13. If the access request originates from the second interconnect port 26 of the remote computing card, then the second counter 24 of the target statistical entry is incremented.

[0038] This embodiment, by distinguishing the incremental operation of the source, can accurately differentiate between local access and access from different remote ports, providing a reliable data foundation for accurate identification of the target page in the future.

[0039] Optionally, the access data includes at least: the number of times the remote computing card accesses the memory address range through the inter-card interconnection link and the number of times the local computing card accesses the memory address range.

[0040] Specifically, in each statistical entry of the hardware statistics unit, the first counter 23 and the second counter 24 record the number of times the remote computing card accesses the memory address segment through the inter-card interconnection link, and the local access counter records the number of times the local computing card accesses the memory address segment. After reading these count values, the software layer can obtain the bidirectional access data between the remote and local terminals.

[0041] This embodiment uses a hardware statistics unit to simultaneously collect remote access counts and local access counts, providing complete data support for subsequent calculation of net access demand using the difference.

[0042] S20. Determine the access frequency of each memory address segment based on the access data.

[0043] See Figure 4 In some embodiments, step S20 includes: S21. Poll the statistical entries of the hardware statistical unit at preset time intervals, read the count value of each counter and clear it to zero.

[0044] Specifically, the software layer polls each statistical entry of the hardware statistics unit at a preset time interval. In one embodiment, the preset time interval is 10ms, that is, the software layer reads the current value of all counters in each statistical entry every 10ms, and immediately clears these counters after reading, thereby using 10ms as a time window to count the access frequency of each memory address segment within the time window.

[0045] S22. For each statistical entry, subtract the local access count from its remote access count, and use the difference as the access popularity of the corresponding memory address segment within the current time window.

[0046] Specifically, after reading the count values ​​of each counter, the software layer subtracts the local access count value from the sum of the remote access count value (i.e., the first counter 23 and the second counter 24) for each statistical entry. The difference is used as the access frequency of the corresponding memory address segment within the current time window. This difference represents the net access demand of the remote computing card for the memory address segment: when the difference is positive and large, it indicates that remote access is much more than local access, and the data on the memory address segment should be migrated to the remote end to optimize access performance; when the difference is negative or close to zero, it indicates that local access is dominant or the demand for remote access is not high, and there is no need to trigger migration.

[0047] S23. The memory address segments whose access popularity exceeds a preset threshold within the current time window, or whose access popularity ranks among the top N memory address segments in all the statistically analyzed memory address segments, are determined to meet the preset conditions.

[0048] Specifically, the software layer determines the access frequency of each memory address segment within the current time window. Memory address segments whose access frequency exceeds a preset threshold (e.g., threshold equals 10) or ranks in the top N (e.g., N=5) are identified as meeting preset conditions, i.e., hot pages that require page migration. The preset threshold and the value of N can be flexibly adjusted according to the actual application scenario, and this application does not impose specific limitations on them.

[0049] In summary, this embodiment obtains the net access demand by subtracting the local access count from the remote access count, which can accurately identify the main users of the target page and avoid misjudging pages that are frequently accessed locally but rarely accessed remotely as hot pages that need to be migrated.

[0050] Optionally, the target computing card is a computing card whose remote access popularity to the page meets preset migration conditions.

[0051] Specifically, when multiple remote computing cards access the same page simultaneously, one of them needs to be selected as the target card for migration. The selection criteria for the target computing card are: the computing card whose remote access popularity to the page meets the preset migration conditions. The preset migration conditions can be any one of the following: remote access popularity exceeding a preset threshold, ranking first in remote access popularity, or ranking in the top N. In a preferred embodiment, the computing card whose remote access popularity meets the preset migration conditions is selected as the target computing card to maximize the performance benefits brought by migration.

[0052] This embodiment clarifies the selection criteria for the target computing card by setting preset migration conditions, thus providing clear decision-making standards for the migration operation.

[0053] S30. Report the page information corresponding to the memory address segment whose access popularity meets the preset conditions to the software layer.

[0054] Specifically, the software layer determines the access popularity of each memory address segment within the current time window, identifies memory address segments whose access popularity exceeds a preset threshold or ranks in the top N as hot pages that need to trigger page migration, and reports their page information.

[0055] S40. The software layer migrates the corresponding page from the source computing card memory to the target computing card memory based on the reported page information.

[0056] See Figure 5 In some embodiments, step S40 includes: S41. The software layer selects the computing card whose remote access count value meets the preset migration conditions as the target computing card based on the remote access count value of each remote computing card to the target page.

[0057] For example, see Figure 2 If page P is currently located in memory 15 of GPU A11, and GPU B12 and GPU C13 frequently access page P, and the statistical results of the hardware statistics unit show that the remote access count value of GPU A11 to page P is 100 and the remote access count value of GPU B12 is 60, then the software layer selects GPU A11 as the target computing card.

[0058] S42. Migrate the target page from the source computing card's video memory to the target computing card's video memory.

[0059] Specifically, the software layer copies the data of the target page from the source compute card's video memory to the target compute card's video memory using DMA or other data transfer methods. After the migration is complete, the page copy in the source compute card's video memory can be marked as invalid or retained as a backup.

[0060] S43. After the migration is complete, update the global address mapping of the multi-card system and notify all computing cards of the address mapping change.

[0061] Specifically, after the migration is complete, the system needs to update the global address mapping table, remapping the logical address of the target page to the physical address of the target computing card. Simultaneously, the address mapping change is communicated to all computing cards via inter-card interconnection links. After each computing card updates its local address mapping table, subsequent access to the page will be correctly routed according to the new address mapping.

[0062] This embodiment selects the computing card whose remote access count value meets the preset migration conditions as the target card, ensuring that the page is migrated to the computing card with the greatest access demand, thereby maximizing the performance benefits brought by the migration.

[0063] Optionally, during the page migration process, concurrent access to the page by other compute cards is suspended until a retransmission signal is received after the migration is complete. To ensure data consistency, when the page migration operation begins, the hardware blocks all concurrent access requests to the page from other compute cards, suspending these requests. After the page data is completely copied to the target compute card's video memory and the address mapping is updated, the system sends a retransmission signal to the blocked access requests. These requests are then retransmitted and correctly routed to the target compute card according to the new address mapping.

[0064] Secondly, the target page that has been migrated to the target computing card does not need to be migrated back to the original computing card after its usage decreases. The page is either released in the target computing card's video memory or reallocated by the software layer. In multi-GPU computing systems, since the video memory capacity of each computing card is usually large, and the main purpose of page migration is to optimize memory access efficiency rather than solve the problem of insufficient video memory capacity, the migrated page does not need to be migrated back to the original computing card after its usage decreases in subsequent calculations. The page can be released in the target computing card's video memory, that is, reclaimed as usable video memory space, or reallocated by the software layer for use by other computing tasks.

[0065] Furthermore, during a preset cooling-off period after the page migration is completed, the page is prohibited from undergoing migration again. To prevent the page from frequently migrating between multiple computing cards due to instantaneous fluctuations in access popularity, the page is prohibited from undergoing migration again during a preset cooling-off period after the page migration is completed. In one embodiment, the preset cooling-off period is 10ms. This cooling-off period can be set to be the same as or different from the polling period, and those skilled in the art can configure it flexibly according to system stability requirements.

[0066] This embodiment ensures that no data inconsistency or access errors occur during the migration process by suspending concurrent access during the migration process and triggering a retransmission after the migration is completed. The design that eliminates the need for reverse migration simplifies the page management logic and avoids complex reverse migration judgments and additional migration overhead. The preset cooldown time effectively ensures the operational stability of the multi-card system and avoids additional overhead and system jitter caused by frequent page migrations.

[0067] Optionally, the hardware statistics unit distinguishes access from different remote computing cards by recording the physical port through which the access request passes, and the software layer restores the identity of the computing card from which the access originates based on the mapping relationship between the physical port and the computing card identifier.

[0068] Specifically, since the hardware layer only obtains physical port information and cannot directly determine the identity of the specific computing card from which the access originates, the software layer pre-maintains a mapping table between physical ports and computing card identifiers. When the software layer reads the count value of the hardware statistics unit, it can restore the physical port to the corresponding remote computing card ID by looking up the mapping table based on the physical port information corresponding to each counter.

[0069] This embodiment maintains the mapping relationship between physical ports and computing card identifiers through software, achieving accurate traceability of remote access sources without increasing hardware complexity. It is an efficient software and hardware co-design scheme.

[0070] See Figure 6 This application also provides a page migration apparatus for remote memory access between multiple cards, comprising: The acquisition module 61 is used to acquire access data of each computing card to the memory address segment through the hardware statistics unit; The access popularity determination module 62 is used to determine the access popularity of each memory address segment based on the access data; The reporting module 63 reports the page information corresponding to the memory address range whose access popularity meets the preset conditions to the software layer; The migration control module 64, the software layer, migrates the corresponding page from the source computing card memory to the target computing card memory based on the reported page information.

[0071] Optionally, in some embodiments, the acquisition module 61 includes: The counting submodule is used to increment the counter in the target statistics entry according to the source of the access request when the target address of the memory access request falls into the memory address segment corresponding to the target statistics entry.

[0072] Optionally, in some embodiments, the heat determination module 62 includes: The numerical statistics submodule is used to poll each statistical entry of the hardware statistics unit at a preset time interval, read the count value of each counter and clear it to zero; The heat calculation submodule is used to subtract the local access count from the remote access count for each statistical entry, and the difference is used as the access heat of the corresponding memory address segment in the current time window. The condition determination submodule is used to determine the memory address segments whose access popularity exceeds a preset threshold within the current time window, or whose access popularity ranks among the top N memory address segments in all the statistically analyzed memory address segments, as meeting the preset condition.

[0073] Optionally, in some embodiments, the migration control module 64 includes: The target selection submodule is used by the software layer to select the computing card whose remote access count value meets the preset migration conditions as the target computing card based on the remote access count value of each remote computing card to the target page. The migration execution submodule is used to migrate the target page from the source compute card's video memory to the target compute card's video memory; The address update submodule is used to update the global address mapping of the multi-card system after the migration is completed, and to notify all compute cards of the address mapping change.

[0074] The page migration device for multi-card remote memory access provided in this embodiment can be used to perform... Figure 1 The technical solutions of the method embodiments shown are similar in principle and technical effect to those of the foregoing embodiments, and will not be described in detail here, but can be referred to each other.

[0075] Figure 7 This is a schematic block diagram illustrating the architecture of one embodiment of the electronic device of this application; based on the same technical concept as the foregoing Embodiment 1, the electronic device provided in this application embodiment, such as... Figure 4 As shown, the steps and flow of the method described in any of the embodiments in Embodiment 1 of this application can be implemented.

[0076] The aforementioned electronic device may include: a housing 71, a processor 72, a memory 73, a circuit board 74, and a power supply circuit 75, wherein the circuit board 74 is disposed inside the space enclosed by the housing 71, and the processor 72 and the memory 73 are disposed on the circuit board 74; the power supply circuit 75 is used to supply power to the various circuits or devices of the aforementioned electronic device; the memory 73 is used to store executable program code; the processor 72 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 73, for executing the page migration method for remote memory access between multiple cards as described in any of the preceding embodiments.

[0077] For details on the specific execution process of the above steps by the processor 72 and the steps further executed by the processor 72 by running executable program code, please refer to the description of Embodiment 1 of this application, which will not be repeated here.

[0078] This application also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the page migration method for remote memory access between multiple cards as described in any of the preceding embodiments.

[0079] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0080] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0081] For ease of description, if systems, servers, etc. are involved, they may be described separately as various units / modules based on their functions. Of course, in implementing this application, the functions of each unit / module can be implemented in one or more software and / or hardware.

[0082] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A page migration method for remote memory access between multiple GPUs, characterized in that, include: Access data of each computing card to the memory address range is obtained through a hardware statistics unit; The access frequency of each memory address segment is determined based on the access data; The page information corresponding to the memory address range whose access popularity meets the preset conditions is reported to the software layer. The software layer migrates the corresponding page from the source computing card's memory to the target computing card's memory based on the reported page information.

2. The page migration method for remote memory access between multiple cards according to claim 1, characterized in that, Before obtaining the access data of each computing card to the memory address segment through the hardware statistics unit, the method further includes: configuring each statistical entry of the hardware statistics unit and specifying the memory address segment monitored by each statistical entry.

3. The page migration method for remote memory access between multiple cards according to claim 2, characterized in that, The statistical entries include the address segment register, the local access counter, the first counter, and the second counter.

4. The page migration method for remote memory access between multiple cards according to claim 1, characterized in that, The step of obtaining access data of each computing card to the memory address segment through the hardware statistics unit includes: When it is detected that the target address of a memory access request falls into the memory address segment corresponding to the target statistics entry, the counter in the target statistics entry is incremented according to the source of the access request.

5. The page migration method for remote memory access between multiple cards according to claim 4, characterized in that, The step of incrementing the counter corresponding to the target statistics entry based on the source of the access request includes: If the access request originates from the local computing card, then the local access counter for the target statistical entry is incremented; If the access request originates from the first interconnect port of the remote computing card, then the first counter of the target statistical entry is incremented; If the access request originates from the second interconnect port of the remote computing card, then the second counter of the target statistics entry is incremented.

6. The page migration method for remote memory access between multiple cards according to claim 1, characterized in that, The access data includes at least: The number of times the remote computing card accesses the memory address range through the inter-card interconnection link and the number of times the local computing card accesses the memory address range.

7. The page migration method for remote memory access between multiple cards according to claim 1, characterized in that, Determining the access frequency of each memory segment based on the access data includes: The hardware statistical unit polls each statistical entry at a preset time interval, reads the count value of each counter, and clears it to zero. For each statistical entry, the remote access count value is subtracted from the local access count value, and the difference is used as the access popularity of the corresponding memory address segment in the current time window; The memory address segments whose access popularity exceeds a preset threshold within the current time window, or whose access popularity ranks among the top N memory address segments in all the statistically analyzed memory address segments, are determined to meet the preset conditions.

8. The page migration method for remote memory access between multiple cards according to claim 1, characterized in that, The target computing card is a computing card whose remote access popularity to the page meets preset migration conditions.

9. The page migration method for remote memory access between multiple cards according to claim 1, characterized in that, The step of migrating the corresponding page from the current source compute card memory to the target compute card memory that issued the main remote access request includes: The software layer selects the computing card whose remote access count value meets the preset migration conditions as the target computing card based on the remote access count value of each remote computing card to the target page. The target page is migrated from the source compute card's video memory to the target compute card's video memory; After the migration is complete, update the global address mapping of the multi-card system and notify all compute cards of the address mapping change.

10. The page migration method for remote memory access between multiple cards according to claim 1, characterized in that, The method further includes: The hardware statistics unit distinguishes access from different remote computing cards by recording the physical ports through which the access requests pass, and the software layer restores the identity of the computing card from which the access originates based on the mapping relationship between the physical port and the computing card identifier.

11. A page migration device for remote memory access between multiple cards, characterized in that, include: The acquisition module is used to acquire access data of each computing card to the memory address segment through the hardware statistics unit; The access popularity determination module is used to determine the access popularity of each memory address segment based on the access data; The reporting module reports the page information corresponding to the memory address range whose access popularity meets the preset conditions to the software layer; The migration control module, in the software layer, migrates the corresponding page from the source computing card's video memory to the target computing card's video memory based on the reported page information.

12. An electronic device, characterized in that, The electronic device includes: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside the space enclosed by the housing, and the processor and the memory are disposed on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the page migration method for remote memory access between multiple cards as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the page migration method for remote memory access between multiple cards as described in any one of claims 1 to 10.

Images