Memory management method and apparatus, electronic device, storage medium, and program product

Abstract

The present disclosure provides a memory management method, device, electronic equipment, storage medium and program product. A memory management method can include: obtaining first topology information between non-uniform memory access (NUMA) nodes; performing topology detection between a graphics processing unit and the NUMA nodes; obtaining second topology information between the graphics processing unit and the NUMA nodes based on a result of the topology detection; obtaining a global topology relationship graph based on the first topology information and the second topology information, wherein the global topology relationship graph includes an access distance of a central processing unit to the NUMA nodes and an access distance between the graphics processing unit and the NUMA nodes; and determining a migration strategy of a target page in a target NUMA node based on the global topology relationship graph.

Description

Technical Field

[0001] This disclosure relates to the fields of storage management and device management, and more particularly to a memory management method, a memory management device, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Modern high-performance servers typically employ Non-Uniform Memory Access (NUMA) architecture and are equipped with multiple graphics processing units (GPUs) to accelerate computation. In this heterogeneous system, both the central processing unit (CPU) and the GPU need to access memory, and the memory is distributed across multiple NUMA nodes. System performance is highly dependent on the efficiency of memory access.

[0003] However, existing memory management systems are primarily designed for CPUs and lack deep integration with GPUs, resulting in low efficiency in GPU-CPU collaborative work. Summary of the Invention

[0004] According to a first aspect of the present disclosure, a memory management method is provided, the memory management method comprising: obtaining first topology information between NUMA nodes with non-uniform memory access; performing topology probing between a graphics processing unit and NUMA nodes; obtaining second topology information between the graphics processing unit and NUMA nodes based on the results of the topology probing; obtaining a global topology graph based on the first and second topology information, wherein the global topology graph includes the access distance between the central processing unit and NUMA nodes and the access distance between the graphics processing unit and NUMA nodes; and determining a migration strategy for a target page in a target NUMA node based on the global topology graph.

[0005] Optionally, topology probing between the graphics processing unit and the NUMA node is performed, including: for each graphics processing unit in the system, determining the central processing unit associated with the current graphics processing unit; and determining the NUMA node where the central processing unit associated with the current graphics processing unit is located as the local NUMA node of the current graphics processing unit.

[0006] Optionally, based on the results of topology probing, a second topology information between the graphics processing unit (GPU) and the NUMA node is obtained, including: determining whether the current NUMA node is a local NUMA node of the current GPU for the current NUMA node and the current GPU; if the current NUMA node is determined to be a local NUMA node of the current GPU, writing the access distance between the CPU corresponding to the current GPU and the current NUMA node into the affinity matrix; if the current NUMA node is determined not to be a local NUMA node of the current GPU, calculating the access distance between the current GPU and the current NUMA node and writing it into the affinity matrix; and using the affinity matrix as the second topology information.

[0007] Optionally, determining the migration strategy for the target page in the target NUMA node based on the global topology graph includes: for the target page in the target NUMA node, determining the candidate NUMA node corresponding to each graphics processing unit and each central processing unit accessing the target page; for each candidate NUMA node, calculating a first access cost for each graphics processing unit to access the current candidate NUMA node and a second access cost for each central processing unit to access the current candidate NUMA node based on the global topology graph, wherein the first access cost represents the cumulative access distance between each graphics processing unit and the current candidate NUMA node, and the second access cost represents the cumulative access distance between each central processing unit and the current candidate NUMA node; determining the total access cost for the current candidate NUMA node based on the first access cost and the second access cost; and determining the target candidate NUMA node to which the data of the target page will be migrated based on the total access cost.

[0008] Optionally, the method may further include: performing a migration operation on a target page in a target NUMA node based on the migration strategy; locking a page table entry corresponding to the target page based on the virtual address corresponding to the target page; updating the page table entry corresponding to the target page in the page table used by the central processing unit; updating the page table entry corresponding to the target page in the page table used by the graphics processing unit based on the virtual address used by the graphics processing unit; and unlocking the locked page table entry.

[0009] Optionally, the target page comprises multiple pages. Locking page table entries corresponding to the target page based on the virtual address of the target page includes: locking multiple page table entries corresponding to the multiple pages in batches based on the virtual addresses of the multiple pages. Updating page table entries corresponding to the target page in the page table of the central processing unit includes: updating multiple page table entries corresponding to the multiple pages in the page table of the central processing unit in batches based on the virtual addresses of the multiple pages. Updating page table entries corresponding to the target page in the page table of the graphics processing unit based on the virtual address of the graphics processing unit includes: updating page table entries corresponding to the multiple pages in the page table of the graphics processing unit in batches based on the virtual address of the multiple pages of the graphics processing unit.

[0010] Optionally, updating the page table entry corresponding to the target page in the page table of the graphics processing unit (GPU) based on the virtual address used by the GPU includes: sending an asynchronous update notification after the page table entry corresponding to the virtual address in the page table of the CPU is updated; and updating the page table entry corresponding to the virtual address in the page table of the GPU based on the virtual address used by the GPU, and refreshing the translation back buffer for the cached page table, based on the asynchronous update notification.

[0011] Optionally, the method may further include: performing bandwidth contention detection on concurrent memory access requests from the central processing unit and the graphics processing unit; scheduling the memory access requests based on the detected bandwidth contention; and forming access execution requests based on the scheduled memory access requests to perform memory access.

[0012] Optionally, bandwidth contention detection is performed on concurrent memory access requests from the central processing unit and the graphics processing unit, including: reading a preset number of memory access requests from the memory access request queue; merging the preset number of memory access requests according to memory channels; and performing bandwidth contention detection on the memory access requests corresponding to each memory channel.

[0013] According to a second aspect of the present disclosure, a memory management apparatus is provided, the memory management apparatus including: a topology information acquisition module configured to: acquire first topology information between NUMA nodes with inconsistent memory access; perform topology probing between a graphics processing unit and NUMA nodes; and acquire second topology information between the graphics processing unit and NUMA nodes based on the results of the topology probing; a graph construction module configured to acquire a global topology graph based on the first and second topology information, wherein the global topology graph includes the access distance between the central processing unit and NUMA nodes and the access distance between the graphics processing unit and NUMA nodes; and a strategy determination module configured to determine a migration strategy for a target page in a target NUMA node based on the global topology graph.

[0014] Optionally, the topology information acquisition module is configured to: for each graphics processing unit in the system, determine the central processing unit associated with the current graphics processing unit; and determine the NUMA node where the central processing unit associated with the current graphics processing unit is located as the local NUMA node of the current graphics processing unit.

[0015] Optionally, the topology information acquisition module is configured to: determine whether the current NUMA node is a local NUMA node of the current graphics processing unit for the current NUMA node and the current graphics processing unit; if the current NUMA node is determined to be a local NUMA node of the current graphics processing unit, write the access distance between the CPU corresponding to the current graphics processing unit and the current NUMA node into the affinity matrix; if the current NUMA node is determined not to be a local NUMA node of the current graphics processing unit, calculate the access distance between the current graphics processing unit and the current NUMA node and write it into the affinity matrix; and use the affinity matrix as the second topology information.

[0016] Optionally, the strategy determination module is configured to: for a target page in a target NUMA node, determine candidate NUMA nodes corresponding to each graphics processing unit and each central processing unit accessing the target page; for each candidate NUMA node, calculate a first access cost for each graphics processing unit accessing the current candidate NUMA node and a second access cost for each central processing unit accessing the current candidate NUMA node based on a global topology graph, wherein the first access cost represents the cumulative access distance between each graphics processing unit and the current candidate NUMA node, and the second access cost represents the cumulative access distance between each central processing unit and the current candidate NUMA node; determine the total access cost for the current candidate NUMA node based on the first access cost and the second access cost; and determine the target candidate NUMA node to which the data of the target page will be migrated based on the total access cost.

[0017] Optionally, the apparatus may further include: a page table update module configured to: perform a migration operation of a target page in a target NUMA node based on the migration strategy; lock a page table entry corresponding to the target page based on the virtual address corresponding to the target page; update the page table entry corresponding to the target page in the page table used by the central processing unit; update the page table entry corresponding to the target page in the page table used by the graphics processing unit based on the virtual address used by the graphics processing unit; and unlock the locked page table entry.

[0018] Optionally, the target page is multiple pages, wherein the page table update module is configured to: lock multiple page table entries corresponding to the multiple pages in batches based on the virtual addresses corresponding to the multiple pages; update multiple page table entries corresponding to the multiple pages in the page table of the central processing unit in batches based on the virtual addresses corresponding to the multiple pages; and update page table entries corresponding to the multiple pages in the page table of the graphics processing unit in batches based on the virtual addresses corresponding to the multiple pages used by the graphics processing unit.

[0019] Optionally, the page table update module is configured to: send an asynchronous update notification based on the completion of updating the page table entry corresponding to the virtual address in the page table for the central processing unit; and, based on the asynchronous update notification, update the page table entry corresponding to the virtual address in the page table for the graphics processing unit, and refresh the translation back buffer for caching the page table, based on the graphics processing unit using the virtual address.

[0020] Optionally, the apparatus may further include: a scheduling module configured to: perform bandwidth contention detection on concurrent memory access requests from the central processing unit and the graphics processing unit; schedule the memory access requests based on the detected bandwidth contention; and form access execution requests based on the scheduled memory access requests to perform memory access.

[0021] Optionally, the scheduling module is configured to: read a preset number of memory access requests from the memory access request queue; merge the preset number of memory access requests according to memory channels; and perform bandwidth contention detection for the memory access requests corresponding to each memory channel.

[0022] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device may include: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the memory management method as described above.

[0023] According to a fourth aspect of the present disclosure, a computer-readable storage medium for storing instructions is provided, which, when executed by at least one processor, cause the at least one processor to perform the memory management method described above.

[0024] According to a fifth aspect of the present disclosure, a computer program product is provided, wherein instructions in the computer program product are executed by at least one processor in an electronic device to perform the memory management method as described above.

[0025] The technical solutions provided by the embodiments of this disclosure bring at least the following beneficial effects: by performing topology probing between the graphics processing unit and the NUMA node, the graphics processing unit can perceive the NUMA topology to form a global topology graph, thereby providing a more suitable migration location for the migrated page and reducing access latency.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0028] Figure 1 This is a flowchart of a memory management method according to an embodiment of the present disclosure.

[0029] Figure 2 This is a flowchart of constructing a global topology graph according to an embodiment of the present disclosure.

[0030] Figure 3 A flowchart of GPU topology probing according to an embodiment of this disclosure is shown.

[0031] Figure 4 A flowchart illustrating the construction of an affinity matrix for a GPU-NUMA node according to an embodiment of this disclosure is shown.

[0032] Figure 5 A flowchart illustrating the affinity measurement between a GPU and a NUMA node according to an embodiment of this disclosure is shown.

[0033] Figure 6 A schematic diagram of a global topology map according to an embodiment of the present disclosure is shown.

[0034] Figure 7 A flowchart illustrating a page data placement decision according to an embodiment of this disclosure is shown.

[0035] Figure 8A flowchart illustrating bandwidth scheduling coordination according to an embodiment of this disclosure is shown.

[0036] Figure 9 This is a flowchart illustrating the synchronous updating of page tables for the CPU and GPU according to embodiments of this disclosure.

[0037] Figure 10 A schematic diagram of a fine-grained page table locking control process according to an embodiment of the present disclosure is shown.

[0038] Figure 11 A schematic diagram of a batch page table synchronization process according to an embodiment of the present disclosure is shown.

[0039] Figure 12 A schematic diagram of an asynchronous page table synchronization process according to an embodiment of the present disclosure is shown.

[0040] Figure 13 A schematic diagram of a GPU page table cache access process according to an embodiment of the present disclosure is shown.

[0041] Figure 14 This is an architecture diagram of a GPU-CPU collaborative NUMA topology-aware memory management device according to an embodiment of the present disclosure.

[0042] Figure 15 This is a block diagram of a memory management apparatus according to embodiments of the present disclosure.

[0043] Figure 16 This is a schematic diagram of the structure of a memory management device in the hardware operating environment of an embodiment of this disclosure.

[0044] Figure 17 This is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0045] Throughout the accompanying drawings, it should be noted that the same reference numerals are used to denote the same or similar elements, features, and structures. Detailed Implementation

[0046] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of embodiments of the present disclosure as defined by the claims and their equivalents. Various specific details are included to aid understanding, but these details are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted.

[0047] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0048] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0049] Currently, NUMA memory management is mainly implemented by the operating system kernel, with typical solutions as follows: 1. NUMA topology probing During system startup, the kernel obtains NUMA topology information through ACPI tables (such as SRAT and SLIT), including: the number of NUMA nodes, the number of CPU cores per node, the memory capacity of each node, and the distance matrix between nodes. In this disclosure, a NUMA node may be referred to as a node.

[0050] 2. Memory allocation strategy The kernel provides several NUMA memory allocation strategies, including: Local allocation, which prioritizes allocating memory from the CPU node where the process is running; Interleave allocation, which allocates memory among multiple nodes in turn; Bind allocation, which allocates memory only from a specified node; and Preferred allocation, which prioritizes allocating memory from a specified node and allocates from other nodes if the specified node is insufficient.

[0051] 3. Page migration The kernel's AutoNUMA mechanism automatically migrates pages, such as periodically scanning the page table, statistically analyzing access patterns, migrating pages frequently accessed by a particular node to that node, and using the CPU to perform page copying.

[0052] 4. GPU Memory Management GPU drivers manage GPU memory independently. For example, GPUs have independent video memory (VRAM), access system memory via PCIe, maintain independent GPU page tables, and transfer data between the CPU and GPU through explicit data copying (cudaMemcpy).

[0053] 5. Unified Memory NVIDIA CUDA provides a unified memory mechanism, such as CPU and GPU sharing the same virtual address space, and data can be automatically migrated between CPU and GPU. However, the migration strategy is simple and is not aware of NUMA topology.

[0054] However, existing NUMA memory management schemes have the following problems: 1. The GPU is unaware of the NUMA topology. For example, the GPU driver is unaware of the distribution and distance of NUMA nodes; when the GPU accesses system memory, it cannot select the nearest NUMA node, resulting in high latency when the GPU accesses remote node memory.

[0055] 2. CPU and GPU memory management are disconnected. For example, the CPU core's NUMA management does not take into account the GPU's access patterns; the GPU driver's memory management does not take into account the CPU's NUMA strategy; the two make independent decisions, which may lead to conflicts (such as the CPU migrating page data to node A, but the GPU accessing it from node B is faster).

[0056] 3. Page table synchronization overhead is high. For example, the CPU and GPU maintain independent page tables. After a page migration (such as moving data from one NUMA node's physical memory to another), the CPU page table and GPU page table need to be updated separately. The synchronization mechanism is inefficient and has high latency.

[0057] 4. Lack of bandwidth coordination. For example, when multiple GPUs and CPUs access memory simultaneously, they compete for bandwidth; the lack of a global bandwidth scheduling mechanism leads to low bandwidth utilization and increased access latency.

[0058] 5. The unified memory strategy is simple. For example, CUDA's unified memory migration strategy is based on page faults, which are passively triggered; it does not consider NUMA topology, resulting in poor selection of migration target nodes; and it does not consider the collaborative access mode of CPU and GPU.

[0059] 6. Lack of optimization during initialization. For example, the relationship between the GPU and NUMA topology is not fully utilized during system startup; the initial data placement is poor, requiring extensive page migrations later.

[0060] The main reason for these problems is that the memory management systems of the CPU and GPU are independent of each other, lacking coordination mechanisms and NUMA topology awareness.

[0061] To address the aforementioned issues, this disclosure provides a GPU-CPU collaborative NUMA topology-aware memory management device, which achieves efficient heterogeneous memory management by establishing a unified topology-aware layer and a collaborative decision-making mechanism. The present disclosure will now be described in detail with reference to specific embodiments.

[0062] Figure 1 This is a flowchart of a memory management method according to an embodiment of the present disclosure. Figure 1 The method shown can be executed by, for example, the system kernel, CPU, GPU, or other devices.

[0063] Reference Figure 1 In step S101, first topology information between NUMA nodes with non-uniform memory access is obtained. According to an embodiment, at system startup, the kernel can detect the topological relationship between NUMA nodes (i.e., NUMA topology) to obtain NUMA topology information (i.e., first topology information). For example, the first topology information may include the number of nodes N, the list of CPU cores for each node, the memory capacity of each node, the distance matrix between nodes, etc.

[0064] In this disclosure, NUMA nodes, nodes, and memory nodes are used interchangeably, as are CPUs and CPU cores. The distance between nodes can be understood as the access distance between a CPU's local node and another node. The node closest to the CPU, or the node that the CPU can access fastest, can be called the CPU's local node. The CPU's local node can be called the node where the CPU resides.

[0065] In step S102, topology probing between the graphics processing unit and the NUMA node is performed. The system may include one or more GPUs. The connection relationship between each GPU and each NUMA node can be probed.

[0066] According to an embodiment, for each graphics processing unit in the system, the central processing unit associated with the current graphics processing unit can be determined, and the NUMA node where the central processing unit associated with the current graphics processing unit is located can be determined as the local NUMA node of the current graphics processing unit.

[0067] It can perform GPU topology probing, that is, detect the topological relationship between each GPU and each NUMA node.

[0068] For example, a GPU driver can detect the connection relationships between GPUs and NUMA nodes. The GPU driver can iterate through each GPU. For each GPU, it can query the PCIe bus currently connected to the GPU, find the corresponding CPU socket, map that CPU socket to a local NUMA node, and record that NUMA node as the GPU's local NUMA node. The GPU's local NUMA node can represent the NUMA node closest to the GPU or the node with the shortest access distance. If there are unprocessed GPUs, the above operations can continue until the local NUMA nodes of all GPUs are determined.

[0069] In step S103, based on the results of topology probing, the second topology information between the graphics processing unit and the NUMA node is obtained.

[0070] According to the embodiment, for the current NUMA node and the current graphics processing unit, it can be determined whether the current NUMA node is a local NUMA node of the current graphics processing unit; if it is determined that the current NUMA node is a local NUMA node of the current graphics processing unit, the access distance between the central processing unit corresponding to the current graphics processing unit and the current NUMA node is written into the affinity matrix; if it is determined that the current NUMA node is not a local NUMA node of the current graphics processing unit, the access distance between the current graphics processing unit and the current NUMA node is calculated and written into the affinity matrix; the affinity matrix is ​​used as the second topology information.

[0071] As an example, each GPU can be traversed. For each GPU, each NUMA node can be traversed. For each NUMA node, the GPU can determine whether the current NUMA node is its local NUMA node. If it is a local NUMA node, the local access latency is written to the affinity matrix. If it is not a local NUMA node, the remote access latency is calculated and written to the affinity matrix, i.e., the access distance between the GPU and the NUMA node (i.e., the remote NUMA node) is calculated, thereby updating the affinity matrix. If there are still unprocessed NUMA nodes for the current GPU, the above operations of writing local access latency or calculating remote access latency to update the affinity matrix can continue. If all NUMA nodes for the current GPU have been processed, it can be checked whether there are still unprocessed GPUs, until all GPUs have been traversed in the above manner.

[0072] In step S104, a global topology map is obtained based on the first topology information and the second topology information. The global topology map includes the access distance between the central processing unit and the NUMA node and the access distance between the graphics processing unit and the NUMA node.

[0073] Figure 2 This is a flowchart of constructing a global topology graph according to an embodiment of the present disclosure.

[0074] Reference Figure 2 In step S201, NUMA topology probing is performed.

[0075] According to the embodiment, when the system starts, the kernel can detect the NUMA topology to obtain NUMA topology information, such as the number of nodes N, the list of CPU cores of each node, the memory capacity of each node, and the distance matrix D[i][j] between nodes (D[i][j] represents the access distance / latency from node i to node j).

[0076] In step S202, GPU topology detection is performed.

[0077] According to an embodiment, the GPU driver can detect the connection relationship between the GPU and the NUMA node. Figure 3 A flowchart of GPU topology probing according to an embodiment of this disclosure is shown.

[0078] Reference Figure 3 The GPU driver can traverse each GPU. For each GPU, it can query the PCIe bus currently connected to the GPU, find the CPU socket corresponding to that PCIe bus, map that CPU socket to a local NUMA node, and record that NUMA node as the GPU's local NUMA node. In this text, the NUMA node closest to the CPU, or the NUMA node where the CPU resides, can be referred to as the CPU's local NUMA node. The GPU's local NUMA node can be represented by the NUMA node closest to the GPU.

[0079] If there are any GPUs that have not yet been processed, the above operations can continue until the local NUMA nodes of all GPUs are determined.

[0080] According to an embodiment, dedicated interconnect technologies such as NVLink can be used to connect the GPU and CPU.

[0081] In step S203, an affinity matrix for GPU-NUMA nodes is constructed. For example, an access latency matrix from each GPU to each NUMA node can be constructed, which can also be called an affinity matrix or a distance matrix. The access latency corresponds to the node distance; the larger the node distance, the higher the access latency and the lower the affinity. Figure 4 A flowchart illustrating the construction of an affinity matrix for a GPU-NUMA node according to an embodiment of this disclosure is shown.

[0082] Reference Figure 4It can iterate through all GPUs. For each GPU, it can iterate through all NUMA nodes. For each NUMA node, the GPU can determine whether the current NUMA node is its local NUMA node. If it is a local NUMA node, the local access latency is written into the affinity matrix. If it is not a local NUMA node, the remote access latency is calculated and written into the affinity matrix. That is, the access distance between the GPU and the NUMA node (i.e., the remote NUMA node) is calculated, thereby updating the affinity matrix.

[0083] If there are still unprocessed NUMA nodes for the current GPU, the above operations of writing local access latency or calculating remote access latency to update the affinity matrix can continue; if all NUMA nodes for the current GPU have been processed, it can be checked whether there are still unprocessed GPUs, until all GPUs have been traversed in the above manner.

[0084] Figure 5 A flowchart illustrating the affinity measurement between a GPU and a NUMA node according to an embodiment of this disclosure is shown.

[0085] Reference Figure 5 It iterates through the combinations of each GPU and each NUMA node. For example, a GPU and a NUMA node can be considered as a combination.

[0086] For the current combination (current NUMA node and current GPU), allocate test memory on the current NUMA node, control the GPU to access the test memory and record the start and end times (i.e., access times), calculate the access latency based on the start and end times and write the access latency into the affinity matrix, then release the test memory. Perform the above operations for each combination until the affinity test for all GPU and NUMA node combinations is completed, and output the affinity measurement results.

[0087] According to embodiments of this disclosure, a software-based NUMA awareness approach can be adopted. For example, NUMA awareness can be implemented at the application layer by manually binding threads and memory.

[0088] In step S204, a global topological relationship graph is constructed.

[0089] According to the embodiments, a global topology graph can be constructed, which may include multiple CPU cores, GPUs, NUMA nodes and the connections between them, and can characterize the access distance from each CPU core to each NUMA node and the access affinity from each GPU to each NUMA node. Figure 6 A schematic diagram of a global topology map according to an embodiment of the present disclosure is shown. Figure 6In this framework, the global topology graph can display the weights (i.e., distance or affinity) between each CPU core (such as CPU Core0, CPU Core1, CPU Core2, CPU Core3) and each NUMA node (such as NUMA0, NUMA1, NUMA2, NUMA3), as well as the weights between each GPU device (such as GPU0, GPU1) and each NUMA node, and the weights between NUMA nodes themselves. Furthermore, the global topology graph can also show which NUMA node is a local NUMA node of the GPU. Figure 6 In a NUMA node, the node can be understood as a memory node, that is, the memory included in the NUMA node. The weight can represent the distance that the CPU or GPU needs to access the memory in a NUMA node. Figure 6 The diagrams shown are merely exemplary, and this disclosure is not limited thereto.

[0090] According to the embodiments, this disclosure provides GPU-NUMA topology detection and affinity modeling, that is, detecting the connection relationship and access latency between GPU and NUMA nodes, and constructing GPU-NUMA affinity matrix and topology model.

[0091] According to the embodiment, steps S101-S104 described above can be executed during the system initialization phase to complete global NUMA topology detection. The above method can be applied to optimize page data placement during the system initialization phase to reduce subsequent migrations. The following operations can be performed during the system runtime phase.

[0092] In step S105, the migration strategy of the target page in the target NUMA node is determined based on the global topology graph.

[0093] After obtaining the global topology graph, the placement of migrated pages (i.e., the migration strategy) can be determined based on the access pattern analysis results and / or the global topology graph. Here, access pattern analysis can be performed using the NUMA management module or other methods can be used to obtain the results.

[0094] According to an embodiment, for a target page in a target NUMA node, candidate NUMA nodes corresponding to each graphics processing unit (GPU) and each central processing unit (CPU) accessing the target page can be determined. For each candidate NUMA node, a first access cost for each GPU to access the current candidate NUMA node and a second access cost for each CPU to access the current candidate NUMA node are calculated based on a global topology graph. The first access cost represents the cumulative access distance between each GPU and the current candidate NUMA node, and the second access cost represents the cumulative access distance between each CPU and the current candidate NUMA node. The total access cost for the current candidate NUMA node is determined based on the first and second access costs. The target candidate NUMA node to which the data of the target page will be migrated is determined based on the total access cost.

[0095] Figure 7 A flowchart illustrating a page data placement decision according to an embodiment of this disclosure is shown.

[0096] Reference Figure 7 It then begins traversing the hot pages (i.e., the target pages). Hot pages can be defined as pages that are frequently accessed.

[0097] For each hot page, the CPU visitor distribution and GPU visitor distribution can be read. The CPU visitor distribution represents the distribution of CPUs accessing the hot page, and the GPU visitor distribution represents the distribution of GPUs accessing the hot page. Candidate NUMA nodes for the hot page can be traversed. These candidate NUMA nodes can include the local NUMA nodes of the CPUs accessing the hot page and the local NUMA nodes of the GPUs accessing the hot page.

[0098] For each candidate NUMA node, the CPU access cost (i.e., the sum of distances) can be accumulated based on the distances between each CPU and the candidate NUMA node. Similarly, the GPU access cost can be accumulated based on the distances between each GPU and the candidate NUMA node. Then, the sum of the CPU and GPU access costs for that candidate NUMA node is calculated as the total cost for that node. For example, it can be based on... Figure 6 The global topology graph shown is used to calculate CPU access cost and GPU access cost.

[0099] Calculate and record the total cost for each candidate NUMA node for the hot page, following the method described above.

[0100] Select the NUMA node with the lowest total cost from the candidate NUMA nodes for the hot page as the optimal NUMA node for that hot page. Determine whether the current NUMA node containing the hot page is the optimal NUMA node. If it is, continue processing unprocessed hot pages in the manner described above; otherwise, generate a page migration strategy, that is, migrate the data of the hot page to the page of the optimal NUMA node.

[0101] According to the embodiments, this disclosure provides CPU-GPU collaborative data placement decision, that is, by comprehensively considering CPU-side access cost, GPU-side access cost and topology distance information, the optimal placement location of a page or data object is jointly determined.

[0102] When multiple GPUs and CPUs access memory simultaneously, bandwidth usage can be coordinated, i.e., bandwidth scheduling (coordinating the use of memory bandwidth by multiple devices to avoid contention and congestion).

[0103] According to the embodiment, bandwidth contention detection can be performed on concurrent memory access requests from the central processing unit and the graphics processing unit; based on the detection of bandwidth contention, the memory access requests are scheduled; and based on the scheduled memory access requests, an access execution request is formed to perform memory access.

[0104] This disclosure provides an initialization and runtime optimization mechanism, namely, optimizing data placement during the system initialization phase and dynamically adjusting affinity parameters and management strategies (such as migration strategies) based on performance feedback during the runtime phase.

[0105] According to the embodiment, a preset number of memory access requests can be read from the memory access request queue, the preset number of memory access requests can be merged according to memory channels, and bandwidth contention detection can be performed on the memory access requests corresponding to each memory channel.

[0106] Figure 8 A flowchart illustrating bandwidth scheduling coordination according to an embodiment of this disclosure is shown. Figure 8 The method shown can be executed by, for example, the system kernel, CPU, GPU, or other devices.

[0107] Reference Figure 8When multiple GPUs and CPUs access memory simultaneously, an access request queue is read. For example, a preset number of access requests can be read from the access request queue. Access requests can be merged by memory channel, and bandwidth contention can be determined for each memory channel. If bandwidth contention exists, at least one of priority scheduling, fair scheduling, and batch scheduling can be executed, and then execution requests can be formed. For example, in the case of priority scheduling, competing requests can be sorted according to at least one of request priority, request initiating device type, service real-time level, cross-NUMA access distance, and historical waiting time, and the requests with higher rankings can be executed first. Preferably, page table synchronization requests, hot page migration requests, and GPU access requests corresponding to real-time computing tasks can be given higher priority. In the case of fair scheduling, bandwidth shares, time slices, or round-robin weights can be allocated to different GPU, CPU core groups, NUMA nodes, or task groups, and each competing entity can be controlled to obtain channel access opportunities in turn according to at least one of round-robin, fair queue, weighted round-robin, or gap round-robin, thereby avoiding a single device occupying bandwidth for a long time. In the case of batch scheduling, small-granularity requests with the same source node, same destination node, same access direction, or adjacent address ranges can be merged into batch access transactions before being uniformly distributed for execution. This reduces transaction switching overhead, DMA commit overhead, and the number of bus arbitrations. The above example method is merely exemplary, and this disclosure is not limited thereto.

[0108] If there is no bandwidth contention, an execution request can be formed directly. Requests can be distributed according to device type; requests corresponding to GPUs can be executed by the GPU DMA engine, and requests corresponding to CPUs can be executed by the CPU. The scheduling execution results are then output.

[0109] According to the embodiments, this disclosure provides a multi-device bandwidth coordination mechanism, which performs contention detection, priority scheduling, fair scheduling and batch scheduling on concurrent memory access of CPU and GPU to improve bandwidth utilization.

[0110] According to embodiments, this disclosure provides a unified page table management interface to simultaneously update CPU page tables and GPU page tables. As an example, a migration operation for a target page in a target NUMA node can be performed based on a determined migration strategy; a page table entry corresponding to the target page is locked based on the virtual address corresponding to the target page; the page table entry corresponding to the target page in the page table used by the CPU is updated; the page table entry corresponding to the target page in the page table used by the graphics processing unit is updated based on the virtual address used by the graphics processing unit; and the locked page table entry is unlocked.

[0111] According to an embodiment, during page migration, a GPU DMA engine or other methods can be used to perform page migration, followed by a collaborative page table update between the CPU and GPU.

[0112] Figure 9 This is a flowchart illustrating the synchronous updating of page tables for the CPU and GPU according to embodiments of this disclosure.

[0113] Reference Figure 9 After page migration, the virtual address and the new physical address can be obtained / received, and page table locks can be acquired. Fine-grained page table locks can be used for locking. Here, a fine-grained page table lock can represent a lock on a page table entry. For example, the CPU (such as a unified page table management interface / module on the CPU side, or an operating system kernel module or GPU driver co-management module running on the CPU side) can receive the virtual address and the new physical address, and then perform the locking operation. After receiving the page migration result, the lock operation can be performed on the locks corresponding to the CPU page table entries and GPU page table entries that need to be updated. That is, the initiator and controller of locking can be, for example, the CPU-side software control logic. After page migration, all page table entries corresponding to the migrated page (including CPU page table entries and GPU page table entries) can be locked, and then the page table entries on the CPU side are updated first, followed by the page table entries on the GPU side. After both sides have been updated, all are unlocked.

[0114] Figure 10 A schematic diagram of a fine-grained page table locking control process according to an embodiment of the present disclosure is shown.

[0115] Reference Figure 10 You can input a virtual address, calculate the page index of the page table based on the virtual address, locate the corresponding page table lock based on the page index, lock the page table entry corresponding to the page index using the page table lock, update the corresponding page table entry, and then unlock it.

[0116] return Figure 9 After locking, the CPU page table entries are queried and updated. Next, each GPU is traversed; if a GPU uses the virtual address, its page table entries are queried and updated until all GPUs have been processed, at which point the TLB is marked as needing a refresh. Here, marking the TLB as needing a refresh can be understood as the CPU-side page table synchronization layer performing a refresh on the relevant TLB. The CPU-side and GPU-side TLBs can be refreshed in a preset order or in parallel. For example, the refresh order of the CPU-side and GPU-side TLBs can follow a preset strategy: the CPU-side can be refreshed first, the GPU-side first, or the refresh can be triggered in parallel. Finally, the page table lock is unlocked. For example, it can be unlocked after the relevant TLBs have been refreshed. Alternatively, it can be unlocked after all old page table entries in the TLBs related to the current page migration on both the CPU and GPU sides have been invalidated.

[0117] The above operations can be performed on each migrated page to uniformly update the page tables of the CPU and GPU.

[0118] In cases involving multiple migration pages (i.e., target pages), multiple page table entries corresponding to the multiple migration pages can be locked in batches based on the virtual addresses corresponding to the multiple migration pages; multiple page table entries corresponding to the multiple migration pages in the page table used by the CPU can be updated in batches based on the virtual addresses corresponding to the multiple migration pages; and page table entries corresponding to the multiple migration pages can be updated in batches based on the virtual addresses used by the graphics processing unit. After batch updating the page table entries, the TLB used to cache the page tables can be flushed in batches.

[0119] According to an embodiment, page tables of the CPU and GPU can be updated synchronously in batches to reduce synchronization overhead. Figure 11 A schematic diagram of a batch page table synchronization process according to an embodiment of the present disclosure is shown. Page table synchronization refers to a mechanism for keeping CPU page tables and GPU page tables consistent.

[0120] Reference Figure 11 First, the update queue is initialized, and the page migration results are traversed, adding the virtual address and new physical address corresponding to the migrated page to the update queue. If the size of the update queue after adding the new address (i.e., the queue size) reaches the batch threshold, then batch synchronous updates are performed; if the size of the update queue after adding the new address does not reach the batch threshold, the page migration results are traversed again until the size of the update queue reaches the batch threshold.

[0121] When performing batch synchronous updates, the CPU page tables are updated in batches, followed by the GPU page tables, then the TLB is flushed in batches, and the update queue is cleared. For example, the above method can be used. Figure 9 The method described above updates the page table and refreshes the TLB in batches.

[0122] According to an embodiment, to reduce blocking, an asynchronous mechanism can be used to perform the page table synchronization process between the CPU and GPU. An asynchronous update notification can be sent based on the completion of the update of the page table entry corresponding to the virtual address in the page table used by the CPU. For example, the asynchronous update notification may include the virtual address corresponding to the target page and the physical address corresponding to the virtual address. Based on the asynchronous update notification, and based on the graphics processing unit's use of the virtual address, the page table entry corresponding to the virtual address in the page table used by the graphics processing unit can be updated, and the translation back buffer used for caching page tables can be refreshed.

[0123] Figure 12 A schematic diagram of an asynchronous page table synchronization process according to an embodiment of the present disclosure is shown.

[0124] Reference Figure 12After performing page migration, on the CPU side, the CPU page table is updated, and then a virtual address and / or new physical address notification is sent. The asynchronous update notification is forwarded to the GPU side via an asynchronous notification mechanism. On the GPU side, the GPU page table is updated, then the GPU TLB is flushed, and a confirmation of update completion is returned. When updating the page table, the following methods can be used: Figure 10 The page table is updated in the manner shown.

[0125] In the unified synchronization method for page tables used in CPUs and GPUs, the aforementioned operations such as page table entry updates, batch synchronization, and asynchronous notifications can be implemented individually or in combination.

[0126] According to the embodiments, cache coherency can be implemented at the hardware level. Cache coherency between the CPU and GPU can be implemented in hardware (such as AMD's hUMA).

[0127] According to the embodiments, virtualization layer management can be used. Cooperative management of NUMA and GPU can be implemented at a virtualization layer (such as KVM).

[0128] According to the embodiments, this disclosure provides a unified page table synchronization mechanism, that is, after page migration or placement adjustment, the CPU page table and GPU page table are updated synchronously, and the mapping consistency is guaranteed by combining batch synchronization, asynchronous notification and TLB refresh.

[0129] According to an embodiment, in order to reduce page table lookup latency, a page table cache can be maintained on the GPU side. Figure 13 A schematic diagram of a GPU page table cache access process according to an embodiment of the present disclosure is shown.

[0130] Reference Figure 13 After the GPU initiates a memory access, it can be determined whether the virtual address corresponding to the memory access hits the GPU page table cache.

[0131] If the GPU page table cache is hit, the physical address is read directly from the cache and returned. If the GPU page table cache is not hit, the GPU page table is queried, the mapping between the virtual address and the physical address is inserted into the GPU page table cache, and the corresponding physical address is returned, thus completing the access preparation.

[0132] According to the embodiments, performance metrics can be collected in real time or periodically. Performance metrics may include, for example, CPU and GPU memory access latency, cross-node access ratio, page migration count and benefits, bandwidth utilization, etc. The affinity matrix can be updated, the weights of data placement decisions adjusted, and bandwidth scheduling strategies optimized based on the collected performance metrics. Furthermore, access patterns can be learned, and common access patterns can be saved for subsequent rapid decision-making.

[0133] Figure 14This is an architecture diagram of a GPU-CPU collaborative NUMA topology-aware memory management device according to an embodiment of the present disclosure.

[0134] Reference Figure 14 This architecture may include a topology awareness and modeling layer, a collaborative decision-making layer, a CPU memory management executor, a GPU memory management executor, a bandwidth scheduler, and a page table synchronization layer.

[0135] In the topology awareness and modeling layer, NUMA topology probing, GPU topology probing, and affinity matrix construction can be performed. For example, NUMA topology probing can be performed using step S101. GPU topology probing can be performed using step S102. Figure 2 The method shown constructs the affinity matrix.

[0136] At the collaborative decision-making layer, access pattern analysis, data placement decisions, and migration coordination can be performed. For example, access pattern analysis can be performed in any manner, such as... Figure 7 The method shown performs data placement decisions. Page migration can be performed using a defined data placement decision.

[0137] Page migration can be performed by the CPU memory management executor and the GPU memory management executor.

[0138] In the page table synchronization layer, the CPU memory management executor and the GPU memory management executor can perform the synchronization operations of the CPU page table and the GPU page table and the TLB management.

[0139] In addition, the bandwidth scheduler can follow Figure 8 The method performs bandwidth scheduling.

[0140] Figure 14 The layers and devices shown are for illustrative purposes only and are not limited thereto.

[0141] According to the embodiments of this disclosure, at least the following technical problems can be solved: 1. How to enable the GPU to be aware of NUMA topology and select the optimal memory access path.

[0142] 2. How to achieve collaborative memory management between the CPU and GPU to avoid decision conflicts.

[0143] 3. How to efficiently synchronize the page tables of the CPU and GPU and reduce synchronization overhead.

[0144] 4. How to coordinate the memory bandwidth usage of multiple GPUs and CPUs to avoid contention.

[0145] 5. How to optimize data placement during the system initialization phase to reduce subsequent migrations.

[0146] According to the embodiments of this disclosure, at least the following technical effects can be achieved: 1. Reduced GPU memory access latency. Through topology-aware data placement, the average latency of GPU memory access is reduced by 30-40%. GPUs prioritize accessing local NUMA nodes, avoiding remote access.

[0147] 2. Improved CPU-GPU collaboration efficiency. Unified collaborative decision-making avoids conflicts in CPU and GPU memory management. Data is placed in locations optimal for both CPU and GPU, reducing overall access costs by 25-35%.

[0148] 3. Reduced page table synchronization overhead. Batch synchronization and asynchronous synchronization mechanisms reduce page table synchronization overhead by more than 60%. Fine-grained locking reduces concurrency contention.

[0149] 4. Improved memory bandwidth utilization. The bandwidth scheduler coordinates access from multiple devices, avoiding contention and congestion. Memory bandwidth utilization has increased from 60% to over 85%.

[0150] 5. Improved application performance. End-to-end application performance is improved by 15-42%, with particularly significant effects in GPU-intensive and memory-intensive applications.

[0151] 6. Enhanced system scalability. The topology-aware mechanism is applicable to NUMA systems and GPU configurations of any size, and its performance advantages become more pronounced as the number of nodes and GPUs increases.

[0152] 7. Improved energy efficiency. Reduces remote memory access and bandwidth contention, lowering system energy consumption by approximately 20-30%.

[0153] Figure 15 This is a block diagram of a memory management apparatus according to embodiments of the present disclosure.

[0154] Reference Figure 15 The memory management device 1500 may include: a topology information acquisition module 1501, a map construction module 1502, a strategy determination module 1503, a page table update module 1504, and a scheduling module 1505. Figure 15 The module names and quantities listed are merely illustrative and this disclosure is not limited thereto.

[0155] According to an embodiment, the topology information acquisition module 1501 can obtain first topology information between non-uniform memory access NUMA nodes; perform topology probing between the graphics processing unit and the NUMA nodes; and obtain second topology information between the graphics processing unit and the NUMA nodes based on the results of the topology probing.

[0156] According to an embodiment, the map construction module 1502 can obtain a global topology map based on the first topology information and the second topology information, wherein the global topology map includes the access distance between the central processing unit and the NUMA node and the access distance between the graphics processing unit and the NUMA node.

[0157] According to an embodiment, the strategy determination module 1503 can determine the migration strategy of the target page in the target NUMA node based on the global topology graph.

[0158] According to an embodiment, the topology information acquisition module 1501 can determine the central processing unit associated with each graphics processing unit in the system; and determine the NUMA node where the central processing unit associated with the graphics processing unit is located as the local NUMA node of the graphics processing unit.

[0159] According to an embodiment, the topology information acquisition module 1501 can determine whether the current NUMA node is a local NUMA node of the current graphics processing unit for the current NUMA node and the current graphics processing unit; if it is determined that the current NUMA node is a local NUMA node of the current graphics processing unit, the access distance between the central processing unit corresponding to the current graphics processing unit and the current NUMA node is written into the affinity matrix; if it is determined that the current NUMA node is not a local NUMA node of the current graphics processing unit, the access distance between the current graphics processing unit and the current NUMA node is calculated and written into the affinity matrix; the affinity matrix is ​​used as the second topology information.

[0160] According to an embodiment, the strategy determination module 1503 can, for a target page in a target NUMA node, determine candidate NUMA nodes corresponding to each graphics processing unit and each central processing unit accessing the target page; for each candidate NUMA node, calculate a first access cost for each graphics processing unit to access the current candidate NUMA node and a second access cost for each central processing unit to access the current candidate NUMA node based on a global topology graph, wherein the first access cost represents the cumulative access distance between each graphics processing unit and the current candidate NUMA node, and the second access cost represents the cumulative access distance between each central processing unit and the current candidate NUMA node; determine the total access cost for the current candidate NUMA node based on the first access cost and the second access cost; and determine the target candidate NUMA node to which the data of the target page will be migrated based on the total access cost.

[0161] According to an embodiment, the page table update module 1504 can perform a migration operation of a target page in a target NUMA node based on the migration strategy; lock the page table entry corresponding to the target page based on the virtual address corresponding to the target page; update the page table entry corresponding to the target page in the page table used by the central processing unit; update the page table entry corresponding to the target page in the page table used by the graphics processing unit based on the virtual address used by the graphics processing unit; and unlock the locked page table entry.

[0162] According to an embodiment, the target page is multiple pages. The page table update module 1504 can lock multiple page table entries corresponding to the multiple pages in batches based on the virtual addresses corresponding to the multiple pages; update multiple page table entries corresponding to the multiple pages in the page table used by the central processing unit in batches based on the virtual addresses corresponding to the multiple pages; and update page table entries corresponding to the multiple pages in the page table used by the graphics processing unit in batches based on the virtual addresses corresponding to the multiple pages.

[0163] According to an embodiment, the page table update module 1504 can send an asynchronous update notification based on the completion of updating the page table entry corresponding to the virtual address in the page table used by the central processing unit; based on the asynchronous update notification, the page table entry corresponding to the virtual address in the page table used by the graphics processing unit is updated based on the virtual address used by the graphics processing unit, and the translation back buffer used for caching the page table is refreshed.

[0164] According to an embodiment, the scheduling module 1505 can perform bandwidth contention detection on concurrent memory access requests from the central processing unit and the graphics processing unit; based on the detected bandwidth contention, it can schedule the memory access requests; and based on the scheduled memory access requests, it can form access execution requests to perform memory access.

[0165] According to an embodiment, the scheduling module 1505 can read a preset number of memory access requests from the memory access request queue; merge the preset number of memory access requests according to memory channels; and perform bandwidth contention detection for the memory access requests corresponding to each memory channel.

[0166] Figure 16 This is a schematic diagram of the structure of a memory management device in the hardware operating environment of an embodiment of this disclosure. Figure 16 The described memory management device can perform memory management methods according to embodiments of the present disclosure.

[0167] like Figure 16As shown, the memory management device 1000 may include: a processing component 1001, a communication bus 1002, a network interface 1003, an input / output interface 1004, a memory 1005, and a power supply component 1006. The communication bus 1002 is used to enable communication between these components. The input / output interface 1004 may include a video display (such as a liquid crystal display), a microphone and speaker, and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). Optionally, the input / output interface 1004 may also include a standard wired interface or a wireless interface. The network interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory or a stable non-volatile memory. The memory 1005 may also optionally be a storage device independent of the aforementioned processing component 1001.

[0168] Those skilled in the art will understand that Figure 16 The structure shown does not constitute a limitation on the memory management device 1000, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0169] like Figure 16 As shown, the memory 1005, which serves as a storage medium, may include an operating system (such as a MAC operating system), a data storage module, a network communication module, a user interface module, a program implementing this disclosure, and a database.

[0170] exist Figure 16 In the memory management device 1000 shown, the network interface 1003 is mainly used for data communication with external electronic devices / terminals; the input / output interface 1004 is mainly used for data interaction with users; the processing component 1001 and the memory 1005 in the memory management device 1000 can be set in the memory management device 1000. The memory management device 1000 calls the program stored in the memory 1005 and various APIs provided by the operating system through the processing component 1001 to execute the memory management method provided in the embodiments of this disclosure.

[0171] Processing component 1001 may include at least one processor, and memory 1005 stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by at least one processor, a memory management method according to embodiments of the present disclosure is performed. However, the above examples are merely exemplary, and the present disclosure is not limited thereto.

[0172] The processing component 1001 can control the components included in the memory management device 1000 by executing a program.

[0173] As an example, the memory management device 1000 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned instruction set. Here, the memory management device 1000 is not necessarily a single electronic device, but may be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction set) individually or in combination. The memory management device 1000 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0174] In the memory management device 1000, the processing component 1001 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the processing component 1001 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0175] The processing component 1001 can execute instructions or code stored in memory, wherein memory 1005 can also store data. Instructions and data can also be sent and received over a network via network interface 1003, wherein network interface 1003 can employ any known transport protocol.

[0176] The memory 1005 can be integrated with the processing component 1001, for example, by arranging RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 1005 can include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory and the processing component 1001 can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processing component 1001 to read data stored in the memory 1005.

[0177] According to embodiments of this disclosure, an electronic device may be provided. Figure 17 This is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 1100 may include at least one memory 1102 and at least one processor 1101. The at least one memory 1102 stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one processor 1101, a memory management method according to an embodiment of the present disclosure is performed.

[0178] Processor 1101 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 1101 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0179] The memory 1102, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, a program for executing the methods of this disclosure, and a database.

[0180] The memory 1102 may be integrated with the processor 1101; for example, RAM or flash memory may be arranged within an integrated circuit microprocessor. Alternatively, the memory 1102 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory and processor may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the memory.

[0181] In addition, electronic device 1100 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of electronic device 1100 can be interconnected via a bus and / or network.

[0182] As an example, electronic device 1100 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 1100 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 1100 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0183] As will be understood by those skilled in the art, Figure 17 The structure shown does not constitute a limitation on the structure and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0184] According to embodiments of this disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one processor, they cause at least one processor to perform a memory management method according to this disclosure. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0185] According to embodiments of this disclosure, a computer program product may also be provided, wherein the instructions in the computer program product can be executed by the processor of a computer device to perform the above-described memory management method.

[0186] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0187] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A memory management method, characterized in that, The method includes: Obtain the first topology information between NUMA nodes with inconsistent memory access; Perform topology probing between the graphics processing unit and the NUMA node; Based on the results of topology probing, second topology information between the graphics processing unit and the NUMA node is obtained; Based on the first and second topology information, a global topology graph is obtained, wherein the global topology graph includes the access distance between the central processing unit and the NUMA node and the access distance between the graphics processing unit and the NUMA node. The migration strategy for the target page in the target NUMA node is determined based on the global topology graph.

2. The method according to claim 1, characterized in that, Perform topology probing between the graphics processing unit and the NUMA node, including: For each graphics processing unit in the system, determine the central processing unit associated with the current graphics processing unit; The NUMA node where the central processing unit associated with the current graphics processing unit resides is determined as the local NUMA node of the current graphics processing unit.

3. The method according to claim 2, characterized in that, Based on the results of topology probing, second topology information between the graphics processing unit and the NUMA node is obtained, including: For the current NUMA node and the current graphics processing unit, determine whether the current NUMA node is the local NUMA node of the current graphics processing unit; If it is determined that the current NUMA node is the local NUMA node of the current graphics processing unit, the access distance between the central processing unit corresponding to the current graphics processing unit and the current NUMA node is written into the affinity matrix. If it is determined that the current NUMA node is not a local NUMA node of the current graphics processing unit, calculate the access distance between the current graphics processing unit and the current NUMA node and write it into the affinity matrix. The affinity matrix is ​​used as the second topological information.

4. The method according to claim 1, characterized in that, The migration strategy for the target page in the target NUMA node is determined based on the global topology graph, including: For a target page in a target NUMA node, identify the candidate NUMA node corresponding to each graphics processing unit and each central processing unit that accesses the target page; For each candidate NUMA node, the first access cost for each graphics processing unit to access the current candidate NUMA node and the second access cost for each central processing unit to access the current candidate NUMA node are calculated based on the global topology graph. The first access cost represents the cumulative access distance between each graphics processing unit and the current candidate NUMA node, and the second access cost represents the cumulative access distance between each central processing unit and the current candidate NUMA node. The total access cost for the current candidate NUMA node is determined based on the first access cost and the second access cost. Based on the total access cost, the data of the target page will be migrated to the target candidate NUMA node.

5. The method according to claim 1, characterized in that, The method further includes: Based on the migration strategy, perform the migration operation of the target page in the target NUMA node; Lock the page table entry corresponding to the target page based on the virtual address of the target page; Update the page table entry in the page table used by the central processing unit that corresponds to the target page; Based on the virtual address used by the graphics processing unit, update the page table entry corresponding to the target page in the page table used by the graphics processing unit; Unlock the locked page table entry.

6. The method according to claim 5, characterized in that, The target page consists of multiple pages. Locking page table entries corresponding to the target page based on the virtual address corresponding to the target page includes: locking multiple page table entries corresponding to the multiple pages in batches based on the virtual addresses corresponding to the multiple pages. This includes updating the page table entry corresponding to the target page in the page table used by the central processing unit, including: Based on the virtual addresses corresponding to the multiple pages, the multiple page table entries corresponding to the multiple pages in the page table used by the central processing unit are updated in batches; The step of updating the page table entry corresponding to the target page in the page table used by the graphics processing unit based on the virtual address includes: Based on the virtual addresses corresponding to the multiple pages, the graphics processing unit updates the page table entries corresponding to the multiple pages in the page table used by the graphics processing unit in batches.

7. The method according to claim 5, characterized in that, Based on the virtual address used by the graphics processing unit, updating the page table entry corresponding to the target page in the page table used by the graphics processing unit includes: Once the page table entry corresponding to the virtual address in the page table used by the central processing unit is updated, an asynchronous update notification is sent. According to the asynchronous update notification, based on the virtual address used by the graphics processing unit, the page table entry corresponding to the virtual address in the page table used by the graphics processing unit is updated, and the translation back buffer used for caching the page table is refreshed.

8. The method according to claim 1, characterized in that, The method further includes: Bandwidth contention detection is performed on concurrent memory access requests from the central processing unit and the graphics processing unit. Based on the detection of bandwidth contention, the memory access request is scheduled. The scheduled memory access requests are used to form access execution requests to perform memory access.

9. The method according to claim 8, characterized in that, Bandwidth contention detection is performed on concurrent memory access requests from the central processing unit and the graphics processing unit, including: Read a preset number of memory access requests from the memory access request queue; The preset number of memory access requests are merged according to memory channels; Bandwidth contention detection is performed for memory access requests corresponding to each memory channel.

10. A memory management device, characterized in that, The device includes: The topology information acquisition module is configured to: obtain first topology information between NUMA nodes with non-uniform memory access; perform topology probing between the graphics processing unit and the NUMA nodes; and obtain second topology information between the graphics processing unit and the NUMA nodes based on the results of the topology probing. The graph construction module is configured to obtain a global topological relationship graph based on the first topological information and the second topological information. The global topological relationship graph includes the access distance between the central processing unit and the NUMA node and the access distance between the graphics processing unit and the NUMA node. The strategy determination module is configured to determine the migration strategy for the target page in the target NUMA node based on the global topology graph.

11. An electronic device, characterized in that, include: At least one processor; At least one memory that stores computer-executable instructions. The computer-executable instructions, when executed by the at least one processor, cause the at least one processor to perform the memory management method as described in any one of claims 1 to 9.

12. A computer-readable storage medium for storing instructions, characterized in that, When the instruction is executed by at least one processor, it causes the at least one processor to perform the memory management method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The instructions in the computer program product are executed by at least one processor in an electronic device to perform the memory management method as claimed in any one of claims 1 to 9.

Images