Method and device for managing display card storage resources, electronic equipment and readable medium
By migrating virtual address blocks step by step within the UVM framework to form a three-level storage pool, the bottleneck problem of graphics card storage resources is solved, enabling efficient management and access of graphics card storage resources and improving computing performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING TENAFE ELECTRONIC TECHNOLOGY CO LTD
- Filing Date
- 2026-01-12
- Publication Date
- 2026-05-15
AI Technical Summary
In large-scale model training and inference scenarios, GPU capacity becomes a key performance bottleneck. Existing memory virtualization technology has failed to effectively solve the problem of expanding and managing GPU storage resources, resulting in access latency and decreased computing performance.
By migrating data between the graphics card, host memory, and solid-state drive in stages according to the access history of virtual address blocks within the UVM framework, a three-level storage pool is formed, optimizing data heat gradient management. Combined with the device interface that directly accesses the solid-state drive, the I/O path is shortened, and asynchronous data prefetching is used to improve access efficiency.
It expands the addressable space of the graphics card, optimizes the utilization of storage resources, ensures the efficiency and reliability of graphics card access, avoids hot data from sinking to the low-speed storage layer, and ensures the efficient execution of model training and inference.
Smart Images
Figure CN121501520B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of graphics card technology, and more specifically, to a method, apparatus, electronic device, and readable medium for managing graphics card storage resources. Background Technology
[0002] Currently, in large-scale model training and inference scenarios, GPU capacity has become a key performance bottleneck. To overcome hardware limitations, the industry often uses memory virtualization technology to expand available GPU resources, such as solutions based on NVIDIA UVM (Unified Virtual Memory) drivers. This driver supports seamless mapping of host memory to device-addressable space and can automatically manage the migration of data between the GPU and host memory, thus providing flexible and efficient underlying support for models with extremely high GPU requirements.
[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0004] The purpose of this disclosure is to provide a method, apparatus, electronic device, and readable medium for managing graphics card storage resources, in order to overcome the bottleneck problem of graphics card storage resources caused by the limitations and defects of related technologies.
[0005] A first aspect of this disclosure provides a method for managing graphics card storage resources, comprising:
[0006] Retrieve the access history of virtual address blocks;
[0007] Based on the access history, within the UVM framework, the virtual address block is migrated step by step between the graphics card, host memory, and solid-state drive.
[0008] In one exemplary embodiment of this disclosure, migrating the virtual address block step-by-step between the graphics card, host memory, and solid-state drive within the UVM framework based on the access history of the virtual address block includes:
[0009] In response to insufficient memory space of the graphics card, a first target virtual address block is determined based on the access history;
[0010] Within the UVM framework, the first target virtual address block is migrated from the graphics card to the host memory.
[0011] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0012] Based on the migration path of the virtual address block, a mapping record is generated.
[0013] The mapping record includes a first mapping record migrated from the graphics card to the host memory, and / or a second mapping record migrated from the host memory to the solid-state drive.
[0014] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0015] In response to the fact that the first target virtual address block has been migrated from the graphics card to the host memory, the first mapping record is updated.
[0016] In one exemplary embodiment of this disclosure, migrating the virtual address block step-by-step between the graphics card, host memory, and solid-state drive within the UVM framework based on the access history of the virtual address block further includes:
[0017] In response to insufficient memory space in the host memory, based on the access history and the mapping record, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory.
[0018] Within the UVM framework, the second target virtual address block is asynchronously written from the host memory to the solid-state drive.
[0019] In one exemplary embodiment of this disclosure, in response to insufficient memory space in the host memory, determining a second target virtual address block from the first target virtual address block that has been migrated to the host memory, based on the access history and the mapping record, includes:
[0020] In response to insufficient memory space in the host memory, migration information for the first target virtual address block is determined based on the first mapping record. This migration information includes migration order and / or timestamps.
[0021] And / or, based on the access history, determine the access frequency of the first target virtual address block;
[0022] And / or, based on the second mapping record, determine the historical storage record of the first target virtual address block in the solid-state drive;
[0023] Based on at least one of the migration information, the access frequency, and the historical storage records, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory.
[0024] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0025] In response to the fact that the second target virtual address block has been asynchronously written from the host memory to the solid-state drive, the first mapping record and the second mapping record are updated.
[0026] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0027] In response to the graphics card needing to access a missing virtual address block, the missing virtual address block is fed back to the graphics card from the current level according to the mapping record.
[0028] The missing virtual address block is a virtual address block that has been migrated from the graphics card to the host memory or the solid-state drive.
[0029] In one exemplary embodiment of this disclosure, in response to the graphics card needing to access a missing virtual address block, feeding back the missing virtual address block from the current level to the graphics card according to the mapping record further includes:
[0030] In response to the graphics card needing to access the missing virtual address block, a page fault is generated;
[0031] Based on the page error, obtain the mapping record;
[0032] Based on the mapping record, the missing virtual address block is fed back from the current level to the graphics card.
[0033] In one exemplary embodiment of this disclosure, feeding back the missing virtual address block from the current level to the graphics card according to the mapping record includes:
[0034] Based on the mapping record, it is determined that the missing virtual address block is located in the host memory, and the host memory feeds back to the graphics card.
[0035] Based on the mapping record, the missing virtual address block is determined to be located on the solid-state drive, and the missing virtual address block is synchronously read to the graphics card.
[0036] In one exemplary embodiment of this disclosure, feeding back the missing virtual address block from the current level to the graphics card according to the mapping record further includes:
[0037] Based on the mapping record, it is determined that the missing virtual address block is located on the solid-state drive, and a specified number of subsequent virtual address blocks of the missing virtual address block are asynchronously pre-read into the pre-read buffer.
[0038] In one exemplary embodiment of this disclosure, feeding back the missing virtual address block from the current level to the graphics card according to the mapping record further includes:
[0039] Based on the mapping record, the missing virtual address block is determined to be located in the pre-read buffer, and the missing virtual address block is fed back to the graphics card from the pre-read buffer.
[0040] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0041] Read and write operations can be performed on the virtual address blocks located on the solid-state drive by directly accessing the device interface of the solid-state drive.
[0042] A second aspect of this disclosure provides a management device for graphics card storage resources, comprising:
[0043] The acquisition module is configured to retrieve the access history of virtual address blocks;
[0044] The migration module is configured to migrate the virtual address block step by step between the graphics card, host memory, and solid-state drive within the UVM framework based on the access history.
[0045] A third aspect of this disclosure provides an electronic device comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the method as described in any of the preceding methods based on instructions stored in the memory.
[0046] A fourth aspect of this disclosure provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the method for managing graphics card storage resources as described in any of the preceding claims.
[0047] A fifth aspect of this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the graphics card storage resource management method as described in any of the above technical solutions.
[0048] In this embodiment of the disclosure, by migrating the virtual address block step by step between the graphics card, host memory, and solid-state drive (SSD) according to the access history within the UVM framework, the SSD is organically integrated into the UVM system as a third-level storage, which expands the addressable space of the graphics card, balances the access frequency and storage medium characteristics, and increases the probability of hot data residing in the high-speed layer. While optimizing the utilization of graphics card storage resources, it also ensures the access efficiency and reliability of the graphics card.
[0049] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0051] Figure 1 A schematic diagram of an exemplary system architecture for a graphics card storage resource management scheme that can be applied to embodiments of the present disclosure is shown.
[0052] Figure 2 This is a flowchart a of a method for managing graphics card storage resources in an exemplary embodiment of this disclosure.
[0053] Figure 3 This is a schematic diagram of the architecture of another graphics card storage resource management scheme in an exemplary embodiment of this disclosure.
[0054] Figure 4 This is flowchart b, which is an exemplary embodiment of the present disclosure of a method for managing graphics card storage resources.
[0055] Figure 5 This is flowchart c, which is an exemplary embodiment of the present disclosure of a method for managing graphics card storage resources.
[0056] Figure 6 This is a flowchart d of a method for managing graphics card storage resources in an exemplary embodiment of this disclosure.
[0057] Figure 7 This is a flowchart e of a method for managing graphics card storage resources in an exemplary embodiment of this disclosure.
[0058] Figure 8 This is a flowchart f of a method for managing graphics card storage resources in an exemplary embodiment of this disclosure.
[0059] Figure 9 This is a flowchart g of a method for managing graphics card storage resources in an exemplary embodiment of this disclosure.
[0060] Figure 10 This is a block diagram of a graphics card storage resource management device according to an exemplary embodiment of the present disclosure.
[0061] Figure 11 This is a block diagram of an electronic device in an exemplary embodiment of this disclosure. Detailed Implementation
[0062] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.
[0063] Furthermore, the accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0064] In large-scale model training and inference scenarios, video memory capacity has become a critical performance bottleneck. To overcome hardware limitations, the industry often uses memory virtualization technology to expand available video memory resources, such as solutions based on the NVIDIA UVM (Unified Virtual Memory) driver. This driver supports seamless mapping of host memory to device-addressable space and can automatically manage the migration of data between video memory and system memory, thus providing flexible and efficient underlying support for models with extremely high video memory requirements.
[0065] To further expand memory capacity and indirectly extend video memory, solid-state drives (SSDs) can be used as the next level of storage expansion. However, the read and write latency of SSDs is much higher than that of DRAM and video memory. If high-volume data is swapped to an SSD, it will directly cause serious access latency, thereby dragging down the overall computing throughput and response performance.
[0066] Therefore, when building a video memory expansion system based on heterogeneous storage, it is necessary to design an effective data placement and migration strategy to avoid hot data sinking to the low-speed storage layer and ensure the efficient execution of model training and inference.
[0067] Based on the aforementioned technical deficiencies in the existing technology, this disclosure proposes a new management scheme for graphics card storage resources. Before describing the management scheme for graphics card storage resources in conjunction with embodiments, the key concepts involved in the scheme are explained as follows:
[0068] 1. UVM: Unified Virtual Memory. UVM is a memory management architecture that allows the CPU and GPU (or other processors) to see a single, unified virtual memory address space. Data only moves between CPU memory and GPU video memory when it is accessed, and the system automatically handles page migration and page faults.
[0069] 2. GPU: Graphics Processing Unit. A GPU is a processor specifically designed for efficient graphics rendering and parallel computing tasks. It has hundreds or thousands of small computing cores, making it suitable for handling a large number of similar tasks simultaneously (such as pixel shading and matrix operations). In addition, GPUs excel at handling data-intensive tasks, prioritizing computational throughput over single-task latency. Hereinafter referred to as a graphics card.
[0070] 3. DRAM: Dynamic Random Access Memory. DRAM is the most common main memory in computers, which is the technology used in memory modules (such as DDR4, DDR5). It serves as a high-speed data cache and working area between the CPU and slow storage devices (such as SSDs / hard drives), and is referred to as host memory below.
[0071] 4. SSD: Solid State Drive. An SSD is a non-volatile storage device that uses flash memory chips to store data. As a primary external storage device, it is used to install operating systems, store applications, and user files.
[0072] 5. Readahead Buffer: Readahead is a performance optimization technique. When the system detects that a program is sequentially reading data, it predicts the data that may be needed next and loads it in advance from slow storage devices (such as SSDs or hard drives) into a cache (such as DRAM).
[0073] 6. VA block: Virtual Address Block. VA stands for Virtual Address. This is the memory address space as seen from the program's perspective; it's a contiguous address space starting from 0. Each process has its own private virtual address space, mapped to actual physical memory by the operating system through the Memory Management Unit (MMU) and page tables. A block refers to a contiguous region of virtual address space of a specific size.
[0074] Figure 1 A schematic diagram of an exemplary system architecture for a graphics card storage resource management scheme that can be applied to embodiments of the present disclosure is shown.
[0075] like Figure 1As shown, system architecture 100 may include one or more of terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.
[0076] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers. For example, server 105 could be a server cluster consisting of multiple servers.
[0077] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers.
[0078] In some embodiments, the graphics card storage resource management method provided in this disclosure is generally executed by server 105, and correspondingly, the graphics card storage resource management device is generally located in terminal device 103 (or terminal device 101 or 102). In other embodiments, some terminals may have functions similar to those of the server device to execute this method.
[0079] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.
[0080] Figure 2 This is a flowchart a of a method for managing graphics card storage resources in an exemplary embodiment of this disclosure.
[0081] refer to Figure 2 Methods for managing graphics card storage resources may include:
[0082] Step S202: Obtain the access history of the virtual address block;
[0083] Step S204: Based on the access history, within the UVM framework, the virtual address block is migrated step by step between the graphics card, host memory, and solid-state drive.
[0084] In one exemplary embodiment of this disclosure, under the UVM architecture, virtual address blocks (VA blocks) exhibit a significant "hot" / cold" stratification characteristic based on their access frequency: frequently accessed "hot" data resides in video memory (GPU) to ensure computational efficiency; while infrequently accessed "cold" data is swapped to host DRAM. Furthermore, even among data blocks located in host memory (DRAM), there are internal heat distinctions due to differences in access patterns, forming multi-level data heat gradients.
[0085] In this embodiment of the disclosure, by distinguishing between virtual address blocks carrying "hot" data and virtual address blocks carrying "cold" data based on the access history within the UVM framework, and migrating the virtual address blocks step by step between the graphics card, host memory, and solid-state drive when the graphics card storage space is insufficient, the solid-state drive is organically integrated into the UVM system as a third-level storage, thereby expanding the addressable space of the graphics card, balancing access frequency and storage medium characteristics, increasing the probability of hot data residing in the high-speed layer, optimizing the utilization of graphics card storage resources, and ensuring the access efficiency and reliability of the graphics card.
[0086] In one exemplary embodiment of this disclosure, a multi-level data heat gradient is determined based on the access history. When the graphics card storage space is insufficient, the virtual address blocks are migrated from the graphics card to the host memory in order of data heat from low to high.
[0087] In one exemplary embodiment of this disclosure, a multi-level data popularity gradient is determined based on the access history. When the host memory storage space is insufficient, the virtual address blocks are migrated from the host memory to the solid-state drive in order of data popularity from low to high.
[0088] Existing UVM driver solutions only manage data migration between GPU memory and CPU memory. However, the implementation of this disclosure, for example... Figure 3 As shown, within the UVM framework, the virtual address block is migrated stepwise between the graphics card, host memory, and solid-state drive, including GPU 302, DRAM 304, and SSD 306.
[0089] Specifically, this disclosure incorporates SSD 306 as a third-level persistent storage within the UVM management framework, forming a coherent "GPU 302 → DRAM 304 → SSD 306" progressively sinking storage pool. This architecture expands the traditional boundaries of UVM, enabling it to manage virtual address spaces several orders of magnitude larger than physical video memory while remaining transparent to applications. For example, when the GPU 302's storage space is full, the system automatically moves the least recently used virtual address block (VA block) out to DRAM 304. This process is automatically completed by the Unified Virtual Memory (UVM) driver.
[0090] The following section details each step of the method for managing graphics card storage resources.
[0091] In one exemplary embodiment of this disclosure, such as Figure 4 As shown, based on the access history of virtual address blocks, within the UVM framework, the step-by-step migration of the virtual address blocks between the graphics card, host memory, and solid-state drive includes:
[0092] Step S402: In response to insufficient memory space of the graphics card, a first target virtual address block is determined based on the access history;
[0093] Step S404: Within the UVM framework, the first target virtual address block is migrated from the graphics card to the host memory.
[0094] In this embodiment of the disclosure, the virtual address block carrying data is determined to be "hot" data, "cold" data, or multi-level hot data by accessing history. Within the UVM framework, the virtual address block carrying "cold" data is preferentially identified as the first target virtual address block and migrated from the graphics card to the host memory.
[0095] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0096] Based on the migration path of the virtual address block, a mapping record is generated.
[0097] The mapping record includes a first mapping record migrated from the graphics card to the host memory, and / or a second mapping record migrated from the host memory to the solid-state drive.
[0098] In one exemplary embodiment of this disclosure, the system maintains a linked list in the host memory based on the Least Recently Used (LRU) algorithm to manage virtual address blocks that have been moved to the host memory, as one implementation of the first mapping record.
[0099] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0100] In response to the fact that the first target virtual address block has been migrated from the graphics card to the host memory, the first mapping record is updated.
[0101] In one exemplary embodiment of this disclosure, such as Figure 5 As shown, based on the access history of virtual address blocks, within the UVM framework, the step-by-step migration of the virtual address blocks between the graphics card, host memory, and solid-state drive also includes:
[0102] Step S502: In response to insufficient memory space in the host memory, based on the access history and the mapping record, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory.
[0103] Step S504: Within the UVM framework, the second target virtual address block is asynchronously written from the host memory to the solid-state drive.
[0104] In this embodiment of the disclosure, when migrating virtual address blocks from DRAM to SSD, not only is the LRU order considered, but also the number of hits (hotness) of the virtual address block and its previous residence time in the SSD are taken into account. This hybrid metric avoids the problem of "hot data being accidentally swapped out to a low-speed SSD" that may be caused by a simple LRU approach. At the same time, the residence time factor prevents the migrated data from frequently thrashing between the SSD and DRAM, making it more suitable for the dynamic management of data lifecycle in the three-level storage of this disclosure.
[0105] In one exemplary embodiment of this disclosure, when the host memory needs to free up more memory space, the system determines the second target virtual address block in the host memory based on the access record and the mapping record, and transfers it to the SSD through an asynchronous write operation. At the same time, the correspondence between the block and the logical block address (LBA) in the SSD is recorded in a mapping table as an implementation of the second mapping record.
[0106] In one exemplary embodiment of this disclosure, such as Figure 6 As shown, in response to insufficient memory space in the host memory, based on the access history and the mapping record, the process of filtering and determining a second target virtual address block from the first target virtual address block that has been migrated to the host memory includes:
[0107] Step S602, in response to insufficient memory space in the host memory.
[0108] Step S604: Based on the first mapping record, determine the migration information of the first target virtual address block, wherein the migration information includes migration order and / or timestamp.
[0109] Step S606: Based on the access history, determine the access frequency of the first target virtual address block;
[0110] Step S608: Based on the second mapping record, determine the historical storage record of the first target virtual address block in the solid-state drive;
[0111] Step S610: Based on at least one of the migration information, the access frequency, and the historical storage records, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory.
[0112] In one exemplary embodiment of this disclosure, when it is necessary to further free up storage space in the host memory, the system selects several blocks from the end of the LRU list and, based on factors such as the number of accesses (hit count) and the time last stored in the solid state drive (SSD), selects a second target virtual address block, transfers it to the SSD through an asynchronous write operation, and records the correspondence between the second target virtual address block and the logical block address (LBA) in the SSD in a mapping table as one implementation of the second mapping record.
[0113] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0114] In response to the fact that the second target virtual address block has been asynchronously written from the host memory to the solid-state drive, the first mapping record and the second mapping record are updated.
[0115] In one exemplary embodiment of this disclosure, such as Figure 7 As shown, the methods for managing graphics card storage resources also include:
[0116] Step S702: In response to the graphics card needing to access a missing virtual address block, the missing virtual address block is fed back to the graphics card from the current level according to the mapping record.
[0117] The missing virtual address block is a virtual address block that has been migrated from the graphics card to the host memory or the solid-state drive.
[0118] In one exemplary embodiment of this disclosure, if the GPU needs a certain virtual address block, but the block is not currently in video memory, it is determined to be a missing virtual address block, and a "page fault" is triggered. At this time, the system will attempt to transfer the block from the host memory to the GPU.
[0119] In one exemplary embodiment of this disclosure, such as Figure 8 As shown, in response to the graphics card needing to access a missing virtual address block, the process of feeding back the missing virtual address block from the current level to the graphics card according to the mapping record further includes:
[0120] Step S802: In response to the graphics card needing to access the missing virtual address block, a page fault is generated.
[0121] Step S804: Based on the page error, obtain the mapping record;
[0122] Step S806: Based on the mapping record, the missing virtual address block is fed back from the current level to the graphics card.
[0123] In this embodiment of the disclosure, if the GPU needs a certain virtual address block, but the block is not currently in video memory, a "page fault" will be triggered. At this time, the system will attempt to transfer the block from the host memory to the GPU.
[0124] In one exemplary embodiment of this disclosure, such as Figure 9 As shown, according to the mapping record, feeding back the missing virtual address block from the current level to the graphics card includes:
[0125] Step S902: Based on the mapping record, determine that the missing virtual address block is located in the host memory, and then feed it back to the graphics card from the host memory;
[0126] Step S904: Based on the mapping record, determine that the missing virtual address block is located on the solid-state drive, and synchronously read the missing virtual address block to the graphics card.
[0127] In one exemplary embodiment of this disclosure, the system first queries the mapping table. If the block is not in the table, it means it is still in host memory and is directly transferred to the GPU. If it is in the table, its current position is determined based on the record. If it is already in the "readahead buffer," it is directly transferred. If it is still in the SSD, a synchronous read operation is triggered to read the block, and simultaneously an asynchronous readahead operation starting from the block's starting address is initiated to read subsequent data into the buffer in advance for later use.
[0128] Furthermore, while existing technical solutions support direct access to SSDs, they are not integrated with the GPU page fault handling process. Although prefetching technology is common in CPU file systems, this solution applies it to the GPU's virtual address block (VA block) granularity and seamlessly integrates it with the UVM's page fault handling mechanism.
[0129] The technical solution disclosed herein uses the NVMe protocol to directly read and write to SSDs, significantly shortening the I / O path, reducing software stack overhead, and completely bypassing the file system. This is particularly important in GPU computing scenarios because GPUs are sensitive to latency, and the additional latency introduced by traditional file systems is unacceptable. Furthermore, asynchronous writing is used when evicting data from the SSD, without blocking the current GPU execution. When reading from the SSD, not only are the required blocks read synchronously, but asynchronous readahead based on address contiguity is also triggered, preloading potentially needed blocks into the buffer, effectively hiding SSD access latency.
[0130] In one exemplary embodiment of this disclosure, feeding back the missing virtual address block from the current level to the graphics card according to the mapping record further includes:
[0131] Based on the mapping record, it is determined that the missing virtual address block is located on the solid-state drive, and a specified number of subsequent virtual address blocks of the missing virtual address block are asynchronously pre-read into the pre-read buffer.
[0132] In one exemplary embodiment of this disclosure, feeding back the missing virtual address block from the current level to the graphics card according to the mapping record further includes:
[0133] Based on the mapping record, the missing virtual address block is determined to be located in the pre-read buffer, and the missing virtual address block is fed back to the graphics card from the pre-read buffer.
[0134] In one exemplary embodiment of this disclosure, the method for managing graphics card storage resources further includes:
[0135] Read and write operations can be performed on the virtual address blocks located on the solid-state drive by directly accessing the device interface of the solid-state drive.
[0136] In this embodiment of the disclosure, when reading and writing to an SSD, the system treats it as a block device and accesses it directly without going through a traditional file system. This method shortens the software path for read and write operations, resulting in faster access speeds and lower latency.
[0137] Corresponding to the above method embodiments, this disclosure also provides a graphics card storage resource management device, which can be used to execute the above method embodiments.
[0138] Figure 10 This is a block diagram of a graphics card storage resource management device according to an exemplary embodiment of the present disclosure.
[0139] refer to Figure 10 The graphics card storage resource management device 1000 may include:
[0140] Module 1002 is configured to retrieve the access history of virtual address blocks;
[0141] The migration module 1004 is configured to migrate the virtual address block step by step between the graphics card, host memory and solid-state drive within the UVM framework based on the access history.
[0142] In one exemplary embodiment of this disclosure, the migration module 1004 is further configured to:
[0143] In response to insufficient memory space of the graphics card, a first target virtual address block is determined based on the access history;
[0144] Within the UVM framework, the first target virtual address block is migrated from the graphics card to the host memory.
[0145] In one exemplary embodiment of this disclosure, the graphics card storage resource management device 1000 is further configured to:
[0146] Based on the migration path of the virtual address block, a mapping record is generated.
[0147] The mapping record includes a first mapping record migrated from the graphics card to the host memory, and / or a second mapping record migrated from the host memory to the solid-state drive.
[0148] In one exemplary embodiment of this disclosure, the graphics card storage resource management device 1000 is further configured to:
[0149] In response to the fact that the first target virtual address block has been migrated from the graphics card to the host memory, the first mapping record is updated.
[0150] In one exemplary embodiment of this disclosure, the migration module 1004 is further configured to:
[0151] In response to insufficient memory space in the host memory, based on the access history and the mapping record, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory.
[0152] Within the UVM framework, the second target virtual address block is asynchronously written from the host memory to the solid-state drive.
[0153] In one exemplary embodiment of this disclosure, the migration module 1004 is further configured to:
[0154] In response to insufficient memory space in the host memory, migration information for the first target virtual address block is determined based on the first mapping record. This migration information includes migration order and / or timestamps.
[0155] And / or, based on the access history, determine the access frequency of the first target virtual address block;
[0156] And / or, based on the second mapping record, determine the historical storage record of the first target virtual address block in the solid-state drive;
[0157] Based on at least one of the migration information, the access frequency, and the historical storage records, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory.
[0158] In one exemplary embodiment of this disclosure, the graphics card storage resource management device 1000 is further configured to:
[0159] In response to the fact that the second target virtual address block has been asynchronously written from the host memory to the solid-state drive, the first mapping record and the second mapping record are updated.
[0160] In one exemplary embodiment of this disclosure, the graphics card storage resource management device 1000 is further configured to:
[0161] In response to the graphics card needing to access a missing virtual address block, the missing virtual address block is fed back to the graphics card from the current level according to the mapping record.
[0162] The missing virtual address block is a virtual address block that has been migrated from the graphics card to the host memory or the solid-state drive.
[0163] In one exemplary embodiment of this disclosure, the migration module 1004 is further configured to:
[0164] In response to the graphics card needing to access the missing virtual address block, a page fault is generated;
[0165] Based on the page error, obtain the mapping record;
[0166] Based on the mapping record, the missing virtual address block is fed back from the current level to the graphics card.
[0167] In one exemplary embodiment of this disclosure, the migration module 1004 is further configured to:
[0168] Based on the mapping record, it is determined that the missing virtual address block is located in the host memory, and the host memory feeds back to the graphics card.
[0169] Based on the mapping record, the missing virtual address block is determined to be located on the solid-state drive, and the missing virtual address block is synchronously read to the graphics card.
[0170] In one exemplary embodiment of this disclosure, the migration module 1004 is further configured to:
[0171] Based on the mapping record, it is determined that the missing virtual address block is located on the solid-state drive, and a specified number of subsequent virtual address blocks of the missing virtual address block are asynchronously pre-read into the pre-read buffer.
[0172] In one exemplary embodiment of this disclosure, the migration module 1004 is further configured to:
[0173] Based on the mapping record, the missing virtual address block is determined to be located in the pre-read buffer, and the missing virtual address block is fed back to the graphics card from the pre-read buffer.
[0174] In one exemplary embodiment of this disclosure, the graphics card storage resource management device 1000 is further configured to:
[0175] Read and write operations can be performed on the virtual address blocks located on the solid-state drive by directly accessing the device interface of the solid-state drive.
[0176] Since the functions of the device 1000 have been described in detail in their corresponding method embodiments, they will not be repeated here.
[0177] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, in embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0178] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.
[0179] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."
[0180] The following reference Figure 11 To describe an electronic device 1100 of this embodiment of the present disclosure. Figure 11 The electronic device 1100 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.
[0181] like Figure 11 As shown, the electronic device 1100 is manifested in the form of a general-purpose computing device. The components of the electronic device 1100 may include, but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including storage unit 1120 and processing unit 1110).
[0182] The storage unit stores program code that can be executed by the processing unit 1110, causing the processing unit 1110 to perform the steps of the various exemplary embodiments of this disclosure described in the "Exemplary Methods" section above. For example, the processing unit 1110 can perform the method shown in the embodiments of this disclosure.
[0183] Storage unit 1120 may include readable media in the form of volatile storage units, such as random access memory (RAM) 11201 and / or cache memory 11202, and may further include read-only memory (ROM) 11203.
[0184] Storage unit 1120 may also include a program / utility 11204 having a set (at least one) program module 11205, such program module 11205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.
[0185] Bus 1130 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.
[0186] Electronic device 1100 can also communicate with one or more external devices 1140 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1100, and / or any device that enables electronic device 1100 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1150. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1160. As shown, network adapter 1160 communicates with other modules of electronic device 1100 via bus 1130. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device migraters, redundant processing units, external disk migration arrays, RAID systems, tape migraters, and data backup storage systems.
[0187] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods of the embodiments of this disclosure.
[0188] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of this disclosure described in the "Exemplary Methods" section above.
[0189] The program product for implementing the above methods according to embodiments of this disclosure can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0190] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0191] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.
[0192] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, or any suitable combination thereof.
[0193] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java and C++, and conventional procedural programming languages such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).
[0194] Furthermore, the above figures are merely illustrative of the processes included in the methods of exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.
[0195] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and concept of this disclosure are indicated by the claims.
Claims
1. A method for managing graphics card storage resources, characterized in that, include: Retrieve the access history of virtual address blocks; Based on the access history, within the UVM framework, the virtual address block is migrated step-by-step between the graphics card, host memory, and solid-state drive, including: In response to insufficient memory space of the graphics card, a first target virtual address block is determined based on the access history; Within the UVM framework, the first target virtual address block is migrated from the graphics card to the host memory; Based on the migration path of the virtual address block, a mapping record is generated. The mapping record includes a first mapping record migrated from the graphics card to the host memory, and / or a second mapping record migrated from the host memory to the solid-state drive; In response to insufficient memory space in the host memory, based on the access history and the mapping record, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory. Within the UVM framework, the second target virtual address block is asynchronously written from the host memory to the solid-state drive; In response to the graphics card needing to access a missing virtual address block, the missing virtual address block is fed back to the graphics card from the current level according to the mapping record. The missing virtual address block is a virtual address block that has been migrated from the graphics card to the host memory or the solid-state drive.
2. The method for managing graphics card storage resources according to claim 1, characterized in that, The methods for managing graphics card storage resources also include: In response to the fact that the first target virtual address block has been migrated from the graphics card to the host memory, the first mapping record is updated.
3. The method for managing graphics card storage resources according to claim 1, characterized in that, In response to insufficient memory space in the host memory, based on the access history and the mapping record, the process of filtering and determining a second target virtual address block from the first target virtual address block that has been migrated to the host memory includes: In response to insufficient memory space in the host memory, migration information for the first target virtual address block is determined based on the first mapping record. This migration information includes migration order and / or timestamps. And / or, based on the access history, determine the access frequency of the first target virtual address block; And / or, based on the second mapping record, determine the historical storage record of the first target virtual address block in the solid-state drive; Based on at least one of the migration information, the access frequency, and the historical storage records, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory.
4. The method for managing graphics card storage resources according to claim 1, characterized in that, The methods for managing graphics card storage resources also include: In response to the fact that the second target virtual address block has been asynchronously written from the host memory to the solid-state drive, the first mapping record and the second mapping record are updated.
5. The method for managing graphics card storage resources according to claim 1, characterized in that, In response to the graphics card needing to access a missing virtual address block, the method of feeding back the missing virtual address block from the current level to the graphics card according to the mapping record further includes: In response to the graphics card needing to access the missing virtual address block, a page fault is generated; Based on the page error, obtain the mapping record; Based on the mapping record, the missing virtual address block is fed back from the current level to the graphics card.
6. The method for managing graphics card storage resources according to claim 5, characterized in that, According to the mapping record, feeding back the missing virtual address block from the current level to the graphics card includes: Based on the mapping record, it is determined that the missing virtual address block is located in the host memory, and the host memory feeds back to the graphics card. Based on the mapping record, the missing virtual address block is determined to be located on the solid-state drive, and the missing virtual address block is synchronously read to the graphics card.
7. The method for managing graphics card storage resources according to claim 5, characterized in that, According to the mapping record, feeding back the missing virtual address block from the current level to the graphics card also includes: Based on the mapping record, it is determined that the missing virtual address block is located on the solid-state drive, and a specified number of subsequent virtual address blocks of the missing virtual address block are asynchronously pre-read into the pre-read buffer.
8. The method for managing graphics card storage resources according to claim 7, characterized in that, According to the mapping record, feeding back the missing virtual address block from the current level to the graphics card also includes: Based on the mapping record, the missing virtual address block is determined to be located in the pre-read buffer, and the missing virtual address block is fed back to the graphics card from the pre-read buffer.
9. The method for managing graphics card storage resources according to claim 1, characterized in that, The methods for managing graphics card storage resources also include: By directly accessing the device interface of the solid-state drive (SSD), read and write operations are performed on the virtual address blocks located on the SSD, including: In response to insufficient memory space of the graphics card, a first target virtual address block is determined based on the access history; Within the UVM framework, the first target virtual address block is migrated from the graphics card to the host memory; Based on the migration path of the virtual address block, a mapping record is generated. The mapping record includes a first mapping record migrated from the graphics card to the host memory, and / or a second mapping record migrated from the host memory to the solid-state drive; In response to insufficient memory space in the host memory, based on the access history and the mapping record, a second target virtual address block is selected from the first target virtual address block that has been migrated to the host memory. Within the UVM framework, the second target virtual address block is asynchronously written from the host memory to the solid-state drive; In response to the graphics card needing to access a missing virtual address block, the missing virtual address block is fed back to the graphics card from the current level according to the mapping record. The missing virtual address block is a virtual address block that has been migrated from the graphics card to the host memory or the solid-state drive.
10. A device for managing graphics card storage resources, characterized in that, include: The acquisition module is configured to retrieve the access history of virtual address blocks; The migration module is configured to migrate the virtual address block step by step between the graphics card, host memory, and solid-state drive within the UVM framework based on the access history.
11. An electronic device, characterized in that, include: Memory; as well as A processor coupled to the memory, the processor being configured to execute the graphics card storage resource management method as described in any one of claims 1-9 based on instructions stored in the memory.
12. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the method for managing graphics card storage resources as described in any one of claims 1-9.
13. A computer program product, characterized in that... It includes a computer program that, when executed by a processor, implements the method for managing graphics card storage resources as described in any one of claims 1-9.