GPU video memory management method, system and equipment based on excess subscription and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a GPU video memory management method, system and device based on excess subscription and a storage medium, and the method comprises the steps: obtaining device and task configuration in response to a video memory allocation request of a GPU task, and calculating an excess subscription proportion; if the GPU task is 0, the VMM constructs a video memory pool based on GPU task configuration, directly allocates a video memory from the video memory pool and releases the video memory back to the video memory pool after the GPU task is completed; if not, the VMM constructs a video memory pool and a lock page memory pool based on GPU task configuration and GPU equipment configuration, and selects the video memory pool or the lock page memory pool to carry out video memory allocation based on a scheduling strategy; whether a migration request exists or not is judged, if yes, migration operation is carried out, and video memory distribution is carried out through the migrated video memory pool or the page locking memory pool; and destroying the video memory pool and the page locking memory pool in response to an exit request of the GPU task. According to the method and the device, the video memory resources of the GPU can be utilized more efficiently under the existing hardware condition.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a GPU memory management method, system, device and storage medium based on oversubscription. Background Art

[0002] With the widespread adoption of computationally intensive tasks such as graphics rendering, deep learning, and neural network model training, GPUs (graphics processing units), as highly parallel general-purpose computing platforms, have become a key computing resource supporting these tasks. GPUs demonstrate significant performance advantages in complex image processing, model calculations, and parallel matrix operations, and have been widely deployed in various high-performance computing scenarios.

[0003] In existing technologies, GPU workloads typically use the physical video memory provided by the GPU device during execution to store information such as intermediate computational data, model parameters, and input and output data. To meet the needs of different tasks, the GPU driver supports dynamic allocation and deallocation of video memory. Based on runtime memory requests, tasks obtain a certain amount of physical video memory resources and perform read and write operations. This memory utilization mechanism offers low latency and high bandwidth, ensuring excellent execution performance for tasks.

[0004] However, with the prevalence of scenarios like multi-task parallel processing and large-scale model training, the physical memory capacity of a single GPU device is increasingly becoming a major constraint on improving system performance. Under existing hardware conditions, memory resources are relatively limited, making it difficult to support the parallel execution of multiple complex tasks. Traditional approaches rely primarily on improving GPU hardware configuration, such as increasing memory capacity or deploying multi-GPU systems. However, such approaches are not only costly but also have limitations in terms of flexibility and resource utilization efficiency. Therefore, how to more efficiently utilize GPU memory resources under existing hardware conditions has become a core technical issue that urgently needs to be addressed. Summary of the Invention

[0005] This application provides a GPU memory management method, system, device, and storage medium based on oversubscription, which can more efficiently utilize GPU memory resources under existing hardware conditions. This application provides the following technical solutions: In a first aspect, the present application provides a GPU memory management method based on oversubscription, the method comprising: In response to a video memory allocation request of a GPU task, obtaining a GPU device configuration and a GPU task configuration, and calculating an oversubscription ratio of the GPU task to the GPU device; If the oversubscription ratio is 0, the VMM builds a video memory pool based on the GPU task configuration, allocates video memory directly from the video memory pool, and releases it back to the video memory pool after the GPU task is completed; If the oversubscription ratio is not 0, the VMM constructs a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration, and selects the video memory pool or the page-locked memory pool for video memory allocation based on a pre-set scheduling policy; determines whether there is a migration request, and if so, performs a migration operation between the video memory pool and the page-locked memory pool, and allocates video memory using the migrated video memory pool or the page-locked memory pool; releases the allocated video memory after the GPU task is completed, and destroys the video memory pool and the page-locked memory pool in response to the exit request of the GPU task.

[0006] In a specific possible implementation scheme, in response to a graphics memory allocation request of a GPU task, obtaining a GPU device configuration and a GPU task configuration, and calculating an oversubscription ratio of the GPU task to the GPU device include: The GPU device configuration represents the physical memory capacity of the target GPU device, and the GPU task configuration represents the memory capacity required by the GPU task during operation; The oversubscription ratio of the GPU task to the current GPU device is calculated based on the obtained GPU device configuration and GPU task configuration. If the GPU task configuration is less than or equal to the GPU device configuration, the oversubscription ratio of the GPU task to the current GPU device is considered to be 0. Otherwise, the calculation method is as follows: Oversubscription ratio = (GPU task configuration - GPU device configuration) / GPU device configuration.

[0007] In a specific possible implementation scheme, if the oversubscription ratio is 0, the VMM builds a video memory pool based on the GPU task configuration, directly allocates video memory from the video memory pool, and releases it back to the video memory pool after the GPU task is completed, including: Based on the obtained GPU device configuration and GPU task configuration, under the control of the VMM, the required video memory resources are pre-applied to the GPU device and a corresponding video memory pool is constructed. The size of the video memory pool is the video memory requirement declared in the GPU task configuration. When the GPU task initiates a video memory application request, the required video memory segment is directly allocated from the video memory pool; after the video memory is used, the video memory segment is released back to the video memory pool; When the GPU task issues a task exit request to the VMM, the video memory pool is destroyed.

[0008] In a specific implementation scheme, if the oversubscription ratio is not 0, the VMM constructing the video memory pool and the page-locked memory pool based on the GPU task configuration and the GPU device configuration includes: Under the control of the VMM, the video memory pool is initialized and built based on the GPU device configuration and GPU task configuration, and a page-locked memory pool is built that is proportional to the video memory pool. The size of the video memory pool is the physical video memory capacity of the GPU device, and the size of the page-locked memory pool is as follows: Page-locked memory pool size = video memory pool size × oversubscription ratio.

[0009] In a specific implementation scheme, selecting a video memory pool or a page-locked memory pool for video memory allocation based on a pre-set scheduling policy includes: When the GPU task initiates a video memory allocation request, the VMM returns a fixed-length virtual address. The virtual address represents the video memory space logically used by the GPU task and is used to abstract the underlying physical memory. The VMM determines the actual physical memory location to which the virtual address should be mapped based on the currently enabled video memory scheduling policy.

[0010] In a specific implementation scheme, the scheduling strategy includes a video memory priority strategy, a memory priority strategy, and a balance strategy: The video memory priority strategy is that the VMM first attempts to map the virtual address to the available fixed-length video memory segment in the video memory pool; if the video memory pool capacity is insufficient, it falls back to allocating page-locked memory from the page-locked memory pool and completes the mapping; The memory priority strategy is that the VMM first attempts to map the virtual address to the available fixed-length page-locked memory in the page-locked memory pool; if the page-locked memory is insufficient, it falls back to allocating video memory resources from the video memory pool and completes the mapping; The balancing strategy is that when allocating video memory resources, the VMM first calculates the real-time usage ratio of the current video memory pool and the page-locked memory pool, that is, the ratio between the current page-locked memory usage and the video memory usage; then compares the real-time usage ratio with the oversubscription ratio obtained by the initial calculation of the task; if the current page-locked memory usage ratio is lower than the target oversubscription ratio, it means that the page-locked memory is underused, and resources are allocated from the page-locked memory pool first; on the contrary, if the page-locked memory usage ratio is higher than the target ratio, resources are allocated from the video memory pool first.

[0011] In a specific implementation scheme, determining whether a migration request exists, and if so, performing a migration operation between a video memory pool and a page-locked memory pool, and allocating video memory using the migrated video memory pool or the page-locked memory pool includes: The VMM determines whether there is a migration requirement for each allocated virtual address segment; If the judgment result is no, skip this step and keep the original mapping relationship unchanged; If there is a migration requirement, the VMM performs the following operations: for the fixed-length virtual address to be migrated, locate its physical memory segment in the original pool, and copy the data of the physical memory segment to the physical area of ​​the corresponding size in the target pool to which it needs to be migrated; after the copy is completed, release the mapping relationship between the virtual address and the physical memory segment in the original pool, and remap the virtual address to the physical address where the newly copied data is located in the target pool to which it needs to be migrated.

[0012] In a second aspect, the present application provides a GPU memory management system based on oversubscription, which adopts the following technical solutions: A GPU memory management system based on oversubscription, comprising: A task start module is used to respond to a video memory allocation request of a GPU task, obtain a GPU device configuration and a GPU task configuration, and calculate an oversubscription ratio of the GPU task to the GPU device; a direct allocation module, configured to, if the oversubscription ratio is 0, cause the VMM to construct a video memory pool based on the GPU task configuration, directly allocate video memory from the video memory pool, and release the video memory back to the video memory pool after the GPU task is completed; an over-allocation module, configured to, if the over-subscription ratio is not 0, cause the VMM to construct a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration, and select the video memory pool or the page-locked memory pool for video memory allocation based on a pre-set scheduling policy; A migration request module is used to determine whether a migration request exists. If so, a migration operation is performed between the video memory pool and the page-locked memory pool, and the video memory is allocated using the migrated video memory pool or the page-locked memory pool. The task ending module is used to release the allocated video memory after the GPU task ends, and destroy the video memory pool and the page lock memory pool in response to the exit request of the GPU task.

[0013] In a third aspect, the present application provides an electronic device comprising a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a GPU memory management method based on oversubscription as described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the storage medium stores a program, and when the program is executed by a processor, it is used to implement a GPU memory management method based on oversubscription as described in the first aspect.

[0015] In summary, the beneficial effects of this application include at least: (1) By introducing a dual-pool mechanism of video memory pool and page-locked memory pool, and combining it with video memory virtual address mapping and scheduling strategies, oversubscription management of GPU video memory resources is achieved. Specifically, for tasks with insufficient video memory, this application uses page-locked memory as an extension of video memory, and indirectly supports data reading and writing of GPU tasks through DMA access, thereby logically expanding the total amount of available video memory. In addition, based on the unified virtual address management mechanism of the VMM, video memory resources can be dynamically mapped, recycled and reused on demand, avoiding the fragmentation problem that occurs in traditional on-demand allocation and improving the efficiency of video memory resource utilization. Through the above mechanism, even if the physical video memory remains unchanged, more tasks can be supported to run in parallel, significantly improving the overall video memory carrying capacity of the system.

[0016] (2) Compared with the traditional method of relying on expanding GPU hardware to meet the video memory requirements of high-concurrency tasks, the oversubscription mechanism proposed in this application uses existing system memory resources (page-locked memory) as a supplementary resource for video memory, which can improve the system's processing power without adding new hardware. Page-locked memory is requested through the collaboration between the driver and the operating system. It has the characteristics of non-paged, strong controllability, and DMA access. It can provide equivalent alternative memory support for GPU tasks without affecting video memory access performance. With the help of this mechanism, more video memory binding tasks can be supported under existing hardware conditions, thereby effectively reducing the equipment cost and energy consumption required by the system to meet computing power requirements and optimizing the system's total cost of ownership.

[0017] (3) The scheduling strategy proposed in this application supports three modes: video memory priority, memory priority, and resource balancing. It can dynamically adjust the resource allocation path according to the task load, the proportion of remaining resources in the pool, and the preset oversubscription ratio. In particular, under the balancing strategy, the deviation between the current usage ratio and the target ratio of the video memory pool and the page-locked memory pool can be compared in real time, and the optimal allocation source can be intelligently selected to effectively maintain resource balance. On this basis, the data migration mechanism between video memory and page-locked memory is further supported. For example, when the CPU is detected to be involved in an operation (such as a video memory snapshot) during a task, the originally allocated video memory mapping can be actively migrated to the CPU-readable page-locked memory, thereby adapting to changes in resource requirements under different operating stages. The above design improves the resource scheduling flexibility of the system in the face of diverse loads and sudden demands, and enhances the flexibility and scalability of the overall system.

[0018] First, determine whether the video memory required by the GPU task exceeds the GPU device capacity. If there is no excess, pre-allocate and reuse the video memory by building a task-specific video memory pool. If there is an excess, the video memory pool is built simultaneously with a proportionally expanded page-locked memory pool, and a schedulable allocation strategy is introduced to dynamically select the source of physical resources. To meet the migration needs of some tasks, a secondary migration and remapping mechanism for virtual addresses is further designed. Finally, through a unified resource release logic driven by task exit, the pool resources are recycled and the pool body is destroyed. Through video memory pool pre-allocation, page-locked memory compensation, flexible scheduling, and dynamic migration, the logical video memory capacity is effectively expanded, fragmentation overhead is eliminated, and resource balance is maintained based on real-time usage characteristics. This significantly improves the overall utilization efficiency of the GPU video memory without increasing the hardware video memory, effectively solving the performance bottleneck caused by insufficient video memory.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and to implement it in accordance with the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of a GPU memory management method based on oversubscription in an embodiment of the present application.

[0021] Figure 2 This is a schematic diagram of the overall process of the GPU memory management method based on oversubscription in an embodiment of the present application.

[0022] Figure 3 This is a structural block diagram of a GPU memory management system based on oversubscription in an embodiment of the present application.

[0023] Figure 4 This is a block diagram of an electronic device for oversubscribed GPU memory management in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0025] First, several terms involved in this application are introduced.

[0026] A GPU (Graphics Processing Unit) is a specialized processor designed to handle graphics and image computing tasks. Although originally designed to accelerate graphics rendering, modern GPUs have evolved into powerful computing devices that are widely used for a variety of computationally intensive tasks.

[0027] GPU workload: This refers to the total amount of computing, memory, and other resources consumed by various applications or tasks running on the GPU. It reflects the GPU's workload and resource requirements within a specific time period.

[0028] System memory: general-purpose high-speed memory directly accessed and managed by the CPU on the system, characterized by high-speed reading and writing and power-off clearing.

[0029] GPU physical memory: The memory storage that comes with the GPU device, usually directly accessed by the GPU graphics processing chip, has lower access latency and higher bandwidth than system memory.

[0030] Page-locked memory: It is part of the system memory and is locked in the system page table. It will not be paged to the disk by the system, and the corresponding physical memory will not be released to other processes. Its priority and performance are higher than those of ordinary memory.

[0031] VMM: Virtual Memory Manager, responsible for the management and maintenance of virtual addresses and underlying memory address mappings, used to map the underlying video memory and physical memory to virtual addresses, ensuring that the location of the underlying memory data (graphics card or physical memory) is dynamically changed without changing the application address.

[0032] DMA: Memory pass-through allows bus devices to directly access system memory without going through the CPU, improving performance and reducing system overhead.

[0033] Optionally, the present application uses the oversubscription-based GPU memory management method provided in each embodiment as an example for explanation in an electronic device, where the electronic device is a terminal or a server. The terminal may be a computer, a tablet computer, etc. This embodiment does not limit the type of electronic device.

[0034] Reference Figure 1 , is a flow chart of a method for managing GPU memory based on oversubscription provided by an embodiment of the present application, the method comprising at least the following steps: Step S101 : In response to a video memory allocation request of a GPU task, obtain a GPU device configuration and a GPU task configuration, and calculate an oversubscription ratio of the GPU task to the GPU device.

[0035] In step S101, for the video memory allocation request initiated by the GPU task, relevant resource configuration information is obtained, and it is determined whether the video memory requested by the task exceeds the available capacity of the current GPU device, thereby calculating the oversubscription ratio of the GPU task to the current GPU device.

[0036] Specifically, first obtain the GPU device configuration and GPU task configuration. The GPU device configuration represents the physical memory capacity of the target GPU device, and the GPU task configuration represents the memory capacity required by the GPU task during operation, which is usually determined by the upper-level task scheduling system during the task issuance phase. Then, based on the obtained GPU device configuration and GPU task configuration, the oversubscription ratio of the GPU task for the current GPU device is calculated. If the GPU task configuration is less than or equal to the GPU device configuration, the oversubscription ratio of the GPU task for the current GPU device is deemed to be 0. Otherwise, the calculation method is as follows: Oversubscription ratio = (GPU task configuration - GPU device configuration) / GPU device configuration; For example, if a GPU task requests 1.5GB of video memory and the physical video memory capacity of the target GPU device is 1.0GB, the corresponding oversubscription ratio is 0.5.

[0037] Step S102: If the oversubscription ratio is 0, the VMM builds a video memory pool based on the GPU task configuration, allocates video memory directly from the video memory pool, and releases it back to the video memory pool after the GPU task is completed.

[0038] In step S102, when the oversubscription ratio of the GPU task to the GPU device is 0, the video memory allocation request corresponding to the GPU task is completed by initializing the video memory pool, and the allocated video memory is released and recycled after the GPU task is completed.

[0039] Specifically, based on the acquired GPU device and task configurations, under the control of the Virtual Memory Manager (VMM), the GPU device pre-application for required video memory resources is initiated and a corresponding memory pool is constructed. The size of the memory pool is the memory requirement specified in the GPU task configuration. The memory pool serves as a memory resource buffer for the GPU task, reserving a certain size of physical memory segments for use by subsequent tasks. Subsequently, during GPU task execution, whenever the task initiates a memory request, the required memory segments are directly allocated from the constructed memory pool to meet the task's real-time memory needs. Compared to traditional on-demand allocation, this allocation operation eliminates the need to request physical memory from the GPU driver layer, effectively reducing the system overhead of kernel-mode memory allocation and release. When the GPU task completes a memory usage session, the memory segment is released back to the memory pool, ensuring memory resource recycling and providing free space for subsequent memory requests. Finally, when the GPU task issues a task exit request to the VMM, the memory pool is destroyed.

[0040] Step S103: If the oversubscription ratio is not 0, the VMM constructs a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration, and selects the video memory pool or the page-locked memory pool for video memory allocation based on a pre-set scheduling policy.

[0041] In step S103, combined Figure 2 , in response to the situation where the GPU task's request for GPU memory exceeds the physical memory capacity of the GPU device, that is, the oversubscription ratio is not 0, a scheduling strategy is introduced to dynamically allocate memory resources by constructing an underlying memory pool consisting of a memory pool and a page-locked memory pool. Under the control of the VMM, the memory pool is initialized and constructed based on the GPU device configuration and the GPU task configuration, and a page-locked memory pool with a proportional relationship to the memory pool is constructed. At this time, the size of the memory pool is the physical memory capacity of the GPU device, and the size of the page-locked memory pool is determined by the following formula: Page-locked memory pool size = memory pool size × oversubscription ratio. Page-locked memory is usually requested by the GPU driver through interaction with the operating system. It is a type of memory that can be accessed by DMA, has the characteristics of non-paged and strong controllability, and is suitable for indirect access scenarios of the GPU. Therefore, it is used as an auxiliary resource for GPU memory in this application.

[0042] Specifically, during the execution of a GPU task, whenever it attempts to initiate a video memory allocation request, the VMM first returns a fixed-length virtual address. This virtual address is uniformly managed by the VMM and represents the video memory space logically used by the GPU task. It is used to abstract the underlying physical memory resources and ensure that subsequent resource access processes have good logical consistency and address isolation. Subsequently, the VMM determines the actual physical memory location to which the virtual address should be mapped based on the currently enabled video memory scheduling policy. Specifically, there are three scheduling policies: Memory priority strategy: The VMM first attempts to map the virtual address to the available fixed-length memory segment in the memory pool. If the memory pool capacity is insufficient, it falls back to allocating page-locked memory from the page-locked memory pool and completes the mapping. Memory priority strategy: The VMM first attempts to map the virtual address to the available fixed-length page-locked memory in the page-locked memory pool. If the page-locked memory is insufficient, it falls back to allocating video memory resources from the video memory pool and completes the mapping. Balance strategy: When allocating video memory resources, the VMM first calculates the real-time usage ratio of the current video memory pool and the page-locked memory pool, that is, the ratio between the current page-locked memory usage and the video memory usage; then, it compares the real-time usage ratio with the target oversubscription ratio obtained by the initial calculation of the task. If the current page-locked memory usage ratio is lower than the target oversubscription ratio, it means that the page-locked memory is underused. In order to get closer to the target ratio, resources should be allocated from the page-locked memory pool first; on the contrary, if the page-locked memory usage ratio is higher than the target ratio, resources should be allocated from the video memory pool first. In this way, the usage of the two types of memory can be continuously adjusted during the dynamic allocation process, so that their actual usage ratio is as close as possible to the preset oversubscription ratio, thereby maintaining the balance and rationality of resource usage.

[0043] After resource selection is completed, the VMM establishes a mapping relationship between the virtual address and the acquired underlying physical memory (video memory or page-locked memory). Subsequent access operations of the GPU task can transparently read and write the actual physical resources through the virtual address.

[0044] Step S104: determine whether there is a migration request. If so, perform a migration operation between the video memory pool and the page-locked memory pool, and allocate video memory using the migrated video memory pool or the page-locked memory pool.

[0045] In step S104 , for the completed video memory allocation of the GPU task, the VMM performs a secondary determination on the allocated memory pool type to determine whether resource allocation needs to be switched between the video memory pool and the page-locked memory pool.

[0046] Specifically, the VMM determines whether there is a migration requirement for each allocated virtual address segment. If the judgment result is no, this step is skipped and the original mapping relationship is retained unchanged; if there is a migration requirement, the VMM performs the following operations: for the fixed-length virtual address to be migrated, it locates its physical memory segment in the video memory pool or the page-locked memory pool, that is, the original pool, and copies the data of the physical memory segment to the video memory pool or the page-locked memory pool to be migrated, that is, the physical area of ​​the corresponding size in the target pool. After the copy is completed, the mapping relationship between the virtual address and the physical memory segment in the original video memory pool or the page-locked memory pool is released, and the virtual address is remapped to the physical address where the newly copied data is located in the video memory pool or the page-locked memory pool to be migrated.

[0047] It should be noted that the analysis of whether there is a migration request includes but is not limited to: the GPU task uses task parameters to identify its access type during the allocation phase, and the VMM uses this to determine whether it needs to be migrated to page-locked memory first; or, if the VMM detects that the task includes data processing processes in which the CPU actively participates, such as typical scenarios such as snapshot saving, log generation, and exporting intermediate model results, and the current resources have been allocated in the CPU-unreadable video memory pool, a migration request will be automatically triggered.

[0048] Taking the video memory snapshot function as an example, this function requires that the video memory data currently used by the GPU task be exported and made available to the CPU for reading and analysis. However, since GPU video memory generally does not support direct CPU access, if the video memory is first scheduled to be allocated by the video memory pool, if the video memory pool mapping is retained, it will be difficult to meet the readability requirements during the snapshot process. Therefore, after detecting that the snapshot task is triggered, the VMM will automatically copy the target data mapped in the video memory pool to the page-locked memory pool and re-establish the mapping relationship between the virtual address and the page-locked memory, thereby ensuring that the data area corresponding to the address can be directly accessed by the CPU.

[0049] Step S105 : After the GPU task is completed, the allocated video memory is released, and in response to the exit request of the GPU task, the video memory pool and the page-locked memory pool are destroyed.

[0050] In step S105 , when the GPU task is completed, the VMM releases the allocated video memory and destroys the video memory pool and the page-locked memory pool in sequence according to a unified release mechanism, ensuring that the underlying resources are effectively recovered.

[0051] Specifically, after the GPU task completes its execution, the VMM receives a request to release the virtual address. For each fixed-length virtual address allocated by the GPU task, the VMM releases its mapping relationship with the underlying physical memory (including the corresponding physical address segment in the video memory pool or the page-locked memory pool); the unmapped physical address segment is returned to the corresponding memory pool for reuse by subsequent tasks; at the same time, the mapping status of the virtual address is reset to ensure that there will be no conflict or overwriting in subsequent mapping relationships. Subsequently, when the GPU task issues a task exit request to the VMM, the video memory pool and page-locked memory pool bound to the GPU task are destroyed. This process includes: releasing the remaining unused memory segments in the pool, deregistering all control structures related to the pool, and clearing the resource allocation and mapping table entries of the task recorded in the VMM, thereby completing the recycling and cleanup of the entire set of memory resources.

[0052] To summarize, the first step is to determine whether the video memory required by the GPU task exceeds the GPU device capacity. If there is no excess, a task-specific video memory pool is constructed for video memory pre-allocation and recycling. If there is an excess, a video memory pool is constructed simultaneously with a proportionally expanded page-locked memory pool, and a schedulable allocation strategy is introduced to dynamically select the source of physical resources. To accommodate the migration needs of some tasks, a secondary migration and remapping mechanism for virtual addresses is further designed. Finally, unified resource release logic driven by task exit is used to reclaim resources within the pool and destroy the pool body. Through video memory pool pre-allocation, page-locked memory compensation, flexible scheduling, and dynamic migration, the logical video memory capacity is effectively expanded, fragmentation overhead is eliminated, and resource balance is maintained based on real-time usage characteristics. This significantly improves the overall utilization efficiency of the GPU video memory without increasing hardware video memory, effectively resolving the performance bottleneck caused by insufficient video memory.

[0053] Figure 3 This is a structural block diagram of a GPU memory management system based on oversubscription provided by an embodiment of the present application. The system includes at least the following modules: The task start module is used to respond to the memory allocation request of the GPU task, obtain the GPU device configuration and GPU task configuration, and calculate the oversubscription ratio of the GPU task to the GPU device; Direct allocation module, used when the oversubscription ratio is 0, VMM builds a memory pool based on GPU task configuration, allocates video memory directly from the memory pool, and releases it back to the memory pool after the GPU task is completed; Over-allocation module, used for constructing a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration if the over-subscription ratio is not 0, and selecting the video memory pool or the page-locked memory pool for video memory allocation based on a pre-set scheduling policy; A migration request module is used to determine whether a migration request exists. If so, a migration operation is performed between the video memory pool and the page-locked memory pool, and the video memory is allocated using the migrated video memory pool or the page-locked memory pool. The task ending module is used to release the allocated video memory after the GPU task ends, and destroy the video memory pool and the page-locked memory pool in response to the exit request of the GPU task.

[0054] For relevant details, please refer to the above method embodiment.

[0055] Figure 4 4 is a block diagram of an electronic device provided in one embodiment of the present application. The device includes at least a processor 401 and a memory 402.

[0056] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content displayed on the display screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0057] Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 402 is used to store at least one instruction, which is executed by processor 401 to implement the oversubscription-based GPU memory management method provided in the method embodiment of the present application.

[0058] In some embodiments, the electronic device may optionally include a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface via a bus, signal lines, or circuit boards. Illustratively, the peripheral devices include, but are not limited to, radio frequency circuitry, a touchscreen display, audio circuitry, and a power supply.

[0059] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.

[0060] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored. The program is loaded and executed by a processor to implement the GPU memory management method based on oversubscription of the above method embodiment.

[0061] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the oversubscription-based GPU memory management method of the above-mentioned method embodiment.

[0062] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A GPU memory management method based on oversubscription, characterized in that: The method comprises: In response to a video memory allocation request of a GPU task, obtaining a GPU device configuration and a GPU task configuration, and calculating an oversubscription ratio of the GPU task to the GPU device; If the oversubscription ratio is 0, the VMM builds a video memory pool based on the GPU task configuration, allocates video memory directly from the video memory pool, and releases it back to the video memory pool after the GPU task is completed; If the oversubscription ratio is not 0, the VMM constructs a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration, and selects the video memory pool or the page-locked memory pool for video memory allocation based on a pre-set scheduling policy; determines whether there is a migration request, and if so, performs a migration operation between the video memory pool and the page-locked memory pool, and allocates video memory using the migrated video memory pool or the page-locked memory pool; releases the allocated video memory after the GPU task is completed, and destroys the video memory pool and the page-locked memory pool in response to the exit request of the GPU task.

2. The GPU memory management method based on oversubscription according to claim 1, characterized in that: The step of obtaining the GPU device configuration and the GPU task configuration in response to the video memory allocation request of the GPU task, and calculating the oversubscription ratio of the GPU task to the GPU device includes: The GPU device configuration represents the physical memory capacity of the target GPU device, and the GPU task configuration represents the memory capacity required by the GPU task during operation; The oversubscription ratio of the GPU task to the current GPU device is calculated based on the obtained GPU device configuration and GPU task configuration. If the GPU task configuration is less than or equal to the GPU device configuration, the oversubscription ratio of the GPU task to the current GPU device is considered to be 0. Otherwise, the calculation method is as follows: Oversubscription ratio = (GPU task configuration - GPU device configuration) / GPU device configuration.

3. The GPU memory management method based on oversubscription according to claim 1, characterized in that: If the oversubscription ratio is 0, the VMM builds a video memory pool based on the GPU task configuration, directly allocates video memory from the video memory pool, and releases it back to the video memory pool after the GPU task is completed, including: Based on the obtained GPU device configuration and GPU task configuration, under the control of the VMM, the required video memory resources are pre-applied to the GPU device and a corresponding video memory pool is constructed. The size of the video memory pool is the video memory requirement declared in the GPU task configuration. When the GPU task initiates a video memory application request, the required video memory segment is directly allocated from the video memory pool; after the video memory is used, the video memory segment is released back to the video memory pool; When the GPU task issues a task exit request to the VMM, the video memory pool is destroyed.

4. The GPU memory management method based on oversubscription according to claim 1, characterized in that: If the oversubscription ratio is not 0, the VMM constructs a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration, including: Under the control of the VMM, the video memory pool is initialized and built based on the GPU device configuration and GPU task configuration, and a page-locked memory pool is built that is proportional to the video memory pool. The size of the video memory pool is the physical video memory capacity of the GPU device, and the size of the page-locked memory pool is as follows: Page-locked memory pool size = video memory pool size × oversubscription ratio.

5. The GPU memory management method based on oversubscription according to claim 4, characterized in that: The selecting of a video memory pool or a page-locked memory pool for video memory allocation based on a preset scheduling policy includes: When the GPU task initiates a video memory allocation request, the VMM returns a fixed-length virtual address. The virtual address represents the video memory space logically used by the GPU task and is used to abstract the underlying physical memory. The VMM determines the actual physical memory location to which the virtual address should be mapped based on the currently enabled video memory scheduling policy.

6. The GPU memory management method based on oversubscription according to claim 5, characterized in that: The scheduling strategies include video memory priority strategy, memory priority strategy and balance strategy: The video memory priority strategy is that the VMM first attempts to map the virtual address to the available fixed-length video memory segment in the video memory pool; if the video memory pool capacity is insufficient, it falls back to allocating page-locked memory from the page-locked memory pool and completes the mapping; The memory priority strategy is that the VMM first attempts to map the virtual address to the available fixed-length page-locked memory in the page-locked memory pool; if the page-locked memory is insufficient, it falls back to allocating video memory resources from the video memory pool and completes the mapping; The balancing strategy is that when the VMM allocates video memory resources, it first calculates the real-time usage ratio of the current video memory pool and the page-locked memory pool, that is, the ratio between the current page-locked memory usage and the video memory usage; The real-time usage ratio is then compared with the oversubscription ratio obtained by the initial calculation of the task; if the current page-locked memory usage ratio is lower than the target oversubscription ratio, it means that the page-locked memory is insufficiently used, and resources are allocated from the page-locked memory pool first; on the contrary, if the page-locked memory usage ratio is higher than the target ratio, resources are allocated from the video memory pool first.

7. The GPU memory management method based on oversubscription according to claim 5, characterized in that: The determining whether there is a migration request, and if so, performing a migration operation between the video memory pool and the page-locked memory pool, and allocating video memory using the migrated video memory pool or the page-locked memory pool includes: The VMM determines whether there is a migration requirement for each allocated virtual address segment; If the judgment result is no, skip this step and keep the original mapping relationship unchanged; If there is a migration requirement, the VMM performs the following operations: for the fixed-length virtual address to be migrated, locate its physical memory segment in the original pool, and copy the data of the physical memory segment to the physical area of ​​the corresponding size in the target pool to which it needs to be migrated; after the copy is completed, release the mapping relationship between the virtual address and the physical memory segment in the original pool, and remap the virtual address to the physical address where the newly copied data is located in the target pool to which it needs to be migrated.

8. A GPU memory management system based on oversubscription, characterized in that: include: A task start module is used to respond to a video memory allocation request of a GPU task, obtain a GPU device configuration and a GPU task configuration, and calculate an oversubscription ratio of the GPU task to the GPU device; a direct allocation module, configured to, if the oversubscription ratio is 0, cause the VMM to construct a video memory pool based on the GPU task configuration, directly allocate video memory from the video memory pool, and release the video memory back to the video memory pool after the GPU task is completed; an over-allocation module, configured to, if the over-subscription ratio is not 0, cause the VMM to construct a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration, and select the video memory pool or the page-locked memory pool for video memory allocation based on a pre-set scheduling policy; A migration request module is used to determine whether a migration request exists. If so, a migration operation is performed between the video memory pool and the page-locked memory pool, and the video memory is allocated using the migrated video memory pool or the page-locked memory pool. The task ending module is used to release the allocated video memory after the GPU task ends, and destroy the video memory pool and the page lock memory pool in response to the exit request of the GPU task.

9. An electronic device, characterized in that: The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the GPU memory management method based on oversubscription according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores a program, which, when executed by a processor, is used to implement a GPU memory management method based on oversubscription according to any one of claims 1 to 7.

Images