Method, system and device for GPU memory management based on excess subscription and storage medium

Abstract

The application relates to the technical field of data processing, in particular to a GPU display memory management method, system and device based on over-subscription and a storage medium. In response to a display memory allocation request of a GPU task, device and task configurations are acquired, and an over-subscription ratio is calculated. If the over-subscription ratio is 0, a VMM constructs a display memory pool based on the GPU task configuration, directly allocates display memory from the display memory pool, and releases the display memory back to the display memory pool after the GPU task is completed. If the over-subscription ratio is not 0, the VMM constructs a display memory pool and a lock page memory pool based on the GPU task configuration and the GPU device configuration, and selects the display memory pool or the lock page memory pool based on a scheduling strategy to perform display memory allocation. Whether a migration request exists is judged. If the migration request exists, a migration operation is performed, and display memory allocation is performed by the migrated display memory pool or the lock page memory pool. In response to an exit request of the GPU task, the display memory pool and the lock page memory pool are destroyed. The application can more efficiently utilize the display memory resources of the GPU under the existing hardware conditions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a GPU memory management method, system, device and storage medium based on excess subscription. BACKGROUND

[0002] With the wide application of computationally intensive tasks such as graphics rendering, deep learning and neural network model training, GPU (Graphics Processing Unit) as a general-purpose computing platform with high parallelism has gradually become a key computing resource supporting the running of the above tasks. GPU shows significant performance advantages in completing complex image processing, model calculation, parallel matrix operation, etc., and has been widely deployed in various high-performance computing scenarios.

[0003] In the prior art, GPU load usually directly uses the physical memory provided by the GPU device during execution for storing intermediate calculation data, model parameters and input and output data, etc. In order to meet the needs of different tasks, the GPU driver supports dynamic allocation and recovery of memory. The task obtains a certain capacity of physical memory resources according to the runtime memory application and performs read and write operations. This memory usage mechanism has the characteristics of low delay and high bandwidth, and can provide good performance guarantee for task execution.

[0004] However, with the popularity of multi-task parallel processing and large-scale model training scenarios, the physical memory capacity of a single GPU device has become a major constraint on system performance improvement. Under the existing hardware conditions, the memory resources are relatively limited and difficult to support the parallel execution of multiple complex tasks. The traditional way mainly relies on improving the GPU hardware configuration, such as increasing the memory capacity or deploying a multi-card system, but such means not only have high costs, but also have certain limitations in flexibility and resource utilization efficiency. Therefore, how to more efficiently use the GPU memory resources under the existing hardware conditions has become a core technical problem to be solved at the present stage. SUMMARY

[0005] The present application provides a GPU memory management method, system, device and storage medium based on excess subscription, which can more efficiently use the GPU memory resources under the existing hardware conditions. The present application provides the following technical solutions:

[0006] In a first aspect, the present application provides a GPU memory management method based on excess subscription, comprising:

[0007] In response to a memory allocation request of a GPU task, obtaining GPU device configuration and GPU task configuration, calculating the excess subscription ratio of the GPU task to the GPU device;

[0008] If the excess subscription ratio is 0, the VMM constructs a display memory pool based on the GPU task configuration, directly allocates display memory from the display memory pool, and releases the display memory back to the display memory pool after the GPU task is completed.

[0009] If the excess subscription ratio is not 0, the VMM constructs a display memory pool and a lock page memory pool based on the GPU task configuration and the GPU device configuration, selects the display memory pool or the lock page memory pool for display memory allocation based on a preset scheduling strategy, judges whether there is a migration request, and if so, performs a migration operation between the display memory pool and the lock page memory pool, and performs display memory allocation by the migrated display memory pool or lock page memory pool; the allocated display memory is released after the GPU task is completed, and the display memory pool and the lock page memory pool are destroyed in response to the exit request of the GPU task.

[0010] In a specific implementation, in response to a display memory allocation request of a GPU task, the GPU device configuration and the GPU task configuration are obtained, and the excess subscription ratio of the GPU task to the GPU device is calculated, including:

[0011] The GPU device configuration represents the physical display memory capacity of a target GPU device, and the GPU task configuration represents the display memory capacity required by the GPU task during running;

[0012] The excess subscription ratio of the GPU task to the current GPU device is calculated according to the obtained GPU device configuration and GPU task configuration, if the GPU task configuration is less than or equal to the GPU device configuration, it is determined that the excess subscription ratio of the GPU task to the current GPU device is 0, otherwise the calculation method is as follows:

[0013] Excess subscription ratio = (GPU task configuration - GPU device configuration) / GPU device configuration.

[0014] In a specific implementation, if the excess subscription ratio is 0, the VMM constructs a display memory pool based on the GPU task configuration, directly allocates display memory from the display memory pool, and releases the display memory back to the display memory pool after the GPU task is completed.

[0015] Based on the obtained GPU device configuration and GPU task configuration, the VMM pre-applies for the required display memory resources to the GPU device and constructs a corresponding display memory pool under the control of the VMM, and the size of the display memory pool is the display memory requirement declared in the GPU task configuration;

[0016] When the GPU task initiates a display memory application request, the required display memory segment is directly allocated from the display memory pool; after the display memory is used, the display memory segment is released back to the display memory pool;

[0017] When the GPU task sends a task exit request to the VMM, the display memory pool is destroyed.

[0018] In one specific implementation, if the excess subscription ratio is not 0, the VMM builds a video memory pool and a lock page memory pool based on GPU task configuration and GPU device configuration, including:

[0019] Under the control of the VMM, the video memory pool is initialized and built based on the GPU device configuration and the GPU task configuration, and a lock page memory pool is built in proportion to the video memory pool;

[0020] The size of the video memory pool is the physical video memory capacity of the GPU device, and the size of the lock page memory pool is as follows:

[0021] The size of the lock page memory pool = the size of the video memory pool x the excess subscription ratio.

[0022] In one specific implementation, the video memory allocation is performed based on a pre-set scheduling strategy to select the video memory pool or the lock page memory pool, including:

[0023] When the GPU task initiates a video memory allocation request, the VMM returns a fixed-length virtual address, which represents the video memory space logically used by the GPU task and is used to abstract the underlying physical memory; the VMM determines the actual physical memory location to which the virtual address should be mapped according to the currently enabled video memory scheduling strategy.

[0024] In one specific implementation, the scheduling strategy includes a video memory priority strategy, a memory priority strategy, and a balanced strategy:

[0025] The video memory priority strategy is that the VMM preferentially attempts to map the virtual address to a usable fixed-length video memory segment in the video memory pool; if the video memory pool capacity is insufficient, the VMM falls back to allocate lock page memory from the lock page memory pool and complete the mapping;

[0026] The memory priority strategy is that the VMM preferentially attempts to map the virtual address to a usable fixed-length lock page memory in the lock page memory pool; if the lock page memory is insufficient, the VMM falls back to allocate video memory resources from the video memory pool and complete the mapping;

[0027] The balanced strategy is that the VMM first calculates the real-time usage ratio of the current video memory pool and the lock page memory pool, i.e., the ratio between the usage amount of the current lock page memory and the usage amount of the video memory; then compares the real-time usage ratio with the excess subscription ratio calculated initially by the task; if the current lock page memory usage ratio is lower than the target excess subscription ratio, it indicates that the lock page memory usage is insufficient, and resources are preferentially allocated from the lock page memory pool; on the contrary, if the lock page memory usage ratio is higher than the target ratio, resources are preferentially allocated from the video memory pool.

[0028] In a specific possible implementation scheme, determining whether a migration request exists, and if so, performing a migration operation between a video memory pool and a page-locked memory pool, and allocating video memory using the migrated video memory pool or the page-locked memory pool includes:

[0029] The VMM determines whether there is a migration requirement for each allocated virtual address segment;

[0030] If the judgment result is no, skip this step and keep the original mapping relationship unchanged;

[0031] If there is a migration requirement, the VMM performs the following operations: for the fixed-length virtual address to be migrated, locate its physical memory segment in the original pool, and copy the data of the physical memory segment to the physical area of ​​the corresponding size in the target pool to which it needs to be migrated; after the copy is completed, release the mapping relationship between the virtual address and the physical memory segment in the original pool, and remap the virtual address to the physical address where the newly copied data is located in the target pool to which it needs to be migrated.

[0032] In a second aspect, the present application provides a GPU memory management system based on oversubscription, which adopts the following technical solutions:

[0033] A GPU memory management system based on oversubscription, comprising:

[0034] A task start module is used to respond to a video memory allocation request of a GPU task, obtain a GPU device configuration and a GPU task configuration, and calculate an oversubscription ratio of the GPU task to the GPU device;

[0035] a direct allocation module, configured to, if the oversubscription ratio is 0, cause the VMM to construct a video memory pool based on the GPU task configuration, directly allocate video memory from the video memory pool, and release the video memory back to the video memory pool after the GPU task is completed;

[0036] an over-allocation module, configured to, if the over-subscription ratio is not 0, cause the VMM to construct a video memory pool and a page-locked memory pool based on the GPU task configuration and the GPU device configuration, and select the video memory pool or the page-locked memory pool for video memory allocation based on a pre-set scheduling policy;

[0037] A migration request module is used to determine whether a migration request exists. If so, a migration operation is performed between the video memory pool and the page-locked memory pool, and the video memory is allocated using the migrated video memory pool or the page-locked memory pool.

[0038] The task ending module is used to release the allocated video memory after the GPU task ends, and destroy the video memory pool and the page lock memory pool in response to the exit request of the GPU task.

[0039] In a third aspect, the present application provides an electronic device, comprising a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement the GPU memory management method based on excess subscription as described in the first aspect.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a program for implementing the GPU memory management method based on excess subscription as described in the first aspect when executed by a processor.

[0041] In summary, the beneficial effects of the present application include at least:

[0042] (1) By introducing the double-pool mechanism of the video memory pool and the lock page memory pool, and combining the video memory virtual address mapping and scheduling strategy, the excess subscription type management of GPU memory resources is realized. Specifically, for tasks with insufficient video memory, the present application uses lock page memory as an extension of video memory, indirectly supports the data read and write of GPU tasks through DMA accessible mode, thereby logically expanding the total amount of available video memory. In addition, based on the virtual address unified management mechanism of VMM, video memory resources can be dynamically mapped, recycled and reused in a timely manner, avoiding the fragmentation problem in traditional on-demand allocation, and improving the use efficiency of video memory resources. Through the above mechanism, even under the premise of constant physical video memory, more tasks can be supported to run in parallel, significantly improving the overall video memory carrying capacity of the system.

[0043] (2) Compared with the traditional way of relying on expanding GPU hardware to meet the video memory needs of high-concurrency tasks, the excess subscription mechanism proposed in the present application uses the existing system memory resources (lock page memory) as a supplementary resource for video memory, without the need to add new hardware to improve the processing capacity of the system. The lock page memory is applied through the cooperation of the driver and the operating system, and has the characteristics of non-paging, strong controllability, DMA access, etc., and can provide equivalent alternative memory support for GPU tasks without affecting the performance of video memory access. With this mechanism, more video memory bound tasks can be supported under existing hardware conditions, thereby effectively reducing the device cost and energy consumption overhead required by the system to meet the computing power demand, and optimizing the overall total cost of ownership of the system.

[0044] (3) The scheduling strategy proposed in the application supports three modes of display memory priority, memory priority and resource balance, and can dynamically adjust the resource allocation path according to the task load, the proportion of remaining resources in the pool and the preset excess subscription proportion. Especially in the balanced strategy, the deviation between the current use proportion and the target proportion of the display memory pool and the lock page memory pool can be compared in real time, and the optimal allocation source can be intelligently selected to effectively maintain resource balance. On this basis, further support is provided for the data migration mechanism between display memory and lock page memory, for example, when CPU participation operation (such as display memory snapshot) is detected in the task, the original allocated display memory mapping can be actively migrated to the lock page memory readable by the CPU, thereby adapting to the changes in resource demand in different running stages. The above design improves the resource scheduling flexibility of the system when facing diversified loads and burst demand, and enhances the flexibility and scalability of the overall system.

[0045] First, it is determined whether the GPU task requires display memory exceeding the GPU device capability; in the case of no excess, display memory pre-allocation and cyclic reuse are performed through the construction of a task-specific display memory pool; in the case of excess, a display memory pool and a lock page memory pool expanded in proportion are constructed synchronously, and a schedulable allocation strategy is introduced to dynamically select the physical resource source; to adapt to the migration requirements of part of the tasks, a secondary migration and remapping mechanism of virtual addresses is further designed; finally, through the unified resource release logic driven by task exit, the pool resource is recycled and the pool body is destroyed. Through display memory pooling pre-allocation, lock page memory compensation, flexible scheduling and dynamic migration, the logical display memory capacity is effectively expanded, the fragmentation overhead is eliminated, and the resource balance is maintained according to the real-time use characteristics, thereby significantly improving the overall utilization efficiency of the GPU display memory without increasing the hardware display memory, and effectively solving the performance bottleneck caused by insufficient display memory.

[0046] The above description is only a summary of the technical solutions of the application. In order to more clearly understand the technical means of the application, and to implement the content of the description, the following will be described in detail with reference to the preferred embodiments of the application and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of the GPU display memory management method based on excess subscription in the embodiment of the application.

[0048] Figure 2 is a flowchart of the GPU display memory management method based on excess subscription in the embodiment of the application.

[0049] Figure 3 is a structure block diagram of the GPU display memory management system based on excess subscription in the embodiment of the application.

[0050] Figure 4is a block diagram of an electronic device based on the excess subscription of GPU video memory management in the embodiments of the present application. DETAILED DESCRIPTION

[0051] The specific embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0052] Firstly, several terms related to the present application are introduced.

[0053] GPU (Graphics Processing Unit) is a kind of processor specially used for processing graphics and image computing tasks. Although it is originally designed to accelerate graphics rendering, modern GPU has developed into a powerful computing device, which is widely used in various computing-intensive tasks.

[0054] GPU business load: refers to the total amount of computing, memory and other resources consumed by various applications or tasks running on the GPU. It reflects the work intensity and resource demand of the GPU in a certain period of time.

[0055] System memory: general high-speed memory on the system that is directly accessed and managed by the CPU, characterized by high-speed read and write and power-off emptying.

[0056] GPU physical memory: memory storage provided by the GPU device, usually directly accessed by the GPU graphics processing chip, with lower access delay and higher bandwidth compared to system memory.

[0057] Locked page memory: part of the system memory, locked into the system page table, not exchanged to the disk by the system paging, and the corresponding physical memory is not released to other processes, with higher priority and performance than ordinary memory.

[0058] VMM: Virtual Memory Manager, responsible for the management and maintenance of virtual address and underlying memory address mapping, used to map the underlying video memory and physical memory to virtual addresses, ensuring that the underlying memory data location (video card or physical memory) is dynamically changed without changing the application program address.

[0059] DMA: memory direct, allowing bus devices to directly access system memory without CPU, improving performance and reducing system overhead.

[0060] Optionally, the excess subscription based GPU memory management method provided in various embodiments of the present application is used in an electronic device, which is a terminal or a server. The terminal can be a computer, a tablet computer, etc. The present embodiment does not limit the type of electronic device.

[0061] Reference Figure 1FIG. 1 is a flowchart of a GPU memory management method based on excess subscription according to an embodiment of the present application. The method comprises the following steps.

[0062] In step S101, in response to a GPU task memory allocation request, obtain GPU device configuration and GPU task configuration, and calculate the excess subscription ratio of the GPU task to the GPU device.

[0063] In step S101, for the GPU task initiated memory allocation request, obtain the relevant resource configuration information, and determine whether the task requested memory exceeds the available capacity of the current GPU device, so as to calculate the excess subscription ratio of the GPU task to the current GPU device.

[0064] Specifically, first, obtain the GPU device configuration and the GPU task configuration. The GPU device configuration represents the physical memory capacity of the target GPU device, and the GPU task configuration represents the memory capacity required by the GPU task during running, which is usually determined by the upper task scheduling system in the task issuing stage. Then, according to the obtained GPU device configuration and GPU task configuration, calculate the excess subscription ratio of the GPU task to the current GPU device. If the GPU task configuration is less than or equal to the GPU device configuration, it is determined that the excess subscription ratio of the GPU task to the current GPU device is 0, otherwise, the calculation method is as follows:

[0065] Excess subscription ratio = (GPU task configuration - GPU device configuration) / GPU device configuration

[0066] For example, if the GPU task requests 1.5GB memory, and the physical memory capacity of the target GPU device is 1.0GB, then the corresponding excess subscription ratio is 0.5.

[0067] In step S102, if the excess subscription ratio is 0, the VMM constructs a memory pool based on the GPU task configuration, directly allocates memory from the memory pool, and releases the allocated memory back to the memory pool after the GPU task is completed.

[0068] In step S102, for the case that the excess subscription ratio of the GPU task to the GPU device is 0, the memory allocation request corresponding to the GPU task is completed by initializing the memory pool, and the release and recovery of the allocated memory are realized after the GPU task is completed.

[0069] Specifically, first, based on the acquired GPU device configuration and GPU task configuration, under the control of the VMM (virtual memory manager), the required GPU memory resources are pre-applied to the GPU device, and the corresponding GPU memory pool is constructed. At this time, the size of the GPU memory pool is the GPU memory requirement declared in the GPU task configuration. The GPU memory pool serves as a GPU memory resource buffer for the GPU task, and reserves a certain scale of physical GPU memory segment for subsequent task calling. Subsequently, during the execution of the GPU task, whenever the task attempts to initiate a GPU memory application request, the required GPU memory segment is directly allocated from the constructed GPU memory pool to meet the real-time GPU memory calling requirement of the task. Compared with the traditional on-demand allocation method, this allocation operation no longer needs to apply for physical GPU memory from the GPU driver layer, thereby effectively reducing the system overhead of the GPU memory allocation and release process in the kernel mode. When the GPU task completes a GPU memory usage, the GPU memory segment is released back to the GPU memory pool, ensuring that the GPU memory resources can be recycled and providing available space for subsequent GPU memory application requests. Finally, when the GPU task issues a task exit request to the VMM, the GPU memory pool is destroyed.

[0070] In step S103, if the excess subscription ratio is not 0, the VMM constructs a GPU memory pool and a lock page memory pool based on the GPU task configuration and the GPU device configuration, and selects the GPU memory pool or the lock page memory pool for GPU memory allocation based on a pre-set scheduling strategy.

[0071] In step S103, in combination with Figure 2 For the case where the GPU task's GPU memory request to the GPU device exceeds the physical GPU memory capacity of the GPU device, i.e., the excess subscription ratio is not 0, a scheduling strategy is introduced to dynamically allocate GPU memory resources by constructing a bottom-layer memory pool composed of a GPU memory pool and a lock page memory pool. Under the control of the VMM, the GPU memory pool is initialized and constructed based on the GPU device configuration and the GPU task configuration, and a lock page memory pool with a proportional relationship with the GPU memory pool is also constructed. At this time, the size of the GPU memory pool is the physical GPU memory capacity of the GPU device, and the size of the lock page memory pool is determined by the following formula: lock page memory pool size = GPU memory pool size x excess subscription ratio. The lock page memory is usually applied by the GPU driver through interaction with the operating system, and belongs to the memory type that can be accessed by DMA, with the characteristics of non-paging and strong controllability, and is suitable for indirect access scenarios of the GPU, so it is used as an auxiliary resource for GPU memory in this application.

[0072] Specifically, during the execution of the GPU task, each time the GPU task attempts to initiate a GPU memory allocation request, a fixed-length virtual address is first returned by the VMM. The virtual address is uniformly managed by the VMM and represents the GPU memory space logically used by the GPU task, which is used to abstract the underlying physical memory resources and ensure good logical consistency and address isolation in subsequent resource access processes. Subsequently, the VMM determines the actual physical memory location to which the virtual address should be mapped according to the currently enabled GPU memory scheduling strategy. Specifically, the following three scheduling strategies are included:

[0073] GPU memory priority strategy: the VMM first attempts to map the virtual address to a fixed-length GPU memory segment available in the GPU memory pool; if the GPU memory pool capacity is insufficient, the VMM then allocates a lock page memory from the lock page memory pool and completes the mapping;

[0074] Memory priority strategy: the VMM first attempts to map the virtual address to a fixed-length lock page memory available in the lock page memory pool; if the lock page memory is insufficient, the VMM then allocates GPU memory resources from the GPU memory pool and completes the mapping;

[0075] Balanced strategy: when allocating GPU memory resources, the VMM first calculates the real-time usage ratio of the current GPU memory pool and the lock page memory pool, i.e., the ratio between the current usage of the lock page memory and the usage of the GPU memory; then, the real-time usage ratio is compared with the target oversubscription ratio obtained by the initial calculation of the task. If the current lock page memory usage ratio is lower than the target oversubscription ratio, it indicates that the lock page memory usage is insufficient, and in order to approach the target ratio, the VMM should preferentially allocate resources from the lock page memory pool; on the contrary, if the lock page memory usage ratio is higher than the target ratio, the VMM should preferentially allocate resources from the GPU memory pool. In this way, the usage of the two types of memory can be continuously adjusted during dynamic allocation, so that the actual usage ratio is as close as possible to the preset oversubscription ratio, maintaining the balance and rationality of resource usage.

[0076] After the resource selection is completed, the VMM establishes a mapping relationship between the virtual address and the obtained underlying physical memory (GPU memory or lock page memory), and the subsequent access operation of the GPU task can be transparently read and written to the actual physical resources through the virtual address.

[0077] In step S104, for the GPU memory allocation completed by the GPU task, the VMM performs a secondary determination on the type of the allocated memory pool to determine whether resource allocation needs to be switched between the GPU memory pool and the lock page memory pool.

[0078] In step S104, for the GPU memory allocation completed by the GPU task, the VMM performs a secondary determination on the type of the allocated memory pool to determine whether resource allocation needs to be switched between the GPU memory pool and the lock page memory pool.

[0079] Specifically, the VMM determines whether there is a migration requirement for each allocated virtual address. If the determination result is no, the original mapping relationship is kept unchanged; if there is a migration requirement, the VMM performs the following operations: for the fixed-length virtual address that needs to be migrated, the physical video memory segment in the original pool is located in the video memory pool or the lock page memory pool, and the data of the physical video memory segment is copied to the corresponding size of the physical region in the target pool. After the copying is completed, the mapping relationship between the virtual address and the physical video memory segment in the original video memory pool or the lock page memory pool is released, and the virtual address is remapped to the physical address where the newly copied data is located in the video memory pool or the lock page memory pool that needs to be migrated.

[0080] It should be noted that the analysis of whether there is a migration request includes but is not limited to: the GPU task identifies its access type through a task parameter in the allocation stage, and the VMM determines whether it needs to be migrated to the lock page memory in priority; or if the VMM detects that the task contains a CPU-initiated data processing process, such as snapshot saving, log generation, model intermediate result export, and the like, and the current resource is allocated in the CPU-unreadable video memory pool, a migration request is automatically triggered.

[0081] Taking the video memory snapshot function as an example, this function requires the video memory data currently used by the GPU task to be exported and read and analyzed by the CPU. However, since the GPU video memory generally does not support direct access by the CPU, if it is first scheduled to the video memory pool for video memory allocation, it is difficult to meet the readability requirement in the snapshot process if the video memory pool mapping is continued. Therefore, after detecting the snapshot task trigger, the VMM will automatically copy the target data mapped to the video memory pool to the lock page memory pool, and re-establish the mapping relationship between the virtual address and the lock page memory, thereby ensuring that the data region corresponding to the address can be directly accessed by the CPU.

[0082] In step S105, the allocated video memory is released after the GPU task is completed, and the video memory pool and the lock page memory pool are destroyed in response to the exit request of the GPU task.

[0083] In step S105, for the scenario where the GPU task is executed and completed, the VMM completes the release of the allocated video memory and the destruction of the video memory pool and the lock page memory pool in sequence according to a unified release mechanism, so as to ensure that the underlying resources are effectively recycled.

[0084] Specifically, after the GPU task completes running, the VMM receives a release request for a virtual address, and for each fixed-length virtual address allocated to the GPU task, the VMM unmaps the virtual address from the underlying physical memory (including a corresponding physical address segment in the video memory pool or the lock page memory pool); the unmapped physical address segment is returned to the corresponding memory pool for subsequent task reuse; at the same time, the mapping state of the virtual address is reset to ensure that subsequent mapping relationships do not conflict or overlap. Subsequently, when the GPU task issues a task exit request to the VMM, the video memory pool and the lock page memory pool bound to the GPU task are destroyed. This process includes: releasing the remaining memory segments in the pool, unregistering all control structures related to the pool, and clearing the resource allocation and mapping table entries of the task recorded in the VMM, thereby completing the recovery and cleaning of the entire set of memory resources.

[0085] To sum up, first, it is determined whether the GPU task requires video memory exceeding the GPU device capacity; in the case of no excess, video memory pre-allocation and cyclic reuse are performed by constructing a task-specific video memory pool; in the case of excess, a video memory pool and a lock page memory pool expanded in proportion are constructed synchronously, and a schedulable allocation strategy is introduced to dynamically select the source of physical resources; to adapt to the migration needs of part of the tasks, a secondary migration and remapping mechanism of virtual addresses is further designed; finally, through the unified resource release logic driven by the task exit, the pool resources are recovered and the pool body is destroyed. Through video memory pool pre-allocation, lock page memory compensation, flexible scheduling, and dynamic migration, the logical video memory capacity is effectively expanded, the fragmentation overhead is eliminated, and resource balance is maintained according to real-time usage characteristics, thereby significantly improving the overall utilization efficiency of the GPU video memory without increasing the hardware video memory, and effectively solving the performance bottleneck caused by insufficient video memory.

[0086] Figure 3 is a structural block diagram of a GPU video memory management system based on excess subscription provided by an embodiment of the present application, and the system at least includes the following modules:

[0087] A task start module is configured to, in response to a video memory allocation request of a GPU task, acquire GPU device configuration and GPU task configuration, and calculate an excess subscription ratio of the GPU task to the GPU device;

[0088] A direct allocation module is configured to, if the excess subscription ratio is 0, the VMM constructs a video memory pool based on the GPU task configuration, allocates video memory directly from the video memory pool, and releases the video memory back to the video memory pool after the GPU task completes running;

[0089] An excess allocation module is configured to, if the excess subscription ratio is not 0, the VMM constructs a video memory pool and a lock page memory pool based on the GPU task configuration and the GPU device configuration, and selects the video memory pool or the lock page memory pool for video memory allocation based on a pre-set scheduling strategy;

[0090] A migration request module is used to determine whether a migration request exists. If so, a migration operation is performed between the video memory pool and the page-locked memory pool, and the video memory is allocated using the migrated video memory pool or the page-locked memory pool.

[0091] The task ending module is used to release the allocated video memory after the GPU task ends, and destroy the video memory pool and the page lock memory pool in response to the exit request of the GPU task.

[0092] For relevant details, please refer to the above method embodiment.

[0093] Figure 4 4 is a block diagram of an electronic device provided in one embodiment of the present application. The device includes at least a processor 401 and a memory 402.

[0094] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content displayed on the display screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0095] Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 402 is used to store at least one instruction, which is executed by processor 401 to implement the oversubscription-based GPU memory management method provided in the method embodiment of the present application.

[0096] In some embodiments, the electronic device can further optionally comprise a peripheral device interface and at least one peripheral device. The processor 401, the memory 402 and the peripheral device interface can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface through a bus, a signal line or a circuit board. Illustratively, the peripheral devices include, but are not limited to, a radio frequency circuit, a touch display screen, an audio circuit, a power supply and the like.

[0097] Of course, the electronic device can also comprise fewer or more components, and the present embodiment is not limited in this regard.

[0098] Optionally, the present application further provides a computer readable storage medium, and the computer readable storage medium stores a program, the program is loaded and executed by a processor to implement the GPU memory management method based on excess subscription of the above method embodiments.

[0099] Optionally, the present application further provides a computer product, and the computer product comprises a computer readable storage medium, and the computer readable storage medium stores a program, the program is loaded and executed by a processor to implement the GPU memory management method based on excess subscription of the above method embodiments.

[0100] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0101] The above embodiments only express several implementation manners of the present application, and the description is specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the patent of the present application should be subject to the appended claims.

Claims

1. A method for GPU memory management based on over-subscription, the method comprising: The method comprises: In response to a GPU task memory allocation request, obtaining a GPU device configuration and a GPU task configuration, and calculating an oversubscription ratio of the GPU task to the GPU device; If the oversubscription ratio is 0, a VMM constructs a memory pool based on the GPU task configuration, directly allocates memory from the memory pool, and releases the allocated memory back to the memory pool after the GPU task is completed; If the oversubscription ratio is not 0, the VMM constructs a memory pool and a lock page memory pool based on the GPU task configuration and the GPU device configuration, selects the memory pool or the lock page memory pool for memory allocation based on a preset scheduling strategy, judges whether there is a migration request, and if so, performs a migration operation between the memory pool and the lock page memory pool, and performs memory allocation by the migrated memory pool or lock page memory pool; and after the GPU task is completed, the allocated memory is released, and in response to a GPU task exit request, the memory pool and the lock page memory pool are destroyed; The scheduling strategy comprises a memory priority strategy, a memory priority strategy, and a balanced strategy: The memory priority strategy is that the VMM preferentially attempts to map a virtual address to a fixed-length memory segment available in the memory pool; if the memory pool capacity is insufficient, the VMM retreats to allocate lock page memory from the lock page memory pool and completes the mapping; The memory priority strategy is that the VMM preferentially attempts to map a virtual address to a fixed-length lock page memory available in the lock page memory pool; if the lock page memory is insufficient, the VMM retreats to allocate memory resources from the memory pool and completes the mapping; The balanced strategy is that the VMM first calculates a real-time usage ratio of the current memory pool and the lock page memory pool, i.e. the ratio between the usage amount of the current lock page memory and the usage amount of the memory; then compares the real-time usage ratio with the oversubscription ratio calculated initially by the task; if the current lock page memory usage ratio is lower than the target oversubscription ratio, it indicates that the lock page memory usage is insufficient, and resources are preferentially allocated from the lock page memory pool; on the contrary, if the lock page memory usage ratio is higher than the target ratio, resources are preferentially allocated from the memory pool.

2. The method of claim 1, wherein, The method comprises: The GPU device configuration represents the physical memory capacity of the target GPU device, and the GPU task configuration represents the required memory capacity of the GPU task during running; According to the obtained GPU device configuration and GPU task configuration, the oversubscription ratio of the GPU task to the current GPU device is calculated; if the GPU task configuration is less than or equal to the GPU device configuration, it is determined that the oversubscription ratio of the GPU task to the current GPU device is 0, otherwise the calculation method is as follows: Oversubscription ratio = (GPU task configuration - GPU device configuration) / GPU device configuration.

3. The method of claim 1, wherein, If the oversubscription ratio is 0, the VMM constructs a memory pool based on the GPU task configuration, directly allocates memory from the memory pool, and releases the allocated memory back to the memory pool after the GPU task is completed. Based on the acquired GPU device configuration and GPU task configuration, under the control of the VMM, pre-apply for the required GPU memory resource to the GPU device and build the corresponding GPU memory pool, the size of the GPU memory pool is the GPU memory requirement declared in the GPU task configuration; When the GPU task initiates the GPU memory application request, directly allocate the required GPU memory segment from the GPU memory pool; after the GPU memory is used, release the GPU memory segment back to the GPU memory pool; When the GPU task sends the task exit request to the VMM, destroy the GPU memory pool.

4. The method of claim 1, wherein, If the excess subscription ratio is not 0, the VMM builds the GPU memory pool and the lock page memory pool based on the GPU task configuration and the GPU device configuration, which includes: Under the control of the VMM, initialize and build the GPU memory pool based on the GPU device configuration and the GPU task configuration, and at the same time, build the lock page memory pool which has a proportional relationship with the GPU memory pool; The size of the GPU memory pool is the physical GPU memory capacity of the GPU device, and the size of the lock page memory pool is as follows: Lock page memory pool size = GPU memory pool size × excess subscription ratio.

5. The method of claim 4, wherein, The GPU memory allocation is based on the pre-set scheduling strategy to select the GPU memory pool or the lock page memory pool, which includes: When the GPU task initiates the GPU memory allocation request, the VMM returns a fixed-length virtual address, which represents the GPU memory space used logically by the GPU task and is used to abstract the underlying physical memory; the VMM determines the actual physical memory location to which the virtual address should be mapped according to the currently enabled GPU memory scheduling strategy.

6. The GPU memory management method based on excess subscription according to claim 5, characterized in that, The judgment of whether there is a migration request, if there is, the migration operation between the GPU memory pool and the lock page memory pool is performed, and the GPU memory allocation is performed by the migrated GPU memory pool or the lock page memory pool, which includes: The VMM judges whether there is a migration demand for each allocated virtual address; If the result of the judgment is no, skip this step to keep the original mapping relationship unchanged; If there is a migration demand, the VMM performs the following operations: for the fixed-length virtual address that needs to be migrated, locate the physical GPU memory segment in the original pool, copy the data of the physical GPU memory segment to the corresponding size of the physical region in the target pool that needs to be migrated; after the copying is completed, the mapping relationship between the virtual address and the physical GPU memory segment in the original pool is removed, and the virtual address is remapped to the physical address where the newly copied data in the target pool that needs to be migrated is located.

7. A GPU memory management system based on over-subscription, the system comprising: It includes: The task start module is configured to, in response to a GPU memory allocation request of a GPU task, acquire a GPU device configuration and a GPU task configuration, and calculate an excess subscription ratio of the GPU task to the GPU device; The direct allocation module is configured to, if the excess subscription ratio is 0, the VMM builds a GPU memory pool based on the GPU task configuration, directly allocates GPU memory from the GPU memory pool, and releases it back to the GPU memory pool after the GPU task is completed; The excess allocation module is configured to, if the excess subscription ratio is not 0, the VMM builds a GPU memory pool and a lock page memory pool based on the GPU task configuration and the GPU device configuration, and selects the GPU memory pool or the lock page memory pool for GPU memory allocation based on a pre-set scheduling strategy; The scheduling strategy includes a GPU memory priority strategy, a memory priority strategy, and a balanced strategy: The GPU memory management method comprises the following steps: a GPU task is received; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; a memory pool is allocated for the GPU task; ​ ​ ​ ​ ​ 8. An electronic device, comprising: ​ 9. A computer-readable storage medium, characterized in that, ​

Images