Method and device for fitting GPU video memory channel mapping and storage medium
By adding API to the GPU driver to obtain physical addresses and using deep neural network to fit the mapping of physical addresses to the video memory channel, the problem of memory access conflicts in GPU virtualization is solved, and the isolation of video memory channels and the improvement of resource utilization is achieved.
Patent Information
- Application Number
- CN202410114268.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-28
- Publication Date
- 2025-08-01
AI Technical Summary
Existing GPU virtualization technology cannot effectively avoid memory access conflicts between different tasks, especially for non-integer power GPU devices, the existing hash mapping function cannot be adapted, resulting in performance interference and low resource utilization.
By adding API to the GPU driver to obtain physical addresses, randomly selecting address sets, using deep neural networks to fit the mapping of physical addresses to the video memory channel, avoiding memory access conflicts, and using gradient descent algorithm to train deep neural networks to achieve isolation of video memory channels.
The isolation of video memory channels in the multi-tenant GPU virtualization system is realized, which avoids performance interference, improves GPU resource utilization, and enhances the security of L2 cache.
Smart Images
Figure CN120407133A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of cloud computing and GPU virtualization, and particularly relates to a method, device, and storage medium for fitting GPU video memory channel mapping. Background Art
[0002] In recent years, with the booming development of industries such as artificial intelligence, big data, and scientific computing, the demand for computer computing power in all walks of life has been increasing continuously. Since the CPU architecture is difficult to cope with large-scale parallel computing, the contradiction between hardware resources and computing power requirements has gradually emerged, and the wide application of GPUs has effectively alleviated this contradiction. As GPUs perform increasingly well in simple repetitive calculations of large-scale data, GPUs have been widely used in compute-intensive applications, and GPU parallel computing technology has become a popular research direction.
[0003] To reduce operation and maintenance costs, more and more enterprises deploy their GPU applications on cloud services. Cloud service providers can virtualize one GPU into multiple sub-GPUs, thereby deploying multiple GPU applications and improving GPU resource utilization. However, resource conflicts will occur between different tasks on one GPU, resulting in performance interference, which in turn affects the quality of service of each task. Therefore, how to virtualize a GPU and allocate the resources on the GPU to different tasks for use is particularly important.
[0004] Currently, GPU virtualization technologies mainly fall into two categories: time-division multiplexing and space-division multiplexing. The GPU virtualization scheme based on time-division multiplexing follows a time-slicing rotation strategy, allowing each GPU task to run in the form of exclusive use of GPU resources, thus avoiding performance interference between different tasks. However, because the computing power requirements of different tasks are different, many small GPU tasks cannot fully utilize all GPU resources alone. Therefore, the GPU utilization rate of this type of virtualization scheme is generally low. The GPU virtualization scheme based on space-division multiplexing uses multi-streaming or MPS (Multi-process Service) technology, enabling multiple tasks to be executed in parallel on one GPU simultaneously, thereby maximizing the utilization rate of GPU resources. However, they cannot avoid memory access conflicts between different tasks (L2 cache line conflicts and DRAM row conflicts caused by sharing video memory channels), so this type of scheme cannot effectively isolate the performance interference generated between different tasks.
[0005] In order to avoid memory access conflicts between different tasks in GPU virtualization technology based on spatial division multiplexing, page coloring technology needs to be used to map the video memory applied by different tasks to different video memory channels. However, the existing GPU video memory page coloring technology is based on a hash mapping function from GPU physical addresses to video memory channel numbers. This hash mapping function is a linear function composed solely of exclusive OR operations, so that this function can be parsed by solving a binary linear equation system through Gaussian elimination. This means that this technology only supports GPUs with the number of channels being a power of 2 and is not applicable to most GPUs on the market. Therefore, how to fit the mapping between GPU physical addresses and video memory channel numbers so that the page coloring technology can be adapted to all GPUs is a technical problem to be solved urgently. Summary of the Invention
[0006] In view of this, the purpose of the present invention is to provide a method, device and storage medium for fitting GPU video memory channel mapping, and realize video memory channel isolation of the GPU virtualization system by establishing a mapping from the physical address of the video memory to the video memory channel number. The specific technical solutions are as follows:
[0007] In a first aspect, an embodiment of the present invention provides a method for fitting GPU video memory channel mapping, including the following steps: Apply for a continuous physical address space in the GPU video memory and obtain the physical addresses of this address space; Randomly select some physical addresses from the address space and obtain the set of physical addresses that conflict with them; Randomly select some physical addresses from the address space again and mark the corresponding channel numbers; Fit the mapping of all physical addresses to video memory channel numbers.
[0008] Further, the obtaining of the physical addresses of this address space specifically includes: Add an application programming interface (API) for GPU virtual-to-physical address conversion in the GPU driver program to convert the virtual address space of the video memory into a physical address space.
[0009] Further, the randomly selecting some physical addresses from the address space and obtaining the set of physical addresses that conflict with them specifically includes: Traverse the selected part of the physical addresses and obtain the first set of physical addresses that conflict with them in DRAM row; For each physical address in the first set of physical addresses, obtain the second set of physical addresses that conflict with it in L2 cache line; Merge the second sets of physical addresses that conflict with each physical address in the first set of physical addresses to obtain the third set of physical addresses.
[0010] Further, randomly selecting a part of physical addresses in the address space again and marking the corresponding channel numbers specifically includes: Determine whether there is an L2 cache line conflict between the selected physical address and the physical addresses in the third physical address set. If there is a conflict, it means that the two physical addresses belong to the same channel, otherwise they do not belong to the same channel; Establish a mapping table from physical addresses to channel numbers.
[0011] Further, fitting the mapping of all physical addresses to the video memory channel numbers specifically includes: Convert the physical addresses in the mapping table into binary as the input features of the deep neural network; Use the gradient descent algorithm to train the deep neural network to fit the mapping relationship between physical addresses and video memory channels; Traverse each physical address in the physical address space of the entire GPU and use the deep neural network to predict the video memory channel number mapped by each physical address.
[0012] In a second aspect, an embodiment of the present invention provides an electronic device, including: A memory for storing a computer program; A processor for implementing the method according to any one of the above first aspects when executing the program stored in the memory.
[0013] In a third aspect, an embodiment of the present invention provides a storage medium. When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the method according to any one of the above first aspects.
[0014] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program, and the computer program implements the method according to any one of the above first aspects when executed by a processor.
[0015] The technical solutions provided by the embodiments of the present invention at least bring the following beneficial effects: In the embodiments of the present invention, by adding a GPU driver application programming interface (API) to obtain the physical address corresponding to a given virtual address and fitting the mapping of the physical address of the GPU to the video memory channel, the video memory channel isolation of the multi-tenant GPU virtualization system is realized, which helps to split the video memory channels of the GPU and allocate them to different tasks for use, avoiding performance interference caused by memory access conflicts between different tasks, and also helps to perform side-channel attacks on the L2 cache of the GPU.
[0016] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments can be obtained based on these drawings.
[0018] Figure 1 It is a schematic diagram of the overall process of a method for fitting GPU video memory channel mapping provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the operation of obtaining the physical address of the address space provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of the operation of obtaining the conflicting physical address provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of the operation of marking the channel number provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of the process of fitting the physical address to the video memory channel number mapping provided by an embodiment of the present invention; Figure 6 It is a schematic diagram of the device for fitting GPU video memory channel mapping provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0019] In order to better understand the above technical solutions, the exemplary embodiments of the present invention will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.
[0020] An embodiment of the present invention takes NVIDIA GPU as an object and provides a method for fitting GPU video memory channel mapping. The schematic diagram of its process is as Figure 1 shown, including the following steps:
[0021] Step S100: Apply for a continuous physical address space in the GPU video memory and obtain the physical address of the address space.
[0022] Step S200: Randomly select some physical addresses from the address space and obtain the set of physical addresses that conflict with them.
[0023] Step S300: Randomly select some physical addresses from the address space again and mark the corresponding channel numbers.
[0024] Step S400: Fit the mapping of all physical addresses to video memory channel numbers.
[0025] In this embodiment, the operation process of step S100 is as Figure 2 shown and includes:
[0026] Step S110: Call the API (Application Programming Interface) cuMemCreate() provided by NVIDIA CUDA to apply for a continuous physical address space in the GPU video memory.
[0027] In this embodiment, the size of the applied physical address space is the size of the GPU video memory.
[0028] Step S120: Call the APIs (Application Programming Interfaces) cuMemAddressReserve() and cuMemMap() provided by NVIDIA CUDA to map the physical address space applied in step S110 to a continuous virtual address space VirtMemSpace starting from the virtual address VirtAddr.
[0029] Step S130: Call the newly added API (Application Programming Interface) GetPhysAddrFromVirtAddr() in the GPU driver, pass in the starting address VirtAddr of the virtual address, and obtain the starting address of the corresponding physical address.
[0030] The function of the GetPhysAddrFromVirtAddr() API in this embodiment is to implement the conversion between virtual addresses and physical addresses. The function takes in a virtual address and returns the corresponding physical address. GetPhysAddrFromVirtAddr() is not included in the NVIDIA driver but is a newly added API to achieve fitting the GPU video memory channel mapping. In the GetPhysAddrFromVirtAddr() API, first retrieve the page table from the video memory and find the page table entry corresponding to the virtual address VirtAddr. Then parse the page table entry to obtain the physical address PhysAddr corresponding to VirtAddr.
[0031] Step S140: Convert the virtual address space VirtMemSpace into a physical address space PhysMemSpace according to the obtained starting address PhysAddr of the physical address and the space size.
[0032] In this embodiment, the operation process of step S200 is as Figure 3 shown and includes:
[0033] Step S210: Randomly extract a physical address from the physical addresses in the physical address space VirtMemSpace that have not been extracted, and assign it to Phys_Addr _ i
[0034] Step S220: Obtain the set DRAM_Row_Conflict_Addr_List of physical addresses that have a DRAM row conflict with Phys_Addr _ i i
[0035] Step S230: Initialize the set Cacheline_Conflict_Addr_List of physical addresses that have an L2 cache line conflict with the addresses in the DRAM_Row_Conflict_Addr_List i set to be an empty set i
[0036] Step S240: Sequentially take out a physical address Phys_Addr_j from the DRAM_Row_Conflict_Addr_List i set, and repeatedly execute Steps S250 to S260 until all the physical addresses in the DRAM_Row_Conflict_Addr_List i set have been taken out
[0037] Step S250: Obtain the set Cacheline_Conflict_Addr_List of physical addresses that have an L2 cache line conflict with Phys_Addr_j ij
[0038] Step S260: Incorporate the set Cacheline_Conflict_Addr_List ij into the set Cacheline_Conflict_Addr_List i
[0039] Step S270: Repeat Steps S210 to S260 until the addresses in all the Cacheline_Conflict_Addr_List sets completely cover all the video memory channels
[0040] The set Cacheline_Conflict_Addr_List in step S270 has N subsets: Cacheline_Conflict_Addr_List0, Cacheline_Conflict_Addr_List1, Cacheline_Conflict_Addr_List2, ..., Cacheline_Conflict_Addr_List N-1 , where N is the number of GPU video memory channels.
[0041] In this embodiment, the operation process of step S300 is as Figure 4 shown, including:
[0042] Step S310: Initialize the sample set Collected_Samples_List of address conflicts to an empty set.
[0043] Step S320: Randomly obtain a physical address Target_Phys_Addr from the physical address space PhysMemSpace.
[0044] Step S330: Detect whether there is an L2 cache line conflict between Target_Phys_Addr and each physical address on each video memory channel generated in step S200. If there is an L2 cache line conflict between Target_Phys_Addr and the physical address on the i-th video memory channel generated in step S200, it indicates that the video memory channel number where Target_Phys_Addr is located is i, and add (Target_Phys_Addr, i) to the data set Collected_Samples_List.
[0045] Step S340: Repeat steps S320 to S330 until the number of samples in the sample set Collected_Samples_List exceeds the size of the target data set.
[0046] The size of the target data set in this embodiment is set to 20000.
[0047] In this embodiment, the operation process of step S400 is as Figure 5 shown, including:
[0048] Step S410: Convert each physical address Target_Phys_Addr in the conflict sample data set Collected_Samples_List collected in step S330 into binary.
[0049] Step S420: Split the new dataset Collected_Samples_List obtained in step S410 into a training set, a validation set, and a test set according to a certain ratio. The ratio can be adjusted as needed. The ratio in this embodiment is 7:2:1.
[0050] Step S430: Use the gradient descent method to train the deep neural network on the training set until the neural network converges to obtain the highest accuracy, and save the network parameters.
[0051] Step S440: Verify the highest accuracy obtained in step S430 on the test set.
[0052] Step S450: Save the parameters and accuracy of the deep neural network obtained in this round of training.
[0053] Step S460: After multiple rounds of training, select the parameters of the deep learning model with the highest accuracy on the test set.
[0054] Step S470: Traverse each physical address in the entire GPU video memory space, and use the deep neural network to predict their video memory channel numbers.
[0055] Based on the same inventive concept, according to another aspect of the present invention, as Figure 6 shown, an embodiment of the present invention further provides a computer device 500, including: At least one processor 520; and A memory 510, where the memory 510 stores a computer program 511 that can run on the processor, and when the processor 520 executes the program, it executes the method for fitting the GPU video memory channel mapping in the above embodiment of the present invention.
[0056] The above are exemplary embodiments disclosed by the present invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of the present invention defined by the claims. The functions, steps, and / or actions of the method claims according to the disclosed embodiments here do not need to be executed in any specific order. In addition, although the elements disclosed in the embodiments of the present invention can be described or claimed in individual form, they can also be understood as multiple unless explicitly limited to the singular.
Claims
1. A method for fitting GPU video memory channel mapping, characterized in that It includes the following steps: Apply for a continuous physical address space in the GPU video memory and obtain the physical address of the address space; Randomly select some physical addresses from the address space and obtain the set of physical addresses that conflict with them; Randomly select some physical addresses from the address space again and mark the corresponding channel numbers; Fit the mapping of all physical addresses to the video memory channel numbers.
2. The method according to claim 1, wherein The obtaining of the physical address of the address space includes: Add a GPU virtual-to-physical address conversion API (application programming interface) in the GPU driver to convert the virtual address space of the video memory into a physical address space.
3. The method according to claim 1, wherein The randomly selecting some physical addresses from the address space and obtaining the set of physical addresses that conflict with them includes: Traverse the selected part of the physical addresses and obtain the first set of physical addresses that conflict with them in the DRAM row; For each physical address in the first set of physical addresses, obtain the second set of physical addresses that conflict with it in the L2 cache line; Merge the second sets of physical addresses that conflict with each physical address in the first set of physical addresses to obtain the third set of physical addresses.
4. The method according to claim 1, characterized in that The randomly selecting some physical addresses from the address space again and marking the corresponding channel numbers includes: Determine whether there is an L2 cache line conflict between the selected physical address and the physical addresses in the third set of physical addresses. If there is a conflict, it means that the two physical addresses belong to the same channel, otherwise they do not belong to the same channel; Establish a mapping table from physical addresses to channel numbers.
5. The method according to claim 1, characterized in that The fitting of the mapping of all physical addresses to the video memory channel numbers includes: Convert the physical addresses in the mapping table into binary as the input features of the deep neural network; Use the gradient descent algorithm to train the deep neural network to fit the mapping relationship between physical addresses and video memory channels; Traverse each physical address in the entire GPU physical address space and use the deep neural network to predict the video memory channel number mapped by each physical address.
6. An electronic device, characterized in that, It includes: A memory for storing a computer program; A processor, when executing the program stored on the memory, implements the method for fitting the GPU video memory channel mapping according to any one of claims 1 to 5.
7. A computer-readable storage medium, when the instructions in the storage medium are executed by the processor of an electronic device, enable the electronic device to execute the method for fitting the GPU video memory channel mapping according to any one of claims 1 to 5.
8. A computer program product, including a computer program, which when executed by a processor implements the method for fitting the GPU video memory channel mapping according to any one of claims 1 to 5.