Memory reuse method and device based on streaming ordered memory allocation and storage medium

Abstract

The present application relates to chip design technical field, especially to the memory reuse method, equipment and storage medium based on streaming ordered video memory allocation, which pre-allocates a block of continuous virtual address space for each task of the current process, and obtains the virtual address space of each task, when a task initiates a video memory allocation request, a virtual address in the corresponding virtual address space is applied through a virtual address application interface, and no physical address is automatically allocated, when the applied virtual address is a reusable virtual address which has established a mapping relationship with a physical address and is marked as released, the established mapping relationship is directly reused to complete the reuse, which greatly reduces the probability of discontinuous virtual address space, and improves the efficiency of video memory application by reusing the established mapping relationship.

Description

Technical Field

[0001] This invention relates to the field of chip design technology, and in particular to a method, apparatus and storage medium for memory reuse based on streaming ordered memory allocation. Background Technology

[0002] Stream Ordered Memory Allocator (SOMA) is a memory allocation strategy for streaming data processing and computation tasks. Based on the dynamic flow characteristics of data, it allocates and releases memory space in real-time and on demand during program execution to adapt to streaming processing models and reduce memory fragmentation. It allows applications to execute memory allocation and release operations in the order they are issued, along with other tasks within the same CUDA stream.

[0003] SOMA-based memory allocation methods include synchronous and asynchronous modes. In asynchronous mode, the user needs to specify the size of the memory to be allocated or released, as well as the stream to which that memory belongs. The user sends the operations to be performed on the stream, such as kernel launch, memory copying, memory allocation, and memory release. Operations on the same stream are executed on the GPU in the order they are sent by the CPU. The kernel launch is used to implement the mapping from virtual addresses to physical addresses. Typically, an operation sent on the stream is asynchronous. For example, calling the memory allocation interface does not mean that memory will be allocated immediately, and calling the memory release interface does not mean that memory will be released immediately. Calling these interfaces only indicates that the relevant task has been sent; the host side does not need to wait for the operation on the stream to complete. The allocation and release are only considered complete after the GPU side has completed its corresponding task.

[0004] The SOMA-based video memory allocation method handles the following asynchronous operations sequentially on a stream: Virtual address va1 is allocated via the video memory allocation interface, mapped to a physical address; virtual address va2 is allocated via the same interface, mapped to a physical address; the video memory of va1 is marked for release; the video memory of va2 is marked for release; virtual address va3 is allocated via the same interface, mapped to a physical address; and the video memory of va3 is marked for release. The release of va1 and va2 occurs before va3. The virtual address space of va3 is larger than the sum of the virtual address spaces of va1 and va2. Theoretically, va3 can reuse the video memory space of va1 and va2. However, this requires that the virtual addresses of ptr1 and ptr2 be contiguous. Existing video memory allocation methods cannot guarantee the contiguousness of virtual addresses va1 and va2. Because each virtual memory location needs to be allocated independently, and is affected by the memory allocation requests of other threads in the current process at the same time and the underlying allocation algorithm of the GPU driver, the continuity and correlation of virtual addresses cannot be guaranteed. Therefore, va3 cannot reuse the virtual address spaces of va1 and va2 and their mapping relationship with physical addresses, resulting in low reuse rate. Thus, a memory reuse method that can reuse multiple virtual addresses to improve reuse rate is urgently needed. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention adopts the following technical solution: a video memory reuse method based on SOMA, the method comprising the following steps:

[0006] S100 pre-allocates a contiguous virtual address space for each task of the current process, thus obtaining the virtual address space for each task.

[0007] S300, when the i-th task initiates the j-th memory allocation request Mreq i,j When calling the virtual address request interface, the Mreq is used. i,j Allocate virtual address range va i,j The virtual address request interface performs the following steps when invoked:

[0008] S310, Get Mreq i,j Requested video memory size i,j and the virtual address space VA pre-allocated for the i-th task i .

[0009] S320, when the VA i There exists a value greater than or equal to the stated size. i,j When the allocatable virtual address range is Mreq, i,j The target virtual address segment va is obtained by allocating virtual addresses within the allocatable virtual address segment. i,jThe allocatable virtual address range includes reusable virtual addresses, which are addresses that have been mapped to physical addresses and marked as released.

[0010] S500, according to the va i,j Query the mapping table between virtual addresses and physical addresses to obtain the virtual address (VA). i,j A reusable virtual address segment that has been mapped to a physical address.

[0011] S700, the first type of virtual address segment directly reuses the established mapping relationship to obtain the va i,j physical address range pa i,j This completes the reuse of video memory.

[0012] Furthermore, the present invention also provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, wherein the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the above-described method.

[0013] In addition, the present invention provides an electronic device including a processor and the aforementioned non-transitory computer-readable storage medium.

[0014] The present invention has at least the following beneficial effects:

[0015] The present invention provides a video memory reuse method, device, and storage medium based on streaming ordered video memory allocation. It requests virtual addresses from a pre-allocated virtual address space for the current task through a virtual address request interface. This interface only requests virtual addresses and does not automatically allocate physical addresses. When the requested virtual address is a reusable virtual address that has already been mapped to a physical address and is marked as released, the established mapping relationship is directly reused, completing the reuse. Since the current task requests virtual addresses from the pre-allocated virtual address space each time, the probability of non-contiguous virtual address space is significantly reduced. Furthermore, reusing the established mapping relationship not only improves the efficiency of video memory allocation but also significantly saves system resource overhead because it eliminates the need to re-establish mapping relationships. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a video memory reuse method based on SOMA provided for an embodiment of the present invention;

[0018] Figure 2 The flowchart illustrates the execution steps of the virtual address application interface provided in this embodiment of the invention when it is invoked. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Unless otherwise defined, all technical and scientific terms used in the embodiments of this invention have the same meaning as commonly understood by those skilled in the art.

[0021] Please refer to the figure, which shows a flowchart of a video memory reuse method based on streaming ordered video memory allocation. The method includes the following steps:

[0022] S100 pre-allocates a contiguous virtual address space for each task of the current process, thus obtaining the virtual address space for each task.

[0023] Each process generates multiple tasks, and each task may include multiple operations that require access to video memory. Therefore, a contiguous virtual address space needs to be allocated to each task. This virtual address space is private to the current task and will not be allocated to other tasks.

[0024] In one implementation, the host device initiates a request for video memory. The host calls the video memory allocation interface to allocate a virtual address. Then, the kernel driver module of the slave device establishes a mapping relationship between the virtual address and the physical address of the slave device. Simultaneously, when the slave device accesses a virtual address, the kernel driver module of the slave device queries the mapping table to translate the virtual address into a physical address, and then completes the corresponding video memory read / write operation.

[0025] In one implementation, the master device is a central processing unit (CPU), but other types of master devices used to initiate video memory requests to slave devices also fall within the scope of protection of this invention.

[0026] In one implementation, the slave device is an accelerator (GPU), but other types of slave devices used to receive memory requests from the master device also fall within the scope of protection of this invention.

[0027] It should be noted that when operations to be performed are sent to the slave device's stream, such as kernel driver module startup, memory copying, memory allocation, and memory deallocation, the operations on the same stream will be executed on the slave device in the order they are sent. Furthermore, the operations on the stream are asynchronous, meaning they do not need to wait for the corresponding operation to actually complete.

[0028] It should be noted that the virtual address space is theoretically infinite, and since virtual addresses are virtual, they do not occupy actual physical video memory. Pre-allocating a contiguous virtual address space can significantly reduce the probability of non-contiguous virtual address spaces.

[0029] S300, when the i-th task initiates the j-th memory allocation request Mreq i,j When calling the virtual address request interface, the Mreq is used. i,j Allocate virtual address range va i,j It should be noted that this virtual address request interface only returns a virtual address and does not allocate a physical address. The physical address is allocated and mapped when the stream is actually executed, which delays the allocation of physical video memory and reduces video memory usage.

[0030] For further information, please refer to [link / reference]. Figure 2 The virtual address request interface performs the following steps when invoked:

[0031] S310, Get Mreq i,j Requested video memory size i,j and the virtual address space VA pre-allocated for the i-th task i .

[0032] Among them, from VA i Mreq i,j The allocated virtual address range can reduce the probability of the allocated virtual address range being non-contiguous.

[0033] S320, when the VA i There exists a value greater than or equal to the stated size. i,j When the allocatable virtual address range is Mreq, i,j The target virtual address segment va is obtained by allocating virtual addresses within the allocatable virtual address segment. i,j The allocatable virtual address range includes reusable virtual addresses, which are addresses that have been mapped to physical addresses and marked as released.

[0034] In one implementation, in S320, the va i,jThe starting address is the starting address of the first allocatable virtual address segment found. This allocation method can improve the allocation efficiency of virtual addresses. In one embodiment, the starting point of the search is the starting point for allocating virtual addresses in the virtual address space. Reusable virtual addresses are concentrated in historically allocated virtual address segments. Searching from the allocation starting point can prioritize the allocation of reusable virtual addresses, reduce virtual address fragmentation, and increase the probability of reusing video memory.

[0035] In another embodiment, in S320, the va i,j The steps for obtaining the starting address include: finding all values ​​greater than or equal to the size. i,j The allocable virtual address ranges are obtained, and the size of each of the allocable virtual address ranges is obtained in relation to the size. i,j The difference is taken as the starting address of the allocatable virtual address segment with the smallest difference. i,j The starting address. For example, when there happens to be a value in the virtual address space that is the same size as the stated size. i,j If two equally available virtual address ranges are available, that range will be allocated first. Prioritizing the allocation of the virtual address range with the smallest difference can reduce the generation of virtual address fragmentation.

[0036] It should be noted that other methods for obtaining VA i,j All methods for determining the starting address fall within the protection scope of this invention.

[0037] In one implementation, in S320, the va i,j end address i,j Satisfy: end i,j =start i,j +size i,j -1.

[0038] In one implementation, in S320, when the VA i There is no value greater than or equal to the stated size. i,j When an allocatable virtual address range is available, the VA is extended from the tail address of the virtual address space. i The size is obtained by expanding VA i And according to S320 based on the extended VA i Assign the va i,j Extending the virtual address from the end ensures contiguous virtual addresses before and after the extension, reducing memory fragmentation. In one implementation, the size of each extension is a preset size. Other types of extension methods also fall within the scope of this invention.

[0039] In one implementation, in S320, the allocatable virtual address segment further includes free addresses, which are virtual addresses that have been released but not reclaimed, virtual addresses that have been reclaimed, and virtual addresses that have never been allocated. Virtual addresses allocated to requests from a task are all allocated from the same reserved virtual address space, which improves the continuity of virtual addresses, reduces the probability of virtual address fragmentation, and increases the number of reusable virtual addresses, further improving the probability of video memory reuse.

[0040] It should be noted that multiple states of virtual addresses exist simultaneously in the system, including: allocated virtual addresses that are not marked as released, virtual addresses that have been mapped to physical addresses and marked as released, released addresses that have not been reclaimed, reclaimed virtual addresses, and virtual addresses that have never been allocated. Among these, allocated virtual addresses that are not marked as released are requested virtual addresses and cannot be reused. Virtual addresses that have been mapped to physical addresses and marked as released are virtual addresses whose use has been completed and whose video memory needs to be released; at this time, the mapping relationship has not been broken and they are free, making them reusable virtual addresses. Addresses that have been released but not reclaimed and virtual addresses that have been reclaimed have had their physical address mapping relationship broken, while virtual addresses that have never been allocated have not had a mapping relationship established with physical addresses. Therefore, these three types of virtual addresses are all free virtual addresses without established mapping relationships and can be allocated to the corresponding requests. Compared to reusable virtual addresses, these free virtual addresses need to have their mapping relationship with physical addresses re-established.

[0041] S500, according to the va i,j Query the mapping table between virtual addresses and physical addresses to obtain the virtual address (VA). i,j A reusable virtual address range that has already been mapped to a physical address. It should be noted that va... i,j The mapping relationship between reusable virtual address segments and physical addresses needs to be obtained by querying a mapping table. This mapping table stores all currently established and unresolved mapping relationships; resolved mapping relationships are deleted from the table.

[0042] S700, the first type of virtual address segment directly reuses the established mapping relationship to obtain the va i,j physical address range pa i,j This completes the reuse of video memory.

[0043] By reusing the established mapping relationships, not only can the efficiency of video memory allocation be improved, but also the system's resource consumption can be greatly reduced because there is no need to re-establish the mapping relationships.

[0044] In one embodiment, the method further includes: S900, releasing the pa i,j The video memory includes the following steps:

[0045] S910, initiate the release of the pa i,j The host device, upon receiving the video memory release request, calls the virtual address release interface to release the video memory. i,j Mark it as released and bind a callback function, which is triggered when the video memory physical page is actually unmapped.

[0046] S930, from the device in stream i Register a callback function in the middle to track the status of the video memory and monitor the pa in real time. i,j The released state.

[0047] S950, when the stream i Execution to the pa i,j When the callback function is called, the mapping unmapping interface is invoked to unmapping the pa. i,j With the va i,j The mapping relationship is then established; and the physical memory release interface is called to release the p... i,j .

[0048] Once the mapping and memory release operations are complete, the released memory will be reclaimed at an appropriate time. It's important to note that because virtual address segments do not actually occupy physical memory resources, they do not require the same actual release process as physical memory. Marking a virtual address segment as released indicates that it has been freed and can be reused.

[0049] It should be noted that the method provided in this embodiment of the invention uses separate interfaces for allocating virtual addresses, allocating physical addresses, establishing mapping relationships, releasing virtual addresses, releasing physical addresses, and demapping relationships. By debinding the allocation of virtual addresses and physical addresses, the method can achieve the following: when calling the virtual address allocation interface, only the virtual address is returned without allocating a physical address. When the stream is actually executed, the physical address allocation interface is called to allocate the physical address, thus delaying the allocation of physical video memory and significantly reducing video memory usage.

[0050] In one implementation, in S500, the va i,jThe method also includes idle virtual address segments that have not yet been mapped to physical addresses. The method further includes: S600, for the idle virtual address segments, sequentially calling the physical address request interface to request physical addresses, and calling the mapping establishment interface to establish the mapping relationship between the virtual addresses in the idle virtual address segments and the requested physical addresses. That is, when the requested virtual addresses include both reusable virtual addresses and idle virtual address segments, the physical addresses of the reusable virtual addresses can be used directly, while the idle virtual address segments need to have a separate mapping relationship established with physical addresses.

[0051] In one implementation, S600 further includes: S610, when a specified size of video memory cannot be allocated, adjusting the physical address allocation unit to the smallest allocation unit. When allocating a large block of video memory, multiple non-contiguous small blocks of physical addresses can be allocated sequentially, and then the mapping relationship with the virtual address segment can be re-established in turn to achieve the purpose of allocating a large block of video memory, thereby improving the success rate of video memory allocation and reducing physical address fragmentation.

[0052] In one embodiment, the method further includes: S1000, when the i-th task initiates the r-th video memory allocation request Mreq i,r Reuse the aforementioned va i,j When, if the Mreq i,r Requested video memory size i,k Smaller than the size i,j Then release the va i,j This reduces excess video memory. Because virtual address segments are mapped to multiple smallest units of physical address space, each smallest unit of physical address space can be released independently, which can significantly reduce video memory usage.

[0053] In one embodiment, S900 further includes: S920, when the va i,j With va marked as released i,k When the virtual addresses are consecutive, the virtual addresses are merged. i,j and the va i,k The virtual address; wherein, the va i,k The k-th memory allocation request Mreq initiated for the i-th task i,k The allocated virtual address segment. This method can merge multiple contiguous free video memory addresses into a single contiguous free video memory block.

[0054] In summary, the present invention provides a video memory reuse method based on SOMA. This method requests virtual addresses from a pre-allocated virtual address space for the current task via a virtual address request interface. This interface only requests virtual addresses and does not automatically allocate physical addresses. When the requested virtual address is a reusable virtual address that has already been mapped to a physical address and is marked as released, the established mapping relationship is directly reused, thus completing the reuse. Since the current task requests virtual addresses from the pre-allocated virtual address space each time, the probability of non-contiguous virtual address space is significantly reduced. Furthermore, reusing the established mapping relationship not only improves the efficiency of video memory allocation but also significantly saves system resource overhead because it eliminates the need to re-establish the mapping relationship.

[0055] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiments.

[0056] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0057] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0058] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0059] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of this invention is defined by the appended claims.

Claims

1. A video memory reuse method based on streaming ordered video memory allocation, characterized in that, The method includes the following steps: S100: Pre-allocate a contiguous virtual address space for each task of the current process to obtain the virtual address space of each task; S300, when the i-th task initiates the j-th memory allocation request Mreq i,j When calling the virtual address request interface, the Mreq is used. i,j Allocate virtual address range va i,j The virtual address request interface performs the following steps when invoked: S310, Get Mreq i,j Requested video memory size i,j and the virtual address space VA pre-allocated for the i-th task i ; S320, when the VA i There exists a value greater than or equal to the stated size. i,j When the allocatable virtual address range is Mreq, i,j The target virtual address segment va is obtained by allocating virtual addresses within the allocatable virtual address segment. i,j The allocatable virtual address range includes reusable virtual addresses, which are addresses that have been mapped to physical addresses and marked as released. S500, according to the va i,j Query the mapping table between virtual addresses and physical addresses to obtain the virtual address (VA). i,j The reusable virtual address range that has already been mapped to physical addresses; S700, the first type of virtual address segment directly reuses the established mapping relationship to obtain the va i,j physical address range pa i,j This completes the reuse of video memory.

2. The method according to claim 1, characterized in that, In S320, the va i,j The starting address is the starting address of the first allocatable virtual address segment found.

3. The method according to claim 1, characterized in that, In S320, the va i,j The steps for obtaining the starting address include: finding all values ​​greater than or equal to the size. i,j The allocable virtual address ranges are obtained, and the size of each of the allocable virtual address ranges is obtained in relation to the size. i,j The difference is taken as the starting address of the allocatable virtual address segment with the smallest difference. i,j The starting address.

4. The method according to claim 1, characterized in that, In S500, the va i,j The method also includes free virtual address segments that have not been mapped to physical addresses; the method further includes: S600, for the free virtual address segment, the physical address application interface is called sequentially to apply for physical addresses, and the mapping establishment interface is called to establish the mapping relationship between the virtual addresses in the free virtual address segment and the applied physical addresses.

5. The method according to claim 4, characterized in that, The S600 also includes the S610, which adjusts the physical address allocation unit to the minimum allocation unit when a specified size of video memory cannot be allocated.

6. The method according to claim 1, characterized in that, The method further includes: S1000, when the i-th task initiates the r-th video memory allocation request Mreq i,r Reuse the aforementioned va i,j When, if the Mreq i,r Requested video memory size i,k Smaller than the size i,j Then release the va i,j The excess video memory.

7. The method according to claim 1, characterized in that, The method further includes: S900, release the pa i,j The video memory includes the following steps: S910, the master device initiates the release of the pa i,j The system receives a video memory release request and, based on the video memory release request, calls the virtual address release interface to release the video memory. i,j Mark it as released and bind a callback function, which is to be triggered after the actual unmapping; S930, from the device in stream i Register a callback function in the middle to track the status of the video memory and monitor the pa in real time. i,j The released state; S950, when the stream i Execution to the pa i,j When the callback function is called, the mapping unmapping interface is invoked to unmapping the pa. i,j With the va i,j The mapping relationship is then established; and the video memory release interface is called to release the memory. i,j .

8. The method according to claim 7, characterized in that, The S900 also includes: S920, when the va i,j With va marked as released i,k When the virtual addresses are consecutive, the virtual addresses are merged. i,j and the va i,k The virtual address; wherein, the va i,k The k-th memory allocation request Mreq initiated for the i-th task i,k The allocated virtual address range.

9. A non-transitory computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the method as described in any one of claims 1-8.

10. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 9.

Images