Memory management device and memory management method

Abstract

The invention provides a memory management device and method. The memory management device comprises a processor; and a non-transitory computer readable medium storing a memory management program that, when executed by the processor, causes the processor to perform: receiving a memory allocation request including a specified size, allocating a buffer having the specified size in memory according to the memory allocation request, and generating a buffer pointer, the buffer area is a device buffer area or a CPU buffer area; receiving a memory access request including a search pointer, determining whether the search pointer corresponds to the device buffer or the CPU buffer according to the buffer pointer and the specified size, allowing access to the device buffer if the search pointer corresponds to the device buffer, and allowing access to the CPU buffer if the search pointer corresponds to the CPU buffer. And the CPU buffer area is allowed to be accessed.

Description

Technical Field

[0001] The present invention relates to memory management technology, and in particular to a memory management device and a memory management method. Background Art

[0002] In modern computing systems, various processing devices, such as central processing units (CPUs), graphics processing units (GPUs), and domain-specific accelerators (e.g., neural processing units (NPUs) and vision processing units (VPUs)), require efficient memory access to perform computational tasks. To improve performance, data is often stored in device memory to reduce data transfer latency and offload memory operations from the CPU. However, different types of buffers, such as CPU buffers and direct memory access (DMA) buffers, may have different access mechanisms, making memory management complex for software developers.

[0003] For example, when accessing a DMA buffer, developers typically need to handle multiple parameters, including the DMA buffer's file descriptor, the buffer's base address, and an offset value. In contrast, CPU buffers can typically be accessed directly through pointers. These inconsistencies in memory access interfaces across different devices increase programming complexity and reduce software portability.

[0004] Furthermore, some device memory areas (e.g., some device buffers) may have restricted access due to security concerns. Certain proprietary accelerators may not allow direct memory access from the application space to prevent erroneous operations that could lead to system failures or security vulnerabilities. This further increases the complexity of memory management in heterogeneous computing environments.

[0005] Therefore, a memory management device and a memory management method are needed to provide a unified method for managing different types of memory areas. Summary of the Invention

[0006] The invention provides a memory management device and a memory management method.

[0007] In an optional embodiment, the present invention provides a memory management device, comprising: a processor; and a non-transitory computer-readable medium coupled to the processor and storing a memory management program comprising instructions, which, when executed by the processor, causes the processor to execute an allocator module and a parser module; wherein executing the allocator module causes the processor to: receive a memory allocation request from an application, wherein the memory allocation request includes a specified size, allocate a buffer having the specified size in a memory according to the memory allocation request, and generate a buffer pointer for the buffer, and return the buffer pointer to the application, wherein the buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for a CPU; and wherein executing the parser module causes the processor to: receive a memory access request from the application, wherein the memory access request includes a search pointer, determine whether the search pointer corresponds to the device buffer or the CPU buffer based on the buffer pointer and the specified size, and allow access to the device buffer if the search pointer corresponds to the device buffer, and allow access to the CPU buffer if the search pointer corresponds to the CPU buffer.

[0008] In another optional embodiment, the present invention provides a memory management device, comprising: an allocator circuit and a parser circuit, wherein the divider circuit is configured to: receive a memory allocation request from an application, wherein the memory allocation request includes a specified size, allocate a buffer with the specified size in a memory according to the memory allocation request, and generate a buffer pointer for the buffer, and return the buffer pointer to the application, wherein the buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for a CPU; and the parser circuit is configured to: receive a memory access request from the application, wherein the memory access request includes a search pointer, determine whether the search pointer corresponds to the device buffer or the CPU buffer according to the buffer pointer and the specified size, and allow access to the device buffer if the search pointer corresponds to the device buffer, and allow access to the CPU buffer if the search pointer corresponds to the CPU buffer.

[0009] In another optional embodiment, the present invention provides a memory management method applied to a computer system that executes an application, the method comprising: a memory allocation phase, comprising: receiving a memory allocation request from the application, wherein the memory allocation request includes a specified size, allocating a buffer with the specified size in the memory according to the memory allocation request; and generating a buffer pointer for the buffer, and returning the buffer pointer to the application; wherein the device buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for the CPU; and an address resolution phase, comprising: receiving a memory access request from the application, wherein the memory access request includes a search pointer, determining whether the search pointer corresponds to the device buffer or the CPU buffer according to the buffer pointer and the specified size, and allowing access to the device buffer if the search pointer corresponds to the device buffer, and allowing access to the CPU buffer if the search pointer corresponds to the CPU buffer.

[0010] Embodiments of the memory management apparatus and memory management method provided herein address the challenges of managing heterogeneous memory accesses by introducing a unified approach for allocating and resolving memory addresses. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A A flowchart showing steps of a memory management method 10 according to an embodiment of the present invention is shown.

[0012] Figure 1B is a corresponding system architecture diagram illustrating the interactions between various components involved in the memory management method 10 .

[0013] Figure 2 is a flow chart according to an embodiment of the present invention, illustrating a detailed process of determining whether a search pointer corresponds to a device buffer or a CPU buffer during the address resolution phase.

[0014] Figures 3A to 3C FIG. 1 is a schematic diagram of an embodiment of the present invention, illustrating different situations in which it is determined whether a search pointer corresponds to a device buffer or a CPU buffer.

[0015] Figure 4 is a block diagram of a memory management device 40 according to an embodiment of the present invention.

[0016] Figure 5 is a block diagram of a memory management device 50 according to another embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following description is made for the purpose of illustrating the general principles of the invention and should not be construed in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

[0018] In each of the following embodiments, the same reference numerals represent the same or similar elements or components.

[0019] The ordinal numbers used in the claims, such as "first", "second", "third", etc., are only for convenience of interpretation and do not imply any priority relationship between each other.

[0020] The descriptions provided below for embodiments of an apparatus or system also apply to embodiments of a method, and vice versa.

[0021] Figure 1A A flowchart showing the steps of a memory management method 10 is shown according to an embodiment of the present invention. Figure 1A As shown, the memory management method 10 includes a memory allocation phase 11 and an address resolution phase 12. The memory allocation phase 11 includes steps S111-S113, and the address resolution phase 12 includes steps S121-S123. Figure 1B is a corresponding system architecture diagram, which illustrates the interaction between various components involved in the memory management method 10. In order to better understand this embodiment, reference should also be made to Figure 1A and Figure 1B .

[0022] like Figure 1A The memory management method 10 shown is composed of Figure 1B The memory manager 100 shown performs the following operations: The memory manager 100 further comprises an allocator 101 and a resolver 102. The memory allocation phase 11 and the address resolution phase 12 are performed by the allocator 101 and the resolver 102, respectively.

[0023] In step S111, allocator 101 receives memory allocation request 104 from application 103. Memory allocation request 104 includes a specified size N indicating the amount of memory required by application 103.

[0024] In step S112, in response to the memory allocation request 104, the allocator 101 allocates a buffer 110 of a specified size N in the memory 105. The buffer 110 may be a device buffer 106 allocated for and accessible by one or more accelerator devices 150, including but not limited to a graphics processing unit (GPU), a neural processing unit (NPU), a vision processing unit (VPU), a DMA, and a deep learning accelerator (DLA). Furthermore, the buffer 110 may be a CPU buffer 107 allocated for and accessible by the CPU 160. In some embodiments, the device buffer 106 may be a GPU buffer, a VPU buffer, a DMA buffer, a DLA buffer, etc. More generally, the accelerator device 150 in the computer system may be any device other than a CPU. In some embodiments, the memory 105 is system memory, such as the computer system's random access memory (RAM), and the device buffer 106 and the CPU buffer 107 share space in the system memory. In some embodiments, the memory 105 may include multiple hardware memories located in different hardware devices. For example, the memory 105 may include a system memory, and an area of ​​the system memory may be allocated as a CPU buffer 107 of the CPU. The memory 105 may also include a video random access memory (VRAM), and an area of ​​the VRAM may be allocated as a GPU buffer (which is a device buffer 106).

[0025] In step S113, allocator 101 generates a buffer pointer 108P for buffer 110 and returns buffer pointer 108P to application 103. Buffer pointer 108P serves as a unified address pointer, helping application 103 access buffer 110 in a consistent manner. In some embodiments, buffer pointer 108P indicates the starting address of allocated buffer 110. In some other embodiments, buffer pointer 108P indicates the ending address of allocated buffer 110. In some other embodiments, buffer pointer 108P indicates an offset address corresponding to a fixed position (e.g., a starting address, an intermediate address, or an ending address) of allocated buffer 110.

[0026] In step S121, the parser 102 receives a memory access request 109 from the application 103. The memory access request 109 includes a search pointer P*, which indicates a memory address for data access requested by the application 103. For example, during neural network inference, the search pointer P* may correspond to a memory location storing input feature maps, intermediate tensors, or model weights required for computation.

[0027] In step S122, the parser 102 determines whether the search pointer P* corresponds to the device buffer 106 or the CPU buffer 107 based on the buffer pointer 108 and the designated size N. Specifically, if it is determined that the search pointer P* falls within the address range of the device buffer 106, then the search pointer P* is determined to correspond to the device buffer 106. Otherwise, the search pointer P* corresponds to the CPU buffer 107.

[0028] In step S123, parser 102 allows application 103 to access the corresponding memory area. Specifically, if search pointer P* corresponds to device buffer 106, access to device buffer 106 is granted. For example, accelerator device 150 or CPU 160 may be allowed to read or write data in device buffer 106 via application 103. If search pointer P* corresponds to CPU buffer 107, access to CPU buffer 107 is granted. For example, CPU 160 or accelerator device 150 may be allowed to read data in CPU buffer 106 associated with CPU 160 or write data associated with CPU 160 to the CPU buffer via application 103.

[0029] It should be noted that although Figure 1B Only a single device buffer 106 is shown; in actual implementations, multiple portions of memory 105 may be allocated as device buffers in response to different memory allocation requests. Each allocated device buffer is associated with a unique buffer pointer, allowing application 103 to reference and access different memory regions as needed. Thus, memory manager 100 may maintain multiple buffer pointers, each corresponding to an allocated device buffer or CPU buffer and a corresponding designated size for the buffer.

[0030] In one embodiment, the memory allocation stage 11 further includes storing a mapping entry between a buffer pointer of the device buffer, the device buffer, and a specified size associated with the device buffer in a data structure. The data structure may be implemented as a <keyword, value> pair (<key,value> pairs), lookup tables, linked lists, or tree-based structures that allow efficient searching and retrieval of buffer information, but the present disclosure is not limited thereto. In some embodiments, the data structure may include a plurality of mapping entries, each mapping entry being a mapping between a buffer pointer of a device buffer, the device buffer, and a specified size associated with the device buffer. In one implementation, a mapping entry may be represented as<key,value> For, the buffer pointer 108 (denoted as P) is used as the key, the corresponding device buffer 106 (denoted as M) and the specified size N form the value, that is,<key,value> Yes, it can be <P,<M,N> >. This mapping enables the parser 102 to efficiently parse memory access requests to the data buffer 106.

[0031] In addition, step S122 uses this data structure to determine whether the search pointer corresponds to the device buffer or the CPU buffer. More detailed steps about this process will be referred to later. Figure 2 Explanation: The figure expands Figure 1A Step S122.

[0032] Figure 2 FIG. 1 is a flow chart according to an embodiment of the present invention, illustrating a detailed process of determining whether a search pointer corresponds to a device buffer or a CPU buffer during the address resolution phase. Figure 2 As shown, this process, Figure 1A Step S122 further includes steps S201-S203.

[0033] In step S201, the parser 102 checks whether the data structure contains at least one mapping entry. For example, the parser 102 checks whether the size of the data structure is non-zero. As previously described, the data structure stores mapping entries between buffer pointers for device buffers, device buffers, and their corresponding specified sizes. Each mapping entry in the data structure may be a mapping between a buffer pointer for a device buffer, a device buffer (e.g., an ID of a device buffer), and a specified size of the device buffer. If the data structure does not contain any mapping entries, meaning that no device buffer has been allocated, the parser 102 determines that the search pointer corresponds to a CPU buffer and ends the process. Otherwise, the process continues to step S202.

[0034] In step S202, parser 102 identifies the largest buffer pointer from the data structure that is no larger than search pointer P* included in memory access request 109. The identified buffer pointer serves as a reference for determining whether the search pointer falls within the device's buffer allocation range. This step ensures that parser 102 selects the buffer pointer that is closest to search pointer P* and smaller than search pointer P*.

[0035] In step S203, parser 102 determines whether the search pointer is within the range of the device buffer associated with the identified buffer pointer. Specifically, parser 102 checks whether the search pointer is less than or equal to the sum of the identified buffer pointer P and the specified size N associated with the buffer pointer. If this condition is met, search pointer P* corresponds to the device buffer and access is granted accordingly. Otherwise, parser 102 determines that search pointer P* corresponds to the CPU buffer.

[0036] Figures 3A to 3C 1 is a schematic diagram illustrating different scenarios for determining whether a search pointer corresponds to a device buffer or a CPU buffer according to an embodiment of the present invention. The three scenarios represent various scenarios for evaluating a search pointer based on a buffer pointer stored in a data structure and its associated allocation size.

[0037] Figure 3A This illustrates the case where the search pointer points directly to the start address of the allocated device buffer. Figure 3A In the example, the search pointer is 0x20, which completely matches the start address of the device buffer (e.g., area A of memory 30) indicated by the identified maximum buffer pointer. The device buffer (e.g., area A) has a size of 16 bytes. Since the search pointer is within the address range of area A (from 0x20 to 0x30), it is determined that the search pointer corresponds to the device buffer.

[0038] Figure 3B This describes the situation where the search pointer is within the allocated range of the device buffer. Figure 3B In the example, the search pointer is 0x28, which falls within the range of region A (0x20 to 0x30). For example, the parser recognizes that the largest buffer pointer not greater than 0x28 is 0x20, and then checks whether the search pointer is within the allocated size range of region A. Since 0x28 ≤ 0x20 + 16, it is determined that the search pointer corresponds to the device buffer.

[0039] Figure 3C Describes a situation where the search pointer falls outside the allocated device buffer area, resulting in a default mapping to the CPU buffer. Figure 3CIn this example, the search pointer is 0x92, which is not within the range of any allocated device buffer. In this example, the parser recognizes that the largest buffer pointer not greater than 0x92 is 0x70 (corresponding to region B). However, since 0x92 > 0x70 + 32, the search pointer does not correspond to any device buffer, and therefore the search pointer is determined to correspond to the CPU buffer.

[0040] In one embodiment, the data structures described above (e.g.,<key,value> The buffer pointers are stored as a red-black tree, and a binary search is performed on the red-black tree to identify the largest buffer pointer that is no larger than the search pointer included in the memory access request. A red-black tree is a self-balancing binary search tree that maintains a balanced structure by enforcing specific color-based properties during insertion and deletion operations. These properties ensure that the tree remains approximately balanced and keeps its height in O(log n), where n is the number of nodes in the tree. By using a red-black tree to store the mapping between buffer pointers and device buffers, the parser can efficiently identify the largest buffer pointer that is no larger than the search pointer through a binary search, which runs in O(log n) time complexity. The binary search works by repeatedly comparing the search pointer with the buffer pointers stored in the tree and navigating left or right based on the comparison result. This approach ensures that the lookup operation is performed in logarithmic time, significantly reducing the overhead of address resolution compared to unstructured or linear search methods.

[0041] In one embodiment, the address resolution phase also includes determining whether the memory access request is issued by a process authorized to access the device buffer or CPU buffer. If it is determined that the memory access request is issued by an unauthorized process, access to the device buffer or CPU buffer is denied. To ensure secure memory access, the resolver may maintain an access control list (ACL) or use a process authentication mechanism to verify whether the requesting process has the necessary permissions to access the device buffer or CPU buffer. This verification can be performed by checking the process identifier (PID), security token, or cryptographic authentication key associated with the memory access request. By incorporating access control into the address resolution phase, the system can prevent unauthorized access and reduce the risk of accidental memory corruption, data leakage, or malicious attacks. This security measure is particularly important in multi-process environments or shared computing platforms, where different applications or processes may request access to the same memory area. Denying access to unauthorized processes helps maintain data integrity and ensure that memory resources are securely managed across multiple computing devices.

[0042] Figure 4FIG. 4 is a block diagram of a memory management device 40 according to an embodiment of the present invention. Figure 4 As shown, the memory management device 40 includes a processor 41 and a non-transient computer readable medium 42. The non-transient computer readable medium 42 may include a flash memory, a solid-state drive (SSD), a hard disk drive (HDD) or other storage media, but the present invention is not limited thereto. The non-transient computer readable medium 42 stores a memory management program 400, which can be implemented using various programming languages, including but not limited to C, C++, Python, Java or assembly language, and can be executed as part of an operating system, a device driver or an application-level library. The memory management program 400 corresponds to Figure 1B The memory manager 100 in FIG. 1 includes instructions that, when executed by the processor, cause the processor 41 to execute the allocator module 401 and the parser module 402. The allocator module 401 corresponds to Figure 1B The allocator 101 in the memory is responsible for processing memory allocation requests to allocate buffers (including device buffers and CPU buffers), generating buffer pointers, and returning the buffer pointers to the application. The parser module 402 corresponds to Figure 1B The parser 102 in is responsible for processing memory access requests and determining whether a given search pointer corresponds to a device buffer or a CPU buffer.

[0043] Figure 5 is a block diagram of a memory management device 50 according to another embodiment of the present invention. The memory management device 50 is implemented as a hardware circuit, including an allocator circuit 501 and a resolver circuit 502. These circuits can be implemented using dedicated hardware logic, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a system-on-chip (SoC) design to optimize memory allocation and address resolution operations. The allocator circuit 501 is configured to process memory allocation requests to allocate buffers (including device buffers and CPU buffers), generate buffer pointers, and return the buffer pointers to the application, corresponding to Figure 1B The parser circuit 502 is responsible for processing memory access requests, determining whether a given search pointer corresponds to a device buffer or a CPU buffer, and Figure 1B The parser 102 in .

[0044] Embodiments of the memory management apparatus and memory management method provided herein address the challenges of managing heterogeneous memory access by introducing a unified approach for allocating and resolving memory addresses. The techniques of the present invention simplify memory access by efficiently processing memory allocation requests and storing mappings between buffer pointers and buffers. Using structured data organizations, such as red-black trees, fast address resolution is achieved through binary searches, reducing lookup latency and improving scalability. Compared to traditional approaches that require device-specific memory access mechanisms, embodiments of the present invention improve software portability, reduce programming complexity, and optimize memory access efficiency in heterogeneous computing environments.

[0045] The above paragraphs describe various aspects. Clearly, this can be implemented in a variety of ways, as described in the specification. Any specific structure or function disclosed in the examples represents only one example. Those skilled in the art will appreciate that any disclosed aspect can be implemented independently, or two or more aspects can be combined, as described in the specification.

[0046] Although the present invention has been described in terms of examples and preferred embodiments, it should be understood that the invention is not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar arrangements (which are obvious to those skilled in the art). Therefore, the appended claims should be given the broadest interpretation so as to cover all such modifications and similar arrangements.

Claims

1. A memory management device, characterized in that: include: processor; as well as a non-transitory computer-readable medium coupled to the processor and storing a memory management program including instructions that, when executed by the processor, cause the processor to execute an allocator module and a parser module; Wherein executing the allocator module causes the processor to: receiving a memory allocation request from an application, the memory allocation request including a specified size, Allocating a buffer having the specified size in memory according to the memory allocation request, and Generate a buffer pointer for the buffer and return the buffer pointer to the application, The buffer is a device buffer allocated for one or more accelerator devices, or CPU buffer allocated for CPU; Which executes the parser module so that the processor: receiving a memory access request from the application, the memory access request including a search pointer, determining, based on the buffer pointer and the specified size, whether the search pointer corresponds to the device buffer or the CPU buffer, If the search pointer corresponds to the device buffer, access to the device buffer is permitted, and if the search pointer corresponds to the CPU buffer, access to the CPU buffer is permitted.

2. The memory management device according to claim 1, wherein: Executing the allocator module further causes the processor to store, in a data structure, a mapping entry between a plurality of buffer pointers to a plurality of device buffers, the plurality of device buffers, and a plurality of specified sizes associated with the plurality of device buffers; as well as Executing the parser module further causes the processor to: checking whether the data structure contains at least one mapping entry, and if the data structure does not contain any mapping entry, determining that the search pointer corresponds to the CPU buffer; If the data structure contains at least one mapping entry, identifying from the data structure a largest buffer pointer that is no larger than the search pointer contained in the memory access request, and If the search pointer is not greater than the sum of the identified buffer pointer and a specified size associated with the identified buffer pointer, the search pointer is determined to correspond to the device buffer; otherwise, the search pointer is determined to correspond to the CPU buffer.

3. The memory management device according to claim 2, wherein: The data structure is structured as a red-black tree, and executing the parser module further causes the processor to perform a binary search on the red-black tree to identify the largest buffer pointer that is not larger than the search pointer included in the memory access request.

4. The memory management device according to claim 1, wherein: Executing the parser module further causes the processor to determine whether the memory access request is issued by a process authorized to access the buffer, and deny access to the buffer if it is determined that the memory access request is issued by an unauthorized process.

5. The memory management device according to claim 1, wherein: The buffer pointer indicates a start address of the buffer, an end address of the buffer, or an offset address corresponding to a fixed position of the buffer; and The search pointer indicates the memory address requested by the application program.

6. The memory management device according to claim 1, wherein: The device buffer includes a graphics processing unit GPU buffer, a neural processing unit NPU buffer, a vision processing unit VPU buffer, a direct memory access DMA buffer, or a deep learning accelerator DLA buffer; and Each of the one or more accelerator devices includes a GPU, an NPU, a VPU, a DMA, or a DLA.

7. A memory management device, characterized in that: Including distributor circuit and resolver circuit, The distributor circuit configuration is: receiving a memory allocation request from an application, the memory allocation request including a specified size, Allocating a buffer having the specified size in memory according to the memory allocation request, and Generate a buffer pointer for the buffer and return the buffer pointer to the application, The buffer is a device buffer allocated for one or more accelerator devices, or CPU buffer allocated for CPU; The resolver circuit configuration is: receiving a memory access request from the application, wherein the memory access request includes a search pointer, determining whether the search pointer corresponds to the device buffer or the CPU buffer based on the buffer pointer and the specified size, and If the search pointer corresponds to the device buffer, access to the device buffer is permitted, and if the search pointer corresponds to the CPU buffer, access to the CPU buffer is permitted.

8. The memory management device according to claim 7, wherein: The allocator circuit is further configured to store in a data structure a plurality of buffer pointers to a plurality of device buffers, the plurality of device buffers, and map entry buffer pointers between a plurality of specified sizes associated with the plurality of device buffers; and The resolver circuit is further configured as follows: checking whether the data structure contains at least one mapping entry, and if the data structure does not contain any mapping entry, determining that the search pointer corresponds to the CPU buffer; If the data structure contains at least one mapping entry, identifying from the data structure a largest buffer pointer that is no larger than the search pointer contained in the memory access request, and If the search pointer is not greater than the sum of the identified buffer pointer and a specified size associated with the identified buffer pointer, the search pointer is determined to correspond to the device buffer; otherwise, the search pointer is determined to correspond to the CPU buffer pointer.

9. The memory management device according to claim 8, wherein: The data structure is structured as a red-black tree, and the parser circuit is further configured to perform a binary search on the red-black tree to identify the largest buffer pointer that is not larger than the search pointer included in the memory access request.

10. The memory management device according to claim 7, wherein: The parser circuit is further configured to determine whether the memory access request is issued by a process that is authorized to access the buffer, and deny access to the buffer if it is determined that the memory access request is issued by an unauthorized process.

11. The memory management device according to claim 7, wherein: The buffer pointer indicates a start address of the buffer, an end address of the buffer, or an offset address corresponding to a fixed position of the buffer; and The search pointer indicates a memory address buffer pointer requested by the application program.

12. The memory management device according to claim 7, wherein: The device buffer includes a graphics processing unit GPU buffer, a neural processing unit NPU buffer, a vision processing unit VPU buffer, a direct memory access DMA buffer, or a deep learning accelerator DLA buffer; and Each of the one or more accelerator devices includes a GPU, an NPU, a VPU, a DMA, or a DLA.

13. A memory management method, applied to a computer system that executes an application program, characterized in that: The method includes a memory allocation phase and an address resolution phase. The memory allocation stage includes: receiving a memory allocation request from the application, wherein the memory allocation request includes a specified size, allocating a buffer having the specified size in a memory according to the memory allocation request; and Generate a buffer pointer for the buffer and return the buffer pointer to the application; The device buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for a CPU; and The address resolution phase includes: receiving a memory access request from the application, wherein the memory access request includes a search pointer, determining whether the search pointer corresponds to the device buffer or the CPU buffer based on the buffer pointer and the specified size, and If the search pointer corresponds to the device buffer, access to the device buffer is permitted, and if the search pointer corresponds to the CPU buffer, access to the CPU buffer is permitted.

14. The memory management method according to claim 13, wherein: The memory allocation phase further includes storing, in a data structure, a mapping entry between a plurality of buffer pointers to a plurality of device buffers, the plurality of device buffers, and a plurality of specified sizes associated with the plurality of device buffers; Wherein determining whether the search pointer corresponds to the device buffer or the CPU buffer during the address resolution stage further comprises: checking whether the data structure contains at least one mapping entry, and if the data structure does not contain any mapping entry, determining that the search pointer corresponds to the CPU buffer; If the data structure contains at least one mapping entry, identifying from the data structure a largest buffer pointer that is no larger than the search pointer contained in the memory access request, and If the search pointer is not greater than the sum of the identified buffer pointer and a specified size associated with the identified buffer pointer, the search pointer is determined to correspond to the device buffer; otherwise, the search pointer is determined to correspond to the CPU buffer.

15. The memory management method according to claim 14, wherein: The data structure is constructed as a red-black tree, and a binary search is performed on the red-black tree to identify the largest buffer pointer that is no larger than the search pointer included in the memory access request.

16. The memory management method according to claim 13, wherein: The address resolution stage further includes determining whether the memory access request is issued by a process that is authorized to access the buffer, and if it is determined that the memory access request is issued by an unauthorized process, denying access to the buffer.

17. The memory management method according to claim 13, wherein: The buffer pointer indicates a start address of the buffer, an end address of the buffer, or an offset address corresponding to a fixed position of the buffer; and The search pointer indicates the memory address requested by the application program.

18. The memory management method according to claim 13, wherein: The device buffer includes a graphics processing unit GPU buffer, a neural processing unit NPU buffer, a vision processing unit VPU buffer, a direct memory access DMA buffer, or a deep learning accelerator DLA buffer; and Each of the one or more accelerator devices includes a GPU, an NPU, a VPU, a DMA, or a DLA.

Images