Memory management device and memory management method

Abstract

A memory management device includes: a processor; and a non-transitory computer-readable storage medium storing a memory management program that, when executed by the processor, causes the processor to: receive a memory allocation request including a specified size; allocate a buffer of the specified size in memory according to the memory allocation request; and generate a buffer pointer, wherein the buffer is a device buffer or a CPU buffer; receive a memory access request including a search pointer; and, based on the buffer pointer and the specified size, determine whether the search pointer corresponds to the device buffer or the CPU buffer; if the search pointer corresponds to the device buffer, allow access to the device buffer; and if the search pointer corresponds to the CPU buffer, allow access to the CPU buffer.

Description

Technical Field

[0001] This invention relates to memory management technology, and in particular to a memory management device and a memory management method. Prior Technology

[0002] In modern computing systems, various processing units, such as the Central Processing Unit (CPU), Graphics Processing Unit (GPU), and domain-specific accelerators (e.g., Neural Processing Unit (NPU) and Vision Processing Unit (VPU)), require efficient memory access to perform computational tasks. To improve performance, data is typically stored in device memory, reducing data transfer latency and offloading memory operations from the CPU. However, different types of buffers, such as CPU buffers and Direct Memory Access (DMA) buffers, may have different access mechanisms, complicating memory management for software developers.

[0003] For example, when accessing a DMA buffer, developers typically need to handle multiple parameters, including the DMA buffer's file descriptor, base address, and offset value. In contrast, CPU buffers can usually be accessed directly via pointers. This inconsistency in memory access interfaces between different devices increases programming complexity and reduces software portability.

[0004] Furthermore, access to some device memory regions (e.g., some device buffers) may be restricted for security reasons. Some proprietary accelerators may not allow direct memory access from application space to prevent erroneous operations that could lead to system failures or security vulnerabilities. This further increases the complexity of memory management in heterogeneous computing environments.

[0005] Therefore, a memory management device and a memory management method are needed to provide a unified approach to managing different types of memory regions. Summary of the Invention

[0006] This invention provides a memory management device and a memory management method.

[0007] In an optional embodiment, the present invention provides a memory management device, comprising: a processor; and a non-transitory computer-readable storage coupled to the processor and storing a memory management program including instructions, wherein when the memory management program is executed by the processor, the processor executes an allocator module and a parser module; wherein executing the allocator module causes the processor to: receive a memory allocation request from an application, wherein the memory allocation request includes a specified size, allocate a buffer having the specified size in memory according to the memory allocation request, and generate a buffer pointer for the buffer, and set the buffer pointer... The pointer is returned to the application, wherein the buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for the CPU; and wherein executing the parser module causes the processor to: receive a memory access request from the application, wherein the memory access request includes a search pointer, determine, based on the buffer pointer and the specified size, whether the search pointer corresponds to the device buffer or the CPU buffer, and allow access to the device buffer if the search pointer corresponds to the device buffer, and allow access to the CPU buffer if the search pointer corresponds to the CPU buffer.

[0008] In another optional embodiment, the present invention provides a memory management device, including: an allocator circuit and a resolver circuit, wherein the allocator circuit is configured to: receive a memory allocation request from an application, wherein the memory allocation request includes a specified size; allocate a buffer of the specified size in memory according to the memory allocation request; generate a buffer pointer for the buffer; and return the buffer pointer to the application, wherein the buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for a CPU; and the resolver circuit is configured to: receive a memory access request from the application, wherein the memory access request includes a search pointer; determine, based on the buffer pointer and the specified size, whether the search pointer corresponds to the device buffer or the CPU buffer; and if the search pointer corresponds to the device buffer, allow access to the device buffer; and if the search pointer corresponds to the CPU buffer, allow access to the CPU buffer.

[0009] In another optional embodiment, the present invention provides a memory management method applied to a computer system executing an application. The method includes: a memory allocation stage, comprising: receiving a memory allocation request from the application, wherein the memory allocation request includes a specified size; allocating a buffer of the specified size in memory according to the memory allocation request; generating a buffer pointer for the buffer; and returning the buffer pointer to the application; wherein the device buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for a CPU; and an address resolution stage, comprising: receiving a memory access request from the application, wherein the memory access request includes a search pointer; determining whether the search pointer corresponds to the device buffer or the CPU buffer according to the buffer pointer and the specified size; and allowing access to the device buffer if the search pointer corresponds to the device buffer, and allowing access to the CPU buffer if the search pointer corresponds to the CPU buffer.

[0010] The embodiments of the memory management device and memory management method provided herein address the challenges of managing heterogeneous memory access by introducing a unified method for allocating and resolving memory addresses. Simple Explanation of the Diagram

[0011] Figure 1A shows a flowchart of the steps of a memory management method 10 according to an embodiment of the present invention. Figure 1B is the corresponding system architecture diagram, illustrating the interaction between various components involved in memory management method 10. Figure 2 is a flowchart of an embodiment of the present invention, illustrating the detailed process of determining whether the search pointer corresponds to the device buffer or the CPU buffer during the address resolution phase. Figures 3A to 3C are schematic diagrams of embodiments according to the present invention, illustrating different cases of determining whether the search pointer corresponds to the device buffer or the CPU buffer. Figure 4 is a block diagram of a memory management device 40 according to an embodiment of the present invention. Figure 5 is a block diagram of a memory management device 50 according to another embodiment of the present invention. Implementation

[0012] The following description is for illustrative purposes only and should not be construed as limiting. The scope of the invention is best determined by referring to the appended claims.

[0013] In each of the following embodiments, the same reference numerals represent the same or similar elements or components.

[0014] The ordinal numbers used in the request items, such as “first,” “second,” “third,” etc., are for convenience of interpretation only and do not imply any priority relationship between them.

[0015] The descriptions provided below for embodiments of the apparatus or system also apply to embodiments of the method, and vice versa.

[0016] Figure 1A illustrates a flowchart of the steps of a memory management method 10 according to an embodiment of the present invention. As shown in Figure 1A, the memory management method 10 includes a memory allocation phase 11 and an address resolution phase 12. The memory allocation phase 11 includes steps S111-S113, while the address resolution phase 12 includes steps S121-S123. Figure 1B is a corresponding system architecture diagram illustrating the interaction between the various components involved in the memory management method 10. For a better understanding of this embodiment, reference should be made to both Figures 1A and 1B.

[0017] The memory management method 10 shown in Figure 1A is executed by the memory manager 100 shown in Figure 1B. The memory manager 100 further includes an allocator 101 and a resolver 102. The memory allocation stage 11 and the address resolution stage 12 are executed by the allocator 101 and the resolver 102, respectively.

[0018] In step S111, allocator 101 receives memory allocation request 104 from application 103. Memory allocation request 104 includes a specified size N, indicating the amount of memory required by application 103.

[0019] In step S112, in response to memory allocation request 104, allocator 101 allocates a buffer 110 of a specified size N in memory 105. Buffer 110 may be a device buffer 106 allocated for and accessible by one or more accelerator devices 150, including but not limited to graphics processing units (GPUs), neural processing units (NPUs), visual processing units (VPUs), DMA, and deep learning accelerators (DLAs). Alternatively, buffer 110 may be a CPU buffer 107 allocated for and accessible by CPU 160. In some embodiments, device buffer 106 may be a GPU buffer, VPU buffer, DMA buffer, DLA buffer, etc. More generally, the accelerator devices 150 in a computer system can be any device other than a CPU. In some embodiments, memory 105 is system memory, such as the random access memory (RAM) of a computer system, and device buffer 106 and CPU buffer 107 share space in the system memory. In some embodiments, memory 105 may include multiple hardware memories located in different hardware devices. For example, memory 105 may include system memory, a region of which may be allocated as CPU buffer 107 of the CPU. Memory 105 may also include video area random access memory (VRAM), and a region of the VRAM may be allocated as its GPU buffer (which is a device buffer 106).

[0020] In step S113, allocator 101 generates a buffer pointer 108P for buffer 110 and returns the buffer pointer 108P to application 103. The buffer pointer 108P serves as a uniform address pointer, helping application 103 access buffer 110 in a consistent manner. In some embodiments, the buffer pointer 108 indicates the start address of the allocated buffer 110. In some other embodiments, the buffer pointer 108 indicates the end address of the allocated buffer 110. In some other embodiments, the buffer pointer 108 indicates an offset address corresponding to a fixed location (e.g., start address, middle address, or end address) of the allocated buffer 110.

[0021] In step S121, parser 102 receives memory access request 109 from application 103. Memory access request 109 includes a search pointer P*, which indicates the memory address requested by application 103 for data access. For example, during neural network inference, search pointer P* may correspond to the memory location storing input feature maps, intermediate tensors, or model weights required for computation.

[0022] In step S122, the parser 102 determines whether the search pointer P* corresponds to the device buffer 106 or the CPU buffer 107 based on the buffer pointer 108 and the specified size N. Specifically, if the search pointer P* falls within the address range of the device buffer 106, then the search pointer P* corresponds to the device buffer 106. Otherwise, the search pointer P* corresponds to the CPU buffer 107.

[0023] In step S123, the parser 102 allows the application 103 to access the corresponding memory region. Specifically, if the search pointer P* corresponds to device buffer 106, access rights to device buffer 106 are granted; for example, accelerator device 150 or CPU 160 may be allowed to read or write data in device buffer 106 via application 103. If the search pointer P* corresponds to CPU buffer 107, access rights to CPU buffer 107 are granted; for example, CPU 160 or accelerator device 150 may be allowed to read data in CPU buffer 106 associated with CPU 160 or write data associated with CPU 160 to CPU buffer 107 via application 103.

[0024] It should be noted that although Figure 1B shows only a single device buffer 106, in actual implementation, multiple portions of memory 105 may be allocated as device buffers in response to different memory allocation requests. Each allocated device buffer is associated with a unique buffer pointer, allowing application 103 to reference and access different memory regions as needed. Therefore, memory manager 100 may maintain multiple buffer pointers, each corresponding to an allocated device buffer or CPU buffer, and a corresponding specified size of the buffer.

[0025] In one embodiment, memory allocation phase 11 further includes a mapping entry in the data structure between a buffer pointer to a device buffer, a device buffer, and a specified size associated with the device buffer. This data structure can be implemented as <key, value> pairs (<key, value> The data structure can be configured using pairs, lookup tables, linked lists, or tree-based structures, allowing for efficient searching and retrieval of buffer information, but this disclosure is not limited thereto. In some embodiments, the data structure may include multiple mapping entries, each mapping entry being a mapping between a buffer pointer to a device buffer, the device buffer, and a specified size associated with the device buffer. In one implementation, a mapping entry may be represented as...<key, value> Yes, where buffer pointer 108 (denoted as P) serves as the key, and the corresponding device buffer 106 (denoted as M) and the specified size N form the value, i.e.<key, value> Yes, it can be <P, <M, N>This mapping enables parser 102 to efficiently resolve memory access requests to data buffer 106.

[0026] Furthermore, step S122 utilizes this data structure to determine whether the search pointer corresponds to a device buffer or a CPU buffer. More detailed steps of this process will be described later with reference to Figure 2, which extends step S122 of Figure 1A.

[0027] Figure 2 is a flowchart according to an embodiment of the present invention, illustrating the detailed process of determining whether the search pointer corresponds to the device buffer or the CPU buffer during the address resolution phase. As shown in Figure 2, this process, namely step S122 of Figure 1A, further includes steps S201-S203.

[0028] In step S201, parser 102 checks whether the data structure contains at least one mapping entry. For example, parser 102 checks whether the size of the data structure is non-zero. As previously described, the data structure stores mapping entries between buffer pointers of device buffers, device buffers, and their corresponding specified sizes. Each mapping entry in the data structure may be a mapping between a buffer pointer of a device buffer, a device buffer (e.g., the ID of the device buffer), and a specified size of the device buffer. If the data structure does not contain any entries, meaning that no device buffer has been allocated, parser 102 determines that the search pointer corresponds to a CPU buffer and terminates the process. Otherwise, the process continues to step S202.

[0029] In step S202, parser 102 identifies from the data structure the largest buffer pointer that is not greater than the search pointer P* contained in memory access request 109. The identified buffer pointer serves as a reference for determining whether the search pointer falls within the device buffer allocation range. This step ensures that parser 102 selects the buffer pointer that is closest to and smaller than the search pointer P*.

[0030] In step S203, parser 102 determines whether the search pointer is within the range of the device buffer associated with the identified buffer pointer. Specifically, parser 102 checks whether the search pointer is less than or equal to the sum of the identified buffer pointer and the specified size associated with that buffer pointer. If this condition is met, the search pointer P* corresponds to the device buffer, and access is granted accordingly. Otherwise, parser 102 determines that the search pointer P* corresponds to the CPU buffer.

[0031] Figures 3A to 3C are schematic diagrams of embodiments according to the present invention, illustrating different cases of determining whether the search pointer corresponds to a device buffer or a CPU buffer. These three cases represent various scenarios for evaluating the search pointer based on the buffer pointer stored in the data structure and its associated allocation size.

[0032] Figure 3A illustrates the case where the search pointer directly points to the starting address of the allocated device buffer. In Figure 3A, the search pointer is 0x20, which perfectly matches the starting address of the device buffer (e.g., region A of memory 30) indicated by the identified maximum buffer pointer. The device buffer (e.g., region A) has a size of 16 bits. Since the search pointer is within the address range of region A (from 0x20 to 0x30), it is determined that the search pointer corresponds to the device buffer.

[0033] Figure 3B illustrates the situation where the search pointer falls within the allocated range of the device buffer. In Figure 3B, the search pointer is 0x28, which falls within the range of region A (0x20 to 0x30). For example, the parser identifies the largest buffer pointer, not greater than 0x28, as 0x20, and then checks if the search pointer is within the allocated size range of region A. Since 0x28 ≤ 0x20 + 16, it is determined that the search pointer corresponds to the device buffer.

[0034] Figure 3C illustrates a scenario where the search pointer falls outside the allocated device buffer region, resulting in a default mapping to the CPU buffer. In Figure 3C, the search pointer is 0x92, which is outside the range of any allocated device buffer. In this embodiment, the parser identifies the largest buffer pointer not greater than 0x92 as 0x70 (corresponding to region B). However, since 0x92 > 0x70 + 32, the search pointer does not correspond to any device buffer, therefore it is determined that the search pointer corresponds to the CPU buffer.

[0035] In one embodiment, the data structure described above (e.g.,<key, value> The buffer pointers are stored as a red-black tree, and a binary search is performed on the red-black tree to identify the largest buffer pointer not greater than the search pointer contained in the memory access request. A red-black tree is a self-balancing binary search tree that maintains a balanced structure by enforcing specific color-based properties during insertion and deletion operations. These properties ensure that the tree remains roughly balanced, keeping its height in O(log n), where n is the number of nodes in the tree. By using a red-black tree to store the mapping between buffer pointers and device buffers, the parser can efficiently identify the largest buffer pointer not greater than the search pointer through a binary search, which runs in O(log n) time complexity. The binary search works by repeatedly comparing the search pointer with the buffer pointers stored in the tree and navigating left or right based on the comparison results. This approach ensures that the lookup operation is performed in logarithmic time, significantly reducing the overhead of address resolution compared to unstructured or linear search methods.

[0036] In one embodiment, the address resolution phase also includes determining whether the memory access request was issued by a process authorized to access the device buffer or CPU buffer. If it is determined that the memory access request was issued by an unauthorized process, access to the device buffer or CPU buffer is denied. To ensure secure memory access, the resolver may maintain an Access Control List (ACL) or use a process authentication mechanism to verify whether the requesting process has the necessary permissions to access the device buffer or CPU buffer. This verification can be performed by checking the process identifier (PID), security token, or cryptographic authentication key associated with the memory access request. By incorporating access control into the address resolution phase, the system can prevent unauthorized access and reduce the risk of accidental memory corruption, data leakage, or malicious attacks. This security measure is particularly important in multi-process environments or shared computing platforms, where different applications or processes may request access to the same memory areas. Denying access to unauthorized processes helps maintain data integrity and ensures secure management of memory resources across multiple computing devices.

[0037] Figure 4 is a block diagram of a memory management device 40 according to an embodiment of the present invention. As shown in Figure 4, the memory management device 40 includes a processor 41 and a non-transitory computer-readable storage 42. The non-transitory computer-readable storage 42 may include flash memory, a solid-state drive (SSD), a hard disk drive (HDD), or other storage media, but the present invention is not limited thereto. The non-transitory computer-readable storage 42 stores a memory management program 400, which can be implemented using various programming languages, including but not limited to C, C++, Python, Java, or assembly languages, and can be executed as part of an operating system, device driver, or application-level library. The memory management program 400 corresponds to the memory manager 100 in Figure 1B, and it includes instructions that, when executed by the processor, cause the processor 41 to execute the allocator module 401 and the parser module 402. Allocator module 401 corresponds to allocator 101 in Figure 1B. It is responsible for processing memory allocation requests to allocate buffers (including device buffers and CPU buffers), generating buffer pointers, and returning the buffer pointers to the application. Parser module 402 corresponds to parser 102 in Figure 1B. It is responsible for processing memory access requests and determining whether a given search pointer corresponds to a device buffer or a CPU buffer.

[0038] Figure 5 is a block diagram of a memory management device 50 according to another embodiment of the present invention. The memory management device 50 is implemented in hardware circuitry, including an allocator circuitry 501 and a resolver circuitry 502. These circuits can be implemented using dedicated hardware logic, such as Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or System-on-Chip (SoC) designs, to optimize memory allocation and address resolution operations. The allocator circuitry 501 is configured to process memory allocation requests to allocate buffers (including device buffers and CPU buffers), generate buffer pointers, and return the buffer pointers to the application, corresponding to the allocator 101 in Figure 1B. The resolver circuitry 502 is responsible for processing memory access requests and determining whether a given search pointer corresponds to a device buffer or a CPU buffer, corresponding to the resolver 102 in Figure 1B.

[0039] The embodiments of the memory management device and method provided herein address the challenges of managing heterogeneous memory access by introducing a unified approach for allocating and resolving memory addresses. By efficiently handling memory allocation requests and storing the mapping between buffer pointers and buffers, the technology of this invention simplifies memory access. Using structured data organization, such as red-black trees, fast address resolution is achieved through binary search, reducing lookup latency and improving scalability. Compared to conventional methods that require device-specific memory access mechanisms, the embodiments of this invention improve software portability, reduce programming complexity, and optimize memory access efficiency in heterogeneous computing environments.

[0040] The preceding paragraphs describe several aspects. Clearly, based on the instructions, this can be done in multiple ways. Any particular structure or function disclosed in the examples represents only one case. As instructed in the instructions, those skilled in the art should note that any disclosed aspect can be performed individually, or two or more aspects can be performed in combination.

[0041] While the invention has been described by way of example and preferred embodiments, it should be understood that the invention is not limited to the disclosed embodiments. Rather, it is intended to cover various modifications and similar arrangements (which will be apparent to those skilled in the art). Therefore, the appended claims should be given the broadest interpretation to cover all such modifications and similar arrangements.

[0042] 10: Memory Management Methods 11: Memory allocation stage 12: Address Resolution Phase S111, S112, S113, S121, S122, S123, S201, S202, S203: Steps 103: Application 104: Memory allocation request 108: Buffer pointer 109: Memory access request 100: Memory Manager 101: Distributor 102: Parser 105,30: Memory 110: Buffer 106: Device Buffer 107: CPU Buffer 150: Accelerator device 160: CPU 40, 50: Memory management device 41: Processor 42: Non-transitory computer-readable storage 400: Memory Management Program 401: Distributor Module 402: Parser Module 501: Distributor Circuit 502: Resolver Circuit

Claims

1. A memory management device, comprising: processor; and a non-transitory computer-readable storage coupled to the processor, storing a memory management program including instructions that, when executed by the processor, cause the processor to execute an allocator module and a parser module; wherein executing the allocator module causes the processor to: receive a memory allocation request from an application, wherein the memory allocation request includes a specified size; allocate a buffer of the specified size in memory according to the memory allocation request; generate a buffer pointer for the buffer; and return the buffer pointer to the application, wherein the buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for the CPU; and wherein executing the parser module causes the processor to: receive a memory access request from the application, wherein the memory access request includes a search pointer; determine, based on the buffer pointer and the specified size, whether the search pointer corresponds to the device buffer or the CPU buffer; and if the search pointer corresponds to the device buffer, allow access to the device buffer; and if the search pointer corresponds to the CPU buffer, allow access to the CPU buffer.

2. The memory management apparatus of claim 1, wherein executing the allocator module further causes the processor to store in a data structure a plurality of buffer pointers of a plurality of device buffers, the plurality of device buffers, and a plurality of specified sizes associated with the plurality of device buffers; and wherein executing the parser module further causes the processor to: check whether the data structure contains at least one mapping entry; if the data structure does not contain any mapping entries, determine that the search pointer corresponds to the CPU buffer; if the data structure contains at least one mapping entry, identify from the data structure the largest buffer pointer not greater than the search pointer contained in the memory access request, and if the search pointer is not greater than the sum of the identified buffer pointer and the specified size associated with the identified buffer pointer, determine that the search pointer corresponds to the device buffer; otherwise, determine that the search pointer corresponds to the CPU buffer.

3. The memory management apparatus as claimed in claim 2, wherein the data structure is constructed as a red-black tree, and the parser module further enables the processor to perform a binary search on the red-black tree to identify the largest buffer pointer not greater than the search pointer contained in the memory access request.

4. The memory management apparatus as claimed in claim 1, wherein executing the parser module further enables the processor to determine whether the memory access request is issued by a process authorized to access the buffer, and if it is determined that the memory access request is issued by an unauthorized process, then access to the buffer is denied.

5. The memory management device as claimed in claim 1, wherein the buffer pointer indicates the start address of the buffer, the end address of the buffer, or an offset address corresponding to a fixed position of the buffer; and the search pointer indicates a memory address requested by the application.

6. The memory management device as claimed in claim 1, wherein the device buffer includes a graphics processing unit (GPU) buffer, a neural processing unit (NPU) buffer, a visual processing unit (VPU) buffer, a direct memory access (DMA) buffer, or a deep learning accelerator (DLA) buffer; and each of the one or more accelerator devices includes a GPU, an NPU, a VPU, a DMA, or a DLA.

7. A memory management device, comprising: An allocator circuit is configured to: receive a memory allocation request from an application, wherein the memory allocation request includes a specified size; allocate a buffer of the specified size in memory according to the memory allocation request; generate a buffer pointer for the buffer; and return the buffer pointer to the application, wherein the buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for a CPU; and a resolver circuit is configured to: receive a memory access request from the application, wherein the memory access request includes a search pointer; determine, based on the buffer pointer and the specified size, whether the search pointer corresponds to the device buffer or the CPU buffer; and if the search pointer corresponds to the device buffer, allow access to the device buffer; and if the search pointer corresponds to the CPU buffer, allow access to the CPU buffer.

8. The memory management apparatus of claim 7, wherein the allocator circuitry is further configured to store in a data structure a plurality of buffer pointers of a plurality of device buffers, the plurality of device buffers, and a plurality of mapping entry buffer pointers between specified sizes associated with the plurality of device buffers; and wherein the parser circuitry is further configured to: check whether the data structure contains at least one mapping entry; if the data structure does not contain any mapping entries, determine that the search pointer corresponds to the CPU buffer; if the data structure contains at least one mapping entry, identify from the data structure the largest buffer pointer not greater than the search pointer contained in the memory access request, and if the search pointer is not greater than the sum of the identified buffer pointer and the specified size associated with the identified buffer pointer, determine that the search pointer corresponds to the device buffer; otherwise, determine that the search pointer corresponds to the CPU buffer pointer.

9. The memory management apparatus of claim 8, wherein the data structure is constructed as a red-black tree, and the parser circuitry is further configured to perform a binary search on the red-black tree to identify the largest buffer pointer not greater than the search pointer contained in the memory access request.

10. The memory management apparatus as claimed in claim 7, wherein the parser circuitry is further configured to determine whether the memory access request is issued by a process authorized to access the buffer, and if it is determined that the memory access request is issued by an unauthorized process, then deny access to the buffer.

11. The memory management device as claimed in claim 7, wherein the buffer pointer indicates the start address of the buffer, the end address of the buffer, or an offset address corresponding to a fixed position of the buffer; and the search pointer indicates a memory address buffer pointer requested by the application.

12. The memory management device as claimed in claim 7, wherein the device buffer includes a graphics processing unit (GPU) buffer, a neural processing unit (NPU) buffer, a visual processing unit (VPU) buffer, a direct memory access (DMA) buffer, or a deep learning accelerator (DLA) buffer; and each of the one or more accelerator devices includes a GPU, an NPU, a VPU, a DMA, or a DLA.

13. A memory management method applied to a computer system executing an application, the method comprising: The memory allocation phase includes: receiving a memory allocation request from the application, wherein the memory allocation request includes a specified size; allocating a buffer of the specified size in memory according to the memory allocation request; generating a buffer pointer for the buffer; and returning the buffer pointer to the application; wherein the device buffer is a device buffer allocated for one or more accelerator devices, or a CPU buffer allocated for the CPU; and the address resolution phase includes: receiving a memory access request from the application, wherein the memory access request includes a search pointer; determining whether the search pointer corresponds to the device buffer or the CPU buffer based on the buffer pointer and the specified size; and allowing access to the device buffer if the search pointer corresponds to the device buffer, and allowing access to the CPU buffer if the search pointer corresponds to the CPU buffer.

14. The memory management method as claimed in claim 13, wherein the memory allocation phase further includes storing in a data structure a plurality of buffer pointers to a plurality of device buffers, the plurality of device buffers, and a plurality of mapping entries between specified sizes associated with the plurality of device buffers; wherein determining whether the search pointer corresponds to the device buffer or the CPU buffer in the address resolution phase further includes: Check whether the data structure contains at least one mapping entry. If the data structure does not contain any mapping entries, determine that the search pointer corresponds to the CPU buffer. If the data structure contains at least one mapping entry, identify the largest buffer pointer in the data structure that is not greater than the search pointer contained in the memory access request. If the search pointer is not greater than the sum of the identified buffer pointer and the specified size associated with the identified buffer pointer, determine that the search pointer corresponds to the device buffer. Otherwise, determine that the search pointer corresponds to the CPU buffer.

15. The memory management method as described in claim 14, wherein the data structure is constructed as a red-black tree, and a binary search is performed on the red-black tree to identify the largest buffer pointer that is not greater than the search pointer contained in the memory access request.

16. The memory management method as described in claim 13, wherein the address resolution phase further includes determining whether the memory access request is issued by a process authorized to access the buffer, and if it is determined that the memory access request is issued by an unauthorized process, then access to the buffer is denied.

17. The memory management method as claimed in claim 13, wherein the buffer pointer indicates the start address of the buffer, the end address of the buffer, or an offset address corresponding to a fixed position of the buffer; and the search pointer indicates a memory address requested by the application.

18. The memory management method of claim 13, wherein the device buffer includes a graphics processing unit (GPU) buffer, a neural processing unit (NPU) buffer, a visual processing unit (VPU) buffer, a direct memory access (DMA) buffer, or a deep learning accelerator (DLA) buffer; and each of the one or more accelerator devices includes a GPU, an NPU, a VPU, a DMA, or a DLA.

Images