Method and device for memory management

Abstract

A method for memory management includes: receiving a request, the request being used to request a memory allocation or memory de-allocation; maintaining a hierarchical bitmap tree, the hierarchical bitmap tree having one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool; and determining blocks corresponding to the request based on the hierarchical bitmap tree. Therefore, the hierarchical bitmap tree having one bit allows directly skipping to the desired bitmap level, which may simplify the memory access mode and have high flexibility and configurability to improve traversal performance. Each block uses only one bit. Therefore, it may be further used in other optimization methods such as the effective application of bit bucket.

Description

METHOD AND DEVICE FOR MEMORY MANAGEMENTTECHNICAL FIELD

[0001] The present disclosure relates to the field of computing technologies, and in particular, to a method for memory management, and a memory management device.BACKGROUND

[0002] Fast and efficient dynamic memory allocation algorithms are one of the most important problems in computer industry. Various memory allocators are the program components which are used in every program or a program part. Their efficiency and performance are the key factors often entirely determining the system behavior.

[0003] However, the existing method for memory management need to use multiple bits for memory access and some kind of pointer-based memory structures, which usually exhibits poor locality and non-deterministic memory access pattern, and is fairly complex to implement and required a significant amount of code; and multiple bits hinder effective use of bit buckets hinder the effective use of the bit bucket.SUMMARY

[0004] The present disclosure provides a method for memory management and a memory management device to solve the problem of poor locality and non-deterministic memory access pattern and the problem that multiple bits hinder effective use of bit buckets hinder the effective use of the bit bucket.

[0005] The following technical solutions are adopted in the present disclosure.

[0006] In a first aspect, a method for memory management is provided. The method includes: receiving a request, the request being used to request a memory allocation or memory de-allocation; maintaining a hierarchical bitmap tree, the hierarchical bitmap tree having one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool; and determining blocks corresponding to the request based on the hierarchical bitmap tree.

[0007] In the embodiments of the present disclosure, the hierarchical bitmap tree having one bit allows directly skipping to the desired bitmap level, which may simplify the memory access mode and have high flexibility and configurability to improve traversal performance. Each block uses only one bit. Therefore, it may be further used in other optimization methods such as the effective application of bit bucket.

[0008] In a possible implementation, maintaining the hierarchical bitmap tree includes organizing the hierarchical bitmap tree as a hierarchical array of subtrees, where each subtree is represented by a bit bucket whose size is equal to a machine register; the hierarchical bitmap tree includes N layers of subtrees where the memory pool is a root bit bucket; each subtree corresponds to one bit bucket; an Nth layer of subtrees includes at least a part of an (N-l)-th layer of subtrees; a block for each subtree corresponds to one bit that controls a status of the block for each subtree; and the one bit is used to indicate statuses of blocks of the Nth layer of subtrees and statuses of blocks of the (N-l)-th layer of subtrees, N being an integer greater than or equal to 2.

[0009] In the method for memory management provided in the embodiments of the present disclosure, all the allocations, despite of their block size logarithm, are rounded up to the minimum allocation granularity. If all the requested allocation sizes happen to be already multiples of the minimum granularity, then the allocated memory waste level will be zero. Thus, the minimum allocated memory waste may be realized.

[0010] Moreover, the method for memory management provided in the embodiments of the present disclosure has the number of layers of bit buckets and the minimum allocation granularity that are required and fully determined during the traversal. The minimum allocation granularity is a lower block size limit, which indicates a low border of a size range. The number of layers of bit buckets is equal to available logarithmic block size steps. In the embodiments of the present disclosure, by changing the above two parameters (i.e., the number of layers of bit buckets and the minimum allocation granularity), the method for memory management may handle any reasonable block size range, thus being easily adaptable to various application fields.

[0011] In a possible implementation, after organizing the hierarchical bitmap tree as the hierarchical array of subtrees, the method further includes: organizing the hierarchical bitmap tree based on a plurality of occupancy bitmaps which is in one-to-one correspondence with a plurality of bit buckets; a size of an occupancy bitmap corresponding to each bit bucket fits the size of the register, and each occupancy bitmap is arranged in a corresponding bit bucket; an occupancy bitmap corresponding to each bit bucket in a current layer is controlled by occupancy bitmaps of bit buckets in an upper layer, and occupancy bitmaps of bit buckets corresponding to all subtrees are controlled by an occupancy bitmap of the root bit bucket.

[0012] In the embodiments of the present disclosure, based on the hierarchical bitmap tree, the reallocation is fast, the coalescing and reuse of the de-allocated memory is improved, and the method for memory management does not need any postponed garbage collection. Therefore, the method formemory management in the embodiments of the present disclosure may provide immediate and automatic coalescing and memory reuse during the de-allocation call, without any additional performance penalty, which may automatically reduce the allocated memory fragmentation and increase the re-allocation speed.

[0013] In a possible implementation, organizing the hierarchical bitmap tree as the hierarchical array of subtrees includes: obtaining a number of layers of bit buckets and a minimum allocation granularity which are set by a user, the number of layers of bit buckets being equal to a number of layers of subtrees; and organizing bit buckets into the hierarchical bitmap tree based on the number of layers of bit buckets and the minimum allocation granularity.

[0014] In a possible implementation, determining blocks corresponding to the request based on the hierarchical bitmap tree, includes: obtaining a mapped size by dividing a requested block size by the minimum allocation granularity, wherein the mapped size is used to indicate a mapping of a size of the requested memory allocation or memory de-allocation to a preset block size; determining a tree level based on the mapped size, wherein the tree level is used to indicate a location in the hierarchical bitmap tree for the free bit search operation; obtaining a bit bucket bit level and a bit bucket layer level by dividing the tree level by the number of tree levels controlled by a single bit bucket; starting from a root bit bucket in the bit bucket layer level, executing the free bit search operation by performing a traversal loop operation on an array of bit buckets corresponding to the root bit bucket, until free bit buckets that satisfy the bit bucket bit level are determined; projecting down a bucket offset and a bit offset corresponding to the free bit buckets to a lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level; determining an occupancy bit mask based on a projected offset and the mapped size as well as the minimum allocation granularity; performing the free bit update operation by enabling an occupancy status or a free status of the occupancy bit mask to be propagated up along a propagation path of the occupancy bit mask; and determining blocks corresponding to updated free bit buckets as the blocks corresponding to the request.

[0015] The method for memory management in the embodiments of the present disclosure fully takes advantage of CPU features, such as performance gain from cache locality, large set of 64-bit registers and their superior performance, bit search and bit count instructions which are able to execute in a single cycle. Most operations during allocator calls are performed inside CPU registers, and only very limited number of memory accesses are needed.

[0016] In a possible implementation, the free bit search operation includes: determining, based on asub-allocation mask, a first bit bucket corresponding to a first free bit in the sub-allocation mask; in a case where a bit level of the first bit bucket satisfies a bit bucket bit level of a block among the blocks corresponding to the request, determining a block corresponding to the first bit bucket as the block among the blocks corresponding to the request; and in a case where the bit level of the first bit bucket does not satisfy the bit bucket bit level of the block among the blocks corresponding to the request, performing a recursive search to search a second bit bucket corresponding to a second free bit in the sub-allocation mask, the second free bit being a free bit next to the first free bit in the sub-allocation mask. The sub-allocation mask is used to indicate statuses of blocks of the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket, and a size of the sub-allocation mask is half of the size of the register; layers except the lowest layer in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level are searched using the sub-allocation mask; and the bit buckets at the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket are searched using the occupancy bitmaps.

[0017] In a possible implementation, performing the free bit update operation by enabling the occupancy status or the free status of the occupancy bit mask to be propagated up along the propagation path of the occupancy bit mask, includes: in a case of requesting the memory allocation, enabling the occupancy status to be propagated up along the propagation path based on the occupancy bit mask; when a bit mask size corresponding to the occupancy bit mask is greater than a single bit size, shifting up the occupancy status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the occupancy status is a free bit value, enabling the occupancy status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

[0018] In a possible implementation, performing the free bit update operation by enabling the occupancy status or the free status of the occupancy bit mask to be propagated up along the propagation path of the occupancy bit mask, includes: in a case of requesting the memory deallocation, enabling the free status to be propagated up along the propagation path based on the occupancy bit mask; when the bit mask size corresponding to the occupancy bit mask is greater than the single bit size, shifting up the free status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the free status is an occupied bit value, enabling the free status to be propagated up along the propagationpath until the buddy bit value of the occupied bit is in the occupancy status.

[0019] In the embodiments of the present disclosure, the free bit update is performed using the fast (single cycle) bit shift and binary logic instructions repeatedly over the same CPU register, and just a single memory read and a single write are required. In addition, due to the compact storage of the array of bit buckets, the neighbor sub-trees (bit buckets) are likely to fall into the same cache line, which may achieve a superior cache locality.

[0020] Therefore, the layout of bit buckets has a compact size, and any bit bucket localizes all the sub-children of its parent node for several layers deep at once, which provides the opportunity for some important code simplifications and optimizations.

[0021] In a second aspect, a memory management device is provided. The memory management device is applied to a computer system or to a computing device that supports the computer system to implement the method for memory management. The memory management device includes units for performing the method for memory management in the first aspect or any implementation of the first aspect. The device includes: a receiving unit, used for receiving a request, the request being used to request a memory allocation or memory de-allocation; a maintaining unit, used for maintaining a hierarchical bitmap tree, wherein the hierarchical bitmap tree has one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool; and a determining unit, used for determining blocks corresponding to the request based on the hierarchical bitmap tree.

[0022] In a possible implementation, the maintaining unit is further used for organizing the hierarchical bitmap tree as a hierarchical array of subtrees, where each subtree is represented by a bit bucket whose size is equal to a machine register; the hierarchical bitmap tree includes N layers of subtrees where the memory pool is a root bit bucket; each subtree corresponds to one bit bucket; an Nth layer of subtrees includes at least a part of an (N-l)-th layer of subtrees; a block for each subtree corresponds to one bit that controls a status of the block for each subtree; and the one bit is used to indicate statuses of blocks of the Nth layer of subtrees and statuses of blocks of the (N-l)-th layer of subtrees, N being an integer greater than or equal to 2.

[0023] In a possible implementation, the device further includes an organizing unit, used for organizing the hierarchical bitmap tree based on a plurality of occupancy bitmaps which is in one-to- one correspondence with a plurality of bit buckets; a size of an occupancy bitmap corresponding to each bit bucket fits the size of the register, and each occupancy bitmap is arranged in a corresponding bit bucket; an occupancy bitmap corresponding to each bit bucket in a current layer is controlled byoccupancy bitmaps of bit buckets in an upper layer, and occupancy bitmaps of bit buckets corresponding to all subtrees are controlled by an occupancy bitmap of the root bit bucket.

[0024] In a possible implementation, the maintaining unit is further used for: obtaining a number of layers of bit buckets and a minimum allocation granularity which are set by a user, wherein the number of layers of bit buckets is equal to a number of layers of subtrees; and organizing bit buckets into the hierarchical bitmap tree based on the number of layers of bit buckets and the minimum allocation granularity.

[0025] In a possible implementation, the maintaining unit is further used for: obtaining a mapped size by dividing a requested block size by the minimum allocation granularity, wherein the mapped size is used to indicate a mapping of a size of the requested memory allocation or memory deallocation to a preset block size; determining a tree level based on the mapped size, wherein the tree level is used to indicate a location in the hierarchical bitmap tree for the free bit search operation; obtaining a bit bucket bit level and a bit bucket layer level by dividing the tree level by the number of tree levels controlled by a single bit bucket; starting from a root bit bucket in the bit bucket layer level, executing the free bit search operation by performing a traversal loop operation on an array of bit buckets corresponding to the root bit bucket, until free bit buckets that satisfy the bit bucket bit level are determined; projecting down a bucket offset and a bit offset corresponding to the free bit buckets to a lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level; determining an occupancy bit mask based on a projected offset and the mapped size as well as the minimum allocation granularity; performing the free bit update operation by enabling an occupancy status or a free status of the occupancy bit mask to be propagated up along a propagation path of the occupancy bit mask; and determining blocks corresponding to updated free bit buckets as the blocks corresponding to the request.

[0026] In a possible implementation, the free bit search operation includes: determining, based on a sub-allocation mask, a first bit bucket corresponding to a first free bit in the sub-allocation mask; in a case where a bit level of the first bit bucket satisfies a bit bucket bit level of a block among the blocks corresponding to the request, determining a block corresponding to the first bit bucket as the block among the blocks corresponding to the request; and in a case where the bit level of the first bit bucket does not satisfy the bit bucket bit level of the block among the blocks corresponding to the request, performing a recursive search to search a second bit bucket corresponding to a second free bit in the sub-allocation mask, the second free bit being a free bit next to the first free bit in the sub-allocation mask. The sub-allocation mask is used to indicate statuses of blocks of the lowest layer of bit bucketsin the array of bit buckets corresponding to the root bit bucket, and a size of the sub-allocation mask is half of the size of the register; layers except the lowest layer in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level are searched using the sub-allocation mask; and the bit buckets at the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket are searched using the occupancy bitmaps.

[0027] In a possible implementation, the maintaining unit is further used for: in a case of requesting the memory allocation, enabling the occupancy status to be propagated up along the propagation path based on the occupancy bit mask, wherein when a bit mask size corresponding to the occupancy bit mask is greater than a single bit size, shifting up the occupancy status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the occupancy status is a free bit value, enabling the occupancy status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

[0028] In a possible implementation, the maintaining unit is further used for: in a case of requesting the memory de-allocation, enabling the free status to be propagated up along the propagation path based on the occupancy bit mask, wherein when the bit mask size corresponding to the occupancy bit mask is greater than the single bit size, shifting up the free status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the free status is an occupied bit value, enabling the free status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

[0029] In a third aspect, a computing device cluster is provided. The computing device cluster includes at least one computing device each including a processor and a memory. The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, to cause the computing device cluster to perform the method in the first aspect or any possible implementation in the first aspect.

[0030] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program or instructions that, when executed by a computing device, cause the computing device to implement the method in the first aspect or any possible implementation of the first aspect.

[0031] In a fifth aspect, a computer program product is provided. The computer program productincludes a computer program or instructions that, when executed by a computing device, cause the computing device to implement the method in the first aspect or any possible implementation of the first aspect.

[0032] Beneficial effects of the above second to fifth aspects can be achieved with reference to either the first aspect or any possible implementation of the first aspects, which will not be repeated here. On the basis of the implementations provided above, the present disclosure can be further combined to provide more implementations.BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG. 1 is a schematic diagram showing a scenario of a buddy allocator;

[0034] FIG. 2 is a schematic diagram showing a structure of a slab allocator;

[0035] FIG. 3 is a schematic diagram showing a hierarchical bitmap having a two-bit entry;

[0036] FIG. 4 is a schematic flow diagram showing a method for memory management, in accordance with embodiments of the present disclosure;

[0037] FIG. 5 is a schematic flow diagram showing another method for memory management, in accordance with embodiments of the present disclosure;

[0038] FIG. 6 is a schematic diagram showing a structure of a hierarchical bitmap tree, in accordance with embodiments of the present disclosure;

[0039] FIG. 7 is a schematic diagram showing an occupation bitmap and a sub-allocation mask, in accordance with embodiment of the present disclosure;

[0040] FIG. 8 is a schematic diagram showing a process of searching a free bit bucket node, in accordance with embodiment of the present disclosure;

[0041] FIG. 9 is a schematic diagram showing a scenario for determining an occupancy bit mask, in accordance with embodiment of the present disclosure;

[0042] FIG. 10 is a schematic diagram of a GPU memory manager to implement a method for memory management in embodiments of the present disclosure;

[0043] FIG. 11 is a schematic diagram showing a layout of an array of bit buckets, in accordance with embodiment of the present disclosure;

[0044] FIG. 12 is a diagram showing a structure of a memory management device, in accordance with embodiment of the present disclosure;

[0045] FIG. 13 is a schematic diagram showing a structure of a computer device, in accordance with embodiments of the present disclosure; and

[0046] FIG. 14 is a schematic diagram showing a structure of a computer device cluster, in accordance with embodiments of the present disclosure.DETAILED DESCRIPTION

[0047] For convenience of understanding, some of technical terms involved in the present disclosure are firstly introduced.

[0048] Cache, refers to a limited amount of a very fast memory that is located directly on a central processing unit (CPU) or graphics processing unit (GPU) chip, and is mapped to the certain parts of the main memory, which have been used lately by the program currently running on the chip. For example, the cache is one or two orders of magnitude faster than the main system random access memory (RAM). The cache serves subsequent requests to the same data much faster than the main memory. A cache hit occurs when the requested data has been found in the cache. A cache miss occurs when the requested data has not been found in the cache, which leads to expensive cache re-mapping and update request into the main RAM, causing the processor execution until either to stall or switch to another execution context. The cache might have several hierarchically organized levels, which are different in size and speed.

[0049] Cache line, refers to the minimum granularity for any operation involving the cache on a chip. The minimum granularity refers to the engaged memory block size. Generally, the size of the cache line is 64 bytes or higher. The starting address of the cache line is aligned to the size of the cache line.

[0050] Bitmap, refers to a compactly stored continuous array of Boolean values (bits).

[0051] Hierarchical bitmap, refers to a one-to-one mapping between some hierarchical task-specific structures (e.g., balanced binary trees) and dedicated-type bitmaps. A logical state of each node in a hierarchical task-specific structure is represented by one or more bits located at an accurately determined offset inside a bitmap. Node bit states usually resemble a hierarchy of the structure, that is, a bit state of a parent node is dependent on the states of the child nodes (e.g., as a result of some Boolean logic function over child node bits, for example, bitwise OR or AND, or a combination of such logic functions).

[0052] Bit bucket, refers a part of a hierarchical bitmap organized in such a way that bits representing several adjacent sub-hierarchical levels are located continuously inside a single memory word. The memory word is treated by a computer program as a single variable of the certain number of bits. For example, up to six adjacent sub-layers of a single bit binary tree can be organized as a bit bucket which fits into a single 64-bit register (the most significant bit represent their children in turn, and so on). Generally, one or two bits of a register or memory variable remain unoccupied by a bucket, and canbe used for some service purposes. Bit bucket concept is particularly used in the cases when multiple reads and / or modifications are performed constantly and repeatedly at the same or nearby offsets inside a hierarchical bitmap, and such operations should be effectively mapped to a modem CPU architecture (e.g., operating on 64-bit registers and a limited amount of cache lines).

[0053] Dynamic memory allocation, refers to a process of obtaining one or more memory blocks from the operating system (OS), driver, or programming language library, occurring on demand at the arbitrary moment of time during the program run time. The allocation procedure accepts the required block size, and returns either a block address which can be used immediately or a block handle which can be passed to a subsequent OS or driver calls that use the allocated memory block.

[0054] Dynamic memory de-allocation, refers to a process reverse to the dynamic memory allocation. The de-allocation procedure accepts the block address or block handle previously returned by the allocation procedure, and performs actions necessary to return the block to the available memory pool so that the block can be reused by further allocations.

[0055] Memory coalescing, refers to the process of merging two or more neighbor free memory blocks into a larger memory block during the de-allocation procedure call, which can be further treated and allocated as a whole.

[0056] Dynamic memory re-allocation, refers to a process of changing (e.g., shrinking or enlarging) the size of previously allocated memory block, which can be performed in-place (keeping the allocated block address the same), or using the subsequent de-allocation / allocation pair of calls.

[0057] Memory allocator, a software (or hardware) component which implements at least the memory allocation and de-allocation calls. Although re-allocation can be replaced by the de- allocation / allocation pair of calls described above, the memory allocator is currently considered a standard allocator function.

[0058] Internal allocator memory, or service allocator memory, refers to the amount of RAM which is required by the memory allocator to store its service information (such as, the internal data structures, the status of the allocated memory blocks, the availability of blocks and so on). For example, the internal allocator memory or service allocator memory is allocated directly from the OS and invisible to the allocator clients, so it can be considered as a memory overhead of the allocator implementation.

[0059] Physical memory granularity, refers to a size of a minimum physical memory allocation unit (measured by bytes) supported by a hardware component (CPU, GPU, memory controller, etc.) of a computer system. The concept of the physical memory granularity is closely related to the physical memory page. Basically, the physical memory granularity and physical memory page have the samemeaning, since the lowest level memory allocation occurs in pages.

[0060] Fast and efficient dynamic memory allocation algorithms are one of the most important problems in the computer industry. Various memory allocators are the program components used in every program or a program part, for example, everywhere. The efficiency and performance of the memory allocators are key factors often entirely determining the performance (or response) of the system.

[0061] For example, current general-purpose memory allocators are mostly based on a very small number of classic algorithms. Thus, the set of the algorithmic building block options available for a custom memory allocator is quite limited

[0062] A buddy allocator is one of the classic basic memory allocators. FIG. 1 shows a scenario of a buddy allocator, which is a dynamically grown / shrunk binary tree of memory blocks. As shown in FIG. 1, each block may be split to form two blocks of a next tree level, and each block (except for a root block) may be coalesced with its buddy block into a block of the upper level (which is twice larger in size). The allocation starts from the root block whose size is the upper limit of possible allocation size. The root block is recursively split in half, until either of two stopping conditions is met. One stopping condition is that the binary logarithm of the resulting block size matches the binary logarithm of the block size requested by the user; and the other stopping condition is that the size of the resulting block reaches a predefined threshold, which is a lower limit of the possible allocation size.

[0063] A slab allocator is another one of the classic basic memory allocators. FIG. 2 shows a structure of a slab allocator. As shown in FIG. 2, in the slab allocator, the memory block cache is implemented as a series of "slabs". The "slabs" refer to hierarchically organized memory block levels of the same certain size. The block size gradually decreases from the top to the bottom of the hierarchy, as shown in FIG. 2. The block sizes from the top to the bottom of the hierarchy are 128 KB, 64 KB, and 32 KB. Each slab has one list for free and one list for allocated blocks. Allocation and deallocation calls move blocks between the two lists. Generally, a slab on a lower level is sub-allocated from the block of one of the upper levels on demand, but in some cases all or some slabs may be statically pre-allocated.

[0064] When the allocator manages a large set of blocks of the same size (for example, as it occurs in some implementations of C++ standard template library (STL) object allocators), the block status may be tracked using a single bit of the plain bitmap located in an internal allocator service memory.

[0065] Most State Of The Art (SOTA) allocators are implemented as the above building blocks.

[0066] In Example 1, the SOTA allocator is a popular open source jeMalloc allocator. The address space of the SOTA allocator is divided into multiple arenas, which are randomly allocated to user threads. Each arena is split into page chunks (up to single page) using a buddy system. Large and huge allocations (larger than a memory page) are handled immediately at a page size, and small allocations are handled by further subdividing page chunks into size class runs. Each class run contains blocks of only one size, which are controlled by simple plain bitmaps.

[0067] In Example 1, the jeMalloc allocator has good multithreaded performance, good cache locality for neighbor allocations, and fair memory waste level because of large supported block sizes range. At the same time, on the upper levels, chunk usage efficiency is moderate due to the large chunk granularity, which increases the memory waste.

[0068] In Example 2, the SOTA allocator is MiMalloc, which is a hybrid slab allocator but is highly optimized for allocating many small objects in a multithreaded environment. Originally, MiMalloc was developed for managing a swarm of tiny objects for internal allocations of an interpreting language engine (such as Python runtime), but later was employed as a general-purpose allocator in many other projects. MiMalloc manages a set of thread-local page directories which are divided into 4 Mb segments, from which, in turn, single pages are allocated that contain blocks of a certain size range only. The long slab control lists are distributed into much smaller ones, which are put directly into the memory pages. Any concurrent list operations are performed using atomics. Actual block free actions are delayed for as long as possible, by grouping them into additional lists for future batch processing when the allocator runs out of the free blocks. MiMalloc has a small code size, good multithreaded performance, and neighbor blocks cache locality. The control list distribution allows to somewhat limit response time and service memory costs. However, the design of MiMalloc tends to use small objects only, and the performance and memory waste numbers on moderate and large blocks, and in the case of a highly varying distribution of the block sizes, are unclear.

[0069] Regarding Example 1 and Example 2, due to rounding up the actually allocated block size either to the closest binary logarithm or to the granularity of the selected slab, the nature of a buddy or slab allocator algorithm has a certain level of the allocated memory waste inherent. The rounding waste can be mitigated by a clever block size selection, but cannot be completely avoided. The need to mitigate the waste often leads to applying various algorithmic tricks which highly increase implementation complexity, often being error-prone, non-obvious to implement and hard to debug. In addition, it is difficult to make the above algorithmic tricks universal, which may be fully applicable in some cases, but may lead to poor performance in others.

[0070] Moreover, most SOTA allocators are implemented using some sort of pointer-based memory structure, such as trees, or sets of the doubly-linked lists, or a combination of them. In addition to the case that pointers consume a large amount of memory (for example, each pointer consumes 8 bytes on modem 64-bit architectures), this structure usually exhibits poor locality and uncertain memory access pattern. For example, during the process of multiple allocations and de-allocations, the items of slab lists tend to be randomly moved between the lists several times. During this process, list item pointers start pointing to more and more random locations, rather than to the neighbor memory locations, resulting in poor address locality and the inability to determine which location in the internal allocator memory the next pointer points to. Thus, the traversal process is somewhat chaotic, which limits the allocator performance on modem CPUs; and the implementation is quite complex and a large amount of code is required.

[0071] Another well-known undesired consequence of using memory structures of such a type is unavoidable and severe performance penalty to freed block coalescing, which has to be performed during freeing and re-allocation calls. Since coalescing algorithms often need to travel a very long and highly branched pointer or bit sequences, typically SOTA allocators, instead of immediate deallocation operation execution, tend to postpone them for as long as possible (often, until allocator completely runs out of free memory), and then perform an expensive batched "garbage collecting" operation at the moment when the SOTA allocator is no longer able to postpone the de-allocation. Besides poor and unpredictable performance, such an approach can lead to the increased memory fragmentation, especially over the multiple repeating allocation / de-allocation cycles.

[0072] The need to traverse an irregular structure and possible need to perform the garbage collection makes it very hard to provide a guaranteed worst case performance bound. For example, the performance bound is a number of CPU instructions (cycles) that need to be executed during the longest allocation or de-allocation call; therefore, it is impossible to predict the performance bound by traversing a highly branched pointer structure. The memory structure irregularity and its dynamic nature makes it hard to provide the deterministic upper bound for an internal allocator memory requirements. This fact usually leads to an internal service memory over-allocation. The above two factors limit or completely prevent usage of such allocators in the embedded and real-time systems with limited resources, where deterministic performance and / or internal service memory consumption bounds are required to be known in advance.

[0073] In Example 3, the memory manager maintains a hierarchical bitmap having a two-bit entry. FIG. 3 shows a structure of a hierarchical bitmap having a two-bit entry which represents allocationstatus of each corresponding memory block and its sub-blocks. The two-bit entry may have the following values: 00 for "sub-allocated" (indicating that the block cannot be allocated as a whole, but rather treated as a heap of sub-blocks), 01 for "free" (the block is available for allocation), 10 for "last" (the block was allocated as a whole), and 11 for "continue" (the block or all of its sub-blocks are currently in use). A bit entry offset of a bitmap strictly corresponds to a memory block offset / size (hierarchical level) pair, so one can be computed from the other. FIG. 3 illustrates an example of bit entry (bit input) state relations for a block hierarchical structure.

[0074] During the allocation call, a free block search is performed by the recursive hierarchical bitmap traversal, until a bit entry with "free" status for a block of the desired size (hierarchy level) is found. If the traversal fails to find such a free block, the allocations (or sub-allocations) are performed by recursively updating the hierarchical bitmap. To de-allocate a previously allocated memory block, the memory manager simply changes the corresponding bitmap entry in to the "free" value.

[0075] The hierarchical bitmap-based design in Example 3 may overcome some of the drawbacks in Example 1 and Example 2. Compared with the pointer-based structure, the service memory consumption of the hierarchical bitmap-based design is orders of magnitude lower. The simple and deterministic "bit entry-block offset" mapping provides an ability for immediate automatic coalescing and memory reuse upon free call, without any performance penalty. Absence of the pointer-based structure avoids highly unpredictable memory access patterns.

[0076] However, since the hierarchical bitmap-based design in Example 3 also uses block size rounding similar to the buddy systems, the hierarchical bitmap-based design inherits the same memory waste problem, and the pure recursive bit-stream nature of the hierarchical bitmap-based design prevents from applying any waste mitigation means. For example, the size of actual memory block returned by the allocator is rounded to the upper binary logarithm, which is always greater than the requested size, resulting in a waste of memory. The layout of the hierarchical bitmap described in Example 3 needs to always start traversal from the root and repeat the traversal steps until a block of desired size with "free" status is encountered. This layout makes the bit masks to flow like a stream, without the ability to skip any intermediate levels and immediately jump to the block of the desired size. In such a layout, it is impossible to allocate any block size other than those aligned with the nearest upper binary logarithm, causing inevitable memory waste.

[0077] Therefore, the recursive nature and two-bit entries prevent effective use of bit buckets, which makes traversal performance and cache access patterns sub-optimal on modern CPUs.

[0078] FIG. 4 shows a flow of a method for memory management provided in the embodiments ofthe present disclosure. Referring to FIG. 4, the method includes the following steps S 101 to SI 03.

[0079] In S 101, a request is received, the request being used to request a memory allocation or memory de-allocation.

[0080] Requesting the memory allocation is to request the allocation of memory blocks in a memory pool (for the user); and requesting the memory de-allocation is to request the de-allocation of free memory blocks in the storage space.

[0081] In SI 02, a hierarchical bitmap tree is maintained, the hierarchical bitmap tree having one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool.

[0082] In the embodiments of the present disclosure, the computer device organizes the hierarchical bitmap tree as a hierarchical array of subtrees, where each subtree is represented by a bit bucket whose size is equal to a machine register.

[0083] In S 103, blocks corresponding to the request are determined based on the hierarchical bitmap tree.

[0084] In the embodiments of the present disclosure, a request is received, the request being used to request a memory allocation or memory de-allocation; a hierarchical bitmap tree is maintained, the hierarchical bitmap tree has one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool; and blocks corresponding to the request based on the hierarchical bitmap tree are determined. Compared with the memory access mode in the related art that a hierarchical bitmap tree having two bits can only access child blocks after accessing the parent block, in the embodiments of the present disclosure, the hierarchical bitmap tree having one bit allows directly skipping to the desired bitmap level, which may simplify the memory access mode and have high flexibility and configurability to improve traversal performance. Each block uses only one bit. Therefore, it may be further used in other optimization methods such as the effective application of bit bucket.

[0085] Referring to FIG. 5, which shows a flow of another method for memory management provided in the embodiments of the present disclosure, the method includes the following steps S201 to S211.

[0086] In S201, a request is received, the request being used to request a memory allocation or memory de-allocation.

[0087] S201 is the same as S 101, which will not be repeated here.

[0088] In S202, the hierarchical bitmap tree is organized as a hierarchical array of subtrees, whereeach subtree is represented by a bit bucket whose size is equal to a machine register.

[0089] The specific implementation process of step S202 may include the following sub-steps.

[0090] In substep Al, the number of layers of bit buckets and a minimum allocation granularity which are set by a user are obtained, where the number of layers of bit buckets is equal to a number of layers of subtrees.

[0091] In the method for memory management provided in the embodiments of the present disclosure, all the allocations, despite of their block size logarithm, are rounded up to the minimum allocation granularity only (typical rounding granularity might be 64 bytes or 256 bytes, for example). If all the requested allocation sizes happen to be already multiples of the minimum granularity (for example, this is often the case for the GPU memory allocations), then the allocated memory waste level will be zero. Thus, the minimum allocated memory waste may be realized.

[0092] Moreover, the method for memory management provided in the embodiments of the present disclosure has the number of layers of bit buckets and the minimum allocation granularity that are required and fully determined during the traversal. The minimum allocation granularity is a lower block size limit, which indicates a low border of a size range. The number of layers of bit buckets is equal to available logarithmic block size steps. In the embodiments of the present disclosure, by changing the above two parameters (i.e., the number of layers of bit buckets and the minimum allocation granularity), the method for memory management may handle any reasonable block size range, thus being easily adaptable to various application fields.

[0093] In sub-step A2, bit buckets are organized into the hierarchical bitmap tree based on the number of layers of bit buckets and the minimum allocation granularity.

[0094] The hierarchical bitmap tree includes N layers of subtrees where the memory pool is a root bit bucket; each subtree corresponds to one bit bucket; an Nth layer of subtrees includes at least a part of an (N-l)-th layer of subtrees; a block for each subtree corresponds to one bit that controls a status of the block for each subtree; and the one bit is used to indicate statuses of blocks of the Nth layer of subtrees and statuses of blocks of the (N-l)-th layer of subtrees, N being an integer greater than or equal to 2.

[0095] Optionally, the lowest layer of bit buckets in the N layers of bit buckets is the memory block with the minimum allocation granularity, such as 64 KB. It should be noted that the minimum allocation granularity can be set by the user, or it can be hard-coded for optimization purposes. The embodiments of the present do not specifically limit the minimum allocation granularity, which can be set according to actual application scenarios.

[0096] FIG. 6 is a schematic diagram showing a structure of a hierarchical bitmap tree provided in the embodiments of the present disclosure. As shown in FIG. 6, the hierarchical bitmap tree may be organized into a hierarchical array of subtrees (i.e., an array of bit buckets). The array of bit buckets is logically subdivided into a plurality of layers. A lower layer of bit buckets represents all subtrees of an upper layer of bit buckets, or part of the upper layer of bit buckets. Therefore, the position of each bit bucket corresponds to the positions and tree sub-levels represented by bits corresponding to each bit bucket. The root bucket 01 corresponds to first several levels of the tree, a next bit bucket layer corresponds to the same number of next tree levels, and so on. In addition, according to the hierarchical bitmap tree structure, each bit bucket B on the lower layer represents the logical status for all child nodes of the single node (bit) from the lowest level of the upper layer of bit buckets. Therefore, the number of bit buckets in a next level is equal to the number of bit buckets in a current level, multiplied by the number of bits in the lowest bit bucket level. For example, the number of bits is half a bit bucket width; if the bit bucket uses 64-bit variables, the number of bits in the lowest bit bucket level will be 32.

[0097] In the embodiments of the present disclosure, the number of bit bucket layers and the minimum allocation granularity may be parameters set by the user. The number of bits in a variable that holds a bit bucket (and the number of bit levels in a bit bucket as well) is a constant, depending on the computer architecture; for example, for a 64-bit variable architecture, it can fit up to 6 bit levels maximum in it. In a case where these three parameters and the required constants are determined, the depth of the hierarchical bitmap tree, the size of the required internal memory, and the range of block sizes that can be allocated may be computed.

[0098] In the embodiments of the present disclosure, based on the hierarchical bitmap tree, the reallocation is fast, the coalescing and reuse of the de-allocated memory is improved, and the method for memory management does not need any postponed garbage collection. Therefore, the method for memory management in the embodiments of the present disclosure may provide immediate and automatic coalescing and memory reuse during the de-allocation call, without any additional performance penalty, which may automatically reduce the allocated memory fragmentation and increase the re-allocation speed.

[0099] In S203, the hierarchical bitmap tree is organized based on a plurality of occupancy bitmaps which is in one-to-one correspondence with a plurality of bit buckets.

[0100] A size of an occupancy bitmap corresponding to each bit bucket fits the size of the register, and each occupancy bitmap is arranged in a corresponding bit bucket. An occupancy bitmapcorresponding to each bit bucket in a current layer is controlled by occupancy bitmaps of bit buckets in an upper layer, and occupancy bitmaps of bit buckets corresponding to all subtrees are controlled by an occupancy bitmap of the root bit bucket.

[0101] In the embodiments of the present disclosure, in the array of bit buckets corresponding to the root bit bucket, all layers of bit buckets except for the lowest layer of bit buckets uses the suballocation mask, and the lowest layer of bit buckets uses the corresponding occupancy bitmap.

[0102] FIG. 7 shows a schematic diagram of an occupation bitmap and a sub-allocation mask provided in the embodiments of the present disclosure. As shown in FIG. 7, the separation is a novel step which allows to employ simple bit logic and fast bit search instructions during allocation and deallocation calls. Referring to FIG. 7, the occupancy bitmap represents the status of corresponding memory blocks in the memory area associated with a bit bucket. In the occupancy bitmap, the bit value "1" indicates that a corresponding memory block and all sub-blocks thereof are fully free and available for allocation.

[0103] Referring to FIG. 7, the sub-allocation mask is mapped to the lowest level of the occupancy bitmap, thus being twice less in size. The sub-allocation mask represents the status of the subtrees corresponding to the lowest bit bucket level. That is, the sub-allocation mask is used to indicate the occupancy status of the lowest layer of memory blocks in the memory area associated with the bit bucket node. The bit value of "1" indicates that at least one free sub-block of small sizes exists in the underlying subtree. The occupancy bits may be propagated up and down, and the block to be allocated may be allocated.

[0104] It should be noted that the "1" and "0" bit meanings in masks of the occupancy bitmap and the sub-allocation mask are reversed. That is, "1" means free, "0" means occupancy; or, "1" means occupancy, "0" means free, which is not limited in the embodiments of the present disclosure, but may be selected according to embodiment requirements and traits.

[0105] In S204, a mapped size is obtained by dividing a requested block size by the minimum allocation granularity.

[0106] The mapped size is used to indicate a mapping of a size of the requested memory allocation or memory de-allocation to a preset block size.

[0107] In S205, a tree level is determined based on the mapped size.

[0108] The tree level is used to indicate a location in the hierarchical bitmap tree for the free bit search operation.

[0109] In the embodiments of the present disclosure, the most significant set bit position in themapped size corresponds to the tree level of the hierarchical bitmap where the free bit search should be performed.

[0110] In S206, a bit bucket bit level and a bit bucket layer level are obtained by dividing the tree level by the number of tree levels controlled by a single bit bucket.

[0111] The preset bit level of each layer of bit buckets is the number of bit levels that fits into a single bit bucket.

[0112] In the embodiments of the present disclosure, the tree level divided by the preset bit level of each layer of bit buckets gives a bit bucket layer index in the array, while the division remainder determines a bit level inside the bit bucket. Since level and layer sizes are always the powers of two, all the above calculations can be performed very fast using bit shifts and masking.

[0113] In S207, starting from a root bit bucket in the bit bucket layer level, the free bit search operation is executed by performing a traversal loop operation on an array of bit buckets corresponding to the root bit bucket, until free bit buckets that satisfy the bit bucket bit level are determined.

[0114] In the embodiments of the present disclosure, the free bit search operation includes the following substeps.

[0115] In substep A3, a first bit bucket corresponding to a first free bit in the sub-allocation mask is determined based on a sub-allocation mask.

[0116] In substep A4, in a case where a bit level of the first bit bucket satisfies a bit bucket bit level of a block among the blocks corresponding to the request, a block corresponding to the first bit bucket is determined as the block among the blocks corresponding to the request.

[0117] When a suitable bit bucket is located, if a bucket bit level index is determined, a single bit search instruction is enough to find the first free bit offset.

[0118] It should be noted that the method for memory management in the embodiments of the present disclosure fully takes advantage of CPU features, such as performance gain from cache locality, large set of 64-bit registers and their superior performance, bit search and bit count instructions which are able to execute in a single cycle. Most operations during allocator calls are performed inside CPU registers, and only very limited number of memory accesses are needed.

[0119] In substep A5, in a case where the bit level of the first bit bucket does not satisfy the bit bucket bit level of the block among the blocks corresponding to the request, a recursive search is performed to search a second bit bucket corresponding to a second free bit in the sub-allocation mask, the second free bit being a free bit next to the first free bit in the sub-allocation mask.

[0120] In the embodiments of the present disclosure, FIG. 8 shows a schematic diagram showing a process of searching a free bit bucket node provided in the embodiments of the present disclosure, starting from the root bit bucket in a layer indicated by the layer index, a bit bucket which has a free bit at a desired level may be found very fast using a small number of traversal loop steps. Referring to FIG. 8, if a level of a first bit bucket node does not meet the level of the block required by the user (i.e., if a free bit for a desired level hasn’t been found in the first found bit bucket), then the traversal procedure steps back recursively, and the next free bit in the appropriate sub-allocation mask is found. That is, a next bit bucket node corresponding to the sub-allocation mask is searched. As shown in FIG. 8, the occupancy bitmap for a layer containing a third bit level is searched.

[0121] In the embodiments of the present disclosure, the free bit search operation is repeated, and if no free bit bucket node is found in the free bit search area, out of memory status information is generated.

[0122] In S208, a bucket offset and a bit offset corresponding to the free bit buckets are projected down to a lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level.

[0123] In the embodiments of the present disclosure, the bit offset and the bucket offset are projected down to the lowest layer of memory blocks in the memory area associated with the bit bucket nodes.

[0124] In the embodiments of the present disclosure, in the case of requesting for memory deallocation, the layer index, bit level index, bit offset and bucket offset may be loaded from the internally allocation record or the allocated block. Alternatively, the layer index, bit level index, bit offset and bucket offset may be computed directly based on the block address and previously computed mapped size.

[0125] In S209, an occupancy bit mask is determined based on a projected offset and the mapped size as well as the minimum allocation granularity.

[0126] In the embodiments of the present disclosure, the projected offset and the mapped size are obtained; and the occupancy bit mask is determined based on the projected offset and the mapped size as well as the minimum allocation granularity. The projected offset is not limited in the embodiments of the present disclosure, which can be set according to actual application scenarios.

[0127] In the embodiments of the present disclosure, FIG. 9 shows a diagram of a scenario for determining the occupancy bit mask provided in the embodiments of the present disclosure. As shown in FIG. 9, after the block (log2(size)) is found, the found bit offset and bucket offset may be projected down to the most bottom of the tree; and after the projected offset and mapped size are obtained, theoccupation bitmask may be constructed based on the minimum allocation granularity, and the actual allocation block size in units of the finest block granularity is obtained as shown in FIG. 9. The occupied bit propagation may be performed while keeping the block search very fast. The projection allows the allocation state switching on each sub-tree level precisely to the actually involved part only, while keeping the rest of the level sub-blocks still available for the allocation. That is, the saved tail sub-blocks in FIG. 9 may be used for future smaller block allocations.

[0128] It should be noted that for the allocation of the saved tail sub-blocks shown in FIG. 9, due to the nature of the free bit search algorithm in the method for memory management in the embodiments of the present disclosure, subsequent smaller block allocations will find and use the remaining tail sub-blocks. Since the allocation always uses the bit search instruction (ffs()) to find a first bit in the appropriate bit level of the bit bucket, the tail sub-blocks will be found and allocated one by one in order, which may not cause any overhead or slowdown to the free bit search process.

[0129] In S210, the free bit update operation is performed by enabling an occupancy status or a free status of the occupancy bit mask to be propagated up along a propagation path of the occupancy bit mask.

[0130] In the embodiments of the present disclosure, the occupancy status or free status is enabled to be propagated up along the propagation path of the occupancy bit mask, so that the bit tree state consistency of the may be kept for the future fast searches.

[0131] For example, in a case of requesting the memory allocation, the occupancy status is enabled to be propagated up along the propagation path based on the occupancy bit mask; when a bit mask size corresponding to the occupancy bit mask is greater than a single bit size, the occupancy status is shifted up to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; and when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the free status is an occupied bit value, the free status is enabled to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

[0132] For example, in a case of requesting the memory de-allocation, the free status is enabled to be propagated up along the propagation path based on the occupancy bit mask; when the bit mask size corresponding to the occupancy bit mask is greater than the single bit size, the free status is shifted up to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; and when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the free status is an occupied bit value, thefree status is enabled to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

[0133] In S211, blocks corresponding to updated free bit buckets are determined as the blocks corresponding to the request.

[0134] In the embodiments of the present disclosure, an allocated memory block address for the request is returned. For example, based on the bit bucket / bit offsets and the tree level index, the newly allocated memory address is computed immediately. The allocated memory address will be automatically aligned to the corresponding block level granularity. Once the allocated memory address is ready, it may be directly returned to the user, or may be stored into an internal allocation record and the user will receive the handle of the allocated memory address.

[0135] The term "handle" means a type-less value uniquely identifying the internal object. This value is usually received by the user via system application program interface (API) call. For example, when a Linux kernel is called to allocate a memory page or create a file, the user will receive the file or memory handle in return. The "handle" may be replaced by any other equivalent term such as "identifier", which is not limited in the embodiments of the present disclosure.

[0136] It should be noted that the system architecture or scenario applicable to the method for memory management in the embodiments of the present disclosure may include a general dynamic memory allocator, a GPU memory manager, a block allocation manager and an embedded system allocator, which is not specifically limited in the embodiments of the present disclosure, and it may also include other adaptive system architectures or scenarios.

[0137] For the general dynamic memory allocator, the method for memory management in the embodiments of the present disclosure may be used as a drop-in replacement for existing memory allocators in many various fields. For example, the method for memory management in the embodiments of the present disclosure is implemented within a standard C library malloc() / free() calls, or standard C++ new / delete operators, Python run-time and so on, thus providing improved performance and memory capacity.

[0138] For the GPU memory manager, GPU has a large physical memory granularity, and the typical minimum GPU allocation unit (page) size is 2 Mb, so that the method for memory management in the embodiments of the present disclosure is very suitable for managing smaller buffer sub-allocations from a single GPU page.

[0139] For the block allocation manager, the method for memory management in the embodiments of the present disclosure improves the performance, defines the response time and the service memoryrange, and has a simple and reliable design, and the allocator may be easily used in demanding applications as hard drive cluster management for a file system driver, database engine storage management and so on.

[0140] For the embedded system allocator, well-defined worst-case performance and service memory structure size bounds make the proposed design a good candidate for a general-purpose dynamic memory allocator for a system with limited resources or real-time response requirements. The clear definition of such bounds is the unconditional demand. For example, the embedded system allocator includes mobile phones, drones, vehicle controllers, remote control blocks, various sensors and detectors with wireless connection, etc.

[0141] In the embodiments of the present disclosure, the method for memory management is used in a GPU memory manager. FIG. 10 shows a schematic diagram of a GPU memory manager provided in the embodiments of the present disclosure to implement the method for memory management in the embodiments of the present disclosure. The physical GPU memory may be allocated in 2 Mb pages only. Based on this, two types of the GPU memory objects, which are the big buffers (larger than a memory page in size), and the small buffers (less than a memory page), may be distinguished. The vast majority of GPU memory allocations are small buffers, while big buffers are typically scarce. Despite of some memory waste, it is more convenient to use a direct driver call to handle those rare big buffers. While, to efficiently sub-allocate the large number of small buffers from GPU VRAM pages, the proposed bit bucket array allocator is employed.

[0142] Referring to FIG. 10, in this example, each page allocated from the GPU kernel driver has an individual bit bucket array attached to it. The arrays of bit bucket nodes are dynamically allocated on demand along with the pages, and are linked to the bits of the dynamically resized bitmap (which is called the page mask). Upon page allocation, the associated page mask bit is set to " 1 " which indicates that the corresponding page has some free blocks. Once all the bits in the page array become zero during the previously performed allocation call, the corresponding page mask bit is also switched to zero. The fact that the associated page is missing and hasn’t been allocated yet is also indicated by the zero page mask bit value.

[0143] In this example, small GPU buffers are always allocated with minimum size of around 64 bytes to 256 bytes. In this case, a 3-layer array of 5-level bit buckets (64-bit word for bucket occupancy mask), with the lowest granularity limit hardcoded to 64 bytes, is a perfect fit for a structure to manage sub-allocations for a single GPU VRAM page. The root bit bucket of the array handles the blocks of sizes in 1 Mb to 64 KB range, the second bucket layer handles the sub-blocks of 32 KB to 2 KB insize, and the third (lowermost) layer handles 1024 bytes to 64 bytes sub-blocks. FIG. 11 shows a schematic diagram of a layout of an array of bit buckets provided in the embodiments of the present disclosure. As shown in FIG. 11, in order to achieve better cache locality, each bit bucket in the intermediate layer is placed right before all sub-bit buckets in a next layer corresponding thereto. In this way, it may be possible to increase the probability for parent and child bit buckets to fall into the same cache line and to avoid extra cache misses during traversal or update loops.

[0144] The allocation process starts from the page mask, from which the first page having free blocks is located. If the corresponding free block is not found, a new page (or array pair) is allocated; and the page mask is enlarged if necessary. It should be noted that, since the page mask consists of 64bit variables, the page mask is not necessarily enlarged for each new page. For example, the page mask is enlarged only once per 64 pages. The enlargement means allocating a bigger piece of memory for the page mask and copy of contents. For example, a simple implementation manner is to use a standard realloc() call or stl::vector<uint64_t> C++ class. Alternatively, since the page mask usually tends to have a handful of 64bit words, another implementation manner is to pre-allocate some reasonable memory chunk for the page mask. The embodiments of the present disclosure do not limit the implementation manner of expanding the page mask.

[0145] The mapped block size may be computed (by a simple bit shift, for hardcoded 64-bit granularity), the set most significant bit (MSB) position is extracted to determine the corresponding block size level, and then the required bucket layer index and bit level index are computed. Then, based on the two computed indices, the free bit search is performed in the corresponding bit bucket array.

[0146] If no free bit is found in the array of bit buckets corresponding to the selected page, then the above steps are repeated until the search is completed successfully. Once a free bit for a desired block size is located, it is projected down to the lowest tree level, the allocation mask is applied here and propagated up inside the array of bit buckets. The found page index and the bit offset are stored into the internal book-keeping record and converted to the newly allocated memory address, which is then returned to the user.

[0147] The rest of allocation and de-allocation procedures are standard. Since the number of layers of the array of bit buckets is hardcoded, all the traversal and mask propagation functionality is implemented as a few nested loops, which may be further unrolled by a clever optimizing compiler.

[0148] It should be noted that the method for memory management described in the embodiments of the present disclosure may be easily customized. For example, if the lower size bound is shifted to256 bytes, and 6-level bit buckets are used instead of 5-level bit buckets, then a single array of 3-layer bit buckets will be able to handle blocks of 256 bytes to 32 megabytes in size, and every "super-page" will occupy 32 MB instead of 2 MB. The described customization renders a big buffer allocation into an extraordinary event, occurring, probably, only once per application run.

[0149] In another possible customization model, the individual GPU pages may be sub-allocated from a huge plain pool (probably, including all the GPU memory available at once). In this case, it is possible to attach the dynamically allocated individual page bucket arrays as the children to the upper level root bucket array (having 2 Mb lower allocation bound and, probably, different number of layers than in a child array).

[0150] Therefore, the method for memory management in the embodiments of the present disclosure has simple algorithm, small amount of computation, and clear internal memory structure design. The method for memory management ip the embodiments of the present disclosure may be implemented in field programmable gate arrays (FPGAs) or in pure hardware circuits. For example, the method for memory management in the embodiments of the present disclosure may be implemented as a part of hardware task scheduler for some future GPU or artificial intelligence (Al) accelerator. In addition, equipped with the method for memory management in the embodiments of the present disclosure, the GPU task scheduler will be able to accomplish previously impossible tasks.

[0151] For example, different nested GPU sub-kernels or subroutines may be created and spawn completely automatically and dynamically on different execution units (EUs); or a sub-task may be dynamically re-scheduled from one EU to another EU, one demand, allocating all necessary memory resources to the sub-kernels (stack, shared memory, constant and temporary memory buffers, etc.) on the fly, without any explicit specification from the programmer. For example, one kernel may be able to compile a just-in-time (JIT) code for the other kernels on the GPU without any user intervention and to execute the code.

[0152] In this way, the GPU or Al accelerator may automatically execute completely divergent code paths on the different EUs, or provide an automatic dynamic load-balancing of the work, offload previously unsuitable algorithms to the acceleration devices, or program such devices with much greater flexibility. In other words, the method for memory management in the embodiments of the present disclosure is easy to implement and debug, so that the development cost of the allocator for the method for memory management is much less than that of the existing SOTA algorithm.

[0153] It should also be noted that the CPU applicable to the method for memory management in the embodiments of the present disclosure may use some CPUs with 64 bit or 32 bit architectures,which makes the development efforts low, the implementation simple and portable, and easy to customize and debug. Machine specific registers (such as Single Instruction Multiple Data (SIMD) registers currently available on many platforms) may also be used in some embodiments. In this case, even wider sub-allocation masks and even deeper bit buckets may be used, which decreases the required number of traversal steps and mask update operations to the absolute minimum (e.g., only one). Of course, using SIMD also requires some significant amount of additional logic (overhead) to handle data, but the method for memory management in the embodiments of the present disclosure may substantially speed up the implementation. In addition, the same goals may be achieved by other manners. If a machine may use very wide registers, which may fit the whole bit tree level in one cycle, then, of course, there will be no need to use neither sub-allocation masks nor bit buckets.

[0154] In summary, the method for memory management in the embodiments of the present disclosure allows to skip the updates for all the fully occupied bit buckets within the large allocations. The term "large allocations" here means that the requested blocks affect several bit buckets or even several bit bucket layers. Since the free block search operation first looks into the sub-allocation masks, there is no need to touch the fully occupied bit buckets. Only the last partially occupied bit bucket and the parent bit bucket need to be updated.

[0155] Next, in a majority of the cases, starting from the root bit bucket node in the layer indicated by the layer index, the traversal loop operation is performed on the bit bucket nodes in the free bit search area to execute the free bit search operation, until the free bit bucket nodes satisfying the bit level are determined from the bit bucket nodes in the free bit search area; the free bit bucket nodes are determined as the requested memory blocks; and a status switching operation is performed on the bit bucket status in the occupancy bitmap of all bit bucket nodes falling within the occupancy bit mask, the status switching operation including switching to the occupancy status in the case of requesting memory allocation, and switching to the free status in the case of requesting memory de-allocation. Therefore, all these steps are performed using the fast (single cycle) bit shift and binary logic instructions repeatedly over the same CPU register, and just a single memory read and a single write are required. In addition, due to the compact storage of the array of bit buckets, the neighbor sub-trees (bit buckets) are likely to fall into the same cache line, which may achieve a superior cache locality.

[0156] Therefore, the layout of bit buckets has a compact size, and any bit bucket localizes all the sub-children of its parent node for several layers deep at once, which provides the opportunity for some important code simplifications and optimizations. Further, the code simplifications and optimizations bring significant performance advantages over the existing SOTA algorithms.

[0157] The method for memory management provided in the embodiments of the present disclosure is described in detail in combination with FIGS. 4 to 11. A memory management device provided in the embodiments of the present disclosure will be described below in combination with FIG. 12. FIG. 12 is a structural diagram of a memory management device provided in the embodiments of the present disclosure. The memory management device 300 can be used to implement the method in the above embodiments, and it may also achieve the beneficial effects of the above method embodiments.

[0158] As shown in FIG. 12, the device 300 may include a receiving unit 310, a maintaining unit 320, and a determining unit 330. In a possible example, the device 300 is used for implementing the method for memory management as described above.

[0159] The receiving unit 310 is used for receiving a request, the request being used to request a memory allocation or memory de-allocation.

[0160] The maintaining unit 320 is used for maintaining a hierarchical bitmap tree, a the hierarchical bitmap tree has one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool.

[0161] The determining unit 330 is used for determining blocks corresponding to the request based on the hierarchical bitmap tree.

[0162] In the embodiments of the present disclosure, the device 300 further includes: an organizing unit 340.

[0163] The organizing unit 340 is used for organizing the hierarchical bitmap tree based on a plurality of occupancy bitmaps which is in one-to-one correspondence with a plurality of bit buckets; a size of an occupancy bitmap corresponding to each bit bucket fits the size of the register, and each occupancy bitmap is arranged in a corresponding bit bucket.

[0164] An occupancy bitmap corresponding to each bit bucket in a current layer is controlled by occupancy bitmaps of bit buckets in an upper layer, and occupancy bitmaps of bit buckets corresponding to all subtrees are controlled by an occupancy bitmap of the root bit bucket.

[0165] In the embodiments of the present disclosure, the maintaining unit 320 is further used for obtaining a number of layers of bit buckets and a minimum allocation granularity which are set by a user, wherein the number of layers of bit buckets is equal to a number of layers of subtrees; and organizing bit buckets into the hierarchical bitmap tree based on the number of layers of bit buckets and the minimum allocation granularity.

[0166] In the embodiments of the present disclosure, the maintaining unit 320 is further used for obtaining a mapped size by dividing a requested block size by the minimum allocation granularity;the mapped size is used to indicate a mapping of a size of the requested memory allocation or memory de-allocation to a preset block size.

[0167] The maintaining unit 320 is further used for determining a tree level based on the mapped size, and the tree level is used to indicate a location in the hierarchical bitmap tree for the free bit search operation.

[0168] The maintaining unit 320 is further used for obtaining a bit bucket bit level and a bit bucket layer level by dividing the tree level by the number of tree levels controlled by a single bit bucket.

[0169] The maintaining unit 320 is further used for: starting from a root bit bucket in the bit bucket layer level, executing the free bit search operation by performing a traversal loop operation on an array of bit buckets corresponding to the root bit bucket, until free bit buckets that satisfy the bit bucket bit level are determined.

[0170] The maintaining unit 320 is further used for projecting down a bucket offset and a bit offset corresponding to the free bit buckets to a lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level.

[0171] The maintaining unit 320 is further used for determining an occupancy bit mask based on a projected offset and the mapped size as well as the minimum allocation granularity.

[0172] The maintaining unit 320 is further used for performing the free bit update operation by enabling an occupancy status or a free status of the occupancy bit mask to be propagated up along a propagation path of the occupancy bit mask.

[0173] The maintaining unit 320 is further used for determining blocks corresponding to updated free bit buckets as the blocks corresponding to the request.

[0174] In the embodiments of the present disclosure, the maintaining unit 320 performs the free bit search operation, and the free bit search operation includes: determining, based on a sub-allocation mask, a first bit bucket corresponding to a first free bit in the sub-allocation mask; in a case where a bit level of the first bit bucket satisfies a bit bucket bit level of a block among the blocks corresponding to the request, determining a block corresponding to the first bit bucket as the block among the blocks corresponding to the request; and in a case where the bit level of the first bit bucket does not satisfy the bit bucket bit level of the block among the blocks corresponding to the request, performing a recursive search to search a second bit bucket corresponding to a second free bit in the sub-allocation mask, the second free bit being a free bit next to the first free bit in the sub-allocation mask.

[0175] The sub-allocation mask is used to indicate statuses of blocks of the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket, and a size of the sub-allocationmask is half of the size of the register; layers except the lowest layer in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level are searched using the sub-allocation mask; and the bit buckets at the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket are searched using the occupancy bitmaps.

[0176] In the embodiments of the present disclosure, the maintaining unit 320 is further used for, in a case of requesting the memory allocation, enabling the occupancy status to be propagated up along the propagation path based on the occupancy bit mask. When a bit mask size corresponding to the occupancy bit mask is greater than a single bit size, shifting up the occupancy status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size.

[0177] The maintaining unit 320 is further used for, when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the occupancy status is a free bit value, enabling the occupancy status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

[0178] In the embodiments of the present disclosure, the maintaining unit 320 is further used for, in a case of requesting the memory de-allocation, enabling the free status to be propagated up along the propagation path based on the occupancy bit mask. When the bit mask size corresponding to the occupancy bit mask is greater than the single bit size, shifting up the free status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size.

[0179] The maintaining unit 320 is further used for, when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the free status is an occupied bit value, enabling the free status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

[0180] In addition, the memory management device 300 shown in FIG. 12 may be implemented as a communication device. The communication device may be a computing device with management services deployed, a computing device with edge services deployed. Alternatively, when the communication device is a chip or chip system applied to a processing device, the memory management device 300 may be implemented as a chip or chip system.

[0181] Some embodiments of the present disclosure further provide a chip system, and the chip system includes a control circuit and an interface circuit. The interface circuit is used for obtaining the identification and operation information of the target edge service. The control circuit is used to realize the function of the computing device with management service deployed in the method according to the identification of the target edge service and the operation information.

[0182] In a possible implementation, the chip system includes a memory for storing program instructions and / or data. The chip system may be composed of chips, and may also include chips and other discrete devices.

[0183] Some embodiments of the present disclosure further provide a computing device. Referring to FIG. 13, FIG. 13 is a schematic diagram showing a structure of a computing device provided in the embodiments of the present disclosure. The computing device 400 includes a bus 402, a processor 404, a memory 406, and a communication interface 408. The processor 404, the memory 406, and the communication interface 408 are communicatively connected to each other through the bus 402. The computing device 400 may be a server or a terminal device. It can be understood that the number of processors and memories in the computing device 400 is not limited in the present disclosure. For example, the above management services are deployed on the computing device 400.

[0184] The bus 402 may be a Peripheral Component Interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be classified into an address bus, a data bus, a control bus, etc. For ease of illustration, only one bold line is used in FIG. 12, but it does not mean that there is only one bus or one type of bus. The bus 402 includes a path for transmitting information between various components (e.g., the processor 404, the memory 406, and the communication interface 408) of the computing device 400.

[0185] The processor 404 may include any one or more of processors such as a CPU, a GPU, a micro processor (MP), or a digital signal processing (DSP).

[0186] The memory 406 may include a volatile memory, such as a random access memory (RAM). The memory 406 may include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD).

[0187] The memory 406 stores an executable program code. The processor 404 executes the executable program code to implement the functions of the receiving unit 310, the maintaining unit 320 and the determining unit 330, thus realizing the method for memory management. That is, the memory 406 stores instructions for executing the method for memory management.

[0188] The communication interface 408 includes but is not limited to transceiver modules such as network interface cards and transceivers, to realize communication between the computing device 400 and other devices or communication networks. The computing device 400 may be a computer (e.g., a server) in a cloud data center, or a computer in an edge data center, or a terminal.

[0189] For the functions of deploying management services and edge services within the different computing devices 400, the different computing devices 400 can communicate through thecommunication network.

[0190] Some embodiments of the present disclosure further provide a computing device cluster. The computing device cluster includes at least one computing device, which may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0191] As shown in FIG. 14, FIG. 14 is a schematic diagram showing a structure of a computing device cluster provided in the embodiments of the present disclosure. The computing device cluster includes at least one computing device 400. A memory 406 in one or more computing devices 400 in the computer cluster may store same instructions for executing the method for memory management.

[0192] In some possible implementations, memories 406 in a plurality of computing devices 400 in the computer cluster may each store a part of the instructions for executing the method for memory management. In other words, the plurality of computing devices 400 can be combined to jointly perform the instructions for executing the method for memory management.

[0193] It should be noted that in the computing device cluster, different memories 406 in different computing devices 400 may store different instructions for executing different parts of functions of the method for memory management. That is, the instructions stored in different memories 406 in different computing devices 400 may implement a function of one or more of the receiving unit 310, maintaining unit 320, and determining unit 330.

[0194] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, such as a wide area network (WAN) or a local area network (LAN).

[0195] Some embodiments of the present disclosure provide a computer program product including instructions. The computer program product may be a software or program product that includes instructions and can be run on a computing device or stored in any available medium. When the computer program product is running on at least one computing device, the at least one computing device can perform the method for memory management.

[0196] Some embodiments of the present disclosure further provide a computer-readable storage medium. The computer-readable storage medium may be any available medium that may be accessed by a computer or a data storage device including one or more available media or a data center including one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk or a magnetic tape), an optical medium (e.g., a digital versatile disk (DVD)), asemiconductor medium (e.g., an SSD), etc. The computer-readable storage medium includes instructions for instructing a computing device (computer) to perform the method for memory management.

[0197] The embodiments described above may be implemented, in whole or in part, by software, hardware (e.g., circuitry), firmware, or any other combination thereof. When implemented using software, the above embodiments may be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When loading or executing the computer instructions or computer programs on a computer produces, in whole or in part, a process or function in accordance with the embodiments of the present disclosure. The computer may be a general purpose computer, a special-purpose computer, a computer network, or another programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g., the computer instructions may be transmitted by wired or wireless (e.g., infrared, Wi-Fi, microwave, etc.) means from one website site, computer, server, or data center to another website site, computer, server, or data center. The computer- readable storage medium may be any usable medium to which a computer is capable of accessing or a data storage device such as a server, data center, etc. that contains a collection of one or more usable media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, and tape), an optical medium (e.g., DVD), or a semiconductor medium, where the semiconductor medium may be a solid state disk.

[0198] It should be understood that the term "and / or" herein is merely a description of the relationship of associated objects, and indicates that three types of relationships may exist, e.g., A and / or B, which may indicate the existence of A alone, the existence of both A and B, and the existence of B alone, where A and B may each be in singular or in plural. In addition, the character " / " herein, generally indicates that associated objects before and after this character are in an "or" relationship, but may also indicate in an "and / or" relationship, specific reference can be made to the preceding and following text for understanding.

[0199] In the embodiments of the present disclosure, "at least one" refers to one or more, and "a plurality of' refers to two or more. The expression "at least one item (one) of the following" or similar expressions refers to any combination of these items, including any combination of singular item (one) or plural items (plurality). For example, at least one item (one) of a, b, or c may be expressed as: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, and c may each be singular or plural.

[0200] In the embodiments of the present disclosure, the words "exemplarily", "for example", and the like are used to denote examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" in the present disclosure should not be construed as being preferred or advantageous over other embodiments or design schemes. Rather, the use of the term "example" is intended to present concepts in a concrete manner.

[0201] The foregoing descriptions are merely specific implementation manners of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art could readily conceive of changes or replacements within the technical scope of the present disclosure, which shall all be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

CLAIMS1. A method for memory management, the method comprising: receiving a request, the request being used to request a memory allocation or memory deallocation; maintaining a hierarchical bitmap tree, wherein the hierarchical bitmap tree has one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool; and determining blocks corresponding to the request based on the hierarchical bitmap tree.

2. The method of claim 1, wherein maintaining the hierarchical bitmap tree, includes: organizing the hierarchical bitmap tree as a hierarchical array of subtrees, where each subtree is represented by a bit bucket whose size is equal to a machine register, wherein the hierarchical bitmap tree includes N layers of subtrees where the memory pool is a root bit bucket; each subtree corresponds to one bit bucket; an Nth layer of subtrees includes at least a part of an (N-l)-th layer of subtrees; a block for each subtree corresponds to one bit that controls a status of the block for each subtree; and the one bit is used to indicate statuses of blocks of the Nth layer of subtrees and statuses of blocks of the (N-l)-th layer of subtrees, N being an integer greater than or equal to 2.

3. The method of claim 2, wherein after organizing the hierarchical bitmap tree as the hierarchical array of subtrees, the method further comprises: organizing the hierarchical bitmap tree based on a plurality of occupancy bitmaps which is in one-to-one correspondence with a plurality of bit buckets, wherein a size of an occupancy bitmap corresponding to each bit bucket fits the size of the register, and each occupancy bitmap is arranged in a corresponding bit bucket; wherein an occupancy bitmap corresponding to each bit bucket in a current layer is controlled by occupancy bitmaps of bit buckets in an upper layer, and occupancy bitmaps of bit buckets corresponding to all subtrees are controlled by an occupancy bitmap of the root bit bucket.

4. The method of claim 3, wherein organizing the hierarchical bitmap tree as the hierarchical array of subtrees, includes: obtaining a number of layers of bit buckets and a minimum allocation granularity which are set by a user, wherein the number of layers of bit buckets is equal to a number of layers of subtrees; and organizing bit buckets into the hierarchical bitmap tree based on the number of layers of bit buckets and the minimum allocation granularity.

5. The method of claim 1, wherein determining blocks corresponding to the request based on the hierarchical bitmap tree, includes: obtaining a mapped size by dividing a requested block size by the minimum allocation granularity, wherein the mapped size is used to indicate a mapping of a size of the requested memory allocation or memory de-allocation to a preset block size; determining a tree level based on the mapped size, wherein the tree level is used to indicate a location in the hierarchical bitmap tree for the free bit search operation; obtaining a bit bucket bit level and a bit bucket layer level by dividing the tree level by the number of tree levels controlled by a single bit bucket; starting from a root bit bucket in the bit bucket layer level, executing the free bit search operation by performing a traversal loop operation on an array of bit buckets corresponding to the root bit bucket, until free bit buckets that satisfy the bit bucket bit level are determined; projecting down a bucket offset and a bit offset corresponding to the free bit buckets to a lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level; determining an occupancy bit mask based on a projected offset and the mapped size as well as the minimum allocation granularity; performing the free bit update operation by enabling an occupancy status or a free status of the occupancy bit mask to be propagated up along a propagation path of the occupancy bit mask; and determining blocks corresponding to updated free bit buckets as the blocks corresponding to the request.

6. The method of claim 5, wherein the free bit search operation includes: determining, based on a sub-allocation mask, a first bit bucket corresponding to a first free bit in the sub-allocation mask; in a case where a bit level of the first bit bucket satisfies a bit bucket bit level of a block among the blocks corresponding to the request, determining a block corresponding to the first bit bucket as the block among the blocks corresponding to the request; and in a case where the bit level of the first bit bucket does not satisfy the bit bucket bit level of the block among the blocks corresponding to the request, performing a recursive search to search a second bit bucket corresponding to a second free bit in the sub-allocation mask, the second free bit being a free bit next to the first free bit in the sub-allocation mask;wherein the sub-allocation mask is used to indicate statuses of blocks of the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket, and a size of the sub-allocation mask is half of the size of the register; layers except the lowest layer in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level are searched using the sub-allocation mask; and the bit buckets at the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket are searched using the occupancy bitmaps.

7. The method of claim 5, wherein performing the free bit update operation by enabling the occupancy status or the free status of the occupancy bit mask to be propagated up along the propagation path of the occupancy bit mask, includes: in a case of requesting the memory allocation, enabling the occupancy status to be propagated up along the propagation path based on the occupancy bit mask, wherein when a bit mask size corresponding to the occupancy bit mask is greater than a single bit size, shifting up the occupancy status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the occupancy status is a free bit value, enabling the occupancy status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

8. The method of claim 5, wherein performing the free bit update operation by enabling the occupancy status or the free status of the occupancy bit mask to be propagated up along the propagation path of the occupancy bit mask, includes: in a case of requesting the memory de-allocation, enabling the free status to be propagated up along the propagation path based on the occupancy bit mask, wherein when the bit mask size corresponding to the occupancy bit mask is greater than the single bit size, shifting up the free status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the free status is an occupied bit value, enabling the free status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

9. A memory management device, the device comprising:a receiving unit, used for receiving a request, the request being used to request a memory allocation or memory de-allocation; a maintaining unit, used for maintaining a hierarchical bitmap tree, wherein the hierarchical bitmap tree has one bit that controls a status of a corresponding block and statuses of all sub-blocks of the corresponding block in a memory pool; and a determining unit, used for determining blocks corresponding to the request based on the hierarchical bitmap tree.

10. The device of claim 9, wherein the maintaining unit is further used for: organizing the hierarchical bitmap tree as a hierarchical array of subtrees, where each subtree is represented by a bit bucket whose size is equal to a machine register, wherein the hierarchical bitmap tree includes N layers of subtrees where the memory pool is a root bit bucket; each subtree corresponds to one bit bucket; an Nth layer of subtrees includes at least a part of an (N-l)-th layer of subtrees; a block for each subtree corresponds to one bit that controls a status of the block for each subtree; and the one bit is used to indicate statuses of blocks of the Nth layer of subtrees and statuses of blocks of the (N-l)-th layer of subtrees, N being an integer greater than or equal to 2.

11. The device of claim 9, further comprising: an organizing unit, used for organizing the hierarchical bitmap tree based on a plurality of occupancy bitmaps which is in one-to-one correspondence with a plurality of bit buckets, wherein a size of an occupancy bitmap corresponding to each bit bucket fits the size of the register, and each occupancy bitmap is arranged in a corresponding bit bucket; wherein an occupancy bitmap corresponding to each bit bucket in a current layer is controlled by occupancy bitmaps of bit buckets in an upper layer, and occupancy bitmaps of bit buckets corresponding to all subtrees are controlled by an occupancy bitmap of the root bit bucket.

12. The device of claim 11, wherein the maintaining unit is further used for: obtaining a number of layers of bit buckets and a minimum allocation granularity which are set by a user, wherein the number of layers of bit buckets is equal to a number of layers of subtrees; and organizing bit buckets into the hierarchical bitmap tree based on the number of layers of bit buckets and the minimum allocation granularity.

13. The device of claim 9, wherein the maintaining unit is further used for:obtaining a mapped size by dividing a requested block size by the minimum allocation granularity, wherein the mapped size is used to indicate a mapping of a size of the requested memory allocation or memory de-allocation to a preset block size; determining a tree level based on the mapped size, wherein the tree level is used to indicate a location in the hierarchical bitmap tree for the free bit search operation; obtaining a bit bucket bit level and a bit bucket layer level by dividing the tree level by the number of tree levels controlled by a single bit bucket; starting from a root bit bucket in the bit bucket layer level, executing the free bit search operation by performing a traversal loop operation on an array of bit buckets corresponding to the root bit bucket, until free bit buckets that satisfy the bit bucket bit level are determined; projecting down a bucket offset and a bit offset corresponding to the free bit buckets to a lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket in the bit bucket layer level; determining an occupancy bit mask based on a projected offset and the mapped size as well as the minimum allocation granularity; performing the free bit update operation by enabling an occupancy status or a free status of the occupancy bit mask to be propagated up along a propagation path of the occupancy bit mask; and determining blocks corresponding to updated free bit buckets as the blocks corresponding to the request.

14. The device of claim 13, wherein the free bit search operation includes: determining, based on a sub-allocation mask, a first bit bucket corresponding to a first free bit in the sub-allocation mask; in a case where a bit level of the first bit bucket satisfies a bit bucket bit level of a block among the blocks corresponding to the request, determining a block corresponding to the first bit bucket as the block among the blocks corresponding to the request; and in a case where the bit level of the first bit bucket does not satisfy the bit bucket bit level of the block among the blocks corresponding to the request, performing a recursive search to search a second bit bucket corresponding to a second free bit in the sub-allocation mask, the second free bit being a free bit next to the first free bit in the sub-allocation mask; wherein the sub-allocation mask is used to indicate statuses of blocks of the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket, and a size of the sub-allocation mask is half of the size of the register; layers except the lowest layer in the array of bit bucketscorresponding to the root bit bucket in the bit bucket layer level are searched using the sub-allocation mask; and the bit buckets at the lowest layer of bit buckets in the array of bit buckets corresponding to the root bit bucket are searched using the occupancy bitmaps.

15. The device of claim 13, wherein the maintaining unit is further used for: in a case of requesting the memory allocation, enabling the occupancy status to be propagated up along the propagation path based on the occupancy bit mask, wherein when a bit mask size corresponding to the occupancy bit mask is greater than a single bit size, shifting up the occupancy status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the occupancy status is a free bit value, enabling the occupancy status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

16. The device of claim 13, wherein the maintaining unit is further used for: in a case of requesting the memory de-allocation, enabling the free status to be propagated up along the propagation path based on the occupancy bit mask, wherein when the bit mask size corresponding to the occupancy bit mask is greater than the single bit size, shifting up the free status to a next bit level by a corresponding number of bits, until an updated bit mask size matches the single bit size; when the updated bit mask size matches the single bit size and a buddy bit value of a buddy bit neighbor to an occupied bit corresponding to the free status is an occupied bit value, enabling the free status to be propagated up along the propagation path until the buddy bit value of the occupied bit is in the occupancy status.

17. A computing device cluster, comprising at least one computing device each including a processor and a memory; wherein the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, to cause the computing device cluster to perform the method of any one of claims 1 to 8.

18. A computer program product, comprising instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 8.

19. A computer-readable storage medium having stored computer instructions, wherein when executed by a computing device, the computer instructions cause the computing device to perform themethod of any one of claims 1 to 8.

Images