Memory allocation method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a memory allocation method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: receiving a memory allocation request of to-be-stored data to a memory database, wherein the memory allocation request comprises a data storage amount of the to-be-stored data; determining a target memory block from a virtual process space of the memory database according to the data storage amount; the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, and data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage amount; and distributing the target memory block for the to-be-stored data, wherein the target memory block is used for storing the to-be-stored data. By adopting the method, the storage capacity and the storage efficiency of the memory database can be improved.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a memory allocation method, device, computer equipment, storage medium and program product. Background Art

[0002] With the rapid development of information technology (IT), the amount of data has exploded, and the demand for data storage has also increased significantly. Generally, in-memory databases such as Redis can be used to centrally manage this data.

[0003] Redis has the characteristics of high performance and flexible data structure, but it stores all data in memory, which makes physical memory a limiting factor. When the data scale exceeds the memory capacity of a single server or virtual machine, expansion becomes complicated and costly.

[0004] To solve the above problems, distributed storage technologies such as Pika have emerged. Pika can support larger data sets by using disk space instead of relying solely on memory to store data. This method effectively breaks through the limitation of memory size on data storage, allowing the system to handle larger amounts of data. However, Pika cannot avoid a large number of I / O operations, and compared with Redis, which works entirely based on memory, Pika will have a decline in performance. Summary of the invention

[0005] Based on this, it is necessary to provide a memory allocation method, device, computer equipment, storage medium and program product to improve the storage performance of the memory database in response to the above technical problems.

[0006] In a first aspect, the present application provides a memory allocation method, the method comprising:

[0007] Receiving a memory allocation request for the memory database from the data to be stored, the memory allocation request including the data storage capacity of the data to be stored;

[0008] According to the data storage capacity, a target memory block is determined from the virtual process space of the memory database; the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, and the data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage capacity;

[0009] The target memory block is allocated for the data to be stored, and the target memory block is used to store the data to be stored.

[0010] In one embodiment, determining the target memory block from the virtual process space of the memory database according to the data storage capacity includes:

[0011] Determine a target free list from a plurality of free lists in the virtual process space according to the data storage capacity; different free lists manage different memory blocks;

[0012] The target memory block is determined from at least one memory block in the target free list according to the data storage capacity.

[0013] In one embodiment, determining a target free list from a plurality of free lists in the virtual process space according to the data storage amount includes:

[0014] Determining a target memory space amount that needs to be allocated to the memory block to store the data according to the memory block size managed by each of the free linked lists and the data storage capacity;

[0015] The target free linked list is determined from the plurality of free linked lists according to the target memory space amount and a preset block table relationship library; the size of the memory block managed by the target free linked list is greater than or equal to the target memory space amount.

[0016] In one embodiment, allocating the target memory block for the data to be stored includes:

[0017] According to the pointer of the head node of the free linked list, the memory block pointed to by the pointer of the head node is used as the target memory block, and the address of the target memory block is obtained;

[0018] The pointer of the head node is updated to the next node of the head node to remove the target memory block from the free linked list.

[0019] In one embodiment, the method further comprises:

[0020] In response to a request for inserting the data to be stored into the memory database, the data to be stored is inserted into the target memory block; wherein the data to be stored is synchronously stored in the disk file.

[0021] In one embodiment, the method further comprises:

[0022] In response to a request to delete the data to be stored, deleting the data to be stored in the target memory block;

[0023] After deleting the data to be stored, the target memory block is reclaimed.

[0024] In a second aspect, the present application provides a memory allocation device, the device comprising:

[0025] A receiving module, configured to receive a memory allocation request for the memory database from the data to be stored, wherein the memory allocation request includes a data storage capacity of the data to be stored;

[0026] A determination module, configured to determine a target memory block from a virtual process space of the memory database according to the data storage capacity; the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, and the data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage capacity;

[0027] The allocation module is used to allocate the target memory block for the data to be stored, and the target memory block is used to store the data to be stored.

[0028] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0030] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0031] The above-mentioned memory allocation method, device, computer equipment, storage medium and program product receive a memory allocation request for the data to be stored to the memory database, the memory allocation request includes the data storage capacity of the data to be stored, and according to the data storage capacity, the target memory block is determined from the virtual process space of the memory database, and the target memory block is allocated for the data to be stored. Since the virtual process space is based on the memory mapping mechanism and is mapped to the storage space of the disk file, the data to be stored stored in the target memory block will be automatically stored in the disk file synchronously, so that data larger than the physical memory is allowed to be stored, which is not limited by the physical memory, and the data to be stored can be directly operated in the virtual process space corresponding to the memory database without the need for conventional file I / O operations, which reduces the system call overhead required for I / O operations, improves the data access speed, and thus improves the memory storage capacity and storage efficiency of the memory database. Compared with the memory database in the related art that uses physical memory to store data, this method is not limited by physical memory, and the memory that can store data is greatly increased; and compared with the distributed storage system in the related art, there is no need to perform I / O operations, and the data processing efficiency is greatly increased, thereby improving the storage performance of the memory database. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 One of the flowcharts of a memory allocation method in one embodiment;

[0033] Figure 2 The second flowchart of a memory allocation method in one embodiment;

[0034] Figure 3 A schematic diagram of the structure of a memory allocation method in one embodiment;

[0035] Figure 4 The third flowchart of a memory allocation method in one embodiment;

[0036] Figure 5 The fourth flowchart of a memory allocation method in one embodiment;

[0037] Figure 6 FIG5 is a fifth flow chart of a memory allocation method in an embodiment;

[0038] Figure 7 FIG6 is a sixth flow chart of a memory allocation method in an embodiment;

[0039] Figure 8 is a structural block diagram of a memory allocation device in one embodiment;

[0040] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In one embodiment, Figure 1 As shown, a memory allocation method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps S102 to S106.

[0043] S102: Receive a memory allocation request for the memory database of the data to be stored, where the memory allocation request includes a data storage capacity of the data to be stored.

[0044] The memory allocation request is used to request the memory database for the memory space required to store the data to be stored. The memory allocation request includes the data storage capacity of the data to be stored. The data storage capacity is used to indicate the actual size of the memory space occupied by the data to be stored, for example, 17 bytes. The size of the data storage capacity can be determined based on the actual storage requirements of the data to be stored and is not limited here.

[0045] S104: Determine a target memory block from a virtual process space of the memory database according to the data storage capacity; the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, and the data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage capacity.

[0046] Virtual Process Space refers to an independent, seemingly continuous memory address space provided by the operating system for each running process. In a virtual memory system, each process has its own virtual address space, which makes it seem that from the perspective of the process, it has the memory resources of the entire computer. In fact, physical memory may be shared by multiple processes and is usually smaller than the sum of the virtual address spaces of all processes. In this way, the operating system can effectively manage and allocate limited physical memory resources to many processes.

[0047] In an embodiment of the present application, the virtual process space refers to a memory address space based on a memory database, for example, a virtual process space based on Redis. In addition, the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, that is, a large block of memory is directly mapped to a disk file through a memory mapping mechanism. Exemplarily, the memory mapping mechanism includes a memory-mapped file (Mmemory-Mapped File, mmap) mechanism. Among them, mmap is a memory-mapped file technology that allows the contents of a file or device to be directly mapped to the address space of a process, so that the process can directly access the file contents through a pointer, just like accessing memory, reducing the system call overhead required for file I / O operations (such as read and write) and improving data access speed, because the data is ultimately stored on the disk, therefore, data sets larger than physical memory can be processed. The data stored in the virtual process space is synchronously stored in the disk file. In other words, the data stored in the virtual process space will be automatically synchronously stored in the disk file. Among them, each memory block of the virtual process space corresponds to a data block mapped to the disk file. For example, Figure 3As shown, memory block 1 (Block_1) corresponds to file data block 1 (File_1), memory block 2 (Block_2) corresponds to file data block 2 (File_2), and memory block n (Block_n) corresponds to file data block n (File_n).

[0048] The size of the target memory block is greater than or equal to the data storage capacity. It is understood that the size of the target memory block is greater than or equal to the data storage capacity, so that the use of the target memory block can meet the storage requirements of the data to be stored. The size of the target memory block is pre-set and fixed, and can be set according to the data storage requirements, for example, 8 bytes, 16 bytes, 32 bytes, etc., which are not specifically limited here. The target memory block can be understood as an unused memory block, or an unoccupied memory block.

[0049] According to the data storage capacity of the data to be stored, a plurality of unused memory blocks can be determined from the virtual process space of the memory database, and a target memory block can be determined from the memory blocks.

[0050] S106: Allocate a target memory block for the data to be stored, the target memory block is used to store the data to be stored. In the application, the address and size of the target memory block can be obtained to store the data to be stored according to the address and size of the target memory block, and to manage and release the target memory block.

[0051] In the application, the disk file of the memory database can be mapped to the virtual process space through mmap, and the virtual process space can be divided into multiple fixed-size memory blocks for use. When a memory allocation request is received, unused memory blocks are found in the virtual process space, and the target storage block is determined according to the data storage capacity, so as to allocate the data to be stored corresponding to the memory allocation request.

[0052] The above-mentioned memory allocation method receives a memory allocation request for the data to be stored to the memory database, the memory allocation request includes the data storage capacity of the data to be stored, and according to the data storage capacity, determines the target memory block from the virtual process space of the memory database (for example, Redis), and allocates the target memory block for the data to be stored. Since the virtual process space is based on the memory mapping mechanism and is mapped to the storage space of the disk file, the data to be stored stored in the target memory block will be automatically stored in the disk file synchronously, so that data larger than the physical memory can be stored without being restricted by the physical memory, and the data to be stored can be directly operated in the virtual process space corresponding to the memory database without the need for conventional file I / O operations, thereby reducing the system call overhead required for I / O operations, improving the data access speed, and thus improving the memory storage capacity and storage efficiency of the memory database (for example, Redis). Compared with the memory database in the related art that uses physical memory to store data, this method is not restricted by physical memory, and the memory that can store data is greatly increased; and compared with the distributed storage system (for example, pika) in the related art, there is no need to perform I / O operations, and the data processing efficiency is greatly increased, thereby improving the data storage performance of the memory database.

[0053] In one embodiment, Figure 2 As shown, step S104, according to the data storage capacity, determines the target memory block from the virtual process space of the memory database, including the following steps S202 and S204.

[0054] S202: Determine a target free list from multiple free lists in the virtual process space according to the data storage capacity; different free lists manage different memory block sizes.

[0055] The memory blocks in the free list (FreeList) are in an idle state or unused state and can be used to store data. The target free list is one of multiple free lists. In addition, the size of the free memory blocks managed by the target free list is greater than or equal to the data storage capacity of the data to be stored.

[0056] Different free lists manage (or link) different memory blocks of different sizes. Each free memory block in the virtual process space of the memory database is connected by multiple free lists. For example, the size of the memory block managed by the free list includes 2 n+ 1 bytes, where n is a positive integer, for example, 8 bytes, 16 bytes, 32 bytes, 64 bytes, etc., which is not limited here. The number of memory blocks managed by the free list can be one or more, which can be determined by the type and number of memory blocks obtained after memory mapping and block division of the disk file, which is not limited here.

[0057] For example, Figure 3 As shown in the figure, FreeList_1 is used to manage free memory blocks with a memory block size of 8 bytes, FreeList_2 is used to manage free memory blocks with a memory block size of 16 bytes, FreeList_3 is used to manage free memory blocks with a memory block size of 32 bytes, and FreeList_n is used to manage free memory blocks with a memory block size of 2 n+1 A free memory block of bytes.

[0058] In the application, the free linked list can be graded in the order of memory blocks from small to large, that is, a multi-level free linked list mechanism is used to classify and manage the memory blocks. Based on this, after receiving a memory allocation request, according to the data storage capacity of the data to be stored, at least one candidate free linked list whose memory block size is greater than or equal to the data storage capacity is determined from the multi-level free linked list, and the candidate free linked list with the smallest managed memory block size is determined as the target free linked list, so as to determine the target memory block of the data to be stored from at least one memory block in the target free linked list according to the data storage capacity.

[0059] S204: Determine a target memory block from at least one memory block in the target free list according to the data storage capacity.

[0060] In the application, a target memory block is determined from at least one memory block in the target free list according to the data storage capacity, and after the target memory block is allocated to the data to be stored, the target memory block can be removed from the target free list. It is understood that the position and size of the target memory block can be recorded while removing the target memory block for subsequent release, recycling and reallocation.

[0061] The above-mentioned memory allocation method determines a target free linked list from multiple free linked lists in the virtual process space according to the data storage capacity, and determines a target memory block from at least one memory block in the target free linked list according to the data storage capacity. The method subdivides the virtual process space of the memory database into multiple memory blocks of fixed sizes, and adopts a free linked list mechanism to classify and manage the memory blocks, thereby realizing effective management of the virtual process space and reducing the memory fragmentation problem. Based on this, when a memory allocation request for the memory database is received from the data to be stored, the corresponding target free linked list can be quickly located according to the data storage capacity of the data to be stored, and a target memory block is selected from the target free linked list and allocated to the data to be stored, so that a suitable target memory block can be quickly allocated to the data to be stored to meet the data storage requirements and improve the memory allocation and recovery efficiency.

[0062] In one embodiment, Figure 4As shown, according to the data storage amount, determining the target free list from multiple free lists in the virtual process space includes the following steps S402 and S404.

[0063] S402: Determine the target memory space amount to be allocated to the memory block to store the data according to the memory block size and data storage amount managed by each free linked list.

[0064] Exemplarily, the memory block size managed by each free linked list is a pre-set fixed value. Based on this, the data storage capacity of the data to be stored is rounded up to determine the target memory space amount that needs to be allocated to the memory block to store the data, or in other words, the memory block size closest to the data storage capacity. For example, if the data storage capacity is 17 bytes, and the memory block sizes managed by each free linked list are 8 bytes, 16 bytes, 32 bytes, etc., respectively, then the target memory space amount that needs to be allocated to the memory block to store the data is determined to be 32 bytes. Among them, the target memory space amount is greater than or equal to the data storage capacity of the data to be stored.

[0065] S404: Determine a target free linked list from multiple free linked lists according to the target memory space amount and a preset block table relationship library; the size of the memory block managed by the target free linked list is greater than or equal to the target memory space amount.

[0066] It can be understood that the target memory space is greater than or equal to the data storage capacity of the data to be stored, so the memory block size managed by the target free list is greater than or equal to the data storage capacity of the data to be stored. The block table relationship in the block table relationship library is used to represent the association between the memory block size and the free list that manages the memory block size. For example, 8 bytes is associated with free list 1 (FreeList_1), 16 bytes is associated with free list 2 (FreeList_2), 32 bytes is associated with free list 3 (FreeList_3), and so on.

[0067] Exemplarily, after determining the target memory space amount, at least one candidate free linked list is determined from multiple free linked lists, wherein the memory block size managed by the candidate free linked list is greater than or equal to the target memory space amount, and the candidate free linked list with the smallest managed memory block size is determined as the target free linked list. For example, after determining that the target memory space amount to be allocated to the memory block to be stored is 32 bytes, each free linked list with a managed memory block size greater than or equal to the target memory space amount can be determined as a candidate free linked list, and the free linked list managing the 32-byte memory block can be determined as the target free linked list, i.e., Free List 3 (FreeList_3).

[0068] The above-mentioned memory allocation method determines the target memory space amount that needs to be allocated to the memory block to be stored according to the memory block size and data storage capacity managed by each free linked list, and determines the target free linked list from multiple free linked lists according to the target memory space amount and a preset block table relationship library, so that the target memory block can be determined from the target free linked list and allocated to the data to be stored. Since the memory block size managed by the target free linked list is greater than or equal to the data storage capacity of the data to be stored, the target memory block selected from the target free linked list can meet the data storage space requirements of the data to be stored, and the target free linked list with the smallest managed memory block can be selected from each candidate free linked list that meets the storage requirements, thereby saving memory space and improving resource utilization.

[0069] In one embodiment, Figure 5 As shown, step S106, allocating a target memory block for the data to be stored, includes the following steps S502 and S504.

[0070] S502: According to the pointer of the head node of the free list, the memory block pointed to by the pointer of the head node is used as the target memory block, and the address of the target memory block is obtained.

[0071] Exemplarily, a free list structure is defined, each memory block in the free list is cascaded in sequence, and each memory block corresponds to a node.

[0072] In the application, you can check whether the target free list is empty. If it is empty, it means that there is no available memory block, and you can return an error (such as NULL) or take other measures (such as selecting the next level free list that manages larger memory blocks). If the target free list is not empty, start from the head node of the target free list and get the address of the memory block pointed to by the node, which means that the first node (i.e., the head node) of the target free list is used as the target memory block for allocation.

[0073] S504: Update the pointer of the head node to the next node of the head node to remove the target memory block from the free linked list.

[0074] After the target memory block is taken out, the pointer of the head node is updated to the next node of the head node, thereby removing the target memory block from the target free list.

[0075] It should be noted that the above process of allocating target memory blocks for data to be stored can be implemented by defining a memory allocation function (for example, malloc function) based on a memory database (such as Redis) according to the above allocation method, wherein the memory allocation function based on the memory database is used to dynamically allocate the target memory block from the virtual process space corresponding to the memory database when the program is running, and return the first address of the allocated target memory block.

[0076] The above memory allocation method, according to the pointer of the head node of the free list, takes the memory block pointed to by the pointer of the head node as the target memory block, obtains the address of the target memory block, updates the pointer of the head node to the next node of the head node to remove the target memory block from the free list, can effectively allocate memory blocks from the free list and ensure the correct update of the free list, avoids allocation problems in memory management, such as memory leaks and double releases, and improves the reliability and accuracy of memory allocation.

[0077] In one embodiment, the memory allocation method further includes: in response to an insertion request of the data to be stored to the memory database, inserting the data to be stored into the target memory block. The data to be stored is synchronously stored in the disk file. Based on this, the operation of the data to be stored can be realized without I / O operation, and the data to be stored in the target memory block of the virtual process space is shared like accessing the memory, which significantly improves the data processing efficiency, and by mapping the memory to the file, it can achieve a storage capacity far exceeding the physical memory, thereby improving the data storage capacity and data storage efficiency of the memory database.

[0078] In one embodiment, Figure 6 As shown, the memory allocation method also includes the following steps S602 and S604.

[0079] S602: In response to a request to delete the data to be stored, delete the data to be stored in the target memory block.

[0080] S604: After deleting the data to be stored, reclaiming the target memory block.

[0081] In the application, after deleting the data to be stored in the target memory block, the corresponding target free list is determined according to the size of the target memory block, and a target free list node corresponding to the target memory block is created, and the new target free list node is inserted into the target free list, thereby realizing the recovery of the target memory block. Exemplarily, the target free list node is inserted into the head of the target free list as the head node, so as to subsequently reallocate the target memory block.

[0082] Exemplarily, a release function (e.g., free function) based on a memory database is defined to delete the data to be stored and reclaim the target memory block. The release function based on the memory database is used to release the dynamic memory previously allocated by the memory allocation function and return it to the operating system for subsequent reallocation to prevent memory leaks.

[0083] The above-mentioned memory allocation method, in response to a deletion request for the data to be stored, deletes the data to be stored in the target memory block, and reclaims the target memory block after deleting the data to be stored, thereby realizing the effective release and recovery of the memory block, ensuring the efficiency and correctness of memory management, and avoiding problems such as memory leaks and double releases.

[0084] In one embodiment, Figure 7 As shown, the memory allocation method includes the following steps S702 to S716.

[0085] S702: Receive a memory allocation request for the memory database of the data to be stored, wherein the memory database is Redis.

[0086] S704: Determine the target memory space amount to be allocated to the memory block to be stored data according to the memory block size and data storage capacity managed by each free linked list.

[0087] S706: Determine a target free linked list from multiple free linked lists in the virtual process space of the memory database according to the target memory space amount and a preset block table relationship library.

[0088] The virtual process space is a storage space mapped to the disk file based on the memory mapping mechanism (mmap), and the memory blocks of the virtual process space correspond to the data blocks mapped to the disk file. The data stored in the virtual process space is stored synchronously in the disk file. In addition, a multi-level free list mechanism is adopted to classify and manage the memory blocks according to the size of the memory blocks, and connect them with free lists. The memory blocks are connected by multiple free lists, and each free list links different memory block sizes, for example, 8bytes, 16byte, 328byte, ... 2 n+1 bytes, n = 1,2,3,..10, etc.

[0089] S708: Determine a target memory block from at least one memory block in the target free list according to the data storage capacity.

[0090] S710: Allocate a target memory block for the data to be stored, where the target memory block is used to store the data to be stored.

[0091] The malloc function based on Redis is used to allocate a target memory block for the data to be stored. The size of the target memory block is greater than or equal to the data storage capacity.

[0092] S712: In response to a request for inserting the data to be stored into the memory database, insert the data to be stored into the target memory block; wherein the data to be stored is synchronously stored into the disk file.

[0093] S714: In response to a deletion request for the data to be stored, the data to be stored in the target memory block is deleted.

[0094] S716: Reclaim the target memory block after deleting the data to be stored.

[0095] Among them, the free function based on Redis is used to release the target memory block, recycle the target memory block, and re-hang it back to the target free list.

[0096] The memory allocation method provided in the embodiment of the present application directly maps a large block of physical memory to the Redis process address space based on mmap, and realizes the efficient allocation and release of memory blocks in the virtual process space based on the malloc function and free function redefined by Redis, and subdivides the mapped memory space into multiple fixed-size memory blocks, each memory block corresponds to a data block in the disk file system, and introduces a multi-level free list mechanism, which is classified and managed according to the size of the memory block, and can more flexibly allocate and recycle memory blocks of different sizes, reducing memory fragmentation. Compared with Pika, the file access speed is close to the memory access speed through memory mapping, which reduces I / O operations, avoids the overhead caused by I / O operations, improves data processing efficiency, and has more advantages in the efficiency of data sets, especially in frequent memory operation scenarios. Compared with Redis in the related art, the limitation of physical memory can be lifted, the amount of data far exceeds the physical memory, and the data storage capacity is improved.

[0097] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0098] Based on the same inventive concept, the embodiment of the present application also provides a memory allocation device for implementing the memory allocation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more memory allocation device embodiments provided below can refer to the limitations on the memory allocation method above, and will not be repeated here.

[0099] In one embodiment, Figure 8As shown, a memory allocation device 800 is provided, including: a receiving module 801, a determining module 802 and an allocating module 803. The receiving module 801 is used to receive a memory allocation request for the data to be stored to the memory database, and the memory allocation request includes the data storage capacity of the data to be stored. The determining module 802 is used to determine the target memory block from the virtual process space of the memory database according to the data storage capacity; the virtual process space is a storage space mapped to the disk file based on the memory mapping mechanism, and the data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage capacity. The allocating module 803 is used to allocate a target memory block for the data to be stored, and the target memory block is used to store the data to be stored.

[0100] The above-mentioned memory allocation device 800 receives a memory allocation request of the data to be stored to the memory database through the receiving module 801, the memory allocation request includes the data storage amount of the data to be stored, determines the target memory block from the virtual process space of the memory database according to the data storage amount through the determination module 802, and allocates the target memory block for the data to be stored through the allocation module 803. Since the virtual process space is a storage space mapped to the disk file based on the memory mapping mechanism, the data to be stored stored in the target memory block will be automatically stored in the disk file synchronously, so that data larger than the physical memory is allowed to be stored, which is not limited by the physical memory, and the data to be stored can be directly operated in the virtual process space corresponding to the memory database, without the need for conventional file I / O operations, reducing the system call overhead required for I / O operations, and improving the data access speed, thereby improving the memory storage capacity and storage efficiency of the memory database. Compared with the memory database in the related art that uses physical memory to store data, this method is not limited by physical memory, and the memory that can store data is greatly increased; and compared with the distributed storage system in the related art, there is no need to perform I / O operations, and the data processing efficiency is greatly increased.

[0101] In one embodiment, the determination module is also used to determine the target free list from multiple free lists in the virtual process space according to the data storage capacity; different free lists manage different memory block sizes; and determine the target memory block from at least one memory block in the target free list according to the data storage capacity.

[0102] In one embodiment, the determination module is also used to determine the target memory space amount that needs to be allocated to the memory block to be stored data based on the memory block size and data storage capacity managed by each free linked list; determine the target free linked list from multiple free linked lists based on the target memory space amount and a preset block table relationship library; the memory block size managed by the target free linked list is the same as the target memory space amount.

[0103] In one embodiment, the allocation module is also used to use the memory block pointed to by the pointer of the head node of the free list as the target memory block and obtain the address of the target memory block; update the pointer of the head node to the next node of the head node to remove the target memory block from the free list.

[0104] In one embodiment, the memory allocation device further includes an insertion module, which is used to insert the data to be stored into the target memory block in response to an insertion request of the data to be stored to the memory database; wherein the data to be stored is synchronously stored in the disk file.

[0105] In one embodiment, the memory allocation device further includes a recycling module, which is used to delete the data to be stored in the target memory block in response to a deletion request for the data to be stored; after deleting the data to be stored, recycling the target memory block.

[0106] Each module in the above-mentioned memory allocation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0107] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a memory allocation method is implemented.

[0108] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above method when executing the computer program.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method are implemented.

[0111] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the aforementioned method when executed by a processor.

[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A memory allocation method, characterized in that: The method comprises: Receiving a memory allocation request for the memory database of the data to be stored, wherein the memory allocation request includes a data storage capacity of the data to be stored; According to the data storage capacity, a target memory block is determined from the virtual process space of the memory database; the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, and the data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage capacity; The target memory block is allocated for the data to be stored, and the target memory block is used to store the data to be stored.

2. The method according to claim 1, characterized in that Determining the target memory block from the virtual process space of the memory database according to the data storage capacity includes: Determine a target free list from a plurality of free lists in the virtual process space according to the data storage capacity; different free lists manage different memory blocks; The target memory block is determined from at least one memory block in the target free list according to the data storage capacity.

3. The method according to claim 2, characterized in that Determining a target free list from a plurality of free lists in the virtual process space according to the data storage amount includes: Determining a target memory space amount that needs to be allocated to the memory block to store the data according to the memory block size managed by each of the free linked lists and the data storage capacity; The target free linked list is determined from the plurality of free linked lists according to the target memory space amount and a preset block table relationship library; the size of the memory block managed by the target free linked list is greater than or equal to the target memory space amount.

4. The method according to claim 2, characterized in that: The allocating the target memory block for the data to be stored includes: According to the pointer of the head node of the free linked list, the memory block pointed to by the pointer of the head node is used as the target memory block, and the address of the target memory block is obtained; The pointer of the head node is updated to the next node of the head node to remove the target memory block from the free linked list.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: In response to a request for inserting the data to be stored into the memory database, the data to be stored is inserted into the target memory block; wherein the data to be stored is synchronously stored in the disk file.

6. The method according to any one of claim 5, characterized in that: The method further comprises: In response to a request to delete the data to be stored, deleting the data to be stored in the target memory block; After deleting the data to be stored, the target memory block is reclaimed.

7. A memory allocation device, characterized in that: The device comprises: A receiving module, configured to receive a memory allocation request for the memory database from the data to be stored, wherein the memory allocation request includes a data storage capacity of the data to be stored; A determination module, configured to determine a target memory block from a virtual process space of the memory database according to the data storage capacity; the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, and data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage capacity; The allocation module is used to allocate the target memory block for the data to be stored, and the target memory block is used to store the data to be stored.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images