Database memory allocation method, device and equipment

Abstract

The invention provides a database memory allocation method, device and equipment, and the method comprises the steps: obtaining a target sub-memory pool allocated by a target thread used for executing a target task under the condition that the memory demand of the target task is smaller than a preset allocation threshold value, the target sub-memory pool is composed of a plurality of linked lists, each linked list comprises a plurality of memory blocks with the same memory capacity, and the memory capacities of the memory blocks linked by the plurality of linked lists are different; under the condition that the memory demand is larger than the memory capacity of the maximum memory block in the multiple linked lists, a first target allocation memory block is obtained from the global memory pool, and the memory capacity of the first target allocation memory block is larger than or equal to the memory demand; extracting a task allocation memory block corresponding to the memory demand from the first target allocation memory block, and allocating the task allocation memory block to a target task; and adding the remaining memory blocks of the first target allocation memory block into the target sub-memory pool.

Description

Technical Field

[0001] The present invention relates to the technical field of memory management, and more specifically to a database memory allocation method, device and equipment. Background Art

[0002] In database management systems, memory management is one of the key factors affecting the overall performance of the system. In order to cope with large-scale data processing needs, database systems usually use a multi-threaded architecture to improve concurrent processing capabilities. Each thread may need to independently allocate memory for operations such as query execution, cache management, and index maintenance. The efficiency of memory management affects the efficiency of data management.

[0003] In a multi-threaded environment, frequent memory allocation and release operations will increase the overhead of memory management, resulting in serious memory fragmentation problems. The system cannot effectively utilize memory resources, and when multiple threads apply for memory, resource requisition and deadlock may occur, limiting the system's concurrency and throughput. Summary of the invention

[0004] In view of the above problems, the present invention provides a database memory allocation method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present invention, a database memory allocation method is provided, the method is applied to a database operating system, the database operating system includes multiple threads, each thread is allocated a sub-memory pool, the method includes: when the memory demand of a target task is less than a preset allocation threshold, obtaining a target sub-memory pool allocated to a target thread for executing the target task, the target sub-memory pool is composed of multiple linked lists, each of the linked lists includes multiple memory blocks with the same memory capacity, and the memory capacities of the memory blocks linked by the multiple linked lists are different; when the memory demand is greater than the memory capacity of the largest memory block in the multiple linked lists, obtaining a first target allocated memory block from a global memory pool, the memory capacity of the first target allocated memory block is greater than or equal to the memory demand; extracting a task allocated memory block corresponding to the memory demand from the first target allocated memory block, and allocating the task allocated memory block to the target task; adding the remaining memory blocks of the first target allocated memory block to the target sub-memory pool.

[0006] According to an embodiment of the present invention, the above-mentioned target sub-memory pool also includes a header link, and the above-mentioned header link is an ordered link of multiple linked list headers based on the memory capacity of the linked memory blocks. The above-mentioned method also includes: generating a linked list header to be inserted based on the above-mentioned task allocation memory block; inserting the above-mentioned linked list header to be inserted into the above-mentioned header link based on the memory capacity of the above-mentioned task allocation memory block; linking the above-mentioned task allocation memory block with the above-mentioned linked list header to be inserted.

[0007] According to an embodiment of the present invention, the above-mentioned linked list header includes an identifier representing the memory capacity of the linked memory block, and the above-mentioned adding the remaining memory blocks of the above-mentioned first target allocated memory blocks to the above-mentioned target sub-memory pool includes: determining the first allocated memory from the above-mentioned remaining memory blocks based on the above-mentioned identifier, the memory capacity of the above-mentioned first allocated memory is the memory capacity corresponding to the first identifier in the above-mentioned target sub-memory pool that is closest to the memory capacity of the above-mentioned remaining memory blocks, and the memory capacity of the above-mentioned first allocated memory is less than or equal to the memory capacity of the above-mentioned remaining memory blocks; determining the first remaining memory block based on the above-mentioned remaining memory blocks and the above-mentioned first allocated memory; linking the above-mentioned first allocated memory to the linked list corresponding to the above-mentioned first identifier; in the case that the memory capacity of the above-mentioned first remaining memory block is greater than or equal to the memory capacity corresponding to the second identifier, determining the second allocated memory block from the above-mentioned first remaining memory block based on the above-mentioned second identifier; and linking the above-mentioned second allocated memory block to the linked list corresponding to the above-mentioned second identifier.

[0008] According to an embodiment of the present invention, the configuration information of the above-mentioned database operating system includes a preset page capacity, and the above-mentioned task allocation memory block corresponding to the above-mentioned memory requirement is extracted from the above-mentioned first target allocation memory block and allocated to the above-mentioned target task, including: based on the above-mentioned preset page capacity and the above-mentioned memory requirement, the above-mentioned task allocation memory block is extracted from the above-mentioned first target allocation memory block, the memory capacity of the above-mentioned task allocation memory block is greater than or equal to the above-mentioned memory requirement, and is an integer multiple of the above-mentioned preset page capacity; and the above-mentioned task allocation memory block is allocated to the above-mentioned target task.

[0009] According to an embodiment of the present invention, the method further includes: when the memory demand is greater than the preset allocation threshold, determining a second target allocation memory block from the global memory pool based on the memory demand; and allocating the second target allocation memory block to the target task.

[0010] According to an embodiment of the present invention, the method further includes: after the target task is completed, releasing the task allocated memory block to the sub-memory pool of the target thread.

[0011] According to an embodiment of the present invention, the method further comprises: after the target task is completed, releasing the second target allocated memory block to the global memory pool.

[0012] According to an embodiment of the present invention, releasing the task allocation memory block to the sub-memory pool of the target thread includes: determining a target linked list based on the memory capacity of the task allocation memory block; and linking the task allocation memory block to the target linked list.

[0013] A second aspect of the present invention provides a database memory allocation device, the device is installed in a database operating system, the database operating system includes multiple threads, each thread is allocated a sub-memory pool, the device includes: an acquisition module, used to acquire a target sub-memory pool allocated to a target thread for executing the target task when the memory demand of the target task is less than a preset allocation threshold, the target sub-memory pool is composed of multiple linked lists, each of the linked lists includes multiple memory blocks with the same memory capacity, and the memory capacities of the memory blocks linked by the multiple linked lists are different; a first target allocated memory block acquisition module, used to acquire a first target allocated memory block from a global memory pool when the memory demand is greater than the memory capacity of the largest memory block in the multiple linked lists, the memory capacity of the first target allocated memory block is greater than or equal to the memory demand; an extraction module, used to extract a task allocated memory block corresponding to the memory demand from the first target allocated memory block, and allocate the task allocated memory block to the target task; a linked list adding module, used to add the remaining memory blocks of the first target allocated memory block to the target sub-memory pool.

[0014] According to an embodiment of the present invention, the above-mentioned target sub-memory pool also includes a header link, and the above-mentioned header link is an ordered link of multiple linked list headers based on the memory capacity of the linked memory blocks. The above-mentioned device also includes: a header generation module, which is used to generate a linked list header to be inserted based on the above-mentioned task allocation memory block; an insertion module, which is used to insert the above-mentioned linked list header to be inserted into the above-mentioned header link based on the memory capacity of the above-mentioned task allocation memory block; and a linking module, which is used to link the above-mentioned task allocation memory block with the above-mentioned linked list header to be inserted.

[0015] According to an embodiment of the present invention, the above-mentioned linked list header includes an identifier representing the memory capacity of the linked memory block, and the above-mentioned linked list joining module includes: a first allocated memory determination submodule, which is used to determine the first allocated memory from the above-mentioned remaining memory blocks based on the above-mentioned identifier, the memory capacity of the above-mentioned first allocated memory is the memory capacity corresponding to the first identifier in the above-mentioned target sub-memory pool that is closest to the memory capacity of the above-mentioned remaining memory block, and the memory capacity of the above-mentioned first allocated memory is less than or equal to the memory capacity of the above-mentioned remaining memory block; a first remaining memory block determination submodule, which is used to determine the first remaining memory block based on the above-mentioned remaining memory block and the above-mentioned first allocated memory; a first allocated memory linking submodule, which is used to link the above-mentioned first allocated memory to the linked list corresponding to the first identifier; a second allocated memory block determination submodule, which is used to determine the second allocated memory block from the above-mentioned first remaining memory block based on the second identifier when the memory capacity of the above-mentioned first remaining memory block is greater than or equal to the memory capacity corresponding to the second identifier; a second linking submodule, which is used to link the above-mentioned second allocated memory block to the linked list corresponding to the above-mentioned second identifier.

[0016] According to an embodiment of the present invention, the configuration information of the above-mentioned database operating system also includes a preset page capacity, and the above-mentioned extraction module includes: a task allocation memory block extraction sub-module, used to extract the above-mentioned task allocation memory block from the above-mentioned first target allocation memory block based on the above-mentioned preset page capacity and the above-mentioned memory requirement, the memory capacity of the above-mentioned task allocation memory block is greater than or equal to the above-mentioned memory requirement and is an integer multiple of the above-mentioned preset page capacity; an allocation sub-module, used to allocate the above-mentioned task allocation memory block to the above-mentioned target task.

[0017] According to an embodiment of the present invention, the above-mentioned device also includes: a second target allocation memory block determination module, which is used to determine the second target allocation memory block from the above-mentioned global memory pool based on the above-mentioned memory demand when the above-mentioned memory demand is greater than the above-mentioned preset allocation threshold; a second allocation module, which is used to allocate the above-mentioned second target allocation memory block to the above-mentioned target task.

[0018] According to an embodiment of the present invention, the device further comprises: a sub-memory pool releasing module, which is used to release the task-allocated memory block to the sub-memory pool of the target thread after the target task is completed.

[0019] According to an embodiment of the present invention, the device further comprises: a global memory pool release module, which is used to release the task allocated memory block to the sub-memory pool of the target thread after the target task is completed.

[0020] According to an embodiment of the present invention, the above-mentioned sub-memory pool release module includes: a target linked list determination sub-module, used to determine the target linked list based on the memory capacity of the above-mentioned task allocation memory block; and a linking sub-module, used to link the above-mentioned task allocation memory block to the above-mentioned target linked list.

[0021] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0022] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0023] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0024] According to an embodiment of the present invention, a sub-memory pool is allocated to each thread. After receiving a target task, a target sub-memory pool allocated to the target thread for executing the target task is obtained. When the memory requirement of the target task is greater than the memory capacity of the largest memory block in a linked list, the first target allocated memory block is obtained from the global memory pool, and the memory corresponding to the memory requirement is extracted and allocated to the target task, and the remaining memory blocks are added to the linked list, thereby subdividing the large memory block into memories of different sizes in different linked lists, meeting the memory requirements of most tasks and reducing memory fragmentation. Furthermore, a sub-memory pool is allocated to each thread. When the target task needs to allocate memory, memory is first allocated from the sub-memory pool, thereby reducing the situation where threads compete for global memory, reducing resource requisition and deadlock, and improving the concurrency performance and throughput of the multi-threaded database operating system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0026] Figure 1 A diagram showing an application scenario of a database memory allocation method and device according to an embodiment of the present invention is shown.

[0027] Figure 2 A flow chart of a database memory allocation method according to an embodiment of the present invention is shown.

[0028] Figure 3 A schematic diagram of a memory pool of a database operating system according to an embodiment of the present invention is shown.

[0029] Figure 4 A schematic diagram of a sub-memory pool after being inserted into a linked list according to an embodiment of the present invention is shown.

[0030] Figure 5 A structural block diagram of a database memory allocation device according to an embodiment of the present invention is shown.

[0031] Figure 6 A block diagram of an electronic device suitable for implementing a database memory allocation method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0033] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0034] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0035] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0036] In database management systems, memory management is one of the key factors affecting the overall performance of the system. In order to cope with large-scale data processing needs, database systems usually adopt a multi-threaded architecture to improve concurrent processing capabilities. Each thread may need to independently allocate memory for operations such as query execution, cache management, and index maintenance. The efficiency of memory management has an important impact on the efficiency of data management.

[0037] In the related technology, frequent memory allocation and release operations in a multi-threaded environment will increase the overhead of memory management, thereby causing system performance to degrade. In addition, due to the lack of refinement in the memory allocation strategy, the problem of internal and external memory fragmentation in the multi-threaded database is relatively serious, resulting in the system being unable to effectively utilize memory resources, thereby affecting the overall performance of the system. When multiple threads need to apply for memory, resource requisition and deadlock may occur, limiting the concurrency and throughput of the system.

[0038] In view of this, an embodiment of the present invention provides a database memory allocation method, which is applied to a database operating system, the database operating system includes multiple threads, each thread is allocated a sub-memory pool, and the method includes: when the memory demand of a target task is less than a preset allocation threshold, obtaining a target sub-memory pool allocated to a target thread for executing the target task, the target sub-memory pool is composed of multiple linked lists, each linked list includes multiple memory blocks with the same memory capacity, and the memory capacities of the memory blocks linked by the multiple linked lists are different; when the memory demand is greater than the memory capacity of the largest memory block in the multiple linked lists, obtaining a first target allocated memory block from a global memory pool, the memory capacity of the first target allocated memory block is greater than or equal to the memory demand; extracting a task allocated memory block corresponding to the memory demand from the first target allocated memory block, and allocating the task allocated memory block to the target task; adding the remaining memory blocks of the first target allocated memory block to the target sub-memory pool.

[0039] Figure 1 A diagram showing an application scenario of a database memory allocation method and device according to an embodiment of the present invention is shown.

[0040] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0041] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0042] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0043] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0044] It should be noted that the database memory allocation method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the database memory allocation device provided in the embodiment of the present invention can generally be set in the server 105. The database memory allocation method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the database memory allocation device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0046] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The database memory allocation method of the embodiment of the invention is described in detail.

[0047] Figure 2 A flow chart of a database memory allocation method according to an embodiment of the present invention is shown.

[0048] like Figure 2 As shown, the database memory allocation method of this embodiment includes operations S210 to S240.

[0049] According to an embodiment of the present invention, the above method is applied to a database operating system, which includes multiple threads, and each thread is allocated a sub-memory pool.

[0050] According to an embodiment of the present invention, when the database operating system is started, a sub-memory pool may be allocated in advance for each thread according to the available memory of the database operating system and expected concurrent threads.

[0051] In operation S210, when the memory requirement of the target task is less than a preset allocation threshold, a target sub-memory pool allocated to a target thread for executing the target task is obtained.

[0052] The target sub-memory pool is composed of a plurality of linked lists, each of which includes a plurality of memory blocks with the same memory capacity, and the memory blocks linked by the plurality of linked lists have different memory capacities.

[0053] According to an embodiment of the present invention, 11 one-way linked lists may be used to form the target sub-memory pool, which correspond to linked lists with memory capacities of 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, and 8192 bytes of memory blocks respectively.

[0054] In operation S220, when the memory demand is greater than the memory capacity of the largest memory block in the multiple linked lists, a first target allocated memory block is obtained from the global memory pool.

[0055] The memory capacity of the first target allocated memory block is greater than or equal to the memory requirement.

[0056] Exemplarily, when the memory capacity of the largest memory block of the linked list is 8 KB (8192 bytes) and the memory requirement is 12 KB, a first target allocation memory block of 16 KB can be obtained from the global memory pool.

[0057] In operation S230, a task allocation memory block corresponding to the memory requirement is extracted from the first target allocation memory block, and the task allocation memory block is allocated to the target task.

[0058] In operation S240, the remaining memory blocks of the first target allocated memory block are added to the target sub-memory pool.

[0059] Exemplarily, when the first allocated memory block is 16 KB and the memory requirement is 12 KB, the memory allocated to the target task is 12 KB and the remaining memory block is 4 KB.

[0060] According to an embodiment of the present invention, by allocating a sub-memory pool to each thread, after receiving a target task, the target sub-memory pool allocated to the target thread for executing the target task is obtained. When the memory requirement of the target task is greater than the memory capacity of the largest memory block in the linked list, the first target allocated memory block is obtained from the global memory pool, and the memory corresponding to the memory requirement is extracted and allocated to the target task, and the remaining memory blocks are added to the linked list, thereby subdividing the large memory block into memories of different sizes in different linked lists, meeting the memory requirements of most tasks and reducing memory fragmentation. Furthermore, by allocating a sub-memory pool to each thread, when the target task needs to allocate memory, memory is first allocated from the sub-memory pool, thereby reducing the situation where threads compete for global memory, reducing resource requisition and deadlock, and improving the concurrency performance and throughput of the multi-threaded database operating system.

[0061] According to an embodiment of the present invention, the above-mentioned target sub-memory pool also includes a header link, which is an ordered link of multiple linked list headers based on the memory capacity of the linked memory blocks. The method also includes: generating a linked list header to be inserted based on the task allocation memory block; inserting the linked list header to be inserted into the header link based on the memory capacity of the task allocation memory block; linking the task allocation memory block with the linked list header to be inserted.

[0062] According to an embodiment of the present invention, when the memory requirement of the target task is less than the memory capacity of the largest memory block in the linked list, a memory block may be selected from the linked list for allocation.

[0063] Exemplarily, when the memory requirement is 3 KB, a 4 KB memory block may be selected from the linked list and allocated to the target task.

[0064] According to an embodiment of the present invention, each thread is allocated a sub-memory pool, and memory blocks are preferentially selected from the sub-memory pool to be allocated to the target task, which can reduce frequent memory allocation and recycling with the global memory pool and improve memory allocation efficiency.

[0065] Figure 3 A schematic diagram of a memory pool of a database operating system according to an embodiment of the present invention is shown.

[0066] Figure 4 A schematic diagram of a sub-memory pool after being inserted into a linked list according to an embodiment of the present invention is shown.

[0067] like Figure 3 As shown, 301 is the global memory pool, 302 is the sub-memory pool allocated to thread A, and 303 is the sub-memory pool allocated to thread N.

[0068] like Figure 4As shown, taking the sub-memory pool corresponding to thread A as an example, when thread A executes the target task, if the memory requirement of 12KB is greater than the memory capacity of the largest memory block of 8192 bytes (8KB), the header of the linked list to be inserted can be generated based on 12288 bytes (12KB), inserted before the header of 8192 bytes, and the memory is linked to the header of 12KB.

[0069] According to an embodiment of the present invention, through multiple linked lists that are linked in order, when the memory demand is less than the memory capacity of the largest memory block, the target memory block to be allocated to the target task can be quickly found and allocated. When the memory demand is greater than the memory capacity of the largest memory block, the memory is connected to the header of the linked list to be inserted, ensuring that the memory blocks are linked in order, thereby optimizing memory usage and allowing the system to respond to memory allocation requests more quickly.

[0070] According to an embodiment of the present invention, the above-mentioned linked list header includes an identifier representing the memory capacity of the linked memory block, and the remaining memory blocks of the first target allocated memory block are added to the target sub-memory pool, including: determining the first allocated memory from the remaining memory blocks based on the identifier, the memory capacity of the first allocated memory is the memory capacity corresponding to the first identifier in the target sub-memory pool that is closest to the memory capacity of the remaining memory block, and the memory capacity of the first allocated memory is less than or equal to the memory capacity of the remaining memory block; determining the first remaining memory block based on the remaining memory block and the first allocated memory; linking the first allocated memory to the linked list corresponding to the first identifier; when the memory capacity of the first remaining memory block is greater than or equal to the memory capacity corresponding to the second identifier, determining the second allocated memory block from the first remaining memory block based on the second identifier; and linking the second allocated memory block to the linked list corresponding to the second identifier.

[0071] Exemplarily, when the remaining memory block is 6KB, it will be determined from the linked list that the first allocated memory is 4096 bytes (4KB), then the first remaining memory block is 2KB, and the memory capacity corresponding to the second identifier is 2048 bytes (2KB), then the second allocated memory block is 2KB; when the remaining memory block is 7KB, the first allocated memory is 4KB, then the first remaining memory block is 3KB, the determined second allocated memory block is 2KB, the second remaining memory block is 1KB, and the third allocated memory determined from the second remaining memory block is 1KB, and so on, the remaining memory blocks can be divided from large to small and added to each linked list of the sub-memory pool.

[0072] According to an embodiment of the present invention, the memory capacity of the linked memory block is determined by the header identifier of the linked list, and the first allocated memory and the first remaining memory block are determined from the remaining memory blocks based on the identifier, the first allocated memory is linked to the linked list corresponding to the first identifier, and the second allocated memory block is determined according to the first remaining memory block. The remaining memory blocks can be divided into memory block sizes in the linked list and added one by one to the corresponding linked list, so that memory management is more refined and memory waste is reduced.

[0073] According to an embodiment of the present invention, the configuration information of the above-mentioned database operating system also includes a preset page capacity, and a task allocation memory block corresponding to the memory requirement is extracted from the first target allocation memory block and allocated to the target task, including: extracting the task allocation memory block from the first target allocation memory block based on the preset page capacity and the memory requirement, the memory capacity of the task allocation memory block is greater than or equal to the memory requirement and is an integer multiple of the preset page capacity; allocating the task allocation memory block to the target task.

[0074] According to an embodiment of the present invention, there are different preset page capacities for different database operating systems. The page capacity refers to the amount of data that can be stored in a page in a virtual memory system. In paging memory management, the physical memory is divided into blocks of fixed size, each block is called a page, and the size of the page is usually a power of 2. When determining the memory allocated to the target task, the memory can be set to an integer multiple of the page, and then the memory can be added to a linked list to reduce memory fragmentation.

[0075] According to an embodiment of the present invention, the memory blocks linked in the linked list may also be set to integer multiples of a preset page capacity, so that in the process of allocating memory to the target task, memory blocks of a fixed size are allocated to reduce memory fragmentation outside the page.

[0076] According to an embodiment of the present invention, the method further includes: when the memory demand is greater than a preset allocation threshold, determining a second target allocation memory block from the global memory pool based on the memory demand; and allocating the second target allocation memory block to the target task.

[0077] According to an embodiment of the present invention, when the memory requirement of the target task is too large and cannot be satisfied by the sub-memory pool, a second target allocated memory block can be directly determined from the global memory pool and directly allocated to the target task.

[0078] According to an embodiment of the present invention, the database operating system can also monitor the current query load, data access mode, memory usage and system resource occupancy in real time, so that the database memory can be regulated according to the monitoring data. When the system load is high, more memory resources can be allocated to key queries, and when the load is low, memory resources can be automatically recycled or reallocated to maximize memory utilization efficiency.

[0079] According to an embodiment of the present invention, the method further comprises: after the target task is completed, releasing the task allocated memory block to the sub-memory pool of the target thread.

[0080] According to an embodiment of the present invention, when the memory demand of the target task is greater than a preset threshold, a second target allocated memory block is determined from the global memory pool to be allocated to the target task, and the memory is released back to the global memory pool after the task is completed. This can avoid adding too much memory to the sub-memory pool and improve memory reuse rate.

[0081] According to an embodiment of the present invention, the method further includes: after the target task is completed, releasing the second target allocated memory block to the global memory pool.

[0082] According to an embodiment of the present invention, the above-mentioned releasing of the task allocation memory block to the sub-memory pool of the target thread includes: determining a target linked list based on the memory capacity of the task allocation memory block; and linking the task allocation memory block to the target linked list.

[0083] Exemplarily, when the memory capacity is 4 KB, a target linked list with a header of 4 KB can be found in the sub-memory pool according to the memory capacity, and the memory is linked to the target linked list.

[0084] Figure 5 A structural block diagram of a database memory allocation device according to an embodiment of the present invention is shown.

[0085] like Figure 5 As shown, the database memory allocation device 500 of this embodiment includes an acquisition module 510 , a first target allocated memory block acquisition module 520 , an extraction module 530 and a linked list adding module 540 .

[0086] The acquisition module 510 is used to acquire the target sub-memory pool allocated to the target thread for executing the target task when the memory requirement of the target task is less than a preset allocation threshold, the target sub-memory pool being composed of a plurality of linked lists, each of which includes a plurality of memory blocks with the same memory capacity, and the memory blocks linked by the plurality of linked lists have different memory capacities. In one embodiment, the acquisition module 510 can be used to execute the operation S210 described above, which will not be described in detail herein.

[0087] The first target allocated memory block acquisition module 520 is used to acquire the first target allocated memory block from the global memory pool when the memory demand is greater than the memory capacity of the largest memory block in the multiple linked lists, and the memory capacity of the first target allocated memory block is greater than or equal to the memory demand. In one embodiment, the first target allocated memory block acquisition module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0088] The extraction module 530 is used to extract the task allocation memory block allocation corresponding to the memory demand from the first target allocation memory block, and assign the task allocation memory block to the target task. In one embodiment, the extraction module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0089] The linked list adding module 540 is used to add the remaining memory blocks of the first target allocated memory block to the target sub-memory pool. In one embodiment, the linked list adding module 540 can be used to perform the operation S240 described above, which will not be described in detail here.

[0090] According to an embodiment of the present invention, the above-mentioned target sub-memory pool also includes a header link, and the above-mentioned header link is an ordered link of multiple linked list headers based on the memory capacity of the linked memory blocks. The above-mentioned device also includes: a header generation module, which is used to generate a linked list header to be inserted based on the above-mentioned task allocation memory block; an insertion module, which is used to insert the above-mentioned linked list header to be inserted into the above-mentioned header link based on the memory capacity of the above-mentioned task allocation memory block; and a linking module, which is used to link the above-mentioned task allocation memory block with the above-mentioned linked list header to be inserted.

[0091] According to an embodiment of the present invention, the above-mentioned linked list header includes an identifier representing the memory capacity of the linked memory block, and the above-mentioned linked list joining module 540 includes: a first allocated memory determination submodule, which is used to determine the first allocated memory from the above-mentioned remaining memory blocks based on the above-mentioned identifier, the memory capacity of the above-mentioned first allocated memory is the memory capacity corresponding to the first identifier in the above-mentioned target sub-memory pool that is closest to the memory capacity of the above-mentioned remaining memory block, and the memory capacity of the above-mentioned first allocated memory is less than or equal to the memory capacity of the above-mentioned remaining memory block; a first remaining memory block determination submodule, which is used to determine the first remaining memory block based on the above-mentioned remaining memory block and the above-mentioned first allocated memory; a first allocated memory linking submodule, which is used to link the above-mentioned first allocated memory to the linked list corresponding to the first identifier; a second allocated memory block determination submodule, which is used to determine the second allocated memory block from the above-mentioned first remaining memory block based on the second identifier when the memory capacity of the above-mentioned first remaining memory block is greater than or equal to the memory capacity corresponding to the second identifier; a second linking submodule, which is used to link the above-mentioned second allocated memory block to the linked list corresponding to the above-mentioned second identifier.

[0092] According to an embodiment of the present invention, the configuration information of the above-mentioned database operating system also includes a preset page capacity, and the above-mentioned extraction module 530 includes: a task allocation memory block extraction sub-module, used to extract the above-mentioned task allocation memory block from the above-mentioned first target allocation memory block based on the above-mentioned preset page capacity and the above-mentioned memory requirement, the memory capacity of the above-mentioned task allocation memory block is greater than or equal to the above-mentioned memory requirement and is an integer multiple of the above-mentioned preset page capacity; an allocation sub-module, used to allocate the above-mentioned task allocation memory block to the above-mentioned target task.

[0093] According to an embodiment of the present invention, the above-mentioned device also includes: a second target allocation memory block determination module, which is used to determine the second target allocation memory block from the above-mentioned global memory pool based on the above-mentioned memory demand when the above-mentioned memory demand is greater than the above-mentioned preset allocation threshold; a second allocation module, which is used to allocate the above-mentioned second target allocation memory block to the above-mentioned target task.

[0094] According to an embodiment of the present invention, the device further comprises: a sub-memory pool releasing module, which is used to release the task-allocated memory block to the sub-memory pool of the target thread after the target task is completed.

[0095] According to an embodiment of the present invention, the device further comprises: a global memory pool release module, which is used to release the task allocated memory block to the sub-memory pool of the target thread after the target task is completed.

[0096] According to an embodiment of the present invention, the above-mentioned sub-memory pool release module includes: a target linked list determination sub-module, used to determine the target linked list based on the memory capacity of the above-mentioned task allocation memory block; and a linking sub-module, used to link the above-mentioned task allocation memory block to the above-mentioned target linked list.

[0097] According to an embodiment of the present invention, any multiple modules among the acquisition module 510, the first target allocated memory block acquisition module 520, the extraction module 530 and the linked list adding module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 510, the first target allocated memory block acquisition module 520, the extraction module 530 and the linked list adding module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the acquisition module 510, the first target allocated memory block acquisition module 520, the extraction module 530 and the linked list adding module 540 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0098] Figure 6 A block diagram of an electronic device suitable for implementing a database memory allocation method according to an embodiment of the present invention is shown.

[0099] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0100] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0101] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0102] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0103] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0104] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the database memory allocation method provided by the embodiment of the present invention.

[0105] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0106] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0107] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0108] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0109] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0111] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A database memory allocation method, characterized in that: The method is applied to a database operating system, the database operating system includes multiple threads, each thread is allocated with a sub-memory pool, and the method includes: When the memory requirement of the target task is less than a preset allocation threshold, obtaining a target sub-memory pool allocated to a target thread for executing the target task, wherein the target sub-memory pool is composed of a plurality of linked lists, each of the linked lists includes a plurality of memory blocks with the same memory capacity, and the memory capacities of the memory blocks linked by the plurality of linked lists are different; When the memory demand is greater than the memory capacity of the largest memory block in the multiple linked lists, obtaining a first target allocated memory block from a global memory pool, wherein the memory capacity of the first target allocated memory block is greater than or equal to the memory demand; Extracting a task allocation memory block corresponding to the memory requirement from the first target allocation memory block, and allocating the task allocation memory block to the target task; Add the remaining memory blocks of the first target allocated memory blocks to the target sub-memory pool.

2. The method according to claim 1, characterized in that The target sub-memory pool further includes a header link, where the header link is an ordered link of multiple linked list headers based on the memory capacity of the linked memory blocks, and the method further includes: Generate a header of a linked list to be inserted based on the task allocation memory block; Based on the memory capacity of the task allocation memory block, inserting the to-be-inserted linked list header into the header link; The task allocation memory block is linked to the header of the linked list to be inserted.

3. The method according to claim 2, characterized in that The linked list header includes an identifier representing the memory capacity of the linked memory block, and the adding the remaining memory blocks of the first target allocated memory block to the target sub-memory pool includes: Determine a first allocated memory from the remaining memory block based on the identifier, wherein a memory capacity of the first allocated memory is a memory capacity corresponding to a first identifier in the target sub-memory pool that is closest to a memory capacity of the remaining memory block, and the memory capacity of the first allocated memory is less than or equal to a memory capacity of the remaining memory block; Determine a first remaining memory block based on the remaining memory block and the first allocated memory; Linking the first allocated memory to a linked list corresponding to the first identifier; In a case where the memory capacity of the first remaining memory block is greater than or equal to the memory capacity corresponding to the second identifier, determining a second allocated memory block from the first remaining memory block based on the second identifier; Link the second allocated memory block to the linked list corresponding to the second identifier.

4. The method according to claim 1, characterized in that: The configuration information of the database operating system includes a preset page capacity, and extracting a task allocation memory block corresponding to the memory requirement from the first target allocation memory block and allocating it to the target task includes: Extracting the task allocation memory block from the first target allocation memory block based on the preset page capacity and the memory requirement, wherein the memory capacity of the task allocation memory block is greater than or equal to the memory requirement and is an integer multiple of the preset page capacity; Allocate the task allocation memory block to the target task.

5. The method according to claim 1, characterized in that The method further comprises: In a case where the memory demand is greater than the preset allocation threshold, determining a second target allocation memory block from the global memory pool based on the memory demand; Allocate the second target allocated memory block to the target task.

6. The method according to claim 1, characterized in that The method further comprises: After the target task is completed, the task allocated memory block is released to the sub-memory pool of the target thread.

7. The method according to claim 5, characterized in that The method further comprises: After the target task is completed, the second target allocated memory block is released to the global memory pool.

8. The method according to claim 6, characterized in that The step of releasing the task allocation memory block to the sub-memory pool of the target thread includes: Determine the target linked list based on the memory capacity of the task allocation memory block; The task allocation memory block is linked to the target linked list.

9. A database memory allocation device, characterized in that: The device is installed in a database operating system, the database operating system includes multiple threads, each thread is allocated with a sub-memory pool, and the device includes: an acquisition module, configured to acquire, when the memory requirement of the target task is less than a preset allocation threshold, a target sub-memory pool allocated to a target thread for executing the target task, wherein the target sub-memory pool is composed of a plurality of linked lists, each of the linked lists comprising a plurality of memory blocks with the same memory capacity, and the memory capacities of the memory blocks linked by the plurality of linked lists are different; A first target allocated memory block acquisition module, configured to acquire a first target allocated memory block from a global memory pool when the memory demand is greater than the memory capacity of the largest memory block in the plurality of linked lists, wherein the memory capacity of the first target allocated memory block is greater than or equal to the memory demand; An extraction module, configured to extract a task allocation memory block corresponding to the memory requirement from the first target allocation memory block, and allocate the task allocation memory block to the target task; A linked list adding module is used to add the remaining memory blocks of the first target allocated memory blocks to the target sub-memory pool.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images