Memory management method, system, electronic device and storage medium

Abstract

One or more embodiments of this specification provide a memory management method, system, electronic device and storage medium. The memory management method is applied to a database instance; the method includes: in response to a memory expansion request initiated by any one of a plurality of processes corresponding to the database instance, loading a first target memory space for each of the plurality of processes, and prohibiting each process from accessing the first target memory space; in response to completing the loading of the first target memory space for each process, allowing each process to access the first target memory space. In the above manner, in the process of online memory expansion or reduction for a database instance corresponding to a database adopting a multi-process architecture, abnormal memory usage of each process can be avoided.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of cloud database technology, and in particular, to a memory management method, system, electronic device, and storage medium, as well as to another memory management method. Background Art

[0002] With the advent of the cloud-native era, cloud database technology is also evolving. Cloud databases are databases optimized or deployed in virtualized computing environments, offering benefits such as pay-as-you-go, on-demand scalability, high availability, and storage consolidation.

[0003] Through online database services based on cloud database technology, users can create and modify instances, migrate and synchronize data, back up and restore data, and perform monitoring and alerting. An instance, or database instance, is a separate database service process that occupies physical memory. Different memory sizes, disk space requirements, and database types can be configured for each instance.

[0004] In actual applications, since the memory specification set for an instance usually affects the performance of the instance, the memory specification of the instance can be adjusted online as needed.

[0005] In related technologies, during the online memory expansion / reduction process for a database instance corresponding to a database that adopts a multi-process architecture, the multiple processes corresponding to the database instance usually independently load the memory space for expansion, or independently unload the memory space that needs to be reduced. If the instance is not restarted, it is difficult to ensure the state consistency between the multiple processes, and abnormal memory space usage will occur; if the instance is restarted, the database service will be interrupted. Summary of the Invention

[0006] This application provides a memory management method, applied to a database instance; the method includes:

[0007] In response to a memory expansion request initiated by any one of a plurality of processes corresponding to the database instance, loading a first target memory space for each of the plurality of processes, and prohibiting each process from accessing the first target memory space;

[0008] In response to completing the loading of the first target memory space for each process, each process is allowed to access the first target memory space.

[0009] This application also provides another memory management method, which is applied to a database instance; the method includes:

[0010] In response to a memory reduction request initiated by any one of a plurality of processes corresponding to the database instance, prohibiting each of the plurality of processes from accessing a target memory space; wherein the target memory space has been loaded for each of the plurality of processes;

[0011] In response to the completion of prohibiting each process from accessing the target memory space, the target memory space is unloaded for each process respectively.

[0012] The present application also provides a memory management device, the device comprising:

[0013] a loading unit, configured to, in response to a memory expansion request initiated by any one of a plurality of processes corresponding to the database instance, load a first target memory space for each of the plurality of processes;

[0014] a first configuration unit, configured to prohibit each process from accessing the first target memory space;

[0015] The first configuration unit is further configured to allow each process to access the first target memory space in response to completing loading of the first target memory space for each process.

[0016] The present application also provides another memory management device, the device comprising:

[0017] a configuration unit configured to, in response to a memory reduction request initiated by any one of a plurality of processes corresponding to the database instance, prohibit each of the plurality of processes from accessing a target memory space; wherein the target memory space has been loaded for each of the plurality of processes;

[0018] The unloading unit is configured to unload the target memory space for each process in response to completion of prohibiting each process from accessing the target memory space.

[0019] The present application also provides a memory management system, which is used to implement any of the above memory management methods.

[0020] The present application also provides an electronic device, comprising a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus;

[0021] The memory stores machine-readable instructions, and the processor executes any of the above-mentioned memory management methods by calling the machine-readable instructions.

[0022] The present application also provides a machine-readable storage medium, which stores machine-readable instructions. When the machine-readable instructions are called and executed by a processor, they implement any of the above-mentioned memory management methods.

[0023] Through the above embodiments, the process of online memory expansion for the database instance corresponding to the database adopting a multi-process architecture is divided into two stages: "loading memory" and "allowing access to the newly expanded memory". Since in the first stage "loading memory", the first target memory space can be loaded for each process separately, and each process can be prohibited from accessing the first target memory space, the state consistency of each process when the second stage "allowing access to the newly expanded memory" starts to be executed does not need to be restarted. Based on this, in the second stage "allowing access to the newly expanded memory", regardless of whether each process synchronously expands the size of the virtual address space managed by each process, since the loading of the first target memory space has been completed for each process, abnormal memory usage can be avoided during the process of memory capacity increase.

[0024] Similarly, by dividing the process of online memory shrinking for the database instance corresponding to the database adopting a multi-process architecture into two stages, "prohibiting access to the memory to be shrunk" and "unloading memory", since in the first stage "prohibiting access to the memory to be shrunk", each process can be prohibited from accessing the target memory space, and the target memory space has not been unloaded for each process, therefore, regardless of whether each process synchronously reduces the size of the virtual address space managed by each process, no memory usage abnormality will occur during the process of memory capacity expansion; and then in the second stage "unloading memory", since each process is prohibited from accessing the target memory space, the target memory space can be unloaded for each process separately, and there is no need to restart the database instance, so the state consistency of each process can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 This is a schematic diagram of memory expansion for different processes shown in the related art;

[0027] Figure 2 This is a schematic diagram of memory reduction for different processes shown in the related art;

[0028] Figure 3 is a schematic diagram of an online memory expansion method shown in an exemplary embodiment;

[0029] Figure 4is a schematic diagram of an online memory shrinking method shown in an exemplary embodiment;

[0030] Figure 5 is a flowchart of a memory management method shown in an exemplary embodiment;

[0031] Figure 6 is a flowchart of another memory management method shown in an exemplary embodiment;

[0032] Figure 7 This is a schematic structural diagram of an electronic device in which a memory management device is located, showing an exemplary embodiment;

[0033] Figure 8 is a block diagram of a memory management device according to an exemplary embodiment;

[0034] Figure 9 is a block diagram showing another memory management device according to an exemplary embodiment. DETAILED DESCRIPTION

[0035] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative work should fall within the scope of protection of this specification.

[0036] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0037] With the advent of the cloud-native era, cloud database technology is also evolving. Cloud databases are databases optimized or deployed in virtualized computing environments, offering benefits such as pay-as-you-go, on-demand scalability, high availability, and storage consolidation.

[0038] Through the online database service provided based on cloud database technology, users can realize functions such as instance creation and modification, data migration and synchronization, data backup and recovery, monitoring and alarm.

[0039] The instance, ie, database instance, may be an independent database service process that occupies physical memory, and different memory sizes, disk spaces, and database types may be set for the instance.

[0040] In practice, since the memory specification set for an instance often affects its performance, the instance's memory specification can be adjusted online as needed. Specifically, users can modify instance configuration parameters to adjust the instance's memory specification as needed; alternatively, the service process corresponding to the instance can automatically determine the need for memory expansion according to a preset memory management policy, although this specification does not specify this.

[0041] For example, when performing online memory expansion for a PostgreSQL database instance, you can modify the configuration parameters related to the memory specifications of the instance, and then immediately restart the PostgreSQL database instance to make the modified configuration parameters take effect, thereby completing the online memory expansion for the PostgreSQL database instance.

[0042] It can be seen that in the embodiment shown above, during the process of performing online memory expansion / reduction for the PostgreSQL database instance, the PostgreSQL database instance needs to be restarted, which will cause the database connection to be interrupted; that is, during the process of restarting the PostgreSQL database instance, the PostgreSQL database instance cannot continue to provide database services normally.

[0043] It should be noted that since the PostgreSQL database is a database that adopts a multi-process architecture, it is difficult to ensure the state consistency between multiple processes without restarting the database instance so that the multiple processes corresponding to the PostgreSQL database instance can synchronously complete memory expansion / contraction.

[0044] For example, see Figure 1 , Figure 1 This is a schematic diagram of memory expansion for different processes shown in the related art. Figure 1As shown, the multiple processes corresponding to the PostgreSQL database instance may include at least process A and process B; wherein, process A and process B can share physical memory 11, the initial memory capacity of physical memory 11 is NBuffers1, and the sizes of virtual address space 21 and virtual address space 31 independently allocated and managed by process A and process B are also NBuffers1. Furthermore, if process A initiates an online memory expansion request and applies for shared physical memory 12 for expansion, and process A has already loaded the memory space corresponding to physical memory 12, the size of process A's virtual address space (i.e., virtual address space 21 + virtual address space 22) will have been expanded to NBuffers2; however, process B may not have loaded the memory space corresponding to physical memory 12, and the size of process B's virtual address space (i.e., virtual address space 31) will still be NBuffers1.

[0045] In this case, if the expanded process A writes data in the newly added physical memory 12 and updates the corresponding index information, and the non-expanded process B attempts to read the data based on the updated index information, a read failure will occur (such as a null pointer exception, an illegal address, etc.).

[0046] Similarly, see Figure 2 , Figure 2 This is a schematic diagram of memory shrinking for different processes shown in the related art. Figure 2 As shown, the multiple processes corresponding to the PostgreSQL database instance may include at least process A and process B; wherein, process A and process B may share physical memory (i.e., physical memory 11+physical memory 12+physical memory 13), the initial memory capacity of the physical memory is NBuffers3, the size of the virtual address space of process A (i.e., virtual address space 21+virtual address space 22+virtual address space 23), and the size of the virtual address space of process B (i.e., virtual address space 31+virtual address space 32+virtual address space 33) are also NBuffers3. Furthermore, if process B initiates an online memory reduction request, and process B has unloaded the memory space corresponding to the shared physical memory 13 that needs to be reduced, the size of process B's virtual address space (that is, virtual address space 31 + virtual address space 32) has been reduced to NBuffers2; while process A may not have yet unloaded the memory space corresponding to the shared physical memory 13 that needs to be reduced, and the size of process A's virtual address space (that is, virtual address space 21 + virtual address space 22 + virtual address space 23) is still NBuffers3.

[0047] In this case, if process A, which has not yet been shrunk, writes data in physical memory 13 and updates the corresponding index information, and process B, which has been shrunk, attempts to read the data based on the updated index information, a read failure will occur (such as a null pointer exception, an illegal address pointed to, etc.).

[0048] It can be seen that in one or more of the embodiments shown above, during the process of online memory expansion / contraction of the PostgreSQL database instance, multiple processes corresponding to the PostgreSQL database instance usually manage the memory space of each process independently. If the instance is not restarted, it is difficult to ensure the state consistency between the multiple processes, which will lead to abnormal usage of shared physical memory.

[0049] In view of this, this specification aims to propose a technical solution that divides the process of online memory management for a database instance into two stages to ensure state consistency among multiple processes corresponding to the database instance.

[0050] Specifically, the process of online memory expansion can be divided into two stages: "loading memory and prohibiting access" and "allowing access to the newly expanded memory". After multiple processes corresponding to the database embodiment have completed the first stage of "loading memory and prohibiting access", each of the multiple processes will start to execute the second stage of "allowing access to the newly expanded memory" to complete the online memory expansion of each process. In response to the completion of the online memory expansion by each process, it can be considered that the database instance has completed the online memory expansion.

[0051] During implementation, in response to a memory expansion request initiated by any one of the multiple processes corresponding to the database instance, the first target memory space can be loaded for each of the multiple processes respectively, and each process can be prohibited from accessing the first target memory space; further, in response to completing the loading of the first target memory space for each process, each process can be allowed to access the first target memory space.

[0052] For example, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an online memory expansion method according to an exemplary embodiment. Figure 3 As shown, the multiple processes corresponding to the PostgreSQL database instance may include at least process A and process B; wherein, process A and process B can share physical memory 11, the initial memory capacity of physical memory 11 is NBuffers1, and the sizes of virtual address spaces 21 and virtual address spaces 31 independently allocated and managed by process A and process B are also NBuffers1.

[0053] In response to the online memory expansion request initiated by process A, in the first stage of "loading memory and prohibiting access", the first target memory space (that is, virtual address space 22 and virtual address space 32) corresponding to the physical memory 12 used for expansion can be loaded for process A and process B respectively, and process A can be prohibited from accessing the virtual address space 22, and process B can be prohibited from accessing the virtual address space 32; since the size of the virtual address space (that is, virtual address space 21) allowed to be accessed by process A is still NBuffers1, and the size of the virtual address space (that is, virtual address space 31) allowed to be accessed by process B is also still NBuffers1, therefore, before the first target memory space corresponding to the physical memory 12 used for expansion is loaded for each process, each process cannot write data in the physical memory 12.

[0054] Furthermore, in response to completing the loading of virtual address space 22 for process A and completing the loading of virtual address space 32 for process B, entering the second stage "allowing access to newly expanded memory", process A can be allowed to access virtual address space 22, and process B can be allowed to access virtual address space 32; based on this, the size of the virtual address space allowed to be accessed by process A (that is, virtual address space 21 + virtual address space 22) can be expanded to NBuffers2, and the size of the virtual address space allowed to be accessed by process B (that is, virtual address space 31 + virtual address space 32) can also be expanded to NBuffers2.

[0055] It should be noted that in this case, regardless of whether each process is allowed to access the first target memory space synchronously, that is, regardless of whether each process synchronously expands the size of the virtual address space it manages, memory usage abnormalities will not occur during the process of memory capacity increase.

[0056] For example, in response to the completion of loading the memory space corresponding to the physical memory 12 used for expansion for both process A and process B, process A can be allowed to access the virtual address space 22 first. The size of the virtual address space of process A that has completed expansion (that is, virtual address space 21 + virtual address space 22) has been expanded to NBuffers2, while the size of the virtual address space allowed to be accessed by process B that has not completed expansion (that is, virtual address space 31) is still NBuffers1; in this case, if process A that has completed expansion writes data in the newly added physical memory and updates the index information, since process B has completed loading the memory space corresponding to physical memory 12, process B can successfully read the data.

[0057] It can be seen that in the embodiment shown above, by dividing the process of online memory expansion for the database instance corresponding to the database adopting a multi-process architecture into two stages, "loading memory" and "allowing access to the newly expanded memory", since in the first stage "loading memory", the first target memory space can be loaded for each process separately, and each process can be prohibited from accessing the first target memory space, there is no need to restart the database instance, and the state consistency of each process when the second stage "allowing access to the newly expanded memory" starts to be executed can be guaranteed; based on this, in the second stage "allowing access to the newly expanded memory", regardless of whether each process synchronously expands the size of the virtual address space managed by each process, since the loading of the first target memory space has been completed for each process, it is possible to avoid abnormal memory usage during the memory capacity increase process.

[0058] Based on similar technical concepts, the process of online memory shrinking can be divided into two stages: "prohibiting access to the memory to be shrunk" and "unloading memory". After multiple processes corresponding to the database embodiment complete the first stage of "prohibiting access to the memory to be shrunk", each of the multiple processes begins to execute the second stage of "unloading memory" to complete the online memory shrinking of each process. In response to the completion of the online memory shrinking by each process, it can be considered that the database instance has completed the online memory shrinking.

[0059] During implementation, in response to a memory reduction request initiated by any one of the multiple processes corresponding to the database instance, each of the multiple processes can be prohibited from accessing the target memory space; wherein, the target memory space has been loaded for each of the multiple processes; further, in response to the completion of prohibiting each process from accessing the target memory space, the target memory space can be unloaded for each process separately to complete the online memory reduction of each process; in response to each process completing the online memory reduction, it can be considered that the database instance has completed the online memory reduction.

[0060] For example, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram showing an online memory shrinking method according to an exemplary embodiment. Figure 4As shown, the multiple processes corresponding to the PostgreSQL database instance may include at least process A and process B; wherein, process A and process B may share physical memory (i.e., physical memory 11+physical memory 12+physical memory 13), the initial memory capacity of the physical memory is NBuffers3, the size of the virtual address space of process A (i.e., virtual address space 21+virtual address space 22+virtual address space 23), and the size of the virtual address space of process B (i.e., virtual address space 31+virtual address space 32+virtual address space 33) are also NBuffers3.

[0061] In response to the memory shrinkage request initiated by process B, in the first stage of "prohibiting access to the memory to be shrunk", process A and process B can be prohibited from accessing the memory space corresponding to the shared physical memory 13 that needs to be shrunk; that is, the size of the virtual address space allowed to be accessed by process B (that is, virtual address space 31 + virtual address space 32) can be shrunk to NBuffers2, and the size of the virtual address space allowed to be accessed by process A (that is, virtual address space 21 + virtual address space 22) can also be shrunk to NBuffers2.

[0062] It should be noted that, in this case, regardless of whether each process synchronously reduces the size of the virtual address space it manages, abnormal memory usage will not occur during the process of memory capacity rebound.

[0063] For example, in response to a memory reduction request initiated by process B, process B can be first prohibited from accessing the memory space (i.e., virtual address space 33) corresponding to the shared physical memory 13 that needs to be reduced. The size of the virtual address space allowed to be accessed by process B (i.e., virtual address space 31 + virtual address space 32) has been reduced to NBuffers2, while the size of the virtual address space allowed to be accessed by process A (i.e., virtual address space 21 + virtual address space 22 + virtual address space 23) may still be NBuffers3; in this case, if process A writes data in the physical memory 13 through the virtual address space 23 and updates the corresponding index information, since the memory space corresponding to the physical memory 13 has not been actually unloaded for process B, it will not cause a memory read exception for process B.

[0064] Furthermore, in response to the completion of prohibiting process A from accessing the virtual address space 23 and prohibiting process B from accessing the virtual address space 33, the memory space corresponding to the physical memory 13 can be unloaded for process A and process B respectively, that is, the virtual address space 23 of process A can be released, the virtual address space 33 of process B can be released, and the physical memory 13 can be released to complete the online memory reduction of the PostgreSQL database instance.

[0065] It can be seen that in the embodiment shown above, the process of online memory shrinking for the database instance corresponding to the database adopting a multi-process architecture is divided into two stages: "prohibiting access to the memory to be shrunk" and "unloading memory". Since in the first stage "prohibiting access to the memory to be shrunk", each process can be prohibited from accessing the target memory space, and the target memory space has not been unloaded for each process, therefore, regardless of whether each process synchronously reduces the size of the virtual address space managed by each process, no memory usage abnormality will occur during the process of memory capacity increase; and then in the second stage "unloading memory", since each process is prohibited from accessing the target memory space, the target memory space can be unloaded for each process separately, and there is no need to restart the database instance, so the state consistency of each process can be guaranteed.

[0066] The present application is described below through specific embodiments and in combination with specific application scenarios.

[0067] See Figure 5 , Figure 5 This is a flowchart of a memory management method shown in an exemplary embodiment.

[0068] In this specification, the memory management method can be applied to a database instance. The database instance can be a database instance corresponding to a database using a multi-process architecture; the database using a multi-process architecture can specifically include but is not limited to databases such as PostgreSQL and Oracle.

[0069] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of this specification, one or more embodiments shown in this specification are described below using the database instance being a PostgreSQL database instance (that is, the database using a multi-process architecture is a PostgreSQL database) as an example. It should be noted that this does not represent a particular limitation on this specification.

[0070] In this specification, the online memory management may include online memory expansion and / or online memory reduction. The following description will be divided into two parts: online memory expansion and online memory reduction.

[0071] (1) Online memory expansion

[0072] like Figure 5 As shown, the method can perform the following steps:

[0073] Step 502: In response to a memory expansion request initiated by any process among a plurality of processes corresponding to the database instance, a first target memory space is loaded for each of the plurality of processes, and each process is prohibited from accessing the first target memory space.

[0074] For example, Figure 3 As shown, in response to a memory expansion request initiated by process A corresponding to the PostgreSQL database instance, first target memory spaces (i.e., virtual address space 22 and virtual address space 32) corresponding to the physical memory 12 for expansion may be loaded for process A and process B, respectively. Process A may be prohibited from accessing virtual address space 22, and process B may be prohibited from accessing virtual address space 32. The size of the virtual address space allowed to be accessed by process A (i.e., virtual address space 21) remains NBuffers1, and the size of the virtual address space allowed to be accessed by process B (i.e., virtual address space 31) also remains NBuffers1.

[0075] It should be noted that, in the above-described embodiments, the specific implementation method for loading the first target memory space for each process is not particularly limited in this specification. For example, the first target memory space can be loaded for each process separately through functions such as shmat and mmap.

[0076] In step 502, the capacity of the first target memory space may be specified by the user or automatically determined by the online database service.

[0077] For example, the user can modify the configuration parameters related to the memory specifications of process A in the instance parameter configuration interface for the database instance, and can submit the modified configuration parameters to the online database service; in response to the modified configuration parameters related to the memory specifications of process A submitted by the user, it can be considered that process A corresponding to the PostgreSQL database instance has initiated a memory expansion request, and the capacity of the first target memory space can also be determined based on the modified configuration parameters indicated in the memory expansion request.

[0078] For another example, in response to a memory expansion request initiated by process A corresponding to the PostgreSQL database instance, the online database service can automatically determine the capacity of the first target memory space based on instance parameters such as the memory specifications and reserved memory capacity of the PostgreSQL database instance, and can also combine with a pre-configured online memory expansion policy.

[0079] It should be noted that in one or more embodiments described in this specification, the storage type of the memory resources corresponding to the memory space is not specifically limited. For example, the storage type may include, but is not limited to, a local SSD (Solid State Disk) or an SSD cloud disk. The local SSD may be an SSD located on the same node as the database engine running the database. The SSD cloud disk may be an elastic block storage device based on a distributed storage architecture.

[0080] In one embodiment shown, in step 502, loading the first target memory space for each of the multiple processes may specifically include: allocating a virtual address space corresponding to the first target memory space for each of the multiple processes, and applying for a shared physical address space corresponding to the first target memory space for the multiple processes; mapping the applied shared physical address space to the virtual address space allocated for each process, so as to complete the loading of the first target memory space for each process.

[0081] For example, Figure 3 As shown, in response to a memory expansion request initiated by process A corresponding to the PostgreSQL database instance, a virtual address space 22 corresponding to the physical memory 12 can be allocated to process A, and a virtual address space 32 corresponding to the physical memory 12 can be allocated to process B, and a shared physical address space 12' corresponding to the physical memory 12 can be applied for process A and process B; further, the applied shared physical address space 12' can be mapped to the virtual address space 22 of process A to complete the loading of the first target memory space for process A; and the applied shared physical address space 12' can be mapped to the virtual address space 32 of process B to complete the loading of the first target memory space for process B.

[0082] In the embodiment shown above, the shared physical address space may specifically include a Buffer Pool; in this case, mapping the shared physical address space to a virtual address space allocated to each process may specifically include: creating an idle cache page in the applied Buffer Pool; and mapping the cache page to a virtual address space allocated to each process.

[0083] Among them, the Buffer Pool is a mechanism that can reduce disk access; due to the principle of locality of data access, when accessing a part of data, it is highly likely that nearby data will also be used, and disk reading and writing are usually read by page. Therefore, the data page corresponding to the disk file can be cached in the cache page created in the Buffer Pool, and the description information of the cache page can be recorded in the Buffer Pool. If the data to be accessed subsequently has been cached in the cache page, disk access can be reduced and data reading efficiency can be improved.

[0084] In a paged virtual memory, both the virtual storage space and the physical storage space can usually be divided into several pages of fixed size, and storage space is allocated to each thread in units of pages.

[0085] For example, Figure 3 As shown, the first target memory space may be a memory space corresponding to the physical memory 12; in response to a memory expansion request initiated by process A corresponding to the PostgreSQL database instance, a Buffer Pool (i.e., a shared physical address space 12') corresponding to the physical memory 12 may be applied for process A and process B, and an idle cache page buffer_page_1 may be created in the applied Buffer Pool; and a virtual address space 22 corresponding to the physical memory 12 may be allocated to process A, and a virtual address space 32 corresponding to the physical memory 12 may be allocated to process B; further, the cache page buffer_page_1 may be mapped to the virtual address space 22 of process A and the virtual address space 32 of process B, respectively.

[0086] It should be noted that in the implementation shown above, for the specific implementation of creating buffer pages in the Buffer Pool, please refer to the relevant technology and will not be described in detail here. For example, the data structure of the Buffer Pool may include buffer meta information, buffer data information, and a buffer hash table; wherein the buffer meta information is the description information of the cache page, the buffer data information is the data stored in the cache page, and the buffer hash table is the index information of the data stored in the cache page.

[0087] Step 504: In response to completing the loading of the first target memory space for each process, each process is allowed to access the first target memory space.

[0088] For example, Figure 3As shown, in response to completing the loading of the memory space corresponding to the physical memory 12 used for expansion for both process A and process B, process A can be allowed to access the virtual address space 22, and process B can be allowed to access the virtual address space 32; wherein, the size of the virtual address space allowed to be accessed by process A (that is, virtual address space 21 + virtual address space 22) can be expanded to NBuffers2, and the size of the virtual address space allowed to be accessed by process B (that is, virtual address space 31 + virtual address space 32) can also be expanded to NBuffers2.

[0089] (2) Online memory shrinking

[0090] The method may further comprise:

[0091] Step 506: In response to a memory reduction request initiated by any of the multiple processes, prohibit each process from accessing a second target memory space; wherein the second target memory space has been loaded for each of the multiple processes;

[0092] Step 508: In response to the completion of prohibiting each process from accessing the second target memory space, the second target memory space is unloaded for each process.

[0093] For example, Figure 4 As shown, the second target memory space can be a memory space that has been loaded for process A and process B respectively and corresponds to the shared physical memory 13 used for shrinking; in response to the memory shrinking request initiated by process B, process A and process B can be prohibited from accessing the memory space corresponding to the shared physical memory 13 that needs to be shrunk; that is, the size of the virtual address space allowed to be accessed by process B (that is, virtual address space 31 + virtual address space 32) can be reduced to NBuffers2, and the size of the virtual address space allowed to be accessed by process A (that is, virtual address space 21 + virtual address space 22) can also be reduced to NBuffers2. Further, in response to the prohibition of process A from accessing virtual address space 23 and the prohibition of process B from accessing virtual address space 33, the memory space corresponding to the physical memory 13 can be unloaded for process A and process B respectively.

[0094] It should be noted that, in step 502 and step 506, any one process may be the same process among the multiple processes, or may be a different process among the multiple processes.

[0095] Among them, in step 506, the capacity of the second target memory space can be specified by the user or automatically determined by the online database service. For the specific determination method, please refer to the relevant embodiment of step 502, which will not be repeated here.

[0096] In one embodiment shown, in step 508, unloading the second target memory space for each process respectively may specifically include: determining the target shared physical address space corresponding to the second target memory space in the shared physical address space of the multiple processes; and determining the target virtual address space of each process to which the target shared physical address space is mapped in the virtual address space of each process; deleting the mapping between the target shared physical address space and the target virtual address space of each process respectively; releasing the target virtual address space of each process; and releasing the target shared physical address space in response to the completion of the release of the target virtual address space of each process.

[0097] For example, Figure 4 As shown, in response to the completion of prohibiting process A and process B from accessing the memory space corresponding to the physical memory 13, the target shared physical address space 13' corresponding to the physical memory 13 can be determined in the shared physical address space of process A and process B; and, the target virtual address space 23 of process A to which the target shared physical address space 13' is mapped can be determined in the virtual address space of process A; and, the target virtual address space 33 of process B to which the target shared physical address space 13' is mapped can be determined in the virtual address space of process B; further, the mapping between the target shared physical address space 13' and the target virtual address space 23 of process A can be deleted, and the mapping between the target shared physical address space 13' and the target virtual address space 33 of process B can be deleted; further, the target virtual address space 23 of process A and the target virtual address space 33 of process B can be released; further, in response to the completion of the release of the target virtual address space 23 of process A and the target virtual address space 33 of process B, the target shared physical address space 13' can be released.

[0098] It should be noted that, in the above embodiments, the specific implementation method of unloading the second target memory space for each process is not particularly limited in this specification. For example, the second target memory space can be unloaded for each process using functions such as shmdt and free.

[0099] In the embodiment shown above, the shared physical address space may specifically include a buffer pool BufferPool; in this case, determining the target shared physical address space corresponding to the second target memory space in the shared physical address space of the multiple processes may specifically include: determining the target cache page corresponding to the second target memory space in the created cache pages included in the Buffer Pool.

[0100] In this case, determining, in the virtual address space of each process, the target virtual address space of each process to which the target shared physical address space is mapped, may specifically include: determining, in the virtual address space of each process, the target virtual address space of each process to which the target cache page is mapped.

[0101] In this case, respectively deleting the mappings between the target shared physical address space and the target virtual address space of each process may specifically include: respectively deleting the mappings between the target cache page and the target virtual address space of each process.

[0102] For example, Figure 4 As shown, in response to the completion of prohibiting process A and process B from accessing the memory space corresponding to the physical memory 13, the target cache page buffer_page_2 corresponding to the physical memory 13 can be determined from the several cache pages created included in the Buffer Pool; and, the target virtual address space 23 of process A to which the target cache page buffer_page_2 is mapped can be determined in the virtual address space of process A; and, the target virtual address space 33 of process B to which the target cache page buffer_page_2 is mapped can be determined in the virtual address space of process B; further, the mapping between the target cache page buffer_page_2 and the target virtual address space 23 of process A can be deleted, and the mapping between the target cache page buffer_page_2 and the target virtual address space 33 of process B can be deleted.

[0103] In one embodiment shown, the database adopting a multi-process architecture may specifically include a PostgreSQL database; the database instance may specifically include a PostgreSQL database instance; in this case, the multiple processes corresponding to the PostgreSQL database instance may include a management process corresponding to the PostgreSQL database instance; and a child process created by the management process as a service process.

[0104] For example, the multiple processes corresponding to the PostgreSQL database instance may specifically include a postmaster process as a management process and one or more user_backend processes as service processes; wherein each user_backend process may be a child process created by the postmaster process through a fork function.

[0105] In this case, the "any process" recorded in step 502 and step 506 may be a postmaster process or any user_backend process corresponding to the PostgreSQL database instance.

[0106] It should be noted that, in the implementation shown above, when memory management is performed for the PostgreSQL database instance, as long as any of the existing multiple processes initiates a memory expansion request or a memory reduction request, the existing multiple processes will complete the online memory expansion or online memory reduction; since the postmaster process as the management process has completed the memory expansion or memory reduction, the new child process as the service process subsequently created and managed by the postmaster process through the fork function can automatically "inherit" the expanded or reduced memory space without the need for additional configuration, thereby improving the efficiency of memory management and improving the user experience of online database services.

[0107] It can be seen from the above technical solution that by dividing the process of online memory expansion for the database instance corresponding to the database adopting a multi-process architecture into two stages, "loading memory" and "allowing access to the newly expanded memory", since in the first stage "loading memory", the first target memory space can be loaded for each process separately, and each process can be prohibited from accessing the first target memory space, there is no need to restart the database instance, and the state consistency of each process when the second stage "allowing access to the newly expanded memory" starts to be executed can be guaranteed; based on this, in the second stage "allowing access to the newly expanded memory", regardless of whether each process synchronously expands the size of the virtual address space managed by each process, since the loading of the first target memory space has been completed for each process, abnormal memory usage can be avoided during the memory capacity increase process.

[0108] See Figure 6 , Figure 6 This is a flowchart illustrating another memory management method according to an exemplary embodiment. The memory management method can be applied to a database instance. The database instance can be a database instance corresponding to a database using a multi-process architecture.

[0109] like Figure 6 As shown, the memory management method can perform the following steps:

[0110] Step 602: In response to a memory reduction request initiated by any one of a plurality of processes corresponding to the database instance, prohibiting each of the plurality of processes from accessing a target memory space; wherein the target memory space has been loaded for each of the plurality of processes;

[0111] Step 604: In response to the completion of prohibiting each process from accessing the target memory space, the target memory space is unloaded for each process.

[0112] In one embodiment shown, in step 604, unloading the target memory space for each process separately may specifically include: determining a target shared physical address space corresponding to the target memory space in the shared physical address space of the multiple processes; and determining a target virtual address space of each process to which the target shared physical address space is mapped in the virtual address space of each process; deleting the mapping between the target shared physical address space and the target virtual address space of each process separately; releasing the target virtual address space of each process; and releasing the target shared physical address space in response to completion of the release of the target virtual address space of each process.

[0113] In the above-described implementation manner, the shared physical address space may specifically include a Buffer Pool.

[0114] In this case, determining the target shared physical address space corresponding to the target memory space in the shared physical address space of the multiple processes may specifically include: determining the target cache page corresponding to the target memory space in the created cache pages included in the Buffer Pool.

[0115] In this case, determining, in the virtual address space of each process, the target virtual address space of each process to which the target shared physical address space is mapped, may specifically include: determining, in the virtual address space of each process, the target virtual address space of each process to which the target cache page is mapped.

[0116] In this case, respectively deleting the mappings between the target shared physical address space and the target virtual address space of each process may specifically include: respectively deleting the mappings between the target cache page and the target virtual address space of each process.

[0117] In one embodiment shown, the database adopting a multi-process architecture may specifically include a PostgreSQL database; the database instance may specifically include a PostgreSQL database instance; in this case, the multiple processes corresponding to the PostgreSQL database instance may include a management process corresponding to the PostgreSQL database instance; and a child process created by the management process as a service process.

[0118] In this specification, the specific implementation method of steps 602 to 604 is similar to that of steps 506 to 508 and will not be repeated here; wherein, the "target memory space" recorded in step 602 or step 604 is also the "second target memory space" recorded in step 506 or step 508.

[0119] Through the above technical solution, it can be seen that by dividing the process of online memory shrinking for the database instance corresponding to the database adopting a multi-process architecture into two stages, "prohibiting access to the memory to be shrunk" and "unloading memory", since in the first stage "prohibiting access to the memory to be shrunk", each process can be prohibited from accessing the target memory space, and the target memory space has not been unloaded for each process, therefore, regardless of whether each process synchronously reduces the size of the virtual address space managed by each process, no memory usage abnormality will occur during the process of memory capacity increase; and then in the second stage "unloading memory", since each process is prohibited from accessing the target memory space, the target memory space can be unloaded for each process separately, and there is no need to restart the database instance, so the state consistency of each process can be guaranteed.

[0120] Corresponding to the embodiment of the memory management method, this specification also provides an embodiment of a memory management system and an embodiment of a memory management device.

[0121] In this specification, the memory management system may execute steps 502 to 508, or the memory management system may execute steps 602 to 604 to implement the technical solution of this specification.

[0122] See Figure 7 , Figure 7 This is a hardware structure diagram of an electronic device in which a memory management device is located, which is shown as an exemplary embodiment. At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, a memory 708, and a non-volatile memory 710, and of course may also include hardware required for other services. One or more embodiments of this specification can be implemented based on software, such as the processor 702 reading the corresponding computer program from the non-volatile memory 710 into the memory 708 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logical devices or a combination of hardware and software, etc., that is, the execution subject of the following processing flow is not limited to each logical unit, but can also be hardware or logical devices.

[0123] See Figure 8 , Figure 8FIG. 1 is a block diagram of a memory management device according to an exemplary embodiment. The memory management device can be applied to Figure 7 In the electronic device shown in FIG. , the technical solution of this specification is implemented. The memory management device may include:

[0124] A loading unit 802 is configured to, in response to a memory expansion request initiated by any one of a plurality of processes corresponding to the database instance, load a first target memory space for each of the plurality of processes;

[0125] A first configuration unit 804 is configured to prohibit each process from accessing the first target memory space;

[0126] The first configuration unit 804 is further configured to allow each process to access the first target memory space in response to completing loading of the first target memory space for each process.

[0127] In this embodiment, the loading unit 802 is specifically configured to:

[0128] Allocating a virtual address space corresponding to the first target memory space for each of the multiple processes, and applying for a shared physical address space corresponding to the first target memory space for the multiple processes;

[0129] The shared physical address space applied for is mapped to the virtual address space allocated to each process, so as to complete the loading of the first target memory space for each process.

[0130] In this embodiment, the shared physical address space includes a buffer pool;

[0131] The loading unit 802 is specifically configured to:

[0132] Create an idle cache page in the requested Buffer Pool;

[0133] The cache pages are mapped to virtual address spaces allocated to the respective processes.

[0134] In this embodiment, the device further includes:

[0135] a second configuration unit, configured to, in response to a memory reduction request initiated by any of the plurality of processes, prohibit each process from accessing a second target memory space; wherein the second target memory space has been loaded for each of the plurality of processes;

[0136] The unloading unit is configured to unload the second target memory space for each process in response to completion of prohibiting each process from accessing the second target memory space.

[0137] In this embodiment, the unloading unit is specifically used to:

[0138] Determining, in the shared physical address spaces of the plurality of processes, a target shared physical address space corresponding to the second target memory space; and determining, in the virtual address space of each process, a target virtual address space of each process to which the target shared physical address space is mapped;

[0139] Deleting mappings between the target shared physical address space and the target virtual address space of each process respectively;

[0140] Release the target virtual address space of each process;

[0141] In response to completion of release of the target virtual address space of each process, the target shared physical address space is released.

[0142] In this embodiment, the shared physical address space includes a buffer pool;

[0143] The unloading unit is specifically used for:

[0144] Determine, from the created cache pages included in the Buffer Pool, a target cache page corresponding to the second target memory space;

[0145] In the virtual address space of each process, determining the target virtual address space of each process to which the target cache page is mapped;

[0146] The mappings between the target cache page and the target virtual address space of each process are deleted respectively.

[0147] In this embodiment, the database instance includes a PostgreSQL database instance;

[0148] The multiple processes include a management process corresponding to the PostgreSQL database instance; and a child process created by the management process as a service process.

[0149] See Figure 9 , Figure 9 FIG. 1 is a block diagram of another memory management device according to an exemplary embodiment. The memory management device can be applied to Figure 7 In the electronic device shown in FIG. , the technical solution of this specification is implemented. The memory management device may include:

[0150] a configuration unit 902 configured to, in response to a memory reduction request initiated by any one of a plurality of processes corresponding to the database instance, prohibit each of the plurality of processes from accessing a target memory space; wherein the target memory space has been loaded for each of the plurality of processes;

[0151] The unloading unit 904 is configured to, in response to prohibiting each process among the multiple processes from accessing the target memory space, unload the target memory space for each process.

[0152] In this embodiment, the unloading unit 904 is specifically configured to:

[0153] Determining, in the shared physical address spaces of the plurality of processes, a target shared physical address space corresponding to the target memory space; and determining, in the virtual address space of each process, a target virtual address space of each process to which the target shared physical address space is mapped;

[0154] Deleting mappings between the target shared physical address space and the target virtual address space of each process respectively;

[0155] Release the target virtual address space of each process;

[0156] In response to the target virtual address space of each process being released, the target shared physical address space is released.

[0157] In this embodiment, the shared physical address space includes a buffer pool;

[0158] The unloading unit 904 is specifically configured to:

[0159] Determine, among the created cache pages included in the Buffer Pool, a target cache page corresponding to the target memory space;

[0160] In the virtual address space of each process, determining the target virtual address space of each process to which the target cache page is mapped;

[0161] The mappings between the target cache page and the target virtual address space of each process are deleted respectively.

[0162] In this embodiment, the database instance includes a PostgreSQL database instance;

[0163] The multiple processes include a management process corresponding to the PostgreSQL database instance; and a child process created by the management process as a service process.

[0164] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0165] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0166] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0167] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0168] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0169] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0170] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0171] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0172] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "an," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0173] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when..." or "in response to determining."

[0174] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A memory management method, applied to a database instance corresponding to a database in a multi-process architecture; the method comprising: In response to a memory expansion request initiated by any one of a plurality of processes corresponding to the database instance, loading a first target memory space for each of the plurality of processes, and prohibiting each process from accessing the first target memory space; In response to completing loading of the first target memory space for each process, allowing each process to access the first target memory space; The step of loading the first target memory space for each of the multiple processes includes: Allocating a virtual address space corresponding to the first target memory space for each of the multiple processes, and applying for a shared physical address space corresponding to the first target memory space for the multiple processes; Mapping the applied shared physical address space to a virtual address space allocated to each process, so as to complete loading of the first target memory space for each process; The shared physical address space includes a buffer pool; Mapping the shared physical address space to the virtual address space allocated to each process includes: Create free cache pages in the requested Buffer Pool; Mapping the cache page to a virtual address space allocated to each process; The database instance includes a PostgreSQL database instance; The multiple processes include a management process corresponding to the PostgreSQL database instance; and a child process created by the management process as a service process.

2. The method according to claim 1, further comprising: In response to a memory reduction request initiated by any of the multiple processes, prohibiting each process from accessing a second target memory space; wherein the second target memory space has been loaded for each of the multiple processes; In response to the completion of prohibiting each process from accessing the second target memory space, the second target memory space is unloaded for each process respectively.

3. The method according to claim 2, wherein the step of unloading the second target memory space for each process comprises: Determining, in the shared physical address spaces of the multiple processes, a target shared physical address space corresponding to the second target memory space; and, determining, in the virtual address space of each process, a target virtual address space of each process to which the target shared physical address space is mapped; Deleting mappings between the target shared physical address space and the target virtual address space of each process respectively; Release the target virtual address space of each process; In response to completion of release of the target virtual address space of each process, the target shared physical address space is released.

4. The method according to claim 3, wherein the shared physical address space comprises a buffer pool; Determining, in the shared physical address spaces of the multiple processes, a target shared physical address space corresponding to the second target memory space includes: Determine, from the created cache pages included in the Buffer Pool, a target cache page corresponding to the second target memory space; Determining, in the virtual address space of each process, the target virtual address space of each process to which the target shared physical address space is mapped includes: In the virtual address space of each process, determining the target virtual address space of each process to which the target cache page is mapped; Deleting the mapping between the target shared physical address space and the target virtual address space of each process respectively includes: The mappings between the target cache page and the target virtual address space of each process are deleted respectively.

5. A memory management method, applied to a database instance corresponding to a database in a multi-process architecture; the method comprising: In response to a memory reduction request initiated by any one of a plurality of processes corresponding to the database instance, prohibiting each of the plurality of processes from accessing a target memory space; wherein the target memory space has been loaded for each of the plurality of processes; In response to the completion of prohibiting each process from accessing the target memory space, unloading the target memory space for each process; The step of unloading the target memory space for each process includes: Determining, in the shared physical address spaces of the plurality of processes, a target shared physical address space corresponding to the target memory space; and determining, in the virtual address space of each process, a target virtual address space of each process to which the target shared physical address space is mapped; Deleting mappings between the target shared physical address space and the target virtual address space of each process respectively; Release the target virtual address space of each process; In response to completion of releasing the target virtual address space of each process, releasing the target shared physical address space; The shared physical address space includes a buffer pool; Determining a target shared physical address space corresponding to the target memory space in the shared physical address spaces of the multiple processes includes: Determine, among the created cache pages included in the Buffer Pool, a target cache page corresponding to the target memory space; Determining, in the virtual address space of each process, the target virtual address space of each process to which the target shared physical address space is mapped includes: In the virtual address space of each process, determining the target virtual address space of each process to which the target cache page is mapped; The respectively deleting the mapping between the target shared physical address space and the target virtual address space of each process includes: Deleting mappings between the target cache page and the target virtual address space of each process respectively; The database instance includes a PostgreSQL database instance; The multiple processes include a management process corresponding to the PostgreSQL database instance; and a child process created by the management process as a service process.

6. A memory management system, configured to implement the method according to any one of claims 1 to 5.

7. An electronic device comprising a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 5 by calling the machine-readable instructions.

8. A machine-readable storage medium storing machine-readable instructions, wherein the machine-readable instructions, when called and executed by a processor, implement the method according to any one of claims 1 to 5.

Images