Dynamic memory capacity adjustment method, system, electronic device and storage medium

Abstract

One or more embodiments of this specification provide a method, system, electronic device, and storage medium for dynamic memory capacity adjustment. The method is applied to a database instance; the method comprises: determining a first memory capacity to be expanded; determining, from a reserved memory space pre-allocated for the database instance, at least a portion of the reserved memory space corresponding to the first memory capacity as a first target memory space, and releasing the first target memory space; the reserved memory space includes a continuous virtual address space that is configured to be in an access-prohibited state during the pre-allocation process; and reallocating the first target memory space to the database instance, and configuring the first target memory space to be in an access-allowed state during the reallocation process.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of cloud database technology, and in particular to a dynamic memory capacity adjustment method, system, electronic device, and machine-readable storage medium, as well as a dynamic memory capacity reduction method. Background Art

[0002] With the advent of the cloud-native era, cloud database technology is also evolving. Cloud databases are databases optimized or deployed in virtualized computing environments, offering benefits such as pay-as-you-go, on-demand scalability, high availability, and storage consolidation.

[0003] Through online database services based on cloud database technology, users can create and modify instances, migrate and synchronize data, back up and restore data, and perform monitoring and alerting. An instance, or database instance, is a separate database service process that occupies physical memory. Different memory sizes, disk space requirements, and database types can be configured for each instance.

[0004] In actual applications, since memory resources are important system resources of the operating system, it is very important for online database services to support elastic scaling and dynamic memory capacity adjustment capabilities based on demand.

[0005] In the related art, during the process of dynamic memory capacity adjustment for a database instance, there are problems such as online database services being unavailable during memory expansion and the original data structure being adjusted. It is impossible to ensure service availability during memory capacity adjustment while avoiding intrusion into the original data structure. Summary of the Invention

[0006] This application provides a dynamic memory capacity adjustment method, applied to a database instance; the method includes:

[0007] Determining a first memory capacity to be expanded;

[0008] Determining, from a reserved memory space pre-allocated for the database instance, at least a portion of the reserved memory space corresponding to the first memory capacity as a first target memory space, and releasing the first target memory space; wherein the reserved memory space includes a continuous virtual address space that is configured as a prohibited access state during a pre-allocation process;

[0009] The first target memory space is reallocated to the database instance, and during the reallocation process, the first target memory space is configured to be in an access-allowed state.

[0010] This application also provides a dynamic memory shrinking method, which is applied to a database instance; the method includes:

[0011] Determine the target memory capacity to be reduced;

[0012] Determining, from the continuous virtual address space allocated to the database instance and configured in an access-allowed state, at least a portion of the continuous virtual address space corresponding to the target memory capacity as a target memory space, and releasing the target memory space;

[0013] The target memory space is reallocated to the database instance, and during the reallocation process, the target memory space is configured to be in an access-prohibited state.

[0014] The present application also provides a dynamic memory capacity adjustment device, the device comprising:

[0015] A first determining unit, configured to determine a first memory capacity to be expanded;

[0016] a first releasing unit, configured to determine, from the reserved memory space pre-allocated for the database instance, at least a portion of the reserved memory space corresponding to the first memory capacity as a first target memory space, and release the first target memory space; wherein the reserved memory space includes a continuous virtual address space that is configured as a prohibited access state during a pre-allocation process;

[0017] The first allocation unit is configured to reallocate the first target memory space to the database instance, and configure the first target memory space to be in an access-allowed state during the reallocation process.

[0018] The present application also provides a dynamic memory capacity reduction device, the device comprising:

[0019] A determination unit, configured to determine a target memory capacity to be reduced;

[0020] a releasing unit, configured to determine, from the continuous virtual address spaces allocated to the database instance and configured in an access-allowed state, at least a portion of the continuous virtual address spaces corresponding to the target memory capacity as a target memory space, and release the target memory space;

[0021] The allocating unit is configured to reallocate the target memory space to the database instance and configure the target memory space to be in an access-prohibited state during the reallocation process.

[0022] The present application also provides a dynamic memory capacity adjustment system, which is used to implement the above-mentioned dynamic memory capacity adjustment method or dynamic memory capacity reduction method.

[0023] The present application also provides an electronic device, comprising a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus;

[0024] The memory stores machine-readable instructions, and the processor executes the above-mentioned dynamic memory capacity adjustment method or dynamic memory capacity reduction method by calling the machine-readable instructions.

[0025] The present application also provides a machine-readable storage medium, which stores machine-readable instructions. When the machine-readable instructions are called and executed by a processor, they implement the above-mentioned dynamic memory capacity adjustment method or dynamic memory shrinkage method.

[0026] Through the above embodiment, the first memory space used for memory expansion is at least a portion of the continuous virtual address space determined from the pre-allocated continuous virtual address space as the reserved memory space, and the first memory space can be converted from a prohibited access state to an allowed access state by reallocating the memory space; through the above method, since the actual expansion of the memory capacity is based on the pre-allocated continuous virtual address space, and no new discontinuous virtual address space is applied for, all data structures in this memory segment are also continuous, thereby avoiding intrusion into the original data structure during the dynamic memory capacity adjustment process. In addition, through the above method, there is no need to copy the data stored in the original memory space to the newly applied memory space, and there is no need for a long locking operation, thereby avoiding the problem of unavailability of online database services during the dynamic adjustment of the memory capacity.

[0027] Similarly, the target memory space used for memory reduction is at least a portion of the continuous virtual address space determined from the continuous virtual address space allocated to the database instance and configured to be in an access-allowed state, and the target memory space can be converted from an access-allowed state to an access-prohibited state by reallocating memory space. In this manner, since the actual reduction of memory capacity is based on the pre-allocated continuous virtual address space, all data structures in this memory segment are also continuous, thereby avoiding intrusion into existing data structures during dynamic memory capacity adjustment. In addition, this manner eliminates the need for lengthy locking operations, thereby avoiding the issue of online database service unavailability during dynamic memory capacity adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0029] Figure 1This is a schematic diagram of memory expansion based on the realloc function shown in the related art;

[0030] Figure 2 This is a schematic diagram of a method for expanding memory capacity with discontinuous data shown in the related art;

[0031] Figure 3 is a schematic diagram of a dynamic memory expansion method shown in an exemplary embodiment;

[0032] Figure 4 is a schematic diagram of a dynamic memory shrinking method shown in an exemplary embodiment;

[0033] Figure 5 is a flow chart showing a method for dynamic memory capacity adjustment according to an exemplary embodiment;

[0034] Figure 6 This is a flow chart of a dynamic memory shrinking method shown in an exemplary embodiment;

[0035] Figure 7 This is a structural diagram of an electronic device in which a dynamic memory capacity adjustment device is located, showing an exemplary embodiment;

[0036] Figure 8 is a block diagram of a dynamic memory capacity adjustment device according to an exemplary embodiment;

[0037] Figure 9 The figure is a block diagram of a dynamic memory shrinking device according to an exemplary embodiment. DETAILED DESCRIPTION

[0038] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative work should fall within the scope of protection of this specification.

[0039] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0040] With the advent of the cloud-native era, cloud database technology is also evolving. Cloud databases are databases optimized or deployed in virtualized computing environments, offering benefits such as pay-as-you-go, on-demand scalability, high availability, and storage consolidation. Through online database services powered by cloud database technology, users can create and modify instances, migrate and synchronize data, back up and restore data, and monitor and alert.

[0041] The instance, ie, database instance, may be an independent database service process that occupies physical memory, and different memory sizes, disk spaces, and database types may be set for the instance.

[0042] In practice, since memory is a critical operating system resource, it's crucial for online database services to support elastic scaling and on-demand dynamic memory capacity adjustment. Specifically, users can adjust instance memory specifications by modifying instance configuration parameters as needed, or the service process corresponding to the instance can automatically determine the need for memory expansion according to pre-set memory management policies. This specification does not specify this requirement.

[0043] For example, see Figure 1 , Figure 1 This is a schematic diagram of memory expansion based on the realloc function shown in the related art. Figure 1 As shown, if memory expansion is required for memory space 11, a continuous new memory space 12 can be applied for. The memory capacity of the new memory space 12 can be the sum of the memory capacity of the original memory space 11 and the memory capacity that needs to be newly expanded; further, all data stored in the original memory space 11 can be copied to the new memory space 12; further, the original memory space 11 can be released.

[0044] It can be seen that in the embodiment shown above, on the one hand, since locking is required in the process of memory expansion based on the realloc function, external processes are not allowed to access the data stored in the original memory space 11, which will cause the problem of unavailability of online database services during memory expansion; on the other hand, since data copying operations are required in the process of memory expansion based on the realloc function, as the amount of data stored in the original memory space continues to increase, the time consumed by each dynamic memory expansion will also continue to increase.

[0045] For example, see Figure 2 , Figure 2 FIG. 1 is a schematic diagram of a method for expanding memory capacity with discontinuous data in the related art. Figure 2As shown, if memory expansion is required for the memory space 21, a new memory space 22 can be applied for, and the memory capacity of the new memory space 22 can be the memory capacity that needs to be newly expanded; wherein, there is no need to lock before applying for the new memory space, nor is there any need to copy the data stored in the original memory space 21 to the new memory space 22, but data can be directly continued to be written in the applied new memory space 22.

[0046] It can be seen that in the embodiment shown above, since the new memory space 22 applied for may be discontinuous with the original memory space 21, the data stored in the original memory space 21 and the data stored in the new memory space 22 may also be discontinuous; in order to be able to normally access the data stored in the discontinuous original memory space 21 and the new memory space 22 respectively, the original data structure needs to be adjusted.

[0047] For example, if the original data structure corresponding to the data stored in the original memory space 21 is a one-dimensional array, after expansion, the data structures corresponding to the data in the original memory space 21 and the new memory space 22 need to be adjusted to two-dimensional data before the data stored in the discontinuous original memory space 21 and the new memory space 22 can be accessed.

[0048] It can be seen that in the embodiments of one or more related technologies shown above, during the process of dynamic memory capacity adjustment for a database instance, there are problems such as online database services being unavailable during memory expansion and the original data structure being adjusted. It is impossible to ensure service availability during memory capacity adjustment while avoiding intrusion into the original data structure.

[0049] In view of this, this specification aims to propose a technical solution of pre-allocating sufficient and continuous memory space for a database instance, and then dynamically adjusting the memory capacity by reallocating the memory space.

[0050] In this specification, before executing the dynamic memory capacity adjustment process, a sufficient amount of continuous virtual address space can be pre-allocated for the database instance, and the continuous virtual address space can be configured to be in an access-prohibited state during the pre-allocation process, so that the continuous virtual address space allocated to the database instance and configured to be in an access-prohibited state can be used as the reserved memory space allocated for the database instance.

[0051] During implementation, during the dynamic memory expansion process, a first memory capacity to be expanded can be determined; further, at least a portion of the reserved memory space corresponding to the first memory capacity can be determined as a first target memory space from the reserved memory space pre-allocated for the database instance, and the first target memory space can be released; wherein, the reserved memory space can specifically include a continuous virtual address space that is configured to be in an access-prohibited state during the pre-allocation process; further, the first target memory space can be reallocated to the database instance, and the first target memory space can be configured to be in an access-allowed state during the reallocation process.

[0052] For example, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a method for dynamic memory expansion according to an exemplary embodiment. Figure 3 As shown, a continuous virtual address space 30 can be pre-allocated for the database instance, and the virtual address space 30 can be configured to be in a prohibited access state during the pre-allocation process so that the virtual address space 30 can be used as a reserved memory space; further, after determining that the first memory capacity to be expanded is NBuffers1, at least a portion of the reserved memory space corresponding to the first memory capacity NBuffers1 (that is, the virtual address space 31 configured to be in a prohibited access state) can be determined from the reserved memory space pre-allocated for the database instance (that is, the virtual address space 30 configured to be in a prohibited access state) as the first target memory space for memory expansion, and the first target memory space (that is, the virtual address space 31 configured to be in a prohibited access state) can be released; further, the virtual address space 31 can be reallocated to the database instance, and the virtual address space 31 can be configured to be in an allowed access state during the reallocation process.

[0053] It can be seen that in the embodiment shown above, the first memory space used for memory expansion is at least a portion of the continuous virtual address space determined from the pre-allocated continuous virtual address space as the reserved memory space, and the first memory space can be converted from a prohibited access state to an allowed access state by reallocating the memory space; through the above method, since the actual expansion of the memory capacity is based on the pre-allocated continuous virtual address space, and no new discontinuous virtual address space is applied for, all data structures in this memory segment are also continuous, thereby avoiding intrusion into the original data structure during the dynamic memory capacity adjustment process. In addition, through the above method, there is no need to copy the data stored in the original memory space to the newly applied memory space, and there is no need for a long locking operation, thereby avoiding the problem of unavailability of online database services during the dynamic adjustment of memory capacity.

[0054] Based on similar technical concepts, dynamic memory shrinkage can also be performed on the basis of pre-allocating the reserved memory space for the database instance.

[0055] During implementation, during the dynamic memory shrinking process, the target memory capacity to be shrunk can be determined; further, at least a portion of the continuous virtual address space corresponding to the target memory capacity can be determined as the target memory space from the continuous virtual address space allocated to the database instance and configured to be in an access-allowed state, and the target memory space can be released; further, the target memory space can be reallocated to the database instance, and during the reallocation process, the target memory space can be configured to be in an access-prohibited state.

[0056] For example, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a dynamic memory shrinking method according to an exemplary embodiment. Figure 4 As shown, the continuous virtual address space allocated to the database instance includes virtual address space 40; wherein, the continuous virtual address space allocated to the database instance may specifically include a continuous virtual address space configured to be in an access-allowed state (i.e., virtual address space 41 + virtual address space 42), and a reserved memory space configured to be in an access-prohibited state (i.e., virtual address space 43) that has been pre-allocated for the database instance; further, after determining that the target memory capacity to be shrunk is NBuffers2, at least a portion of the continuous virtual address space corresponding to the target memory capacity NBuffers2 can be determined as the target memory space (i.e., virtual address space 42 configured to be in an access-allowed state) from the continuous virtual address space allocated to the database instance and configured to be in an access-allowed state, and the target memory space can be released; further, the virtual address space 42 can be reallocated to the database instance, and the virtual address space 42 can be configured to be in an access-prohibited state during the reallocation process.

[0057] Thus, in the embodiment shown above, the target memory space used for memory reduction is at least a portion of the continuous virtual address space determined from the continuous virtual address space allocated to the database instance and configured to be in an access-allowed state, and the target memory space can be converted from an access-allowed state to an access-prohibited state by reallocating the memory space. In this way, since the actual reduction of memory capacity is based on the pre-allocated continuous virtual address space, all data structures in this memory segment are also continuous, thereby avoiding intrusion into the original data structure during the dynamic memory capacity adjustment process. In addition, in this way, there is no need for long-term locking operations, thereby avoiding the problem of unavailability of online database services during the dynamic memory capacity adjustment process.

[0058] The present application is described below through specific embodiments and in combination with specific application scenarios.

[0059] See Figure 5 , Figure 5 This is a flow chart showing a method for dynamic memory capacity adjustment according to an exemplary embodiment.

[0060] In this specification, the dynamic memory capacity adjustment method can be applied to a database instance corresponding to a database. The specific database type of the database is not particularly limited in this specification.

[0061] In this specification, the dynamic memory capacity adjustment may include dynamic memory expansion and / or dynamic memory contraction. The following description will be divided into two parts: dynamic memory expansion and dynamic memory contraction.

[0062] (1) Dynamic memory expansion

[0063] like Figure 5 As shown, the dynamic memory capacity adjustment method can perform the following steps:

[0064] Step 502: Determine a first memory capacity to be expanded.

[0065] For example, it may be determined that the first memory capacity to be expanded is NBuffers1.

[0066] In step 502, the size of the first memory capacity may be specified by the user or automatically determined by the online database service.

[0067] For example, a user can modify the configuration parameters related to memory specifications in the instance parameter configuration interface for the database instance, and can submit the modified configuration parameters to the online database service; in response to the modified configuration parameters related to memory specifications submitted by the user, the database instance can initiate a memory expansion request, and can also determine the first memory capacity based on the modified configuration parameters indicated in the memory expansion request.

[0068] For another example, in response to a memory expansion request initiated by the database instance, the online database service can automatically determine the first memory capacity based on instance parameters such as the memory specifications and reserved memory capacity of the database instance, and can also combine with a pre-configured memory expansion policy.

[0069] Step 504: Determine at least a portion of the reserved memory space corresponding to the first memory capacity from the reserved memory space pre-allocated for the database instance as a first target memory space, and release the first target memory space; wherein the reserved memory space includes a continuous virtual address space that is configured as an access-prohibited state during the pre-allocation process.

[0070] For example, Figure 3 As shown, a continuous virtual address space 30 can be pre-allocated for the database instance, and the virtual address space 30 can be configured to be in an access-prohibited state during the pre-allocation process so that the virtual address space 30 can be used as a reserved memory space; further, after determining that the first memory capacity to be expanded is NBuffers1, at least a portion of the reserved memory space corresponding to the first memory capacity NBuffers1 (that is, the virtual address space 30 configured to be in an access-prohibited state) can be determined from the reserved memory space pre-allocated for the database instance (that is, the virtual address space 30 configured to be in an access-prohibited state) as the first target memory space for memory expansion, and the first target memory space (that is, the virtual address space 31 configured to be in an access-prohibited state) can be released.

[0071] The specific implementation of releasing the first target memory space in step 504 is not described in detail in this specification. For example, the munmap function can be used to specify the starting position, capacity, and other attributes of the first target memory space to release the first target memory space.

[0072] In one illustrated embodiment, prior to step 504, a contiguous virtual address space may be pre-allocated for the database instance to serve as the reserved memory space. During implementation, before determining at least a portion of the reserved memory space corresponding to the first memory capacity from the pre-allocated reserved memory space for the database instance as the first target memory space, the method may further include: allocating the reserved memory space to the database instance, and configuring the reserved memory space to a prohibited access state during the allocation process.

[0073] For example, Figure 3 As shown, a continuous virtual address space 30 may be allocated to the database embodiment, and during the allocation process, the virtual address space 30 may be configured to be in an access-prohibited state.

[0074] The specific implementation of configuring the virtual address space to a prohibited or permitted state during the allocation process is not described in detail in this specification. For example, the virtual address space can be allocated to the database instance using the mmap function, and the allocated virtual address space can be configured to a prohibited state using the PROT_NONE property.

[0075] In a possible embodiment, the database instance specifically includes a database instance corresponding to a database that adopts a multi-process architecture; allocating the reserved memory space to the database instance and configuring the reserved memory space to a prohibited access state during the allocation process may specifically include: responding to a memory pre-allocation request initiated by any process among the multiple processes corresponding to the database instance, allocating a continuous virtual address space to each of the multiple processes, and configuring the continuous virtual address space allocated to each process to a prohibited access state during the allocation process, so as to determine the continuous virtual address space allocated to each process and configured to a prohibited access state as the reserved memory space of each process.

[0076] For example, the database instance may be a PostgreSQL database instance corresponding to a PostgreSQL database; the multiple processes corresponding to the PostgreSQL database instance may include process A and process B; in response to a memory pre-allocation request initiated by process A, continuous virtual address space with the same memory capacity may be allocated to process A and process B respectively, and during the allocation process, the continuous virtual address space allocated to process A and process B respectively may be configured to a prohibited access state to be used as reserved memory space for process A and process B.

[0077] It should be noted that in the embodiments shown above, the database using a multi-process architecture can be a PostgreSQL database, which is merely an exemplary description and is not particularly limited in this specification; in addition, the multiple processes corresponding to the PostgreSQL database instance include process A and process B, which is also merely an exemplary description, and the number of processes corresponding to the PostgreSQL database instance can be any positive integer.

[0078] In an embodiment of a related technology, for a database instance corresponding to a database that adopts a multi-process architecture, since each process in the multiple processes can usually independently manage and allocate the virtual address space of each process, the virtual addresses of the same data in the virtual address spaces of different processes may be different.

[0079] In the above-described embodiment, since the original data structure can be avoided from being invaded during the dynamic memory capacity adjustment process, the dynamic memory capacity adjustment requirements of the database instance corresponding to the database adopting the multi-process architecture can be met.

[0080] In one embodiment shown, the pre-allocated reserved memory space may specifically include: a continuous virtual address space supporting small page storage.

[0081] For example, Figure 3 As shown, a continuous virtual address space 30 can be pre-allocated for a database instance, and the virtual address space 30 can be configured as an access-prohibited state during the pre-allocation process, and the virtual address space 30 can be configured as a continuous virtual address space that supports small page storage so that the virtual address space 30 can be used as a reserved memory space.

[0082] It should be noted that, in the above embodiments, Figure 3 The example of the reserved memory space configured to be in an access-prohibited state including a continuous virtual address space supporting small page storage is provided as an exemplary description and does not specifically limit this specification. In actual applications, the reserved memory space configured to be in an access-prohibited state may also include a continuous virtual address space supporting large page storage.

[0083] In order to help those skilled in the art better understand the technical solutions in this specification, small page storage and large page storage are briefly introduced below.

[0084] In paged virtual memory, both virtual and physical memory are typically divided into fixed-size pages, with each thread allocated memory in pages. The "small page" size is typically 4KB, while the "large page" size can be 2MB, 1GB, or even larger.

[0085] The “small page” may also be generally referred to as a normal page; using a “large page” can effectively reduce the memory occupied by the page table and improve the access efficiency of the TLB (Translation Lookaside Buffer).

[0086] In actual applications, when allocating memory space through the mmap function, a virtual address space that supports small page storage can usually be allocated, and the allocated virtual address space will not be directly mapped to physical memory until a request to access the memory is made. A page fault interrupt will be triggered, and the allocated virtual address space will be mapped to physical memory. If a virtual address space that supports large page storage needs to be allocated, it will be directly mapped to physical memory when allocating the virtual address space that supports large page storage.

[0087] It can be seen that in the implementation mode shown above, the reserved space pre-allocated for the database embodiment can be a continuous virtual address space that supports small page storage. In this way, in the process of pre-allocating the reserved memory space, it will not be directly mapped to the physical memory. Instead, at least part of the reserved memory space will not be mapped to the physical memory until at least part of the reserved memory space is actually used for memory expansion or actual memory access, thereby avoiding the reserved memory space configured as the prohibited access state occupying the physical memory, resulting in a waste of physical memory resources.

[0088] Step 506: reallocate the first target memory space to the database instance, and configure the first target memory space to be in an access-allowed state during the reallocation process.

[0089] For example, Figure 3 As shown, the first target memory space (ie, the virtual address space 31) can be reallocated to the database instance through the mmap function, and the virtual address space 31 can be configured to be in an access-allowed state during the reallocation process.

[0090] In this specification, when the reserved memory space configured to be in an access-prohibited state specifically includes a continuous virtual address space that supports small page storage, during the reallocation process, the first target memory space can be configured as a continuous virtual address space that supports large page storage, or the first target memory space can be configured as a continuous virtual address space that supports small page storage, thereby realizing a dynamic memory capacity adjustment method that is compatible with large page storage and small page storage.

[0091] In one embodiment shown, the pre-allocated reserved memory space may specifically include: a continuous virtual address space that supports small page storage; step 506 may also include: configuring the first target memory space as a continuous virtual address space that supports large page storage during the reallocation process; applying for a physical address space that supports large page storage corresponding to the first memory capacity for the database instance; and mapping the applied physical address space that supports large page storage to the first target memory space that is reallocated to the database instance.

[0092] For example, Figure 3 As shown, a continuous virtual address space 30 can be pre-allocated for the database instance, and the virtual address space 30 can be configured to a prohibited access state during the pre-allocation process, and the virtual address space 30 can be configured as a continuous virtual address space that supports small page storage, so that the virtual address space 30 can be used as a reserved memory space; further, at least part of the reserved memory space corresponding to the first memory capacity NBuffers1 (that is, the virtual address space 31 configured to a prohibited access state) is determined to be the first target memory space for memory expansion, and the first target memory space can be released; further, the virtual address space 31 can be reallocated to the database instance, and the virtual address space 31 can be configured to an allowed access state during the reallocation process, and the virtual address space 31 can be configured as a continuous virtual address space that supports large page storage during the reallocation process; further, a physical address space that supports large page storage corresponding to the first memory capacity NBuffers1 can be applied for; further, the applied physical address space that supports large page storage can be mapped to the reallocated first target memory space (that is, the virtual address space 31 configured to an allowed access state).

[0093] It should be noted that, in the above embodiments, Figure 3 The example of a reserved memory space configured in an access-denied state including a continuous virtual address space supporting small page storage, and a first target memory space configured in an access-allowed state including a continuous virtual address space supporting large page storage, is provided as an exemplary description and does not specifically limit this specification. In actual applications, the first target memory space configured in an access-allowed state may also include a continuous virtual address space supporting small page storage.

[0094] In another embodiment shown, the pre-allocated reserved memory space may specifically include: a continuous virtual address space that supports small page storage; the step 506 may also include: configuring the first target memory space as a continuous virtual address space that supports small page storage during the reallocation process.

[0095] For example, Figure 3 As shown, a continuous virtual address space 30 can be pre-allocated for the database instance, and the virtual address space 30 can be configured to be in an access-prohibited state during the pre-allocation process, and the virtual address space 30 can be configured to be a continuous virtual address space that supports small page storage, so that the virtual address space 30 can be used as a reserved memory space; further, at least a portion of the reserved memory space corresponding to the first memory capacity NBuffers1 (that is, the virtual address space 31 configured to be in an access-prohibited state) is determined to be the first target memory space for memory expansion, and the first target memory space can be released; further, the virtual address space 31 can be reallocated to the database instance, and the virtual address space 31 can be configured to be in an access-allowed state during the reallocation process, and the virtual address space 31 can be configured to be a continuous virtual address space that supports small page storage during the reallocation process.

[0096] (2) Dynamic memory shrinkage

[0097] The method may further comprise:

[0098] Step 508: Determine the second memory capacity to be reduced.

[0099] Step 510: Determine at least a portion of the continuous virtual address space corresponding to the second memory capacity as a second target memory space from the continuous virtual address space allocated to the database instance and configured to be in an access-allowed state, and release the second target memory space.

[0100] Step 512: reallocate the second target memory space to the database instance, and configure the second target memory space to be in an access-prohibited state during the reallocation process.

[0101] For example, Figure 4 As shown, it can be determined that the second memory capacity to be shrunk is NBuffers2; further, at least part of the continuous virtual address space corresponding to the second memory capacity NBuffers2 can be determined as the second target memory space (that is, the virtual address space 42 configured to be in the access-allowed state) from the continuous virtual address space allocated to the database instance (that is, the virtual address space 41 + the virtual address space 42), and the second target memory space can be released; further, the virtual address space 42 can be reallocated to the database instance, and the virtual address space 42 can be configured to be in the access-prohibited state during the reallocation process.

[0102] Among them, in the step 508, the capacity of the second memory capacity can be specified by the user or automatically determined by the online database service. For the specific determination method, please refer to the relevant embodiment of the step 502, which will not be repeated here.

[0103] Among them, in step 510, the continuous virtual address space configured to allow access may specifically include: a continuous virtual address space configured to allow access and supporting small page storage; or a continuous virtual address space configured to allow access and supporting large page storage.

[0104] It should be noted that Figure 4 The example of a continuous virtual address space configured to be in an access-allowed state including a continuous virtual address space supporting large page storage is provided as an example and does not specifically limit this specification. In actual applications, a continuous virtual address space configured to be in an access-allowed state may also include a continuous virtual address space supporting small page storage.

[0105] In one embodiment shown, in step 510, when the continuous virtual address space configured to allow access includes a continuous virtual address space configured to allow access and supporting large page storage, before releasing the second target memory space, the method may further include: determining a physical address space corresponding to the second target memory space that supports large page storage; and deleting the mapping between the physical address space and the second target memory space.

[0106] In one embodiment shown, in step 512, when the continuous virtual address space configured to allow access includes a continuous virtual address space configured to allow access and supporting large page storage, the method may further include: configuring the second target memory space as a continuous virtual address space supporting small page storage during the reallocation process; and releasing the physical address space supporting large page storage corresponding to the second target memory space.

[0107] For example, Figure 4As shown, a continuous virtual address space 41 and a virtual address space 42 can be allocated to the database instance, and the virtual address space 41 and the virtual address space 42 can be configured to be in an access-allowed state during the allocation process, and the virtual address space 41 and the virtual address space 42 can be configured to be continuous virtual address spaces that support large page storage during the allocation process; further, after determining that the second memory capacity to be shrunk is NBuffers2, at least a portion of the continuous virtual address space corresponding to the second memory capacity NBuffers2 can be determined as the second target memory space ( That is, the virtual address space 42 is configured to be in an access-allowed state), and the second target memory space can be released; further, the virtual address space 42 can be reallocated to the database instance, and the virtual address space 42 can be configured to be in an access-prohibited state during the reallocation process, and the virtual address space 42 can be configured to be a continuous virtual address space that supports small page storage during the reallocation process, so that the virtual address space 42 configured to be in an access-prohibited state can be used as a reserved memory space allocated for the database instance; in addition, since the virtual address space 42 has been configured to be a continuous virtual address space that supports small page storage, the physical address space that supports large page storage corresponding to the virtual address space 42 can also be released.

[0108] Through the above technical solution, it can be known that the first memory space used for memory expansion is at least a portion of the continuous virtual address space determined from the pre-allocated continuous virtual address space as the reserved memory space, and the first memory space can be converted from a prohibited access state to an allowed access state by reallocating the memory space; through the above method, since the actual expansion of the memory capacity is based on the pre-allocated continuous virtual address space, and no new discontinuous virtual address space is applied for, all data structures in this memory segment are also continuous, thereby avoiding intrusion into the original data structure during the dynamic memory capacity adjustment process. In addition, through the above method, there is no need to copy the data stored in the original memory space to the newly applied memory space, and there is no need for a long locking operation, thereby avoiding the problem of unavailability of online database services during the dynamic adjustment of memory capacity.

[0109] See Figure 6 , Figure 6 This is a flow chart of a dynamic memory shrinking method according to an exemplary embodiment. The dynamic memory shrinking method can be applied to a database instance. Figure 6 As shown, the dynamic memory shrinking method can perform the following steps:

[0110] Step 602: Determine the target memory capacity to be reduced;

[0111] Step 604: Determine at least a portion of the continuous virtual address space corresponding to the target memory capacity as a target memory space from the continuous virtual address space allocated to the database instance and configured in an access-allowed state, and release the target memory space.

[0112] Step 606: reallocate the target memory space to the database instance, and configure the target memory space to be in an access-prohibited state during the reallocation process.

[0113] In one embodiment shown, the continuous virtual address space configured to allow access includes:

[0114] A contiguous virtual address space configured to allow access and support small page storage; or

[0115] A contiguous virtual address space configured to allow access and support large page storage.

[0116] In one embodiment shown, when the continuous virtual address space configured to be in the access-allowed state includes a continuous virtual address space configured to be in the access-allowed state and supporting large page storage, before releasing the target memory space, the method further includes:

[0117] Determine a physical address space supporting large page storage corresponding to the target memory space;

[0118] The mapping between the physical address space and the target memory space is deleted.

[0119] In one embodiment shown, when the continuous virtual address space configured to be in the access-allowed state includes a continuous virtual address space configured to be in the access-allowed state and supporting large page storage, the method further includes:

[0120] During the reallocation process, the target memory space is configured as a continuous virtual address space supporting small page storage; and the physical address space supporting large page storage corresponding to the target memory space is released.

[0121] Through the above technical solution, it can be seen that the target memory space used for memory reduction is at least a portion of the continuous virtual address space determined from the continuous virtual address space allocated to the database instance and configured to allow access, and the target memory space can be converted from the allowed access state to the prohibited access state by reallocating the memory space; through the above method, since the actual reduction of memory capacity is based on the pre-allocated continuous virtual address space, all data structures in this memory segment are also continuous, thereby avoiding intrusion into the original data structure during the process of dynamic memory capacity adjustment. In addition, through the above method, there is no need for long-term locking operations, thereby avoiding the problem of unavailability of online database services during the dynamic adjustment of memory capacity.

[0122] Corresponding to the embodiment of the dynamic memory capacity adjustment method, this specification also provides an embodiment of a dynamic memory capacity adjustment system, an embodiment of a dynamic memory capacity adjustment device, and an embodiment of a dynamic memory capacity reduction device.

[0123] In this specification, the dynamic memory capacity adjustment system may execute steps 502 to 512, or the dynamic memory capacity adjustment system may execute steps 602 to 604 to implement the technical solution of this specification.

[0124] See Figure 7 , Figure 7 This is a hardware structure diagram of an electronic device in which a dynamic memory capacity adjustment device is located, which is shown as an exemplary embodiment. At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, a memory 708, and a non-volatile memory 710, and of course may also include hardware required for other services. One or more embodiments of this specification can be implemented based on software, such as the processor 702 reading the corresponding computer program from the non-volatile memory 710 into the memory 708 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0125] See Figure 8 , Figure 8 This is a block diagram of a dynamic memory capacity adjustment device according to an exemplary embodiment. The dynamic memory capacity adjustment device can be applied to Figure 7 In the electronic device shown in FIG. , the technical solution of this specification is implemented. The dynamic memory capacity adjustment device may include:

[0126] A first determining unit 802 is configured to determine a first memory capacity to be expanded;

[0127] A first releasing unit 804 is configured to determine, from the reserved memory space pre-allocated for the database instance, at least a portion of the reserved memory space corresponding to the first memory capacity as a first target memory space, and release the first target memory space; wherein the reserved memory space includes a continuous virtual address space that is configured as a prohibited access state during the pre-allocation process;

[0128] The first allocating unit 806 is configured to reallocate the first target memory space to the database instance, and configure the first target memory space to be in an access-allowed state during the reallocation process.

[0129] In this embodiment, the pre-allocated reserved memory space includes:

[0130] A continuous virtual address space that supports small page storage.

[0131] In this embodiment, the first allocating unit 806 is further configured to:

[0132] During the reallocation process, configuring the first target memory space as a continuous virtual address space supporting small page storage;

[0133] or,

[0134] During the reallocation process, the first target memory space is configured as a continuous virtual address space that supports large page storage; a physical address space that supports large page storage corresponding to the first memory capacity is applied for the database instance; and the applied physical address space that supports large page storage is mapped to the first target memory space reallocated to the database instance.

[0135] In this embodiment, the device further includes:

[0136] The pre-allocation unit is configured to allocate the reserved memory space to the database instance and configure the reserved memory space to a prohibited access state during the allocation process.

[0137] In this embodiment, the database instance includes a database instance corresponding to a database using a multi-process architecture;

[0138] The pre-allocation unit is specifically used to:

[0139] In response to a memory pre-allocation request initiated by any one of the multiple processes corresponding to the database instance, a continuous virtual address space is allocated to each of the multiple processes, and during the allocation process, the continuous virtual address space allocated to each process is configured as a prohibited access state, so that the continuous virtual address space allocated to each process and configured as a prohibited access state is determined as the reserved memory space of each process.

[0140] In this embodiment, the device further includes:

[0141] A second determining unit, configured to determine a second memory capacity to be reduced;

[0142] a second releasing unit, configured to determine, from the continuous virtual address space allocated to the database instance and configured in an access-allowed state, at least a portion of the continuous virtual address space corresponding to the second memory capacity as a second target memory space, and release the second target memory space;

[0143] The second allocation unit is configured to reallocate the second target memory space to the database instance, and configure the second target memory space to be in an access-prohibited state during the reallocation process.

[0144] In this embodiment, the continuous virtual address space configured to be in the access-allowed state includes:

[0145] A contiguous virtual address space configured to allow access and support small page storage; or

[0146] A contiguous virtual address space configured to allow access and support large page storage.

[0147] In this embodiment, the second releasing unit is further used to:

[0148] Determine a physical address space supporting large page storage corresponding to the second target memory space;

[0149] The mapping between the physical address space and the second target memory space is deleted.

[0150] In this embodiment, the second allocation unit is further configured to:

[0151] During the reallocation process, the second target memory space is configured as a continuous virtual address space supporting small page storage; and the physical address space supporting large page storage corresponding to the second target memory space is released.

[0152] See Figure 9 , Figure 9This is a block diagram of a dynamic memory shrinking device according to an exemplary embodiment. The dynamic memory shrinking device can be applied to Figure 7 In the electronic device shown in FIG. , the technical solution of this specification is implemented. The dynamic memory capacity reduction device may include:

[0153] A determining unit 902 is configured to determine a target memory capacity to be reduced;

[0154] a releasing unit 904, configured to determine, from the continuous virtual address spaces allocated to the database instance and configured in an access-allowed state, at least a portion of the continuous virtual address spaces corresponding to the target memory capacity as a target memory space, and release the target memory space;

[0155] The allocating unit 906 is configured to reallocate the target memory space to the database instance, and configure the target memory space to be in an access-prohibited state during the reallocation process.

[0156] In this embodiment, the continuous virtual address space configured to be in the access-allowed state includes:

[0157] A contiguous virtual address space configured to allow access and support small page storage; or

[0158] A contiguous virtual address space configured to allow access and support large page storage.

[0159] In this embodiment, the release unit 904 is further used to:

[0160] Determine a physical address space supporting large page storage corresponding to the target memory space;

[0161] The mapping between the physical address space and the target memory space is deleted.

[0162] In this embodiment, the allocation unit 906 is further configured to:

[0163] During the reallocation process, the target memory space is configured as a continuous virtual address space supporting small page storage; and the physical address space supporting large page storage corresponding to the target memory space is released.

[0164] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0165] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0166] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0167] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0168] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0169] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0170] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0171] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0172] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "an," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0173] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when..." or "in response to determining."

[0174] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A method for dynamic memory capacity adjustment, applied to a database instance corresponding to a database using a multi-process architecture; the method comprising: Determining a first memory capacity to be expanded; Allocating the reserved continuous memory space to the database instance and configuring the reserved continuous memory space to a prohibited access state during the allocation process includes: in response to a memory pre-allocation request initiated by any process among multiple processes corresponding to the database instance, allocating a continuous virtual address space to each of the multiple processes, and configuring the continuous virtual address space allocated to each process to a prohibited access state during the allocation process, so as to determine the continuous virtual address space allocated to each process and configured to be in the prohibited access state as the reserved memory space for each process; Determining, from the reserved contiguous memory space pre-allocated for the database instance, at least a portion of the reserved contiguous memory space corresponding to the first memory capacity as a first target memory space, and releasing the first target memory space; wherein the reserved contiguous memory space includes a contiguous virtual address space supporting small page storage that is configured to be in an access-prohibited state during the pre-allocation process; reallocating the first target memory space to the database instance, and configuring the first target memory space to be in an access-allowed state during the reallocation process; During the reallocation process, configuring the first target memory space as a continuous virtual address space supporting small page storage; or, During the reallocation process, the first target memory space is configured as a continuous virtual address space that supports large page storage; a physical address space that supports large page storage corresponding to the first memory capacity is applied for the database instance; and the applied physical address space that supports large page storage is mapped to the first target memory space reallocated to the database instance.

2. The method according to claim 1, further comprising: Determining the second memory capacity to be reduced; Determining, from the continuous virtual address space allocated to the database instance and configured in an access-allowed state, at least a portion of the continuous virtual address space corresponding to the second memory capacity as a second target memory space, and releasing the second target memory space; The second target memory space is reallocated to the database instance, and during the reallocation process, the second target memory space is configured to be in an access-prohibited state.

3. The method according to claim 2, wherein the continuous virtual address space configured to allow access to the state comprises: A continuous virtual address space configured to allow access and support small page storage; or, A contiguous virtual address space configured to allow access and support large page storage.

4. The method according to claim 3, wherein, when the continuous virtual address space configured to be in the access-allowed state includes a continuous virtual address space configured to be in the access-allowed state and supporting large page storage, before releasing the second target memory space, the method further comprises: Determine a physical address space supporting large page storage corresponding to the second target memory space; The mapping between the physical address space and the second target memory space is deleted.

5. The method according to claim 3 , wherein, when the continuous virtual address space configured to be in the access-allowed state includes a continuous virtual address space configured to be in the access-allowed state and supporting large page storage, the method further comprises: During the reallocation process, configuring the second target memory space as a continuous virtual address space supporting small page storage; And, releasing the physical address space supporting large page storage corresponding to the second target memory space.

6. A dynamic memory capacity adjustment system, wherein the dynamic memory capacity adjustment system is used to implement the method according to any one of claims 1 to 5.

7. An electronic device comprising a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 5 by calling the machine-readable instructions.

8. A machine-readable storage medium storing machine-readable instructions, wherein the machine-readable instructions, when called and executed by a processor, implement the method according to any one of claims 1 to 5.

Images