Resource processing method and device for cloud native spatial database

Abstract

The invention discloses a resource processing method and device for a cloud native spatial database, and the method comprises the steps: obtaining the actual state of a resource from each initialized virtual machine at regular time in an open-source container arrangement engine environment, carrying out the resource calculation according to a preset rule and the actual state of the resource, obtaining the expected resource amount of the virtual machine, and carrying out the resource processing of the virtual machine. A request is sent to the resource planner plug-in interface, the resource planner plug-in interface is triggered to return a response result, the request is used for requesting expected resource allocation of each virtual machine, and the response result is determined by the resource planner plug-in interface according to the expected resource quantity and the resource condition of a container arrangement engine node where the virtual machine is located; and according to the response result and the resource number of the container arrangement engine node where the virtual machine is located, generating a corresponding instruction, and sending the instruction to a server of the container arrangement engine to carry out virtual machine resource adjustment.

Description

Technical Field

[0001] The present application relates to the field of database technology, and more specifically, to a resource processing method and device for a cloud native spatial database. Background Art

[0002] In a cloud-native spatial database management system based on a storage-computing separation architecture, horizontal expansion of computing resources, that is, increasing the number of computing nodes of the spatial database instance to improve the responsiveness of high-concurrency and high-computing scenarios, is a conventional processing method.

[0003] However, for a spatial database instance on a single computing node, if the central processing unit (CPU) or memory usage configuration needs to be increased during the task processing peak period, the spatial database instance must be restarted, which will seriously affect the continuity of client services. Similarly, depending on the business volume, if the idle resources occupied by the spatial database instance need to be released, the instance must also be restarted.

[0004] Therefore, how to achieve elastic utilization of resources by the cloud-native spatial database management system without affecting the client's business and usage is an urgent problem to be solved in this application. Summary of the invention

[0005] In view of this, the present application discloses a resource processing method and device for a cloud-native spatial database, which aims to achieve transparent and elastic scaling of the use of resources such as CPU and memory by spatial database instances, so as to achieve elastic utilization of resources by the cloud-native spatial database management system without affecting the client's business and use.

[0006] In order to achieve the above purpose, the disclosed technical solution is as follows:

[0007] In a first aspect, the present application discloses a resource processing method for a cloud native spatial database, the method being applied to a daemon process, the method comprising:

[0008] In the open source container orchestration engine environment, the actual resource status is obtained from each initialized virtual machine at regular intervals;

[0009] Perform resource calculation according to preset rules and the actual status of the resources to obtain the expected resource amount of the virtual machine;

[0010] Send a request to the resource planner plug-in interface, triggering the resource planner plug-in interface to return a response result; wherein the request is used to request the resource allocation expected by each virtual machine; the response result is determined by the resource planner plug-in interface according to the expected resource amount and the resource situation of the container orchestration engine node where the virtual machine is located;

[0011] Generate corresponding instructions according to the response result and the resource amount of the container orchestration engine node where the virtual machine is located;

[0012] The instruction is sent to the server of the container orchestration engine to adjust the virtual machine resources.

[0013] Preferably, performing resource calculation according to preset rules and the actual state of the resources to obtain the expected resource amount of the virtual machine includes:

[0014] Setting resource thresholds; wherein the resource thresholds include at least a minimum resource threshold, a maximum resource threshold and a duration threshold;

[0015] Get the actual usage of the virtual machine;

[0016] If the actual usage during the time period is less than the minimum resource threshold and reaches the duration threshold, the minimum resource threshold is determined as the expected resource amount of the virtual machine;

[0017] If the actual usage during the time period is equal to the preset threshold, or the actual usage is greater than the maximum resource threshold and reaches the duration threshold, the preset value is calculated according to the preset calculation rule to obtain the expected resource amount of the virtual machine.

[0018] Preferably, the process of determining the response result includes:

[0019] Get the current resource configuration of the virtual machine;

[0020] Comparing the expected resource amount with the current resource configuration amount of the virtual machine;

[0021] If the expected resource amount is less than the current resource configuration amount of the virtual machine, a response result of reducing the virtual machine resources is obtained;

[0022] If the expected resource amount is greater than the current resource configuration amount of the virtual machine, a response result of increasing the virtual machine resources is obtained.

[0023] Preferably, the resource planner plug-in interface generates corresponding instructions according to the response result and the amount of resources of the container orchestration engine node where the virtual machine is located, including:

[0024] If the response result is to reduce virtual machine resources, generate a virtual machine patch instruction through the resource planner plug-in interface;

[0025] If the response result is to increase virtual machine resources, and the number of remaining available resources of the container orchestration engine node where the virtual machine is located is greater than or equal to the expected amount of resources to be increased, a virtual machine patch instruction is generated through the resource planner plug-in interface;

[0026] If the response result is to increase the virtual machine resources, and the number of remaining available resources of the container orchestration engine node where the virtual machine is located is less than the expected amount of resources to be increased, a virtual machine migration instruction is generated through the resource planner plug-in interface.

[0027] Preferably, sending the instruction to a server of a container orchestration engine to adjust virtual machine resources includes:

[0028] If the instruction is a virtual machine patch instruction, the virtual machine patch instruction is sent to the server of the container orchestration engine to execute the virtual machine patch execution process to complete the virtual machine resource adjustment;

[0029] If the instruction is a virtual machine migration instruction, the virtual machine migration instruction is sent to the server of the container orchestration engine, and the virtual machine migration is performed according to the minimum unit of the container orchestration engine to complete the virtual machine resource adjustment.

[0030] Preferably, the virtual machine patch execution process includes:

[0031] Notifying the virtual machine controller through the server of the container orchestration engine to adjust the virtual machine resources for the virtual machine patch instruction;

[0032] The virtual operating system simulator is started through the virtual machine controller to adjust the CPU and memory amount to complete the virtual machine patch execution process.

[0033] Preferably, after sending the instruction to the server of the container orchestration engine to adjust the virtual machine resources, the method further includes:

[0034] Synchronizes the actual resource usage of the virtual machine to the resource planner plug-in interface.

[0035] Preferably, the process of initializing the virtual machine includes:

[0036] For each virtual machine, the resource planner plug-in interface is connected to the monitor to obtain the resource status of the CPU and memory of the virtual machine;

[0037] When the response of the monitor is obtained, the resource status of the CPU and memory of the virtual machine is sent to the resource planner plug-in interface so that the resource planner plug-in interface returns an allow response, until the resource status of the CPU and memory of all virtual machines are sent to the resource planner plug-in interface so that the resource planner plug-in interface returns an allow response, thereby completing the virtual machine initialization process; wherein the allow response is used to record the resource status of the virtual machine in the resource planner plug-in interface.

[0038] A second aspect of the present application discloses a resource processing device for a cloud native spatial database, the device being applied to a daemon process, and the device comprising:

[0039] A timed acquisition unit is used to periodically acquire the actual status of resources from each initialized virtual machine in an open source container orchestration engine environment;

[0040] A calculation unit, used to perform resource calculation according to preset rules and the actual state of the resources to obtain the expected resource amount of the virtual machine;

[0041] A sending trigger unit is used to send a request to the resource planner plug-in interface to trigger the resource planner plug-in interface to return a response result; wherein the request is used to request the resource allocation expected by each virtual machine; the response result is determined by the resource planner plug-in interface according to the expected resource amount and the resource situation of the container orchestration engine node where the virtual machine is located;

[0042] A generating unit, configured to generate corresponding instructions according to the response result and the resource amount of the container orchestration engine node where the virtual machine is located;

[0043] The resource adjustment unit is used to send the instruction to the server of the container orchestration engine to adjust the virtual machine resources.

[0044] Preferably, the computing unit comprises:

[0045] A setting module, used to set resource thresholds; wherein the resource thresholds at least include a minimum resource threshold, a maximum resource threshold and a duration threshold;

[0046] A setting module, used to set resource thresholds; wherein the resource thresholds at least include a minimum resource threshold, a maximum resource threshold and a duration threshold;

[0047] A first acquisition module is used to obtain the actual usage of the virtual machine;

[0048] A first determination module, configured to determine the minimum resource threshold as the expected resource amount of the virtual machine if the actual usage amount within the time period is less than the minimum resource threshold and reaches the duration threshold;

[0049] The calculation module is used to calculate the preset value according to the preset calculation rule to obtain the expected resource amount of the virtual machine if the actual usage within the time period is equal to the preset threshold, or the actual usage is greater than the maximum resource threshold and reaches the duration threshold.

[0050] It can be seen from the above technical scheme that the present application discloses a resource processing method and device for a cloud native spatial database. In an open source container orchestration engine environment, the actual status of resources is obtained from each initialized virtual machine at a regular interval, and resource calculation is performed according to preset rules and the actual status of resources to obtain the expected resource amount of the virtual machine. A request is sent to the resource planner plug-in interface to trigger the resource planner plug-in interface to return a response result, wherein the request is used to request the expected resource allocation of each virtual machine, and the response result is determined by the resource planner plug-in interface according to the expected resource amount and the resource situation of the container orchestration engine node where the virtual machine is located. According to the response result and the number of resources of the container orchestration engine node where the virtual machine is located, a corresponding instruction is generated, and the instruction is sent to the server of the container orchestration engine to adjust the virtual machine resources.

[0051] Through the above scheme, in the cloud native spatial database management system, virtualization technology is used to encapsulate virtual machine resources, so that in the open source container orchestration engine environment, resource status is obtained from each initialized virtual machine at regular intervals, and expansion is performed based on the application programming interface of the container orchestration engine to support the control of virtual machine resources in the open source container orchestration engine environment, that is, virtual machine resources are adjusted according to the server and instructions of the container orchestration engine, so that in the open source container orchestration engine K8s environment, the server of the container orchestration engine adjusts virtual machine resources according to instructions, and the spatial database instance uses transparent and elastic scaling of resources such as CPU and memory. Transparent means that it does not affect the client's use, and the client is unaware. Elastic scaling means elastic adjustment of virtual machine resources on demand, resource expansion during busy hours, and resource release during idle hours. This enables the cloud native spatial database management system to achieve elastic utilization of resources without affecting the client's business and use. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0053] Figure 1 A schematic diagram of the overall structure of implementing vertical resource scaling disclosed in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of the storage and computing nodes of YukonDB for Serverless disclosed in the embodiment of the present application;

[0055] Figure 3 A schematic diagram of initializing a virtual machine disclosed in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of the operation phase of the overall structure for implementing vertical resource scaling disclosed in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of the execution flow of the virtual machine patch disclosed in the embodiment of the present application;

[0058] Figure 6 A schematic diagram of a process flow of a resource processing method for a cloud native spatial database disclosed in an embodiment of the present application;

[0059] Figure 7 A schematic diagram of the structure of a resource processing device of a cloud native spatial database disclosed in an embodiment of the present application;

[0060] Figure 8 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0063] It can be seen from the background technology that for a spatial database instance on a single computing node, during the peak period of task processing, if the CPU or memory usage configuration needs to be increased, the spatial database instance must be restarted, which will seriously affect the continuity of the client business. Similarly, depending on the business volume, if it is necessary to release the idle resources occupied by the spatial database instance, the instance must also be restarted. Therefore, how to achieve elastic utilization of resources by the cloud-native spatial database management system without affecting the client's business and use is a problem that this application urgently needs to solve.

[0064] In order to solve the above problems, the present application discloses a resource processing method and device for a cloud native spatial database. In a cloud native spatial database management system, virtualization technology is used to encapsulate virtual machine resources, so that resource status is obtained from each initialized virtual machine in an open source container orchestration engine environment at regular intervals, and expansion is performed based on the application programming interface of the container orchestration engine to support the control of virtual machine resources in an open source container orchestration engine environment, that is, virtual machine resources are adjusted according to the server and instructions of the container orchestration engine, so that in the open source container orchestration engine K8s environment, the server of the container orchestration engine adjusts virtual machine resources according to instructions, and realizes the transparent and elastic scaling of the use of resources such as CPU and memory by the spatial database instance. Transparent means that it does not affect the use of the client, and the client is unaware. Elastic scaling means elastic adjustment of virtual machine resources on demand, resource expansion during busy hours, and resource release during idle hours. So that the cloud native spatial database management system can realize elastic utilization of resources without affecting the business and use of the client. The specific implementation method is specifically described by the following embodiments.

[0065] refer to Figure 1 As shown, it is a schematic diagram of the overall structure of implementing vertical scaling of resources disclosed in an embodiment of the present application. The overall structure of implementing vertical scaling of resources includes a resource planner (Schedule) plug-in interface (yk_schedule) of an extended open source container orchestration engine (K8S), a daemon process (yk_autoscaler), a monitor (yk_vm-monitor) running inside a virtual machine (vm), and a virtual machine controller (yk_vm_controller).

[0066] Among them, yk_schedule is used to process resource requests from yk_autoscaler and limit the increase of resources to avoid over-allocation of node resources.

[0067] yk_autoscaler is deployed on each K8S node and is used to make VM expansion decisions based on the specified resource configuration rules.

[0068] yk_vm-monitor is used to perform VM resource management functions such as VM expansion operations and verifying whether resource release operations are feasible.

[0069] yk_vm_controller is used to execute the VM resource adjustment request sent by the API Server.

[0070] The K8S API Server in the overall structural diagram of implementing vertical resource scaling is the server of the application programming interface of the open source container orchestration engine; YukonDB is the storage and computing node based on the Yugong cloud native spatial database management system (YukonDB for Serverless). Figure 2 shown.

[0071] This application mainly uses virtualization technology to encapsulate YukonDB, and expands it based on the K8s API to support the control of YukonDB virtual machine resources (YukonDB vm) in the K8s environment, and realizes the dynamic adjustment of YukonDB VM memory and CPU; cooperates with the resource planner (scheduler) and virtualization technology (open source simulator and virtualizer (QEMU) + kernel-based virtual machine (KVM)) of each node to realize the smooth online migration of YukonDB VM between cluster nodes.

[0072] Among them, QEMU is used to perform hardware virtualization. Usually Linux comes with a KVM module to provide CPU and memory virtualization. In the QEMU+KVM solution, QEMU provides input / output (I / O) for external applications, and KVM provides access to hardware.

[0073] The specific workflow of each component in the overall structure for implementing vertical resource scaling includes an initialization phase and a running phase.

[0074] Initialization phase Figure 3 shown.

[0075] During the initialization phase, the following steps are performed:

[0076] 1) Connect yk_autoscaler to yk_vm-mornitor and obtain the CPU and memory resource status of the virtual machine vm through yk_vm-mornitor;

[0077] 2) After receiving the response from yk_vm-mornitor, yk_autoscaler sends the resource status of the VM and the request for the current resource allocation to yk_schedule. At this time, Request only serves as a "registration" and does not request any resource scaling;

[0078] 3) When yk_schedule receives the Request from yk_autoscaler, it returns a Permit response, which notifies yk_autoscaler that the CPU and memory of the corresponding vm are exactly equal to the Request. It plays a "registration" role, and the initial state of the resources of the vm in the schedule (i.e., CPU usage, memory usage, etc.) is recorded;

[0079] After performing the above operations on all VMs, initialization is completed.

[0080] Operation phase Figure 4 shown.

[0081] During the operation phase, the collaboration process of each component is as follows:

[0082] (1) yk_autoscaler periodically obtains the actual resource status from each VM, and performs resource calculation based on the preset rules and the actual resource status to obtain the expected resource amount of the new VM; wherein the actual resource status is used to represent the actual usage of the virtual machine; the preset rules are determined by the minimum resource threshold, the maximum resource threshold and the duration threshold in the resource threshold;

[0083] Specifically, resource (including memory and CPU) thresholds are set; wherein the resource thresholds include at least a minimum resource threshold, a maximum resource threshold and a duration threshold, and the actual usage of the virtual machine is obtained. If the actual usage during the time period is less than the minimum resource threshold and reaches the duration threshold, the minimum resource threshold is determined as the expected resource amount of the virtual machine. If the actual usage during the time period is equal to the preset threshold, or the actual usage is greater than the maximum resource threshold, the preset value is calculated according to the preset calculation rule to obtain the expected resource amount of the virtual machine.

[0084] It should be noted that the preset value, minimum resource threshold, maximum resource threshold and duration threshold are set according to actual conditions, and this application does not make specific limitations.

[0085] The time period can be 1 minute, 3 minutes, etc., and there is no specific limitation on the time period.

[0086] The preset calculation rules are configured and defined externally, for example, resource thresholds are set, such as memory thresholds (a, b), where a is the minimum memory threshold and b is the maximum memory threshold. If the actual usage of the VM is less than a and lasts for t minutes (i.e., the actual usage is less than the minimum memory threshold and reaches the duration threshold during the time period), the new expected memory amount is a; if the actual usage reaches 0.8b (i.e., the product of the preset value and b is calculated, and the product of the preset value and b is only a calculation method in the preset calculation rules) or is greater than b, and lasts for t' minutes, that is, the actual usage during the time period is equal to the preset threshold (such as 0.8b), or the actual usage is greater than the maximum memory threshold and reaches the duration threshold, then the new VM's expected memory amount is 1.5*b. The CPU resource amount is calculated in a similar way.

[0087] It should be noted that the preset value is set according to the actual situation and this application does not make any specific limitation.

[0088] (2) yk_autoscaler sends a request to yk_scheduler to request the resource allocation expected by each VM, regardless of whether the resource amount changes;

[0089] (3) yk_scheduler responds (Response) according to the resource situation of the K8s node where the vm is located and the expected resource amount of the vm, and returns the Response to yk_autoscaler; the resource situation of the K8s node where the vm is located is the current resource configuration amount of the K8s node where the vm is located; the current resource configuration amount of the vm is obtained, and the expected resource amount is compared with the current resource configuration amount of the vm. If the expected resource amount is less than the current resource configuration amount of the vm, the response result of reducing the vm resources is obtained; if the expected resource amount is greater than the current resource configuration amount of the vm, the response result of increasing the vm resources is obtained; if the expected resource amount is equal to the current resource configuration amount of the vm, no adjustment is made;

[0090] (4) If VM resources need to be reduced, the response is a virtual machine patch (vmpatch) instruction.

[0091] It should be noted that if the response result is to reduce the virtual machine resources, a vmpatch instruction is generated through the resource planner plug-in interface.

[0092] (5) If you need to add VM resources and the K8s node has sufficient resources, the Response is the vm patch instruction.

[0093] It should be noted that if the response result is to increase virtual machine resources, and the number of remaining available resources of the K8S node where the vm is located is greater than or equal to the expected amount of resources to be increased, it is determined that the vm resources need to be increased and the resources of the K8s node are sufficient (that is, sufficient to accommodate the expected amount of resources), and the Response is generated by yk_scheduler as the vmpatch instruction.

[0094] Among them, the preset number of resources is set according to actual conditions and this application does not make any specific restrictions.

[0095] (6) If additional VM resources are needed and the K8s node has insufficient resources, the Response is the VM migration (vm_migrate) instruction.

[0096] It should be noted that if the response result is to increase vm resources, and the number of remaining available resources of the K8S node where the vm is located is less than the expected amount of resources to be increased, it is determined that the vm resources need to be increased and the resources of the K8s node are insufficient (that is, not enough to accommodate the expected amount of resources), and the Response is generated by yk_scheduler as the vm_migrate instruction.

[0097] vmpatch corresponds to vm_migrate. If the resources of the current K8S node are sufficient, the resources will be directly reduced or increased; if not, they will be migrated to other K8S nodes. Whether the resources of the current K8S node are sufficient is determined by whether the number of resources of the K8S node where the vm is located is less than the preset number of resources.

[0098] (7) After receiving the corresponding Response, yk_autoscaler sends a corresponding request to the K8S API Server according to the instruction. If the instruction is a vmpatch instruction, it requests vmpatch; if the instruction is a vm_migrate instruction, it requests vm_migrate.

[0099] vm_migrate is managed by yk_schedule. The migration process of vm is equivalent to the migration of the smallest unit (pod) based on K8S, that is, selecting a node that meets the expected resource capacity and redeploying the vm.

[0100] Among them, a single K8S node can deploy multiple VMs; a spatial database instance runs inside the VM; when the number of spatial database client requests increases, especially calls to spatial computing functions, the memory and CPU will increase, and when it exceeds the set threshold, a resource expansion request will be triggered.

[0101] The process of requesting vmpatch is as follows:

[0102] vmpatch is performed through the controller (yk_vm_controller). After receiving the API call request, the controller connects back to the virtual operating system emulator (QEMU) running outside the VM to make changes. Figure 5 .

[0103] It should be noted that in the virtualization environment of this application, the memory of the VM is managed by the virtualization management program (QEMU + kernel driver module KVM) on the host machine, rather than the physical memory directly managed by the operating system. When the memory configuration of the VM is increased or decreased, the virtualization management program will adjust the memory size of the virtual machine accordingly, and the operating system and applications in the VM, including the database service (PostgreSQL), will perceive these changes and make corresponding adjustments without restarting the database service.

[0104] Figure 5 In the example, 1) the K8s API Server receives the vmpatch request instruction from yk_autoscaler, which contains the requested amount of the current VM and its corresponding resources (memory and CPU);

[0105] 2) Notify yk_vm_controller to perform virtual machine resource adjustment actions;

[0106] 3) yk_vm_controller starts the virtual operating system simulator QEMU tool of the virtual machine vm to adjust the CPU and memory, and returns the result after execution.

[0107] (8) yk_autoscaler synchronizes the actual resource usage of the VM to yk_schedule.

[0108] It should be noted that yk_autoscaler notifies when resources are reduced and issues requests when resources need to be increased. This means that yk_scheduler always keeps track of the upper limit of resource usage to avoid the problem of over-allocation of resources.

[0109] This application implements vertical scaling of computing nodes in YukonDB for Serverless based on QEMU+KVM virtual machine technology, and provides supporting components for engineering practice in the K8s environment.

[0110] The present application provides a method and device for expanding and reducing the capacity of resources of a spatial database instance, namely a computing node, so as to realize the elastic utilization of resources by a spatial database management system without affecting the connection and use of a database client.

[0111] In an embodiment of the present application, in a cloud-native spatial database management system, virtualization technology is used to encapsulate virtual machine resources, so that resource status is obtained from each initialized virtual machine in an open source container orchestration engine environment at regular intervals, and expansion is performed based on the application programming interface of the container orchestration engine to support control of virtual machine resources in an open source container orchestration engine environment, that is, virtual machine resources are adjusted according to the server and instructions of the container orchestration engine, so that in a K8s environment, the server of the container orchestration engine adjusts virtual machine resources according to instructions, and transparent and elastic scaling of the use of resources such as CPU and memory by the spatial database instance is achieved. Transparent means that it does not affect the use of the client, and the client is unaware. Elastic scaling means elastic adjustment of virtual machine resources on demand, resource expansion during busy hours, and resource release during idle hours. This enables the cloud-native spatial database management system to achieve elastic utilization of resources without affecting the client's business and use.

[0112] refer to Figure 6 As shown, it is a flow chart of a resource processing method of a cloud native spatial database disclosed in an embodiment of the present application. The resource processing method of the cloud native spatial database is applicable to the above-mentioned embodiment. Figure 1 The yk_autoscaler component of the overall structure of implementing vertical resource scaling in the cloud native spatial database mainly includes the following steps:

[0113] S601: In the open source container orchestration engine environment, yk_autoscaler periodically obtains the actual status of resources from each initialized VM;

[0114] The actual resource status includes at least the CPU status and the memory status.

[0115] The process of virtual machine initialization is as follows:

[0116] For each virtual machine, connect the resource planner plug-in interface to the monitor to obtain the CPU and memory resource status of the VM;

[0117] When the monitor's response is obtained, the CPU and memory resource status of the virtual machine is sent to the resource planner plug-in interface so that yk_autoscaler returns a Permit response, until the CPU and memory resource status of all virtual machines are sent to the resource planner plug-in interface so that yk_autoscaler returns a permission response, completing the virtual machine initialization process; wherein the Permit response is used to record the resource status of the virtual machine in the resource planner plug-in interface.

[0118] The virtual machine initialization is the same as above Figure 3 The execution process and execution principle of the initialization phase are consistent, which can be referred to and will not be repeated here.

[0119] S602: yk_autoscaler performs resource calculation according to preset rules and actual resource status to obtain the expected resource amount of the VM.

[0120] Specifically, yk_autoscaler calculates resources according to preset rules and actual resource status to obtain the expected resource amount of the VM as shown in A1-A4.

[0121] A1: Setting resource thresholds; wherein the resource thresholds include at least a minimum resource threshold, a maximum resource threshold and a duration threshold.

[0122] A2: Get the actual usage of the VM.

[0123] A3: If the actual usage during the time period is less than the minimum memory threshold and reaches the duration threshold, the minimum resource threshold is determined as the expected resource amount of the VM.

[0124] A4: If the actual usage during the time period is equal to the preset threshold, or the actual usage is greater than the maximum memory threshold, the preset value is calculated according to the preset calculation rule to obtain the expected resource amount of the VM.

[0125] Among them, the preset value is set according to the actual situation and this application does not make any specific limitation.

[0126] The execution process and execution principle of A1-A4 are the same as those of the above embodiments. Figure 4 The execution process and execution principle are consistent, which can be used as a reference and will not be repeated here.

[0127] S603: yk_autoscaler sends a request to yk_scheduler, triggering yk_scheduler to return a response result; wherein the request is used to request the resource allocation expected by each VM; the response result is determined by yk_scheduler based on the expected resource amount and the resource situation of the K8S node where the virtual machine is located.

[0128] The specific process of yk_scheduler determining the response result based on the expected resource amount and the resource situation of the K8S node where the virtual machine is located is shown in B1-B4.

[0129] B1: yk_scheduler obtains the current resource configuration of the K8S node where the vm is located.

[0130] B2: yk_scheduler compares the expected resource amount with the current resource configuration of the K8S node where the VM is located.

[0131] B3: If the expected amount of resources is less than the current resource configuration of the K8S node where the VM is located, yk_scheduler obtains a response result of reducing the VM resources.

[0132] B4: If the expected amount of resources is greater than the current resource configuration of the K8S node where the VM is located, yk_scheduler obtains a response result of increasing VM resources.

[0133] The execution process and execution principle of B1-B4 are similar to those of the above embodiment. Figure 4 The execution process and execution principle are consistent, which can be referred to and will not be repeated here.

[0134] S604: yk_autoscaler generates corresponding instructions based on the response result and the number of resources of the K8S node where the vm is located.

[0135] The specific process of yk_autoscaler generating corresponding instructions based on the response results and the number of resources of the K8S node where the VM is located is shown in C1-C3.

[0136] C1: If the response result is to reduce VM resources, yk_autoscaler generates a VM patch instruction through yk_scheduler.

[0137] C2: If the response result is to increase the VM resources, and the number of remaining available resources of the K8S node where the VM is located is greater than or equal to the expected increase in resources, yk_autoscaler determines that the resources of the K8S node where the VM is located are sufficient to accommodate the expected increase in resources, that is, the resources are satisfied, and generates a VM patch instruction through the resource planner plug-in interface.

[0138] Among them, the preset number of resources is set according to actual conditions and this application does not make any specific restrictions.

[0139] C3: If the response result is to increase the VM resources, and the remaining available resources of the K8S node where the VM is located are less than the expected amount of resources to be increased, yk_autoscaler determines that the resources of the K8S node where the VM is located are not sufficient to accommodate the expected amount of resources to be increased, that is, the resources are insufficient, and generates a VM migration instruction (vm_migrate instruction) through the resource planner plug-in interface.

[0140] The execution process and execution principle of C1-C3 are similar to those of the above embodiment. Figure 4 The execution process and execution principle are consistent, which can be referred to and will not be repeated here.

[0141] S605: yk_autoscaler sends instructions to the K8S API Server to adjust virtual machine resources.

[0142] In S605, if the instruction is a vm patch instruction, the vm patch instruction is sent to the K8S API Server to execute the virtual machine patch execution process to complete the virtual machine resource adjustment;

[0143] If the instruction is a vm_migrate instruction, the vm_migrate instruction is sent to the K8S API Server, and the VM migration is performed according to the smallest unit of the container orchestration engine to complete the virtual machine resource adjustment.

[0144] The VM patch execution process is as follows:

[0145] The server of the container orchestration engine notifies the virtual machine controller to adjust the virtual machine resources for the VM patch instruction, and starts the virtual operating system simulator through the VM controller to adjust the CPU and memory to complete the VM patch execution process.

[0146] After sending the instruction to the server of the container orchestration engine to adjust the VM resources, the actual resource usage of the VM is synchronized to the resource planner plug-in interface.

[0147] The execution process and execution principle of S601-S605 are similar to those of the above embodiment. Figure 4 and Figure 5 The execution process and execution principle are consistent, which can be referred to and will not be repeated here.

[0148] In the embodiment of the present application, in the cloud-native spatial database management system, virtualization technology is used to encapsulate virtual machine resources, so that in the open source container orchestration engine environment, resource status is obtained from each initialized virtual machine at regular intervals, and expansion is performed based on the application programming interface of the container orchestration engine to support the control of virtual machine resources in the open source container orchestration engine environment, that is, virtual machine resources are adjusted according to the server and instructions of the container orchestration engine, so that in the open source container orchestration engine K8s environment, the server of the container orchestration engine adjusts virtual machine resources according to instructions, and the spatial database instance uses transparent and elastic scaling of resources such as CPU and memory. Transparent means that it does not affect the use of the client, and the client is unaware. Elastic scaling means elastic adjustment of virtual machine resources on demand, resource expansion during busy hours, and resource release during idle hours. This allows the cloud-native spatial database management system to achieve elastic utilization of resources without affecting the client's business and use.

[0149] Based on the above embodiments Figure 6 A resource processing method for a cloud native spatial database is disclosed. The embodiment of the present application also discloses a resource processing device for a cloud native spatial database. Figure 7 As shown, the resource processing device of the cloud native spatial database includes:

[0150] The timing acquisition unit 701 is used to regularly acquire the actual status of resources from each initialized virtual machine in an open source container orchestration engine environment;

[0151] The calculation unit 702 is used to perform resource calculation according to preset rules and actual resource status to obtain the expected resource amount of the virtual machine;

[0152] A sending trigger unit 703 is used to send a request to the resource planner plug-in interface to trigger the resource planner plug-in interface to return a response result; wherein the request is used to request the resource allocation expected by each virtual machine; the response result is determined by the resource planner plug-in interface according to the expected resource amount and the resource situation of the container orchestration engine node where the virtual machine is located;

[0153] A generating unit 704 is used to generate a corresponding instruction according to the response result and the number of resources of the container orchestration engine node where the virtual machine is located;

[0154] The resource adjustment unit 705 is used to send instructions to the server of the container orchestration engine to adjust the virtual machine resources.

[0155] Furthermore, the calculation unit 702 includes:

[0156] A setting module, used to set resource thresholds; wherein the resource thresholds at least include a minimum resource threshold, a maximum resource threshold and a duration threshold;

[0157] A first acquisition module is used to obtain the actual usage of the virtual machine;

[0158] A first determination module, configured to determine the minimum resource threshold as the expected resource amount of the virtual machine if the actual usage amount in the time period is less than the minimum resource threshold and reaches the duration threshold;

[0159] The calculation module is used to calculate the preset value according to the preset calculation rules to obtain the expected resource amount of the virtual machine if the actual usage during the time period is equal to the preset threshold, or the actual usage is greater than the maximum resource threshold and reaches the duration threshold.

[0160] Further, the sending trigger unit 703 for determining the response result includes:

[0161] The second acquisition module is used to obtain the current resource configuration of the virtual machine;

[0162] A comparison module, used to compare the expected resource amount with the current resource configuration amount of the virtual machine;

[0163] A third acquisition module is used to obtain a response result of reducing the virtual machine resources if the expected resource amount is less than the current resource configuration amount of the virtual machine;

[0164] The fourth acquisition module is used to obtain a response result of increasing the virtual machine resources if the expected resource amount is greater than the current resource configuration amount of the virtual machine.

[0165] Furthermore, the generating unit 704 includes:

[0166] A first generating module, configured to generate a virtual machine patch instruction through a resource planner plug-in interface if the response result is to reduce virtual machine resources;

[0167] The second generation module is used to generate a virtual machine patch instruction through the resource planner plug-in interface if the response result is to increase virtual machine resources and the number of remaining available resources of the container orchestration engine node where the virtual machine is located is greater than or equal to the expected amount of resources to be increased;

[0168] The third generation module is used to generate a virtual machine migration instruction through the resource planner plug-in interface if the response result is to increase the virtual machine resources and the number of remaining available resources of the container orchestration engine node where the virtual machine is located is less than the expected amount of resources to be increased.

[0169] Furthermore, the resource adjustment unit 705 includes:

[0170] An execution module, configured to send the virtual machine patch instruction to a server of a container orchestration engine to execute a virtual machine patch execution process to complete the virtual machine resource adjustment if the instruction is a virtual machine patch instruction;

[0171] The migration module is used to send the virtual machine migration instruction to the server of the container orchestration engine if the instruction is a virtual machine migration instruction, and perform virtual machine migration according to the smallest unit of the container orchestration engine to complete the virtual machine resource adjustment.

[0172] Furthermore, the execution module includes:

[0173] A first adjustment submodule is used to notify the virtual machine controller through the server of the container orchestration engine to adjust the virtual machine resources for the virtual machine patch instruction;

[0174] The second adjustment submodule is used to start the virtual operating system simulator through the virtual machine controller to adjust the CPU and memory amount to complete the virtual machine patch execution process.

[0175] Furthermore, the resource processing device of the cloud native spatial database also includes:

[0176] The synchronization unit is used to synchronize the actual resource usage of the virtual machine to the resource planner plug-in interface.

[0177] Furthermore, the timing acquisition unit 701 for initializing the virtual machine includes:

[0178] A fifth acquisition module is used to connect the resource planner plug-in interface to the monitor for each virtual machine to obtain the CPU and memory resource status of the virtual machine;

[0179] A sending module is used to send the resource status of the CPU and memory of the virtual machine to the resource planner plug-in interface when a response from the monitor is obtained, so that the resource planner plug-in interface returns an allow response, until the resource status of the CPU and memory of all virtual machines are sent to the resource planner plug-in interface, so that the resource planner plug-in interface returns an allow response, thereby completing the process of initializing the virtual machine; wherein the allow response is used to record the resource status of the virtual machine in the resource planner plug-in interface.

[0180] In the embodiment of the present application, in the cloud-native spatial database management system, virtualization technology is used to encapsulate virtual machine resources, so that in the open source container orchestration engine environment, resource status is obtained from each initialized virtual machine at regular intervals, and expansion is performed based on the application programming interface of the container orchestration engine to support the control of virtual machine resources in the open source container orchestration engine environment, that is, virtual machine resources are adjusted according to the server and instructions of the container orchestration engine, so that in the open source container orchestration engine K8s environment, the server of the container orchestration engine adjusts virtual machine resources according to instructions, and the spatial database instance uses transparent and elastic scaling of resources such as CPU and memory. Transparent means that it does not affect the use of the client, and the client is unaware. Elastic scaling means elastic adjustment of virtual machine resources on demand, resource expansion during busy hours, and resource release during idle hours. This allows the cloud-native spatial database management system to achieve elastic utilization of resources without affecting the client's business and use.

[0181] An embodiment of the present application also provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the resource processing method of the cloud native spatial database as described above.

[0182] The present application also provides an electronic device, whose structural diagram is as follows: Figure 8 As shown, it specifically includes a memory 801 and one or more instructions 802, wherein the one or more instructions 802 are stored in the memory 801 and are configured to be executed by one or more processors 803 to execute the one or more instructions 802 to execute the resource processing method of the above-mentioned cloud native spatial database.

[0183] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0184] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0185] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0186] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0187] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0188] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A resource processing method for a cloud native spatial database, characterized in that: The method is applied to a daemon process, and the method comprises: In the open source container orchestration engine environment, the actual resource status is obtained from each initialized virtual machine at regular intervals; Perform resource calculation according to preset rules and the actual status of the resources to obtain the expected resource amount of the virtual machine; Send a request to the resource planner plug-in interface, triggering the resource planner plug-in interface to return a response result; wherein the request is used to request the resource allocation expected by each virtual machine; the response result is determined by the resource planner plug-in interface according to the expected resource amount and the resource situation of the container orchestration engine node where the virtual machine is located; Generate corresponding instructions according to the response result and the resource amount of the container orchestration engine node where the virtual machine is located; The instruction is sent to the server of the container orchestration engine to adjust the virtual machine resources.

2. The method according to claim 1, characterized in that The performing resource calculation according to the preset rules and the actual state of the resources to obtain the expected resource amount of the virtual machine includes: Setting resource thresholds; wherein the resource thresholds include at least a minimum resource threshold, a maximum resource threshold and a duration threshold; Get the actual usage of the virtual machine; If the actual usage during the time period is less than the minimum resource threshold and reaches the duration threshold, the minimum resource threshold is determined as the expected resource amount of the virtual machine; If the actual usage during the time period is equal to the preset threshold, or the actual usage is greater than the maximum resource threshold and reaches the duration threshold, the preset value is calculated according to the preset calculation rule to obtain the expected resource amount of the virtual machine.

3. The method according to claim 1, characterized in that The process of determining the response result includes: Get the current resource configuration of the virtual machine; Comparing the expected resource amount with the current resource configuration amount of the virtual machine; If the expected resource amount is less than the current resource configuration amount of the virtual machine, a response result of reducing the virtual machine resources is obtained; If the expected resource amount is greater than the current resource configuration amount of the virtual machine, a response result of increasing the virtual machine resources is obtained.

4. The method according to claim 3, characterized in that The resource planner plug-in interface generates corresponding instructions according to the response result and the resource amount of the container orchestration engine node where the virtual machine is located, including: If the response result is to reduce virtual machine resources, generate a virtual machine patch instruction through the resource planner plug-in interface; If the response result is to increase virtual machine resources, and the number of remaining available resources of the container orchestration engine node where the virtual machine is located is greater than or equal to the expected amount of resources to be increased, a virtual machine patch instruction is generated through the resource planner plug-in interface; If the response result is to increase the virtual machine resources, and the number of remaining available resources of the container orchestration engine node where the virtual machine is located is less than the expected amount of resources to be increased, a virtual machine migration instruction is generated through the resource planner plug-in interface.

5. The method according to claim 4, characterized in that The step of sending the instruction to the server of the container orchestration engine to adjust the virtual machine resources includes: If the instruction is a virtual machine patch instruction, the virtual machine patch instruction is sent to the server of the container orchestration engine to execute the virtual machine patch execution process to complete the virtual machine resource adjustment; If the instruction is a virtual machine migration instruction, the virtual machine migration instruction is sent to the server of the container orchestration engine, and the virtual machine migration is performed according to the minimum unit of the container orchestration engine to complete the virtual machine resource adjustment.

6. The method according to claim 5, characterized in that The virtual machine patch execution process includes: Notifying the virtual machine controller through the server of the container orchestration engine to adjust the virtual machine resources for the virtual machine patch instruction; The virtual operating system simulator is started through the virtual machine controller to adjust the CPU and memory amount to complete the virtual machine patch execution process.

7. The method according to claim 1, characterized in that After sending the instruction to the server of the container orchestration engine to adjust the virtual machine resources, the method further includes: Synchronizes the actual resource usage of the virtual machine to the resource planner plug-in interface.

8. The method according to claim 1, characterized in that The process of initializing a virtual machine includes: For each virtual machine, the resource planner plug-in interface is connected to the monitor to obtain the resource status of the CPU and memory of the virtual machine; When the response of the monitor is obtained, the resource status of the CPU and memory of the virtual machine is sent to the resource planner plug-in interface so that the resource planner plug-in interface returns an allow response, until the resource status of the CPU and memory of all virtual machines are sent to the resource planner plug-in interface so that the resource planner plug-in interface returns an allow response, thereby completing the virtual machine initialization process; wherein the allow response is used to record the resource status of the virtual machine in the resource planner plug-in interface.

9. A resource processing device for a cloud native spatial database, characterized in that: The device is applied to a daemon process, and comprises: A timed acquisition unit is used to periodically acquire the actual status of resources from each initialized virtual machine in an open source container orchestration engine environment; A calculation unit, used to perform resource calculation according to preset rules and the actual state of the resources to obtain the expected resource amount of the virtual machine; A sending trigger unit is used to send a request to the resource planner plug-in interface to trigger the resource planner plug-in interface to return a response result; wherein the request is used to request the resource allocation expected by each virtual machine; the response result is determined by the resource planner plug-in interface according to the expected resource amount and the resource situation of the container orchestration engine node where the virtual machine is located; A generating unit, configured to generate corresponding instructions according to the response result and the resource amount of the container orchestration engine node where the virtual machine is located; The resource adjustment unit is used to send the instruction to the server of the container orchestration engine to adjust the virtual machine resources.

10. The device according to claim 9, characterized in that The computing unit comprises: A setting module, used to set resource thresholds; wherein the resource thresholds at least include a minimum resource threshold, a maximum resource threshold and a duration threshold; A setting module, used to set resource thresholds; wherein the resource thresholds at least include a minimum resource threshold, a maximum resource threshold and a duration threshold; A first acquisition module is used to obtain the actual usage of the virtual machine; A first determination module, configured to determine the minimum resource threshold as the expected resource amount of the virtual machine if the actual usage amount within the time period is less than the minimum resource threshold and reaches the duration threshold; The calculation module is used to calculate the preset value according to the preset calculation rules to obtain the expected resource amount of the virtual machine if the actual usage within the time period is equal to the preset threshold, or the actual usage is greater than the maximum resource threshold and reaches the duration threshold.

Images