Service batch processing method based on Kubernetes cronjob, computing device and storage medium

By using Kubernetes cronjobs for batch processing of services, the resource configuration of cloud-native serverless databases is automatically adjusted, solving the problems of resource waste and high costs in ordinary database clusters during business fluctuations, and achieving efficient resource utilization and cost savings.

CN119045945APending Publication Date: 2024-11-29GROW UP TOGETHER (SHANGHAI) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411059759.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In existing technologies, ordinary database clusters are inflexible in resource allocation when business fluctuates, leading to resource waste or shortage, and cloud-native serverless databases are costly.

Method used

We adopt a service batch processing method based on Kubernetes cronjobs, which automatically adjusts resource configuration by periodically stopping and starting services and leveraging the elasticity of cloud-native serverless databases.

Benefits of technology

It achieves efficient resource utilization, reduces costs by 50%, improves resource utilization and operational efficiency, and is suitable for scenarios with fluctuating business conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119045945A_ABST
    Figure CN119045945A_ABST
Patent Text Reader

Abstract

The invention provides a service batch processing method based on Kubernetes cronjob, a computing device and a storage medium. The service batch processing method based on the Kubernetes cronjob comprises the steps that information of a current task to be executed is acquired; determining a corresponding service account and a command according to the task information; and executing the command by using the service account to process the deployment copy of the target space. According to the scheme, the advantage of elasticity of a cloud native Serverless database is fully utilized, when resources do not need to be used, all links of the database are disconnected in a mode of regularly closing or starting the service, the Serverless database enters the stop state, capacity expansion of cluster resources can be rapidly completed in the business peak period, and therefore the usage amount of the resources is reduced, and the service life of the cluster resources is prolonged. And meanwhile, the cost can be saved by 50% according to 12 hours of actual use every day.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for batch timing stopping and starting of services based on Kubernetes cronjob. Background Art

[0002] Databases are very important infrastructure for Internet products. Considering the traffic characteristics of Internet products, when creating a database, it is often necessary to configure the resources of the database cluster conservatively, including multiple parameter configurations such as CPU, memory, storage, and number of connections, to ensure that the business can run smoothly during peaks and troughs. In this case, the purchased cluster resources may be idle during the business trough period, resulting in high overall costs; but during the stage of increasing business pressure, cluster resources may be insufficient. The cloud-native Serverless database provided by various public cloud vendors can solve this problem well. The database cluster resources can be dynamically and elastically expanded and contracted with the business load, freeing developers from complex business resource evaluation and operation and maintenance work.

[0003] In scenarios where business fluctuations are large, the resource usage and specification changes of common database clusters and Serverless clusters are as follows: Figure 1 As shown in the figure, in a scenario with large business fluctuations:

[0004] In ordinary clusters, more resources are wasted during the trough period, and insufficient resources cause business losses during the peak period.

[0005] Serverless clusters, because their specifications are dynamically adjusted at any time according to business demand, overall waste of resources is very small, which improves resource utilization and reduces resource usage; cluster resource expansion and contraction can be completed within 1 second, and the business is completely unaware, and business needs can be fully met during peak hours, ensuring that the business is not damaged, and improving system stability; breaking the fixed resource payment model, truly expanding the pay-as-you-go model that dynamically matches load and resources; no manual configuration changes are required, which improves operation and maintenance efficiency. Supports automatic start and stop capabilities. When there is no connection, the cluster can automatically stop to release computing costs; when a new connection is established, the cluster automatically starts without feeling.

[0006] However, the biggest disadvantage of cloud-native Serverless databases is their high price, which is much higher than databases with annual or monthly subscriptions. This undoubtedly increases the usage and decision-making costs of enterprises.

[0007] In some scenarios, such as R&D environments, if you can make good use of the cloud-native Serverless data's ability to pause when there is no activity, you can use the powerful advantages of the Serverless database to save half the cost. Summary of the invention

[0008] The purpose of this application is to provide a service batch processing method, computing device and storage medium based on Kubernetes cronjob, which is used to solve the technical problem in the prior art that the resources of ordinary clusters cannot be dynamically adjusted at any time according to the demand of the business.

[0009] In a first aspect, the present application provides a service batch processing method based on Kubernetes cronjob, characterized in that the service batch processing method based on Kubernetes cronjob includes:

[0010] Get the current task information to be executed;

[0011] Determine the corresponding service account and command according to the task information;

[0012] The command is executed using the service account to process the deployment copy of the target space.

[0013] Preferably, before the step of obtaining the information of the current task to be executed, the following steps are included:

[0014] Create a service account in the first namespace.

[0015] Set cluster roles;

[0016] Bind the service account to the cluster role, and specify permissions for the service account.

[0017] Preferably, the permissions include:

[0018] Get the deployments and deployments / scale resources in the apps and extensions api groups; and / or,

[0019] Patching the deployments and deployments / scale resources in the apps and extensions api groups; and / or,

[0020] List the deployments and deployments / scale resources in the apps and extensions api groups; and / or,

[0021] Monitors the deployments and deployments / scale resources in the apps and extensions api groups.

[0022] Preferably, before the step of obtaining the information of the current task to be executed, the method further includes:

[0023] Create task information to be executed, the task information including: task execution time, task execution service account and task execution command.

[0024] Preferably, the processing comprises:

[0025] Reducing the number of deployment replicas in the target namespace to 0; and / or restoring the deployment replicas in the target namespace.

[0026] Preferably, after the step of using the service account to execute the command to process the deployment copy of the target space, the following steps are included:

[0027] The working status of the database is determined according to the processing result.

[0028] Preferably, the step of determining the working status of the database according to the processing result includes:

[0029] When the number of deployed replicas in the target namespace is reduced to 0, the database enters a stopped state; or,

[0030] When the deployment copy in the target namespace is restored to the initial state, the database enters the startup state.

[0031] In a second aspect, the present application provides a computing device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the service batch processing method based on Kubernetes cronjob as described above.

[0032] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the service batch processing method based on Kubernetes cronjob as described above.

[0033] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0034] The service batch processing method, computing device and storage medium based on Kubernetes cronjob provided in this application realizes the service batch processing based on kubernetes, fully utilizing the advantages of cloud native Serverless database elasticity. When resources are not needed, all links of the database are disconnected by shutting down or starting services at a scheduled time, so that the Serverless database enters a stopped state. During business peak periods, cluster resources can be expanded quickly, thereby reducing resource usage and improving resource utilization. At the same time, according to the actual use of 12 hours per day, 50% of the cost can be saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A diagram of resource usage and specification changes of common database clusters and Serverless clusters in scenarios with large business fluctuations provided for this application;

[0037] Figure 2 A flow chart of a service batch processing method based on kubernetes cronjob provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of a process for creating a service account provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application.

[0040] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The purpose of this application is to provide a service batch processing method, computing device and storage medium based on Kubernetes cronjob, aiming to fully utilize the advantages of cloud-native Serverless database elasticity while disconnecting all links to the database by shutting down the service at a scheduled time when not in use.

[0043] In order to make the above-mentioned objects, features and advantages of the present application more obvious, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0044] Example 1

[0045] like Figure 2 As shown, the service batch processing method based on Kubernetes cronjob in this embodiment includes:

[0046] Get the current task information to be executed;

[0047] Determine the corresponding service account and command according to the task information;

[0048] The command is executed using the service account to process the deployment copy of the target space.

[0049] When carrying out research and development work, many services are usually deployed in kubernetes. When cluster services are not needed, all services need to be stopped. In order to automatically stop all deployed services, in an embodiment of the present application, all deployed services are stopped by executing the set task information. Specifically, by obtaining the current task information to be executed, such as the task execution time, the service account for task execution, and the task execution command, and determining the corresponding service account to be used and the command to be executed by the service account according to the task information, by executing the task at the time specified by the task to be executed, such as 22:00 every day, using the service account corresponding to the task to be executed, after executing the kubectl command, the deployment copy in the target space kid17-rc002 is processed, for example, the number of deployment copies in the namespace is reduced to 0, and the number of all deployment copy services in kubernetes is reduced to 0, which can make the services in the entire kubernetes stop faster and less destructive; for example, restore the deployment copy in the target namespace to restore the service.

[0050] refer to Figure 3 As shown, before the step of obtaining the current pending task information, it includes:

[0051] Create a service account in the first namespace.

[0052] Set cluster roles;

[0053] Bind the service account to the cluster role, and specify permissions for the service account.

[0054] In an embodiment of the present application, before obtaining the information of pending tasks, it is necessary to create service accounts with different usage environments according to different task requirements, such as creating a service account for batch stopping services, creating a service account for batch replying services, etc.; further set cluster roles with different permissions, such as setting permissions to obtain deployment and deployment / scale resources in the apps and extensions API groups, setting permissions to patch deployment and deployment / scale resources in the apps and extensions API groups, setting permissions to list deployments and deployments / scale resources in the apps and extensions API groups, and setting permissions to monitor deployments and deployments / scale resources in the apps and extensions API groups, etc.; further bind the created service account to the cluster role, and specify the permissions that can be used for the service account.

[0055] In an embodiment of the present application, after the step of using a service account to execute a command to process the deployment replicas of the target space, further, the working status of the database is determined according to the processing result of the deployment replicas. For example, when the number of deployment replicas in the target namespace is reduced to 0, the cluster service stops. Further, the number of services connected to the Serverless database becomes 0, and the database enters a stopped state at this time. When restoring the service, an exemplary method is to set the parameter replicas=1 in yaml to achieve the expansion of the number of deployment replicas to restore the service; another exemplary method is to use another service account that is pre-created and bound to a cluster role to execute a command for expansion to achieve the expansion of the number of deployment replicas in the target namespace to restore the service. Furthermore, after the number of deployment replicas in the target namespace is expanded and the service is restored, the corresponding connected database also enters the startup state accordingly to wait for the connection service.

[0056] Through the above-mentioned service batch processing method based on kubernetes cronjob, the automatic batch stop and start of services based on kubernetes is realized, making full use of the elasticity of the cloud original sound serverless database. When resources are not needed, all links to the database are disconnected by regularly shutting down or starting services, so that the serverless database enters a stopped state. During business peak periods, cluster resources can be expanded quickly, thereby reducing resource usage and improving resource utilization. At the same time, according to the actual use of 12 hours per day, 50% of the cost can be saved.

[0057] Example 2

[0058] A computing device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the service batch processing method based on Kubernetes cronjob in embodiment 1.

[0059] Example 3

[0060] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the service batch processing method based on Kubernetes cronjob in embodiment 1.

[0061] Example 4

[0062] A computer program product includes a computer program, which, when executed by a processor, implements the service batch processing method based on Kubernetes cronjob in embodiment 1.

[0063] Example 5

[0064] A computer device, which may be a database, may have an internal structure as shown in Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the service batch processing method based on Kubernetes cronjob in Example 1 is implemented.

[0065] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0066] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present invention may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present invention may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0067] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A service batch processing method based on Kubernetes cronjob, characterized in that: The service batch processing method based on Kubernetes cronjob includes: Get the current task information to be executed; Determine the corresponding service account and command according to the task information; The command is executed using the service account to process the deployment copy of the target space.

2. The service batch processing method based on Kubernetes cronjob according to claim 1 is characterized in that: Before obtaining the information of the current task to be executed, include: Create a service account in the first namespace. Set cluster roles; Bind the service account to the cluster role, and specify permissions for the service account.

3. The service batch processing method based on Kubernetes cronjob according to claim 2 is characterized in that: The permissions include: Get the deployments and deployments / scale resources in the apps and extensions api groups; and / or, Patching the deployments and deployments / scale resources in the apps and extensions api groups; and / or, List the deployments and deployments / scale resources in the apps and extensions api groups; and / or, Monitors the deployments and deployments / scale resources in the apps and extensions api groups.

4. The service batch processing method based on Kubernetes cronjob according to claim 1 is characterized in that: Before the step of obtaining the information of the current task to be executed, the method further includes: Create task information to be executed, wherein the task information includes: task execution time and task execution command.

5. The service batch processing method based on Kubernetes cronjob according to any one of claims 1 to 4, characterized in that: The processing includes: reducing the number of deployment replicas in the target namespace to 0; and / or, Restores a replica of the deployment in the target namespace.

6. The service batch processing method based on Kubernetes cronjob according to claim 5 is characterized in that: After the step of using the service account to execute the command to process the deployment copy of the target space, the method includes: The working status of the database is determined according to the processing result.

7. The service batch processing method based on Kubernetes cronjob according to claim 6 is characterized in that: The step of determining the working status of the database according to the processing result comprises: When the number of deployed replicas in the target namespace is reduced to 0, the database enters a stopped state; or, When the deployment copy in the target namespace is restored to the initial state, the database enters the startup state.

8. A computing device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the service batch processing method based on Kubernetes cronjob according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the service batch processing method based on Kubernetes cronjob described in any one of claims 1 to 7 are implemented.