Method and system for automatically, horizontally, vertically and elastically expanding and shrinking capacity of lifting middleware platform

Abstract

The invention discloses a method and system for improving automatic horizontal and vertical elastic capacity expansion and contraction of a middleware platform. The method comprises the following steps that S1, installation of middleware is constructed, and a Helm Chart product package is deployed; s2, visual deployment is carried out through a KubeSphere platform; s3, deploying a horizontal and vertical capacity expansion and contraction trigger module developed by a Golang language to a cluster; and S4, setting a cyclic feedback mechanism, continuously monitoring a resource state and automatically adjusting the resource state. The invention provides an efficient and extensible solution, and provides a simplified and efficient middleware operation and maintenance automatic management technical method for developers and system administrators.

Description

Technical Field

[0001] The present invention relates to the technical field of software automated operation and maintenance management, and in particular to a method and system for improving the automatic horizontal and vertical elastic expansion and contraction of a middleware platform. Background Art

[0002] With the rapid development of information technology, operation and maintenance work is no longer a simple equipment care, but involves multiple levels such as business resource configuration management, business operation and maintenance management, and middleware maintenance. These tasks not only require the operation and maintenance team to have a deep technical foundation, but also require flexible strategic thinking and forward-looking planning.

[0003] As a bridge between application systems and underlying resources, the stability and performance of middleware are crucial to the operation of the entire system. Therefore, the maintenance of middleware is also an important part that the operation and maintenance team cannot ignore. However, existing technologies cannot quickly and repeatably perform automatic elastic expansion and reduction operations on middleware services, resulting in low efficiency in the operation and maintenance management of middleware. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for improving the automatic horizontal and vertical elastic expansion and contraction of a middleware platform, so as to solve the technical problem of how to achieve fast and repeatable automatic elastic expansion and contraction operations on middleware services.

[0005] The present invention is implemented by adopting the following technical solution: a method for improving the automatic horizontal and vertical elastic expansion and contraction of a middleware platform, comprising the following steps: S1: Build the Helm Chart product package for the installation and deployment of the middleware; S2: Visual deployment through the KubeSphere platform; S3: Deploy the horizontal and vertical scaling trigger module developed in Golang to the cluster; S4: Set up a loop feedback mechanism to continuously monitor resource status and make automatic adjustments.

[0006] Furthermore, step S1 is specifically as follows: building a Helm Chart product package for the installation and deployment of the middleware, which includes the services of the middleware itself and the Exporter service for monitoring indicator collection, and uploading the product package to the application store of the KubeSphere platform for subsequent convenient deployment and management.

[0007] Furthermore, step S2 is specifically as follows: through the visual interface of the KubeSphere platform, the middleware and the Exporter service for monitoring indicator collection are quickly deployed to the Kubernetes cluster to ensure that Prometheus can normally collect monitoring indicator data related to the middleware resources to achieve real-time performance monitoring and resource usage analysis.

[0008] Furthermore, step S3 is specifically as follows: deploying the horizontal and vertical scaling trigger module service developed using the Golang language to the cluster, which is responsible for monitoring and receiving resource alarm events sent by the Alertmanager, and the resource alarm events include CPU, memory and disk usage alarms.

[0009] Furthermore, the horizontal and vertical scaling trigger module makes intelligent judgments based on the received alarm events to decide whether to execute the corresponding resource expansion or reduction operations, and dynamically adjusts the middleware service load by calling the KubeSphere API, and finally waits for and processes the execution results of the expansion or reduction operations.

[0010] A system for improving the automatic horizontal and vertical elastic scaling of a middleware platform includes a product package production module, a visual deployment module, a horizontal and vertical scaling trigger module, and a loop feedback module, wherein the product package production module is used to build a Helm Chart product package for the installation and deployment of the middleware; the visual deployment module is used to perform visual deployment through the KubeSphere platform; the horizontal and vertical scaling trigger module is used to deploy the horizontal and vertical scaling trigger module developed in the Golang language to a cluster; the loop feedback module is used to set up a loop feedback mechanism to continuously monitor resource status and automatically adjust it.

[0011] Furthermore, the product package making module is used to install the middleware and deploy the Helm Chart product package, which includes the middleware's own services and the Exporter service for monitoring indicator collection, and upload the product package to the application store of the KubeSphere platform for subsequent convenient deployment and management.

[0012] Furthermore, the visual deployment module quickly deploys the middleware and the Exporter service for monitoring indicator collection to the Kubernetes cluster through the visual interface of the KubeSphere platform, ensuring that Prometheus can normally collect monitoring indicator data related to the middleware resources to achieve real-time performance monitoring and resource usage analysis.

[0013] Furthermore, the horizontal and vertical scaling trigger module is responsible for monitoring and receiving resource alarm events sent by Alertmanager, which include CPU, memory and disk usage alarms, and making intelligent judgments based on the received alarm events to decide whether to execute corresponding resource expansion or reduction operations, and dynamically adjust the middleware service load by calling the KubeSphere API, and finally wait for and process the execution results of the expansion and reduction operations.

[0014] The beneficial effect of the present invention is that the present invention realizes fast and repeatable automatic elastic expansion and contraction of middleware services by containerizing the middleware, developing HelmChart products that can be quickly installed and deployed, and developing services that receive monitoring alarm events and trigger automatic horizontal and vertical expansion and contraction in combination with the Kubesphere platform using the Golang language. The method provides an efficient and scalable solution, and provides developers and system administrators with a simplified and efficient technical method for automated management of middleware operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0016] Figure 1 It is a flow chart of the present invention; Figure 2 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0018] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0019] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] See also Figure 1 , a method for improving the automatic horizontal and vertical elastic expansion and contraction of a middleware platform, comprising the following steps: S1: Build the Helm Chart product package for the installation and deployment of the middleware; S2: Visual deployment through the KubeSphere platform; S3: Deploy the horizontal and vertical scaling trigger module developed in Golang to the cluster; S4: Set up a loop feedback mechanism to continuously monitor resource status and make automatic adjustments.

[0021] In this embodiment, step S1 specifically includes: building a Helm Chart product package for the installation and deployment of the middleware, which includes the services of the middleware itself and the Exporter service for monitoring indicator collection, and uploading the product package to the application store of the KubeSphere platform for subsequent convenient deployment and management.

[0022] It should be pointed out that KubeSphere is an open source containerization platform built on Kubernetes, designed to simplify and centrally manage containerized applications. It provides a complete set of tools and functions to help users deploy, manage and monitor applications on Kubernetes clusters. The following are some of the main features and functions of KubeSphere: One-stop management interface: KubeSphere provides a user-friendly web interface that allows users to easily manage and operate Kubernetes clusters, applications and other related resources. Application Market: KubeSphere has a built-in application market from which users can select and deploy commonly used application templates to simplify the application deployment and configuration process.

[0023] Helm is an open source Kubernetes application package management tool that simplifies and automates the deployment and management of Kubernetes applications. It provides a packaging format called "Chart" and related tools for defining, packaging, and publishing Kubernetes applications. Here are some key concepts and main features of Helm: Chart: Chart is the packaging format of Helm, which contains a set of files that describe Kubernetes applications, including deployment configurations, dependencies, service definitions, etc. Chart can be regarded as a template for an application, which is used to deploy applications in a Kubernetes cluster.

[0024] In this embodiment, step S2 is specifically: through the visual interface of the KubeSphere platform, the middleware and the Exporter service for monitoring indicator collection are quickly deployed to the Kubernetes cluster to ensure that Prometheus can normally collect monitoring indicator data related to the middleware resources to achieve real-time performance monitoring and resource usage analysis.

[0025] It should be pointed out that Kubernetes (often referred to as K8s) is an open source container orchestration and management platform for automating the deployment, expansion and operation of application containers. It provides a rich set of features to help users simplify the management and operation of containerized applications. The following are some key concepts and main features of Kubernetes: Container orchestration: Kubernetes allows users to define and manage the deployment, scheduling and scaling of multiple containerized applications. It can automatically allocate and schedule containers on nodes in the cluster to ensure high availability and load balancing of applications. Storage orchestration: Kubernetes provides the concept of storage volumes, enabling applications to access persistent storage. It supports multiple storage backends, such as local storage, network storage, and cloud storage, and provides flexible storage orchestration capabilities.

[0026] In this embodiment, step S3 is specifically: deploying the horizontal and vertical scaling trigger module service developed in Golang language to the cluster, responsible for monitoring and receiving resource alarm events sent by Alertmanager, and the resource alarm events include CPU, memory and disk usage alarms. For example: triggering an alarm when the CPU usage exceeds 85%, triggering an alarm when the memory usage exceeds 90%, triggering an alarm when the disk available space is less than 10%, etc.

[0027] In this embodiment, the horizontal and vertical scaling trigger module makes intelligent judgments based on the received alarm events to decide whether to execute the corresponding resource expansion or reduction operations, and dynamically adjusts the middleware service load by calling the KubeSphere API, and finally waits for and processes the execution results of the expansion or reduction operations.

[0028] In this embodiment, step S4 is specifically as follows: a loop feedback mechanism is set to form a self-circulating intelligent expansion and contraction system, which continuously monitors resource status and automatically adjusts, significantly improving the elasticity and adaptability of the middleware platform to better cope with dynamic changes in business loads. Monitoring resource status, including: CPU usage: real-time monitoring of CPU load and analysis of resource consumption of each service. Memory usage: monitoring memory usage and timely detection of memory leaks. Disk IO performance: monitoring disk read and write performance and evaluating the health of the storage system. Network traffic and latency: network data obtained through eBPF, showing network traffic, latency and packet loss rate. Pod abnormal information: such as abnormal Pod restart, expulsion, insufficient resources, etc. Indicators of various middleware themselves: such as kafka production and consumption, redis hits. And according to the feedback resource status, automatic expansion or contraction operations are performed.

[0029] See also Figure 2 The present invention also provides a system for improving the automatic horizontal and vertical elastic scaling of a middleware platform, including a product package production module, a visual deployment module, a horizontal and vertical scaling trigger module and a loop feedback module, wherein the product package production module is used to build a Helm Chart product package for the installation and deployment of the middleware; the visual deployment module is used to perform visual deployment through the KubeSphere platform; the horizontal and vertical scaling trigger module is used to deploy the horizontal and vertical scaling trigger module developed in the Golang language to the cluster; the loop feedback module is used to set a loop feedback mechanism to continuously monitor the resource status and automatically adjust it.

[0030] In this embodiment, the product package making module is used to install the middleware and deploy the HelmChart product package, which includes the middleware's own services and the Exporter service for monitoring indicator collection, and upload the product package to the application store of the KubeSphere platform for subsequent convenient deployment and management.

[0031] In this embodiment, the visual deployment module quickly deploys the middleware and the Exporter service for monitoring indicator collection to the Kubernetes cluster through the visual interface of the KubeSphere platform, ensuring that Prometheus can normally collect monitoring indicator data related to the middleware resources to achieve real-time performance monitoring and resource usage analysis.

[0032] In this embodiment, the horizontal and vertical scaling trigger module is responsible for monitoring and receiving resource alarm events sent by Alertmanager, which include CPU, memory and disk usage alarms, and making intelligent judgments based on the received alarm events to decide whether to execute corresponding resource expansion or reduction operations, and dynamically adjust the middleware service load by calling KubeSphereAPI, and finally wait for and process the execution results of the expansion and reduction operations.

[0033] In this embodiment, the loop feedback module is used to set up a loop feedback mechanism to form a self-circulating intelligent expansion and contraction system, which continuously monitors resource status and automatically adjusts, significantly improving the elasticity and adaptability of the middleware platform to better cope with dynamic changes in business load.

[0034] Based on the above embodiments, the present invention has at least the following technical effects: Flexible innovation: This patented technology can be used to flexibly integrate scattered module data to achieve automatic expansion and contraction.

[0035] Reliability and consistency: Automated execution ensures the consistency and correctness of each deployment, reduces the risk of human error, and provides reliable execution and operating results.

[0036] Reduce operating costs: This method can effectively adapt to dynamic business needs, greatly reduce the possibility of resource waste, improve the elasticity and resource utilization of the middleware platform, and reduce operating costs.

[0037] Improve development efficiency: Transform complex and difficult manual black-screen operation and maintenance operations into automatic monitoring and execution, greatly improving operation and maintenance management efficiency.

[0038] In summary, the present invention containerizes the middleware, develops a Helm Chart product that can be quickly installed and deployed, and uses the Golang language in combination with the Kubesphere platform to develop services that receive monitoring alarm events and trigger automatic horizontal and vertical expansion and contraction, thereby achieving fast and repeatable automatic elastic expansion and contraction operations on middleware services. This method provides an efficient and scalable solution, and provides developers and system administrators with a simplified and efficient technical method for automated management of middleware operation and maintenance. The technical solution proposed in the present invention has good practicality and wide applicability, and provides an innovative solution for the elastic expansion and contraction of the middleware platform, adapting to the needs of the modern cloud computing environment.

[0039] For the aforementioned embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the present application.

[0040] The above embodiments describe the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the changes and modifications made by those skilled in the art shall be within the scope of protection of the appended claims of the present invention without departing from the spirit and scope of the present invention.

Claims

1. A method for improving the automatic horizontal and vertical elastic expansion and contraction of a middleware platform, characterized in that: The steps include: S1: Build the Helm Chart product package for the installation and deployment of the middleware; S2: Visual deployment through the KubeSphere platform; S3: Deploy the horizontal and vertical scaling trigger module developed in Golang to the cluster; S4: Set up a loop feedback mechanism to continuously monitor resource status and make automatic adjustments.

2. The method for improving the automatic horizontal and vertical elastic expansion and contraction of the middleware platform according to claim 1, characterized in that: Step S1 is specifically as follows: build the Helm Chart product package for the installation and deployment of the middleware, which includes the services of the middleware itself and the Exporter service for monitoring indicator collection, and upload the product package to the application store of the KubeSphere platform for subsequent convenient deployment and management.

3. The method for improving the automatic horizontal and vertical elastic expansion and contraction of the middleware platform according to claim 2, characterized in that: Step S2 is specifically as follows: through the visual interface of the KubeSphere platform, the middleware and the Exporter service for monitoring indicator collection are quickly deployed to the Kubernetes cluster to ensure that Prometheus can normally collect monitoring indicator data related to the middleware resources to achieve real-time performance monitoring and resource usage analysis.

4. The method for improving the automatic horizontal and vertical elastic expansion and contraction of the middleware platform according to claim 3, characterized in that: Step S3 is specifically as follows: deploying the horizontal and vertical scaling trigger module service developed in Golang language to the cluster, which is responsible for monitoring and receiving resource alarm events sent by Alertmanager, and the resource alarm events include CPU, memory and disk usage alarms.

5. The method for improving the automatic horizontal and vertical elastic expansion and contraction of the middleware platform according to claim 4, characterized in that: The horizontal and vertical expansion and contraction trigger module makes intelligent judgments based on the received alarm events to decide whether to execute the corresponding resource expansion or contraction operations, and dynamically adjusts the middleware service load by calling the KubeSphere API, and finally waits for and processes the execution results of the expansion and contraction operations.

6. Improve the system of automatic horizontal and vertical elastic expansion and contraction of the middleware platform, characterized by: It includes a product package production module, a visual deployment module, a horizontal and vertical scaling trigger module and a loop feedback module. The product package production module is used to build the Helm Chart product package for the installation and deployment of the middleware; the visual deployment module is used to perform visual deployment through the KubeSphere platform; the horizontal and vertical scaling trigger module is used to deploy the horizontal and vertical scaling trigger module developed in the Golang language to the cluster; the loop feedback module is used to set up a loop feedback mechanism to continuously monitor the resource status and automatically adjust it.

7. The system for improving the automatic horizontal and vertical elastic expansion and contraction of the middleware platform according to claim 6, characterized in that: The product package production module is used to install the middleware and deploy the Helm Chart product package, which includes the middleware's own services and the Exporter service for monitoring indicator collection, and upload the product package to the application store of the KubeSphere platform for subsequent convenient deployment and management.

8. The system for improving the automatic horizontal and vertical elastic expansion and contraction of the middleware platform as claimed in claim 7, characterized in that: The visual deployment module quickly deploys the middleware and the Exporter service for monitoring indicator collection to the Kubernetes cluster through the visual interface of the KubeSphere platform, ensuring that Prometheus can normally collect monitoring indicator data related to the middleware resources to achieve real-time performance monitoring and resource usage analysis.

9. The system for improving the automatic horizontal and vertical elastic expansion and contraction of the middleware platform as claimed in claim 8, characterized in that: The horizontal and vertical scaling trigger module is responsible for monitoring and receiving resource alarm events sent by Alertmanager, which include CPU, memory and disk usage alarms, and making intelligent judgments based on the received alarm events to decide whether to perform corresponding resource expansion or reduction operations, and dynamically adjust the middleware service load by calling the KubeSphere API, and finally wait for and process the execution results of the expansion and reduction operations.

Images