A kind of automatic management and optimization method and device

Abstract

The present invention relates to the field of automated governance, and specifically provides an automated management and optimization method and device, which stores resource declarations of multiple Kubernetes clusters in a Git repository, and manages the status of multiple clusters by automatically synchronizing declarations; a configuration source controller continuously monitors configuration and code changes in the Git repository, and notifies a configuration processing controller of the new configuration; the configuration processing controller processes the changed configuration, and finally integrates and generates a final configuration suitable for the Kubernetes cluster; the application deployment controller applies the new configuration changes to the cluster after receiving the configuration list; and the monitoring alarm controller monitors the process and results of the configuration update, and automatically issues an alarm notification. Compared with the prior art, the present invention can achieve efficient, intelligent and safe management of multiple Kubernetes clusters.

Description

Technical Field

[0001] The present invention relates to the field of automated management, and specifically provides an automated management and optimization method and device. Background Art

[0002] As technologies in the fields of cloud computing and microservices continue to mature, Kubernetes has become the preferred platform for managing large-scale containerized applications. However, the management and operation of Kubernetes clusters still face many challenges, such as complex configuration, cumbersome deployment, low resource utilization, slow fault response, etc. Traditional cluster management methods rely on manual intervention and can only manage one cluster at a time, which is not only inefficient but also prone to errors.

[0003] Therefore, how to achieve automated and intelligent management of multiple Kubernetes clusters has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present invention aims at solving the above-mentioned deficiencies of the prior art and provides an automatic management and optimization method with strong practicability.

[0005] A further technical task of the present invention is to provide an automated management and optimization device that is rationally designed, safe and applicable.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] An automated management and optimization method that stores resource declarations of multiple Kubernetes clusters in a Git repository and manages the status of multiple clusters by automatically synchronizing declarations;

[0008] The configuration source controller continuously monitors the configuration and code changes in the Git repository, automatically pulls the changes into the Kubernetes cluster, and notifies the configuration processing controller of the new configuration;

[0009] The configuration processing controller processes the changed configuration, and finally integrates and generates a final configuration suitable for the Kubernetes cluster, and sends it to the application deployment controller;

[0010] After receiving the configuration list, the application deployment controller automatically applies the new configuration changes to the cluster;

[0011] The monitoring alarm controller monitors the process and results of configuration updates, and automatically issues alarm notifications when problems occur during changes.

[0012] Furthermore, each Kubernetes cluster has its own configuration source controller, which connects to the Git repository and other source code repositories that store configurations. Configurations are managed in a declarative YAML manner, and all configurations are stored in the Git repository configuration source in the form of declarative YAML files.

[0013] Store Kubernetes' YAML configuration files in the configuration repository for organization and management, extract common configurations and differential configurations according to different environments, distinguish between common configurations and differential configurations, organize and manage them according to the environment, and manage differential configuration files for each environment independently;

[0014] The configuration source controller includes two synchronization strategies: scheduled synchronization and event-driven synchronization. The scheduled synchronization will synchronize the latest configuration repository configuration according to the defined time. During event-driven synchronization, the configuration source controller will continuously monitor changes in the configuration repository. When the configuration source controller detects changes in the configuration file, it will automatically extract the latest configuration file, including common configuration and differential configuration for this environment, pull it into this Kubernetes cluster, and trigger subsequent processing.

[0015] Furthermore, when changes need to be made to the cluster configuration or application, the developer makes the changes locally and commits the modified changes to the Git repository;

[0016] Each commit needs to pass code review and testing, and by using Git's version control feature, all configuration changes are tracked to specific commit records.

[0017] Further, the configuration processing controller includes a template engine and a dependency management module;

[0018] The template engine module parses the configuration template language and dynamically generates a Kubernetes resource manifest file, including:

[0019] Parse the template file in the configuration file and generate the corresponding Kubernetes resource list based on the configuration values ​​provided by the user and the placeholders in the template file;

[0020] Parameterized configuration is supported through placeholders in the template file. The placeholders in the template file are replaced according to the configuration values ​​provided by the user;

[0021] Before the template generates the resource list, a syntax check is performed to ensure that there are no syntax errors in the template file. In addition, basic logic verification is performed.

[0022] Furthermore, the dependency management module manages the dependencies of the application, ensures that the application and all its dependent components can be correctly deployed, installed, upgraded, and deleted, and handles version compatibility issues, including:

[0023] Read and parse the dependency files in the configuration file; build a dependency graph based on the dependency of the parsed results to represent the relationship between the application and dependent components, and automatically integrate the deployment order of all installed applications; during version management, the dependency management module handles version issues of dependent components; during dependency upgrades, the dependent components are upgraded synchronously.

[0024] Furthermore, when installing an application in the application deployment controller, if it is the first time to install and deploy the application and there is no corresponding application installed in the Kubernetes cluster, the generated Kubernetes resource list is directly applied to the Kubernetes cluster to complete the application installation;

[0025] When upgrading an application, if there is already a deployed application in the Kubernetes cluster, the generated Kubernetes resource list is compared with the currently deployed application to determine the upgrade operation that needs to be performed;

[0026] Then, incrementally update the applications that need to be upgraded, generate an incrementally updated Kubernetes resource list, and update only the changed parts.

[0027] When deleting an application, if there is an application deployed in the Kubernetes cluster but there is no corresponding application in the Kubernetes resource list, the application will be deleted;

[0028] First, perform a dependency check. Check the dependencies before deleting the application to ensure that the deletion operation does not affect other components that depend on this application. If there are dependencies, cancel the deletion operation and clean up related resources after deleting the application.

[0029] Furthermore, the monitoring alarm controller will record all operation logs, including detailed information on configuration extraction, processing and application, and push messages. When configuration changes or application deployment are completed, notifications will be automatically sent to relevant personnel for real-time feedback and monitoring.

[0030] If an error occurs during synchronization or deployment, an alert notification is sent with detailed error information.

[0031] An automated management and optimization device, comprising: at least one memory and at least one processor;

[0032] The at least one memory is used to store a machine-readable program;

[0033] The at least one processor is used to call the machine-readable program to execute an automated management and optimization method.

[0034] Compared with the prior art, the automated management and optimization method and device of the present invention has the following outstanding beneficial effects:

[0035] The present invention is simple and easy to use, and stores the declarative YAML configuration of the cluster in a Git repository, and all changes are managed through Git commits and merge requests.

[0036] Users can configure and manage applications in multiple Kubernetes clusters simply through a YAML declaration in a Git configuration repository.

[0037] The system automatically monitors the specified Git repository, and once configuration changes are detected, these changes are automatically applied to the Kubernetes cluster, without the need for manual user operation.

[0038] When the system detects configuration changes or abnormal situations, it will automatically send notifications and alarms through defined channels to help users understand the system status in a timely manner. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] A best embodiment is given below:

[0041] An automated management and optimization method in this embodiment is intended to provide an efficient, secure, and intelligent management method for multiple Kubernetes clusters, filling the gap in automated deployment and cluster state management of multiple Kubernetes clusters. The system uses declarative configuration and automated synchronization mechanisms to ensure that the actual state of the cluster is always consistent with the expected state, reduce human operational errors, and improve the stability and maintainability of the multi-Kubernetes cluster system. Through modular design and multi-cluster support, the system can adapt to deployment requirements of different environmental scales and complexities, and can be seamlessly integrated with existing development and operation tool chains, improving the overall level of automation and achieving efficient, intelligent, and secure management of multiple Kubernetes clusters.

[0042] This method manages the status of multiple clusters by storing resource declarations of multiple Kubernetes clusters in a Git repository and automatically synchronizing these declarations. The core of the design consists of four controllers: configuration source controller, configuration processing controller, application deployment controller, and monitoring alarm controller.

[0043] Among them, the configuration source controller continuously monitors the configuration and code changes in the Git repository, automatically pulls these changes into the Kubernetes cluster, and notifies the configuration processing controller of these new configurations. The configuration processing controller processes these changed configurations and finally integrates them to generate a final configuration suitable for the Kubernetes cluster and sends it to the application deployment controller. After receiving the configuration list, the application deployment controller automatically applies these new configuration changes to the cluster. At the same time, the monitoring alarm controller monitors the process and results of these configuration updates, and automatically issues alarm notifications when there are problems with the changes. These controllers work together to achieve automated management and deployment of multiple clusters.

[0044] The following further explains:

[0045] 1. Configuring the source controller

[0046] Each Kubernetes cluster has its own configuration source controller, which is used to connect to Git repositories and other source code repositories that store configurations. The controller supports connecting to multiple configuration sources, such as Git repositories, Helm repositories, S3 storage buckets, etc., and obtains configurations from multiple configuration sources. Configurations are managed in a declarative yaml manner, and all configurations are stored in configuration sources such as Git repositories in the form of declarative YAML files, so that team collaboration updates and review of configuration changes can be easily performed.

[0047] Store Kubernetes' YAML configuration files in the configuration repository for organization and management, and extract common configurations and differential configurations according to different environments (such as development, testing, and production), and distinguish between common configurations and differential configurations. Organize and manage by environment, and manage differential configuration files for each environment independently to facilitate customization and adjustment in different environments.

[0048] The configuration source controller supports two synchronization strategies: scheduled synchronization and event-driven synchronization. Scheduled synchronization will synchronize the latest configuration repository configuration according to the defined time. In event-driven synchronization, the configuration source controller will continuously monitor changes in the configuration repository. When the configuration source controller detects changes in the configuration file, it will automatically extract the latest configuration file, including common configuration and differential configuration for this environment, pull it into the Kubernetes cluster, and trigger subsequent processing. This automatic synchronization mechanism ensures that the actual state of the cluster is always consistent with the expected state defined by the user in the configuration repository.

[0049] Through the configuration source controller of each Kubernetes cluster, applications across multiple Kubernetes clusters can be deployed and managed based on a Git configuration repository, making it easier to manage multiple Kubernetes clusters in large-scale and distributed environments.

[0050] When the cluster configuration or application needs to be modified, the developer makes the change locally and submits the modified change to the Git repository. The permission management mechanism of the Git repository ensures that only authorized personnel can modify the configuration. Each submission needs to pass code review and testing to ensure the correctness and stability of the change. And by using Git's version control feature, all configuration changes can be tracked to specific submission records. Each change has a detailed change record, including the change time, change content, and change personnel, so as to ensure that all changes can be tracked and audited.

[0051] (II) Configuring the Processing Controller

[0052] The configuration processing controller processes the extracted configuration files. It provides a non-intrusive way to customize and manage Kubernetes configurations. By overwriting and adjusting common and differential configurations for different environments, it ensures configuration consistency and flexibility and generates a final configuration suitable for the Kubernetes cluster.

[0053] The configuration processing controller includes two modules: template engine and dependency management.

[0054] The template engine module is used to parse the configured template language and supports dynamic generation of Kubernetes resource manifest files, including:

[0055] (1) Template parsing: Parse the template file in the configuration file and generate the corresponding Kubernetes resource list based on the configuration values ​​provided by the user and the placeholders in the template file.

[0056] (2) Parameterized configuration: Parameterized configuration is supported through placeholders in template files. The placeholders in the template files can be replaced according to the configuration values ​​provided by the user to achieve flexible customization of resources.

[0057] (3) Template validation: Before the template generates a resource list, a syntax check is performed to ensure that the template file has no syntax errors. In addition, basic logical validation is performed, for example, checking whether required fields are provided and whether the resource definition complies with the requirements of the Kubernetes API, thereby ensuring that the generated resource list complies with the Kubernetes specification.

[0058] The dependency management module is responsible for managing application dependencies, ensuring that the application and all its dependent components can be correctly deployed, installed, upgraded, and deleted, and handling version compatibility issues, including:

[0059] (1) Dependency parsing: Read and parse the dependency files in the configuration file to determine the dependencies between the applications to be deployed.

[0060] (2) Dependency sorting: Build a dependency graph based on the dependency relationships of the parsed results to represent the relationship between the application and its dependent components. Automatically integrate the deployment order of all installed applications to ensure that all dependent components are installed in the correct order.

[0061] (3) Version management: The dependency management module handles version issues of dependent components and ensures that the installed dependent component versions are compatible with the main application.

[0062] (4) Dependency upgrade: When an application is upgraded, its dependent components are also upgraded to ensure the compatibility of the entire application ecosystem.

[0063] Based on the collaborative work of the template engine module and the dependency management module, the configuration processing controller can efficiently and reliably manage complex Kubernetes applications and provide the capabilities of automated deployment, upgrade, and management.

[0064] By configuring the processing controller, users can generate different resource lists based on different configuration conditions, realize flexible customization and parameterized configuration of Kubernetes resources, and integrate and generate a final Kubernetes resource list for deployment and use by the application deployment controller.

[0065] 3. Application Deployment Controller

[0066] The application deployment controller is a package management tool for Kubernetes, responsible for the installation, upgrade, and deletion of Kubernetes applications. This module supports version management and rolling upgrades of applications, ensures high availability and stability of applications, and achieves efficient management of Kubernetes resources. After obtaining the Kubernetes resource list generated by the configuration processing controller, the application deployment controller is responsible for deploying it to the Kubernetes cluster, including:

[0067] (1) Application installation: If this is the first installation and deployment of the application and there is no corresponding application installed in the Kubernetes cluster, directly apply the generated Kubernetes resource list to the Kubernetes cluster to complete the application installation.

[0068] (2) Application upgrade: If there are already deployed applications in the Kubernetes cluster, compare the generated Kubernetes resource list with the currently deployed applications to determine the upgrade operations that need to be performed.

[0069] Then, incremental updates are performed on the applications that need to be upgraded, and a Kubernetes resource list for incremental updates is generated. Only the changed parts are updated to ensure that the application does not interrupt service during the upgrade process. In addition, compatibility verification is performed before the upgrade to ensure that the configuration and resources of the new version do not damage the existing applications.

[0070] (3) Application deletion: If there is an application that has been deployed in the Kubernetes cluster but there is no corresponding application in the Kubernetes resource list, the application will be deleted.

[0071] First, check the dependencies before deleting the application to ensure that the deletion operation will not affect other components that depend on this application. If there are dependencies, cancel the deletion operation. After deleting the application, related resources such as configuration files and persistent storage will be cleaned up to release system resources.

[0072] Kubernetes applications can be efficiently managed through the application deployment controller. The installation, upgrade, and deletion of applications can be automated to reduce the complexity and error rate of manual operations. At the same time, the application can be rolled back to the previous stable version when the upgrade fails to ensure the stability of the system.

[0073] (IV) Monitoring and alarm controller

[0074] The monitoring and alarm controller system records all operation logs, including detailed information on configuration extraction, processing, and application. Audit logs can help administrators track issues and conduct compliance reviews, ensuring the security and transparency of the system.

[0075] At the same time, the monitoring alarm controller also integrates a message push platform to realize change notifications and alarms. When the configuration change or application deployment is completed, the system will automatically send a notification to the relevant personnel to provide real-time feedback and monitoring. If an error occurs during synchronization or deployment, the system will also send an alarm notification and provide detailed error information to help developers troubleshoot and solve the problem.

[0076] Based on the above method, an automated management and optimization device in this embodiment includes: at least one memory and at least one processor;

[0077] The at least one memory is used to store a machine-readable program;

[0078] The at least one processor is used to call the machine-readable program to execute an automated management and optimization method.

[0079] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0080] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automated management and optimization method, characterized in that: Store resource declarations for multiple Kubernetes clusters in a Git repository and manage the status of multiple clusters by automatically synchronizing declarations; The configuration source controller continuously monitors the configuration and code changes in the Git repository, automatically pulls the changes into the Kubernetes cluster, and notifies the configuration processing controller of the new configuration; The configuration processing controller processes the changed configuration, and finally integrates and generates a final configuration suitable for the Kubernetes cluster, and sends it to the application deployment controller; After receiving the configuration list, the application deployment controller automatically applies the new configuration changes to the cluster; The monitoring alarm controller monitors the process and results of configuration updates and automatically issues alarm notifications when problems occur in the changes. Each Kubernetes cluster has its own configuration source controller, which connects to the Git repository and other source code repositories that store configurations. Configurations are managed in a declarative YAML manner. All configurations are stored in the Git repository configuration source in the form of declarative YAML files. Store Kubernetes' YAML configuration files in the configuration repository for organization and management, extract common configurations and differential configurations according to different environments, distinguish between common configurations and differential configurations, organize and manage them according to the environment, and manage differential configuration files for each environment independently; The configuration source controller includes two synchronization strategies: scheduled synchronization and event-driven synchronization. The scheduled synchronization will synchronize the latest configuration repository configuration according to the defined time. During event-driven synchronization, the configuration source controller will continuously monitor changes in the configuration repository. When the configuration source controller detects changes in the configuration file, it will automatically extract the latest configuration file, including common configuration and differential configuration for this environment, pull it into this Kubernetes cluster, and trigger subsequent processing.

2. The automated management and optimization method according to claim 1, characterized in that: When changes need to be made to the cluster configuration or application, the developer makes the changes locally and commits the modified changes to the Git repository; Each commit needs to pass code review and testing, and by using Git's version control feature, all configuration changes are tracked to specific commit records.

3. The automated management and optimization method according to claim 2, characterized in that: The configuration processing controller includes a template engine and a dependency management module; The template engine module parses the configuration template language and dynamically generates a Kubernetes resource manifest file, including: Parse the template file in the configuration file and generate the corresponding Kubernetes resource list based on the configuration values ​​provided by the user and the placeholders in the template file; Parameterized configuration is supported through placeholders in the template file. The placeholders in the template file are replaced according to the configuration values ​​provided by the user; Before the template generates the resource list, a syntax check is performed to ensure that there are no syntax errors in the template file. In addition, basic logic verification is performed.

4. The automated management and optimization method according to claim 3, characterized in that: The dependency management module manages the dependencies of the application, ensures that the application and all its dependent components can be correctly deployed, installed, upgraded, and deleted, and handles version compatibility issues, including: Read and parse the dependency files in the configuration file; build a dependency graph based on the dependency of the parsed results to represent the relationship between the application and dependent components, and automatically integrate the deployment order of all installed applications; during version management, the dependency management module handles version issues of dependent components; during dependency upgrades, the dependent components are upgraded synchronously.

5. The automated management and optimization method according to claim 4, characterized in that: When installing an application in the application deployment controller, if it is the first time to install and deploy the application and there is no corresponding application installed in the Kubernetes cluster, the generated Kubernetes resource list is directly applied to the Kubernetes cluster to complete the application installation; When upgrading an application, if there is already a deployed application in the Kubernetes cluster, the generated Kubernetes resource list is compared with the currently deployed application to determine the upgrade operation that needs to be performed; Then, incrementally update the applications that need to be upgraded, generate an incrementally updated Kubernetes resource list, and update only the changed parts. When deleting an application, if there is an application deployed in the Kubernetes cluster but there is no corresponding application in the Kubernetes resource list, the application will be deleted; First, perform a dependency check. Check the dependencies before deleting the application to ensure that the deletion operation does not affect other components that depend on this application. If there are dependencies, cancel the deletion operation and clean up related resources after deleting the application.

6. The automated management and optimization method according to claim 5, characterized in that: The monitoring alarm controller will record all operation logs, including detailed information on configuration extraction, processing and application, and push messages. When configuration changes or application deployment are completed, notifications will be automatically sent to relevant personnel for real-time feedback and monitoring. If an error occurs during synchronization or deployment, an alert notification is sent with detailed error information.

7. An automated management and optimization device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 6.