Methods and apparatus for in-situ upgrades across multiple versions of a Kubernetes cluster

Abstract

This invention provides a method and apparatus for in-situ upgrades of a Kubernetes cluster across multiple versions, relating to the field of distributed technology. The method includes: distributing the new version of the Kubernetes installation package and the necessary upgrade scripts to all nodes in the cluster; after stopping the kube-apiserver and ETCD services, performing an in-situ upgrade of ETCD and its associated objects; sequentially stopping the services of the Master node and Node nodes to complete the in-situ upgrades of the Master and Node nodes; sequentially starting the services on the Master node and Node nodes, and synchronously updating the local files on the Node nodes; and starting the kube-controller-manager service to restore Kubernetes' management of the cluster. This invention overcomes the limitations of in-situ upgrades for Kubernetes products by modifying Kubernetes storage object data and local files, eliminating the need for frequent version-by-version upgrades.

Description

Technical Field

[0001] This invention relates to the field of distributed technology, and in particular to a method and apparatus for in-situ upgrade of a Kubernetes cluster across multiple versions. Background Technology

[0002] In recent years, with the continuous development of the digital economy, cloud computing has played a crucial role as the core of new infrastructure. The core cloud computing technology, Kubernetes (k8s), maintains a high-frequency release schedule of one version every three months. To reap the benefits of Kubernetes' development, upgrading the Kubernetes version of one's cluster is an inevitable choice for major cloud vendors.

[0003] The official Kubernetes community currently uses a biased upgrade approach. This method ensures that high-availability clusters will not experience problems due to component version differences, but it requires that the version difference in each upgrade cannot exceed one minor version, and its compute node component kubelet does not support in-place upgrades; all containers must be manually removed before an upgrade. Clearly, this upgrade method has many limitations.

[0004] To address these limitations, the industry has modified the Kubernetes source code based on the official in-place upgrade solution: by adding a new cluster upgrade branch to the existing logic, the range of new and old component versions that the deviation strategy of Kubernetes upgrade can support is expanded, while also handling the in-place upgrade of the kubelet component.

[0005] However, industry-standard solutions can compromise the integrity of Kubernetes products. Furthermore, these solutions can introduce intrusive modifications to the original logic, affecting product security and robustness, introducing significant risks, and incurring high development and testing costs. Additionally, code remnants from a single upgrade process can reduce product maintainability and reusability. Summary of the Invention

[0006] In view of this, the present invention provides a method and apparatus for in-situ upgrade of a Kubernetes cluster across multiple versions, in order to solve at least one of the problems mentioned above.

[0007] To achieve the above objectives, the present invention adopts the following solution:

[0008] According to a first aspect of the present invention, a method for in-situ upgrade of a Kubernetes cluster across multiple versions is provided. The method includes: distributing a new version of the Kubernetes installation package and upgrade scripts to all nodes in the cluster; stopping the kube-apiserver and ETCD services of the Kubernetes cluster to put the Kubernetes cluster into a maintenance phase; performing an in-situ upgrade of ETCD using upgrade scripts, and upgrading the objects stored in ETCD after the in-situ upgrade of ETCD is completed; stopping all services of the Master node and performing an in-situ upgrade of the Master node using upgrade scripts; stopping all services of the Node nodes and performing an in-situ upgrade of the Node nodes using upgrade scripts; sequentially starting the kube-apiserver and kube-scheduler services on the Master node; starting the services on the Node nodes, calling the kube-apiserver interface on the host machine of each Node node to obtain container object information, and rewriting local files based on the container dimension according to the obtained container object information to ensure compatibility with the new version of Kubernetes; and starting the kube-controller-manager service of the Master node to restore Kubernetes management of the cluster.

[0009] As an embodiment of the present invention, after starting the kube-controller-manager service of the Master node to restore k8s management of the cluster in the above method, the method further includes: performing a cluster health check operation by calling the k8s API interface to check for service unavailability or container restart issues; and updating the cluster status and version information in the k8s cluster management system.

[0010] As an embodiment of the present invention, the method described above for distributing the new version of the k8s installation package and the upgrade script to all nodes in the cluster includes: checking the component status of the k8s cluster, determining whether the k8s cluster meets the upgrade conditions, and in response to the k8s cluster meeting the upgrade conditions, distributing the new version of the k8s installation package and the upgrade script to all nodes in the cluster.

[0011] As an embodiment of the present invention, the above method for performing an in-situ upgrade of ETCD using an upgrade script includes: executing an ETCD snapshot, creating a backup of the current ETCD data, and then backing up the relevant Kubernetes cluster configuration files and system configurations; and performing an in-situ upgrade of ETCD using the upgrade script and the new version of the Kubernetes installation package.

[0012] As an embodiment of the present invention, the above method involves performing an ETCD snapshot, creating a backup of the current ETCD data, and then backing up the relevant Kubernetes cluster configuration files and system configurations, including: determining the node address, port number, and required certificate authentication information of the node running the ETCD cluster; using the etcdctl tool to create an ETCD snapshot based on the node address, port number, and certificate path, and checking the status of the snapshot file to ensure that the snapshot is created successfully and the data is complete; storing the successfully created snapshot file on other servers or cloud storage services; and backing up the configuration files of the Master node and Node nodes, the configuration files of static Pods, the network configuration files, the service account keys, and the encrypted configuration files.

[0013] As an embodiment of the present invention, after distributing the new version of the Kubernetes installation package and the upgrade scripts to all nodes in the cluster, the method further includes: assessing the resources required during the upgrade process and auditing the existing resource usage; performing the upgrade during periods of low system load based on the assessment and audit results to reduce the impact on business; and setting resource quotas and limits based on the assessment and audit results to ensure that the upgrade operation does not consume too many resources.

[0014] As an embodiment of the present invention, the above method for assessing the resources required during the upgrade process and auditing the existing resource usage includes: assessing the resources required for the upgrade operation itself, including CPU, memory, storage space and network bandwidth; assessing the changes in business load during the upgrade operation and predicting the system load in different time periods; and using monitoring tools to audit the resource usage of the existing Kubernetes cluster, including node resource utilization, Pod resource requests and limit settings, and service traffic patterns.

[0015] According to a second aspect of the present invention, an in-situ upgrade apparatus for a Kubernetes cluster across multiple versions is provided. The apparatus includes: a distribution unit for distributing a new version of the Kubernetes installation package and upgrade scripts to all nodes in the cluster; a maintenance unit for stopping the kube-apiserver and ETCD services of the Kubernetes cluster to put the Kubernetes cluster into a maintenance phase; an ETCD upgrade unit for performing an in-situ upgrade of the ETCD using upgrade scripts, and upgrading objects stored in the ETCD after the in-situ upgrade is completed; a Master upgrade unit for stopping all services of the Master node and performing an in-situ upgrade of the Master node using upgrade scripts; and a Node upgrade unit for stopping all services of the Node nodes. The service utilizes an upgrade script to perform an in-situ upgrade of the Node node; the Master recovery unit is used to sequentially start the kube-apiserver and kube-scheduler services on the Master node, and after the Node recovery unit restores the services, start the kube-controller-manager service on the Master node to restore Kubernetes' management of the cluster; the Node recovery unit is used to start the services on the Node node, call the kube-apiserver interface on the host machine of each Node node to obtain container object information, and rewrite local files based on the obtained container object information at the container dimension to ensure compatibility with the new version of Kubernetes.

[0016] As an embodiment of the present invention, the above-mentioned apparatus further includes: a health check unit, used to perform a cluster health check operation by calling the K8s API interface after the Master recovery unit starts the kube-controller-manager service of the Master node to restore k8s management of the cluster, to check whether there are service unavailability or container restart problems; and a status update unit, used to update the status and version information of the cluster in the K8s cluster management system.

[0017] As an embodiment of the present invention, the above-mentioned distribution unit distributes the new version of the k8s installation package and the upgrade script to all nodes in the cluster by: checking the component status of the k8s cluster, determining whether the k8s cluster meets the upgrade conditions, and in response to the k8s cluster meeting the upgrade conditions, distributing the new version of the k8s installation package and the upgrade script to all nodes in the cluster.

[0018] In one embodiment of the present invention, the above-mentioned ETCD upgrade unit performs an in-situ upgrade of ETCD using an upgrade script, which includes: executing an ETCD snapshot, creating a backup of the current ETCD data, and then backing up the relevant Kubernetes cluster configuration files and system configurations; and performing an in-situ upgrade of ETCD using the upgrade script and the new version of the Kubernetes installation package.

[0019] In one embodiment of the present invention, the ETCD upgrade unit performs an ETCD snapshot, creating a backup of the current ETCD data, and then backing up the relevant Kubernetes cluster configuration files and system configurations, including: determining the node address, port number, and required certificate authentication information of the node running the ETCD cluster; creating an ETCD snapshot using the etcdctl tool based on the node address, port number, and certificate path; checking the status of the snapshot file to ensure that the snapshot is created successfully and the data is complete; storing the successfully created snapshot file on another server or cloud storage service; and backing up the configuration files of the Master node and Node nodes, the configuration files of static Pods, the network configuration files, the service account keys, and the encrypted configuration files.

[0020] In one embodiment of the present invention, the above-mentioned apparatus further includes: a resource assessment unit, used to assess the resources required during the upgrade process and audit the existing resource usage; an upgrade time period selection unit, used to perform the upgrade during periods of low system load based on the assessment results and audit results to reduce the impact on business; and an upgrade resource configuration unit, used to set resource quotas and limits based on the assessment results and audit results to ensure that the upgrade operation does not consume too many resources.

[0021] In one embodiment of the present invention, the resource assessment unit assesses the resources required during the upgrade process and audits the existing resource usage, including: assessing the resources required for the upgrade operation itself, including CPU, memory, storage space and network bandwidth; assessing the changes in business load during the upgrade operation and predicting the system load in different time periods; and using monitoring tools to audit the resource usage of the existing Kubernetes cluster, including node resource utilization, Pod resource requests and limit settings, and service traffic patterns.

[0022] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0023] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0024] As can be seen from the above technical solution, the in-situ upgrade method and apparatus for Kubernetes clusters across multiple versions provided by this invention breaks through the constraints of in-situ upgrades of Kubernetes products. While ensuring the integrity of the Kubernetes product, it expands the upgradeable span of Kubernetes by modifying Kubernetes storage object data and local files, eliminating the need for frequent version-by-version upgrades. Simultaneously, it solves the drawback of needing to restart containers during Kubernetes version upgrades, achieving seamless and lossless Kubernetes version upgrades. Specifically, it first performs data backup through ETCD snapshots, modifies Kubernetes storage object data to complete the Kubernetes object upgrade transformation, prepares data for local file upgrades, and then calls the kube-apiserver interface to obtain the upgraded container object information. Based on the obtained container object information, it upgrades local files at the container level. Furthermore, the entire upgrade process reasonably applies Kubernetes' deviation strategy by stopping the kube-controller-manager service and adopting a certain start-stop sequence, resolving version compatibility issues between various components and the kube-apiserver component interface during the upgrade process. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0026] Figure 1 This is a flowchart illustrating a method for in-situ upgrades across multiple versions of a Kubernetes cluster, as provided in an embodiment of the present invention.

[0027] Figure 2 This application provides a Kubernetes component access relationship diagram in its embodiments.

[0028] Figure 3 This is a schematic diagram of the process of performing an in-situ upgrade of ETCD using an upgrade script, provided in an embodiment of this application.

[0029] Figure 4 This is a flowchart illustrating the process of executing ETCD snapshots and backing up key files of a Kubernetes cluster, provided in an embodiment of this application.

[0030] Figure 5 This is a schematic diagram of the in-situ upgrade method for a Kubernetes cluster across multiple versions provided in this application embodiment;

[0031] Figure 6 This is a schematic diagram of a cross-version in-situ upgrade device for a Kubernetes cluster provided in an embodiment of the present invention;

[0032] Figure 7 This is a schematic block diagram of the system configuration of the electronic device provided in the embodiments of the invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0034] The information collected in the technical solution of this application is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, and necessary confidentiality measures have been taken. This does not violate public order and good morals, and a corresponding operation entry point is provided for the user to choose to authorize or refuse. A corresponding operation entry point is provided for the user to choose to agree to or refuse the automated decision-making result; if the user chooses to refuse, the process proceeds to the expert decision-making process.

[0035] like Figure 1 The diagram shown is a flowchart illustrating a method for in-situ upgrades across multiple versions of a Kubernetes cluster, as provided in this embodiment of the invention. This embodiment describes the invention from the perspective of the upgrade script. The method includes the following steps:

[0036] Step S101: Distribute the new version of the Kubernetes installation package and the upgrade scripts to all nodes in the cluster.

[0037] This step is a preparatory step for the in-situ upgrade method for Kubernetes clusters across multiple versions in this application. In-situ upgrade means directly replacing the existing software package with the new version and re-running the service based on the new software. This method eliminates the need to migrate the cluster, ensures uninterrupted operation of nodes, and maintains business continuity. Therefore, before performing an in-situ upgrade, the Kubernetes installation package and the necessary upgrade scripts need to be installed on all nodes in the cluster.

[0038] Depending on the architecture of the Kubernetes cluster and the role of each node, different types of nodes require different installation packages and upgrade scripts. A Kubernetes cluster primarily consists of two types of nodes: Master nodes and Node nodes. Each type of node undertakes different responsibilities, therefore the required software packages and configurations differ during the upgrade process. Figure 2The diagram shows the access relationships between Kubernetes components. The Kubernetes component kube-apiserver exposes an API Server interface, providing external interface services and acting as a bridge for communication with other components. It also converts all information into API objects (resources) and stores them in ETCD. The Kubernetes component kubelet internally connects to the Docker runtime, responsible for the management and maintenance of application containers. For the Docker runtime, containers correspond to user processes and local files.

[0039] The Master node is part of the control plane and is responsible for the management and scheduling of the entire cluster. This mainly includes the installation and updating of the following components:

[0040] kube-apiserver: Provides Kubernetes API services and is the front end of the control plane.

[0041] kube-scheduler: Responsible for scheduling decisions, selecting the appropriate Node for newly created Pods.

[0042] kube-controller-manager: Runs the controller and handles cluster-level functions such as node addition and removal.

[0043] etcd: Distributed key-value store used to save the state of the entire cluster.

[0044] Therefore, the upgrade installation package and scripts for the Master node need to include the updated content for these components.

[0045] A Node is a node that runs the actual application container, and mainly includes the installation and updates of the following components:

[0046] kubelet: Runs on each Node and is responsible for starting Pods and containers (such as Dcoker) and reporting to the Master.

[0047] kube-proxy: Maintains network rules on Nodes and enables network connectivity for services.

[0048] The upgrade packages and scripts for Node are primarily designed for these components.

[0049] Therefore, during the upgrade process, this application can select appropriate installation packages and scripts for distribution based on the role of each node: Different installation packages and scripts are prepared: for Master and Node nodes, corresponding upgrade packages and scripts are prepared, ensuring they contain the correct versions of components and the required configuration changes. By carefully preparing and using automated tools to distribute installation packages and upgrade scripts, the upgrade process for the Kubernetes cluster can be ensured to be both efficient and secure.

[0050] Therefore, preferably, this step may specifically include: checking the component status of the Kubernetes cluster, determining whether the Kubernetes cluster meets the upgrade conditions, and in response to the Kubernetes cluster meeting the upgrade conditions, distributing the new version of the Kubernetes installation package and the upgrade scripts to all nodes in the cluster.

[0051] In a further preferred embodiment, in addition to the above-mentioned preparatory work, the following additional preparatory work may be included: performing cluster health check operations by calling the K8s API interface to check for service unavailability or container restart issues; and updating the cluster status and version information in the K8s cluster management system.

[0052] The cluster health check operations here can check items including node status, Pod status, and system component status. Pay particular attention to whether any nodes are in a NotReady state or whether any Pods are frequently restarting (too many restarts), as these may indicate cluster problems. Additionally, perform availability checks on critical services within the cluster to ensure normal service response. This can be done by calling the service's health check endpoint or executing actual service requests. During health checks, log any problems and anomalies found in detail; this information is crucial for diagnosing problems and planning upgrade strategies.

[0053] Before upgrading, this application can also record detailed configuration, status, and version information of the current cluster. This includes, but is not limited to, version information of each node, running service versions, and snapshots of configuration files. For example, cluster version information can be obtained using kubectlversion or by accessing the Kubernetes API. Backing up the current state and configuration information of the cluster to a secure location is crucial in case of problems requiring a rollback during or after the upgrade.

[0054] By performing these supplementary preparatory tasks, the current health status of the cluster can be more comprehensively assessed, providing important basic information for the upgrade, and ensuring that problems during and after the upgrade can be quickly located and resolved. This not only improves the success rate of the upgrade but also helps maintain the long-term stability and security of the cluster.

[0055] Step S102: Stop the kube-apiserver and ETCD services of the k8s cluster to put the k8s cluster into maintenance.

[0056] Stopping the kube-apiserver and ETCD services on the Kubernetes cluster prevents data inconsistencies or state changes during critical operations, ensuring the security of upgrades or maintenance. Furthermore, stopping the kube-apiserver and ETCD services on the Kubernetes cluster does not affect running container instances; they can continue to provide services normally because already started container instances are managed directly by the kubelets on each Node, rather than being dynamically scheduled through the kube-apiserver.

[0057] Step S103: Use the upgrade script to perform an in-situ upgrade on ETCD, and upgrade the objects saved in ETCD after the in-situ upgrade is completed.

[0058] Preferred, such as Figure 3 As shown, this step involves using the upgrade script to perform an in-situ upgrade of ETCD, including:

[0059] Step 301: Execute an ETCD snapshot to create a backup of the current ETCD data, and then back up the relevant Kubernetes cluster configuration files and system configurations.

[0060] More preferably, this step may include, for example: Figure 4 The sub-steps shown:

[0061] Step S3011: Determine the node address, port number, and required certificate authentication information for running the ETCD cluster.

[0062] Step S3012: Create an ETCD snapshot using the etcdctl tool based on the node address, the port number, and the certificate path.

[0063] Step S3013: Check the status of the snapshot file to ensure that the snapshot was created successfully and the data is complete.

[0064] Step S3014: Store the successfully created snapshot file on another server or cloud storage service.

[0065] Step S3015: Back up the configuration files of the Master node and Node nodes, the configuration files of static Pods, the network configuration files, the service account keys, and the encrypted configuration files.

[0066] After completing the above steps, you can obtain a complete ETCD data snapshot and a backup of the cluster configuration file, which provides a solid foundation for in-situ upgrades of ETCD and any necessary disaster recovery operations.

[0067] Step S302: Use the upgrade script and the new version of the k8s installation package to perform an in-situ upgrade of ETCD.

[0068] As can be seen from the above, after performing an in-situ upgrade of ETCD, this application immediately upgrades the objects stored in ETCD. This is because cross-version upgrades in a Kubernetes cluster mean that the data model stored in ETCD may also need to be updated to adapt to the new version of API and resource definitions. By updating the objects stored in ETCD immediately after the ETCD upgrade is completed, it is possible to ensure that the data model is compatible with the new version of Kubernetes, thereby avoiding errors or anomalies caused by version mismatch.

[0069] Objects in ETCD refer to various Kubernetes resource objects, including but not limited to Pods (the basic deployment unit in Kubernetes, each Pod can contain one or more containers), Services (defining a way to access Pods, typically used as load balancers), Deployments and StatefulSets (used to manage the deployment and scaling of Pods), ConfigMaps and Secrets (used to store configuration data and sensitive information), Roles and RoleBindings (defining access control rules for Kubernetes resources), and Network Policies (defining network access policies between Pods). When performing an ETCD upgrade, these objects stored in ETCD are updated for reasons including but not limited to the following:

[0070] 1) API version update: The Kubernetes API evolves with version updates. Some API versions may be deprecated and removed in subsequent versions. If objects in ETCD still use older API versions, they may not be correctly recognized or processed by newer versions of Kubernetes.

[0071] 2) Data model changes: In some Kubernetes version updates, the data model of resource objects may change. In this case, the objects stored in ETCD need to be updated to the new data model to ensure the normal operation of the cluster.

[0072] 3) Leveraging new features: New versions of Kubernetes may introduce new features or fields. Updating objects allows these objects to take advantage of the improvements and features provided by the new version.

[0073] Step S104: Stop all services of the Master node and use the upgrade script to complete the in-situ upgrade of the Master node.

[0074] Specifically, this means stopping services such as kube-scheduler and kube-controller-manager on the Master node. The method for stopping a service depends on how it is run, such as whether it is a system service or a Pod.

[0075] Step S105: Stop all services of the Node and use the upgrade script to complete the in-situ upgrade of the Node.

[0076] Step S106: Start the kube-apiserver and kube-scheduler services on the Master node in sequence.

[0077] Starting kube-apiserver: First, start the kube-apiserver service on the Master node, as it is the server for the Kubernetes API, and other components rely on it for communication. After starting, you also need to verify that the service started successfully and that you can communicate normally with the cluster via kubectl or other API clients.

[0078] Start kube-scheduler: After confirming that kube-apiserver is running normally, start the kube-scheduler service, which is responsible for scheduling Pods to appropriate Nodes. After starting, you need to check the logs or status to confirm that kube-scheduler is running normally.

[0079] Step S107: Start the service on the Node node, call the kube-apiserver interface on the host machine of each Node node to obtain the upgraded container object information, and rewrite the local storage file based on the container dimension according to the obtained container object information to ensure compatibility with the new version of k8s.

[0080] In other words, when upgrading the Kubernetes cluster, this application simultaneously upgrades the local storage files on the host machine and the ETCD storage data on the remote Master node, ensuring that the content represented by both is consistent and matches the Kubernetes version. This allows different versions of Kubernetes component services to smoothly take over the modified resource objects after startup. Here, the resource objects are the operation resource objects of the API interface provided by Kubernetes, and different resource objects correspond to different functions.

[0081] Step S108: Start the kube-controller-manager service on the Master node to restore Kubernetes' management of the cluster.

[0082] As can be seen from steps S106-S108 above, the upgraded startup sequence of this application follows a specific order: first, the kube-apiserver and kube-scheduler services on the Master node are started; then, the services on the Node nodes are started; and finally, the kube-controller-manager service on the Master node is started. This specific startup sequence can bring the following beneficial effects:

[0083] 1. Ensure nodes are ready.

[0084] Starting services on the Nodes (especially kubelets) allows these nodes to rejoin the cluster and be recognized by the kube-apiserver. This ensures that when the kube-controller-manager starts, it can correctly see the latest status of all nodes and make decisions based on this information.

[0085] 2. Avoid scheduling too early.

[0086] If kube-controller-manager starts too early, it may attempt to schedule Pods or execute other control logic based on incomplete or outdated cluster state information, which may lead to unnecessary rescheduling or other conflicts, especially during upgrades where the cluster state changes rapidly.

[0087] 3. Manage resources and workload

[0088] Once the services on the Node nodes are started and these nodes are confirmed to be healthy, kube-controller-manager can manage resources and workloads more accurately. For example, it can automatically scale the number of Pod replicas as needed, or reschedule Pods to healthy nodes when necessary.

[0089] Preferred, such as Figure 4 As shown, after step S101, which distributes the new version of the Kubernetes installation package and the upgrade scripts to all nodes in the cluster, the method may further include the following steps:

[0090] Step S401: Assess the resources required for the upgrade process and audit the current resource usage.

[0091] Specifically, this step may further include, for example: Figure 5 The sub-steps shown:

[0092] Step S4011: Assess the resources required for the upgrade operation itself, including CPU, memory, storage space, and network bandwidth.

[0093] Step S4012: Assess changes in business load during the upgrade operation and predict system load at different time periods.

[0094] Step S4013: Use monitoring tools to audit the resource usage of the existing Kubernetes cluster, including node resource utilization, Pod resource requests and limit settings, and service traffic patterns.

[0095] Step S402: Based on the assessment and audit results, perform the upgrade during periods of low system load to reduce the impact on business, such as at night.

[0096] Step S403: Based on the assessment and audit results, set resource quotas and limits to ensure that the upgrade operation does not consume too many resources.

[0097] These steps ensure that Kubernetes cluster upgrades are not only successfully executed, but also minimize the impact on running services, while guaranteeing the effective utilization of cluster resources and the continuous stability of business operations.

[0098] The schematic diagrams for steps S101-S108 above can be found in [reference needed]. Figure 5 ,Depend on Figure 5 As can be seen, the key to this upgrade solution lies in modifying the local storage files on the host machine and the remote ETCD storage data at the appropriate time before and after the Kubernetes component upgrade, ensuring that the content represented by both is consistent and matches the Kubernetes version. This allows different versions of Kubernetes component services to successfully take over the modified resource objects after startup.

[0099] Therefore, the in-situ upgrade method for Kubernetes clusters across multiple versions provided by this invention breaks through the constraints of in-situ upgrades of Kubernetes products. While ensuring the integrity of the Kubernetes product, it expands the upgradeable scope of Kubernetes by modifying Kubernetes storage object data and local files, eliminating the need for frequent version-by-version upgrades. Simultaneously, it solves the drawback of needing to restart containers during Kubernetes version upgrades, achieving seamless and lossless Kubernetes version upgrades. Specifically, it first performs data backup through ETCD snapshots, modifies Kubernetes storage object data to complete the Kubernetes object upgrade transformation, prepares data for local file upgrades, and then calls the kube-apiserver interface to obtain the upgraded container object information. Based on the obtained container object information, it upgrades local files at the container level. Furthermore, the entire upgrade process reasonably applies Kubernetes' deviation strategy by stopping the kube-controller-manager service and using a specific start-stop sequence, resolving version compatibility issues between various components and the kube-apiserver component interface during the upgrade process.

[0100] like Figure 6The diagram shows a schematic representation of a cross-version in-situ upgrade device for a Kubernetes cluster provided in an embodiment of this application. The device includes: a distribution unit 610, a maintenance unit 620, an ETCD upgrade unit 630, a Master upgrade unit 640, a Node upgrade unit 650, a Master recovery unit 660, and a Node recovery unit 670, which are arranged sequentially and adjacently.

[0101] Distribution unit 610 is used to distribute the new version of the Kubernetes installation package and the scripts required for upgrading to all nodes in the cluster.

[0102] Maintenance unit 620 is used to stop the kube-apiserver and ETCD services of the k8s cluster, so that the k8s cluster enters the maintenance phase;

[0103] The ETCD upgrade unit 630 is used to perform in-situ upgrades on the ETCD using an upgrade script, and to upgrade the objects stored in the ETCD after the in-situ upgrade is completed.

[0104] Master upgrade unit 640 is used to stop all services of the Master node and use the upgrade script to complete the in-situ upgrade of the Master node;

[0105] The Node upgrade unit 650 is used to stop all services of the Node and complete the in-situ upgrade of the Node using an upgrade script.

[0106] Master recovery unit 660 is used to sequentially start the kube-apiserver and kube-scheduler services on the Master node, and after the Node recovery unit 670 restores the services, start the kube-controller-manager service on the Master node to restore Kubernetes' management of the cluster;

[0107] The Node recovery unit 670 is used to start the services on the Node node. On the host machine of each Node node, it calls the kube-apiserver interface to obtain container object information and rewrites local files based on the container dimension according to the obtained container object information to ensure compatibility with the new version of K8s.

[0108] Preferably, the above apparatus further includes: a health check unit, used to perform cluster health check operations by calling the K8s API interface after the Master recovery unit starts the kube-controller-manager service of the Master node to restore k8s management of the cluster, to check for service unavailability or container restart issues; and a status update unit, used to update the cluster status and version information in the K8s cluster management system.

[0109] Preferably, the distribution unit 610 distributes the new version of the Kubernetes installation package and the upgrade scripts to all nodes in the cluster by: checking the component status of the Kubernetes cluster, determining whether the Kubernetes cluster meets the upgrade conditions, and in response to the Kubernetes cluster meeting the upgrade conditions, distributing the new version of the Kubernetes installation package and the upgrade scripts to all nodes in the cluster.

[0110] Preferably, the ETCD upgrade unit 630 performs an in-situ upgrade of ETCD using an upgrade script, which includes: executing an ETCD snapshot, creating a backup of the current ETCD data, and then backing up the relevant Kubernetes cluster configuration files and system configurations; and performing an in-situ upgrade of ETCD using the upgrade script and the new version of the Kubernetes installation package.

[0111] Preferably, the ETCD upgrade unit 630 performs an ETCD snapshot, creating a backup of the current ETCD data, and then backing up the relevant Kubernetes cluster configuration files and system configurations, including: determining the node address, port number, and required certificate authentication information of the node running the ETCD cluster; creating an ETCD snapshot using the etcdctl tool based on the node address, port number, and certificate path; checking the status of the snapshot file to ensure that the snapshot was created successfully and the data is complete; storing the successfully created snapshot file on another server or cloud storage service; and backing up the configuration files of the Master node and Node nodes, the configuration files of static Pods, the network configuration files, the service account keys, and the encrypted configuration files.

[0112] Preferably, the above-mentioned apparatus further includes: a resource assessment unit for assessing the resources required during the upgrade process and auditing the existing resource usage; an upgrade time selection unit for performing the upgrade during periods of low system load based on the assessment and audit results to reduce the impact on business; and an upgrade resource configuration unit for setting resource quotas and limits based on the assessment and audit results to ensure that the upgrade operation does not consume excessive resources.

[0113] Preferably, the resource assessment unit assesses the resources required during the upgrade process and audits the existing resource usage, including: assessing the resources required for the upgrade operation itself, including CPU, memory, storage space, and network bandwidth; assessing changes in business load during the upgrade operation and predicting the system load in different time periods; and using monitoring tools to audit the resource usage of the existing Kubernetes cluster, including node resource utilization, Pod resource requests and limit settings, and service traffic patterns.

[0114] As can be seen from the above technical solution, the in-situ upgrade device for Kubernetes clusters across multiple versions provided by this invention breaks through the constraints of in-situ upgrades of Kubernetes products. While ensuring the integrity of the Kubernetes product, it expands the upgradeable span of Kubernetes by modifying Kubernetes storage object data and local files, eliminating the need for frequent version-by-version upgrades. Simultaneously, it solves the drawback of needing to restart containers during Kubernetes version upgrades, achieving seamless and lossless Kubernetes version upgrades. Specifically, it first performs data backup through ETCD snapshots, modifies Kubernetes storage object data to complete the Kubernetes object upgrade transformation, prepares data for local file upgrades, and then calls the kube-apiserver interface to obtain the upgraded container object information. Based on the obtained container object information, it upgrades local files at the container level. Furthermore, the entire upgrade process reasonably applies Kubernetes' deviation strategy by stopping the kube-controller-manager service and using a specific start-stop sequence, resolving version compatibility issues between various components and the kube-apiserver component interface during the upgrade process.

[0115] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method.

[0116] This invention also provides a computer-readable storage medium storing a computer program for performing the above-described methods.

[0117] like Figure 7 The electronic device 600 may also include: a communication module 110, an input unit 120, an audio processor 130, a display 160, and a power supply 170. It is worth noting that the electronic device 600 does not necessarily need to include these components. Figure 7 All components shown; in addition, electronic device 600 may also include Figure 7 The components shown can be referenced in the prior art.

[0118] like Figure 7The central processing unit 100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device. The central processing unit 100 receives inputs and controls the operation of various components of the electronic device 600.

[0119] The memory 140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 100 may execute the program stored in the memory 140 to perform information storage or processing, etc.

[0120] Input unit 120 provides input to central processing unit 100. Input unit 120 may be, for example, a keypad or touch input device. Power supply 170 provides power to electronic device 600. Display 160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0121] The memory 140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 140 can also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application / function storage unit 142 for storing application and function programs or processes for executing the operation of the electronic device 600 via the central processing unit 100.

[0122] The memory 140 may also include a data storage unit 143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 144 of the memory 140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0123] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processing unit 100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0124] Based on different communication technologies, multiple communication modules 110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide audio output via the speaker 131 and receive audio input from the microphone 132, thereby enabling typical telecommunications functions. The audio processor 130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 130 is coupled to a central processing unit 100, enabling on-device recording via the microphone 132 and on-device playback of stored audio via the speaker 131.

[0125] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0129] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for in-situ upgrades across multiple versions of a Kubernetes cluster, characterized in that, The method includes: Distribute the new version of the Kubernetes installation package and the necessary upgrade scripts to all nodes in the cluster; Stopping the kube-apiserver and ETCD services of the Kubernetes cluster puts the Kubernetes cluster into maintenance mode. The upgrade script is used to perform an in-place upgrade of ETCD, and the objects stored in ETCD are upgraded after the in-place upgrade of ETCD is completed; Stop all services on the Master node and use the upgrade script to complete the in-situ upgrade of the Master node; Stop all services of the Node and use the upgrade script to complete the in-situ upgrade of the Node; Start the kube-apiserver and kube-scheduler services on the Master node in sequence; Start the services on the Node, call the kube-apiserver interface on the host machine of each Node to obtain the upgraded container object information, and rewrite the local files based on the container dimension according to the obtained container object information to ensure compatibility with the new version of K8s; Start the kube-controller-manager service on the Master node to restore Kubernetes' management of the cluster.

2. The method for in-situ upgrade of a Kubernetes cluster across multiple versions as described in claim 1, characterized in that, After starting the kube-controller-manager service on the Master node to restore Kubernetes management of the cluster, the method further includes: Perform cluster health checks by calling the K8s API to check for service unavailability or container restart issues. Update the cluster status and version information in the K8s cluster management system.

3. The method for in-situ upgrade of a Kubernetes cluster across multiple versions as described in claim 1, characterized in that, The process of distributing the new version of the Kubernetes installation package and the necessary upgrade scripts to all nodes in the cluster includes: The component status of the Kubernetes cluster is checked to determine whether the Kubernetes cluster meets the upgrade conditions. In response to the Kubernetes cluster meeting the upgrade conditions, the new version of the Kubernetes installation package and the upgrade scripts are distributed to all nodes in the cluster.

4. The in-situ upgrade method for a Kubernetes cluster across multiple versions as described in claim 1, characterized in that, The in-situ upgrade of ETCD using the upgrade script includes: Execute an ETCD snapshot to create a backup of the current ETCD data, and then back up the relevant Kubernetes cluster configuration files and system configurations; Use the upgrade script and the new version of the Kubernetes installation package to perform an in-place upgrade of ETCD.

5. The method for in-situ upgrade of a Kubernetes cluster across multiple versions as described in claim 4, characterized in that, The process of executing an ETCD snapshot, creating a backup of the current ETCD data, and then backing up the relevant Kubernetes cluster configuration files and system configurations includes: Determine the node address, port number, and required certificate authentication information for running the ETCD cluster; An ETCD snapshot is created using the etcdctl tool based on the node address, the port number, and the certificate path. Check the status of the snapshot file to ensure that the snapshot was created successfully and the data is complete; Store the successfully created snapshot file on another server or cloud storage service; Back up the configuration files of the Master node and Node nodes, the configuration files of static Pods, the network configuration files, the service account keys, and the encrypted configuration files.

6. The method for in-situ upgrade of a Kubernetes cluster across multiple versions as described in claim 1, characterized in that, After distributing the new version of the Kubernetes installation package and the upgrade scripts to all nodes in the cluster, the method further includes: Assess the resources required for the upgrade process and audit the current use of resources; Based on assessment and audit results, upgrades are performed during periods of low system load to minimize the impact on business. Resource quotas and limits are set based on assessment and audit results to ensure that upgrade operations do not consume excessive resources.

7. The method for in-situ upgrade of a Kubernetes cluster across multiple versions as described in claim 6, characterized in that, The assessment of resources required for the upgrade process, and the audit of existing resource usage, include: Assess the resources required for the upgrade operation itself, including CPU, memory, storage space, and network bandwidth; Assess changes in business load during the upgrade process and predict system load at different time periods; Use monitoring tools to audit the resource usage of existing Kubernetes clusters, including node resource utilization, Pod resource requests and limit settings, and service traffic patterns.

8. A cross-version in-situ upgrade device for a Kubernetes cluster, characterized in that, The device includes: The distribution unit is used to distribute the new version of the Kubernetes installation package and the scripts required for upgrades to all nodes in the cluster. The maintenance unit is used to stop the kube-apiserver and ETCD services of the Kubernetes cluster, thus putting the Kubernetes cluster into the maintenance phase. The ETCD upgrade unit is used to perform in-situ upgrades on the ETCD using upgrade scripts, and to upgrade the objects stored in the ETCD after the in-situ upgrade is completed. The Master upgrade unit is used to stop all services of the Master node and use the upgrade script to complete the in-situ upgrade of the Master node. The Node upgrade unit is used to stop all services of a Node and use an upgrade script to complete the in-situ upgrade of the Node. The Master recovery unit is used to sequentially start the kube-apiserver and kube-scheduler services on the Master node, and after the Node recovery unit restores the services, start the kube-controller-manager service on the Master node to restore Kubernetes' management of the cluster; The Node recovery unit is used to start the services on the Node. On the host machine of each Node, it calls the kube-apiserver interface to obtain container object information, and rewrites local files based on the container dimension according to the obtained container object information to ensure compatibility with the new version of Kubernetes.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images