K8s cluster recovery method and device, equipment, medium and program product

Abstract

The invention relates to the technical field of cloud computing containers, and discloses a K8s cluster recovery method, device and equipment, a medium and a program product.The method comprises the steps that the node state of a preset node in a K8s cluster is monitored, whether collapse occurs or not is judged, and if collapse occurs, warning information is generated, the preset nodes comprise a core database cluster node, a master control node and a working node; when it is detected that the recovery interface is triggered, backup data of the node state are obtained, and the backup data comprise historical database data, historical master control data and historical working data; and recovering the K8s cluster according to the backup data. By performing data backup and monitoring on the nodes of the K8s cluster, the nodes can be automatically recovered according to the backup data after the K8s cluster crashes, so that the K8s cluster is recovered to a healthy and available state again, the condition of incomplete recovery due to manual operation is avoided, and the reliability of a cluster recovery result is improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing container technology, and in particular to a K8s cluster recovery method, device, equipment, medium and program product. Background Art

[0002] Container technology is a popular technology and the most cutting-edge technology. Since the launch of Kubernetes (simplified as K8s), a large number of services have been deployed to K8s clusters, and K8s clusters have become the foundation of cloud computing. With a large number of K8s clusters used in production, it is inevitable that the entire K8s cluster will collapse and become unavailable. At this time, the cluster needs to be restored. Currently, the cluster is mainly restored by manual operation, but when the cluster crashes, anything may happen, even complete data loss, and manual operation cannot complete the recovery work. Summary of the invention

[0003] In view of this, the present invention provides a K8s cluster recovery method, device, equipment, medium and program product to solve the problem of incomplete manual recovery after the K8s cluster crashes.

[0004] In a first aspect, the present invention provides a method for recovering a K8s cluster, the method comprising:

[0005] Monitor the node status of the preset nodes in the K8s cluster to determine whether a crash occurs. If a crash occurs, generate an alarm message. The preset nodes include: core database cluster nodes, master nodes, and working nodes.

[0006] When the recovery interface is detected to be triggered, the backup data of the node status is obtained, and the backup data includes: historical database data, historical master control data and historical working data;

[0007] Restore the K8s cluster based on the backup data.

[0008] The K8s cluster recovery method provided by the embodiment of the present invention monitors the node status of the preset nodes in the K8s cluster to determine whether a crash occurs, generates an alarm message if a crash occurs, and obtains the backup data of the node status after detecting that the recovery interface is triggered based on the alarm message, and recovers the K8s cluster according to the backup data. The present invention can automatically recover each node according to the backup data after the K8s cluster crashes by backing up and monitoring the data of the nodes of the K8s cluster, so that the K8s cluster can be restored to a healthy and available state, avoiding the situation where manual recovery is incomplete, and improving the reliability of the cluster recovery result.

[0009] In an optional implementation, the node status of preset nodes in the K8s cluster is monitored to determine whether a crash occurs, including: monitoring the node status of the core database cluster nodes according to a first preset period to determine whether the core database cluster nodes have crashed; monitoring the entry status, communication status, and container status of the master node according to the first preset period to determine whether the master node has crashed; monitoring the communication status and container status of the working node according to the first preset period to determine whether the working node has crashed.

[0010] By performing crash monitoring on the nodes of the K8s cluster, the present invention can determine whether the K8s cluster has crashed or a catastrophic failure that affects the survival of the cluster has occurred, thereby restoring the cluster in time and avoiding the impact of the cluster crash on the operating system.

[0011] In an optional implementation, generation of backup data includes: using a preset automated operation and maintenance tool to obtain historical database data of core database cluster nodes, historical master control data of master control nodes, and historical working data of working nodes according to a second preset period; packaging the historical database data, historical master control data, and historical working data according to a preset format to obtain backup data, and storing the backup data in a backup database.

[0012] The present invention backs up the data of each node in the K8s cluster, and can restore the K8s cluster to a healthy and available state based on the backed-up original data when the K8s cluster crashes, ensuring that the restored cluster is consistent with the initial K8s cluster data.

[0013] In an optional implementation, the generation of backup data further includes: performing data cleanup on the backup database based on a preset cleanup strategy.

[0014] The present invention can delete the backup data which is too old by cleaning the backup database, thereby controlling the data volume of the database and avoiding the data volume being too large.

[0015] In an optional implementation, the K8s cluster is restored according to the backup data, including: deploying the target cluster based on the deployment script of the K8s cluster using a preset automated operation and maintenance tool; copying the historical database data in the core database cluster of the target cluster, and restoring the core database cluster nodes of the target cluster; copying the historical master control data in the master control node of the target cluster, and restoring the master control node of the target cluster; copying the historical working data in the directory corresponding to the working node of the target cluster, and restoring the working node of the target cluster; using the target cluster as the restored K8s cluster.

[0016] The present invention can restore the K8s cluster to a healthy and available initial state by gradually restoring each node of the K8s cluster, thereby ensuring that the operating system can continue to run based on the restored K8s cluster.

[0017] In an optional implementation, determining the preset automated operation and maintenance tool includes: constructing the preset automated operation and maintenance tool based on a recovery program generated in a preset programming language.

[0018] The present invention can automatically implement data backup and cluster recovery by constructing an automated operation and maintenance tool, thereby avoiding incomplete recovery due to manual operation and improving the reliability of cluster recovery results.

[0019] In a second aspect, the present invention provides a K8s cluster recovery device, the device comprising:

[0020] The alarm module is used to monitor the node status of the preset nodes in the K8s cluster to determine whether a crash occurs. If a crash occurs, an alarm message is generated. The preset nodes include: core database cluster nodes, master nodes, and working nodes.

[0021] The acquisition module is used to acquire the backup data of the node status when it is detected that the recovery interface is triggered. The backup data includes: historical database data, historical master control data and historical working data;

[0022] The recovery module is used to restore the K8s cluster based on the backup data.

[0023] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the K8s cluster recovery method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0024] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the K8s cluster recovery method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0025] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the K8s cluster recovery method of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 It is a flowchart of a method for restoring a K8s cluster according to an embodiment of the present invention;

[0028] Figure 2 It is a schematic diagram of the component structure of a K8s cluster recovery method according to an embodiment of the present invention;

[0029] Figure 3 It is a monitoring process diagram of a K8s cluster recovery method according to an embodiment of the present invention;

[0030] Figure 4 It is a flowchart of another K8s cluster recovery method according to an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of a backup process of another K8s cluster recovery method according to an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of backup data of another K8s cluster recovery method according to an embodiment of the present invention;

[0033] Figure 7 It is a flowchart of another K8s cluster recovery method according to an embodiment of the present invention;

[0034] Figure 8 It is a recovery process diagram of another K8s cluster recovery method according to an embodiment of the present invention;

[0035] Fig. 9 This is a structural block diagram of a K8s cluster recovery device according to an embodiment of the present invention;

[0036] Fig.10 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0038] The embodiment of the present invention is applicable to the application scenario of deploying a K8s cluster in a server as a container cloud platform. Container technology is an efficient application deployment method that allows developers to package and run applications in an isolated environment. This process is called containerization. Container technology creates an isolated group on the physical host operating system, divides the resources of the physical host operating system into each group, and balances the conflicting resource usage requirements between the groups.

[0039] Container technology belongs to virtualization at the operating system level. It does not simulate hardware. Instead, it isolates, allocates, and manages the CPU, memory, disk, network and other resources of the host system with the support of a specific operating system kernel, thereby forming independent containers. Container technology is lightweight, portable, and scalable, and can provide consistent and efficient software development and deployment solutions. Containers package applications and their related dependencies in an independent, portable unit, containing the necessary elements for the smooth execution of applications, such as code, environment variables, processes and execution environments, and software dependencies. This allows applications to run consistently in different environments, improving the efficiency of development and deployment.

[0040] Container technology is different from virtual machines. Virtual machines run applications by simulating a complete operating system, while containers run with the support of the operating system kernel and do not require hardware simulation, so they are more efficient and flexible. Container technology solves a series of problems in traditional deployment methods, making software development and deployment more consistent and efficient.

[0041] The Kubernetes cluster is an open source container cluster management system developed by Google for managing containerized workloads and services. It is based on Docker technology and provides a series of functions such as deployment and operation, resource scheduling, service discovery and dynamic scaling, aiming to improve the convenience of large-scale container cluster management. The core concepts of Kubernetes include Kubernetes objects, which represent the state of the cluster, including running containerized applications, available resources, and application behavior policies. By creating and configuring these objects, users can tell the Kubernetes system the desired cluster state, and the system will strive to ensure the realization of this state. The design philosophy of Kubernetes emphasizes declarative configuration and automation, making it a portable and scalable platform. "K8s" is a simplified representation of "Kubernetes" because there are 8 characters between "K" and "s". An embodiment of the present invention provides a method for recovering a K8s cluster, which achieves the effect of automatic recovery when K8s crashes by backing up data of the K8s cluster. The explanations of the terms involved are as follows:

[0042] Kubernetes / K8s: the cluster name of the current container cloud platform;

[0043] ETCD: the core database of K8s;

[0044] Kubelet: worker node daemon in the K8s cluster, used to maintain node status, including containers, heartbeats, etc.

[0045] Containerd: container daemon of K8s cluster, used for container lifecycle management;

[0046] Master: the master control node of K8s;

[0047] Apiserver: the core component on the Master node;

[0048] Ansible: An open source IT automation tool.

[0049] According to an embodiment of the present invention, an embodiment of a recovery method for a K8s cluster is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] In this embodiment, a K8s cluster recovery method is provided, which can be used for mobile terminals, such as computers, etc. Figure 1is a flowchart of a method for restoring a K8s cluster according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0051] Step S101, monitor the node status of the preset node in the K8s cluster to determine whether a crash occurs, and generate an alarm message if a crash occurs.

[0052] Specifically, in an embodiment of the present invention, ETCD (Enhanced Transactional Data Store) is a highly reliable, distributed key-value storage database, which is mainly used for data sharing and coordination in distributed systems. In the K8s cluster, ETCD plays the role of a core component and is the key storage backend for cluster status data. Therefore, it belongs to the core database cluster node of the K8s cluster and stores the shared configuration and service discovery information of the cluster. For example, ETCD stores the status information of all objects in the K8s cluster, which is used for scheduling tasks, network routing, service discovery, etc., and is therefore essential for the normal operation of the cluster.

[0053] In addition, in the K8s cluster, the Master node also plays a vital role. It is the main control node of the K8s cluster and is mainly responsible for managing and scheduling cluster resources. The Master node is the brain of the K8s cluster and is responsible for coordinating and managing various tasks and resources in the cluster. The core functions of the Master node include: (1) Management API interface: The Master node runs kube-apiserver, which is the API server of the K8s cluster and is responsible for providing an interface and control panel for interacting with the cluster. All operations and management tasks are processed through the kube-apiserver, including creating, updating, and deleting resource objects, as well as querying the status and configuration information of the cluster. (2) Controller management: The kube-controller-manager on the Master node is responsible for managing the controllers in the cluster and ensuring that the system is in the required state. For example, the replica controller is responsible for ensuring that the number of replicas of the Pod meets expectations, and the service controller is responsible for ensuring the stability and availability of the service. (3) Scheduler: The kube-scheduler on the Master node is responsible for scheduling the Pod to the nodes in the cluster. According to the resource requirements and scheduling strategy of the Pod, it selects the most suitable node to run the Pod to ensure the rational use of resources and load balancing.

[0054] Worker nodes are working nodes in the K8s cluster. They are responsible for running the host of container applications, receiving instructions from the Master node, and starting, stopping, managing, and monitoring container applications according to the instructions. Worker nodes have the following functions: (1) Running containers: The Docker engine (or other container runtime) hosted on the Worker node is used to run and manage containers. Kubernetes provides advanced management functions for container applications by encapsulating the container runtime on the node. (2) Receiving instructions: Worker nodes receive instructions and policies to deploy, schedule, and manage container applications through communication with the Master node. Worker nodes report their own status information to the Master node, enabling the Master node to maintain the status of the entire cluster. (3) Storage management: Worker nodes may be connected to distributed storage systems, responsible for mounting persistent volumes (Volume) into containers, and managing persistent data through the Pod lifecycle. (4) Network management: Worker nodes are responsible for connecting containers to the cluster network so that containers can communicate with each other and access external resources. They are also responsible for functions such as load balancing, service discovery, and network policies.

[0055] In some optional embodiments, such as Figure 2 As shown, the embodiment of the present invention develops a monitoring alarm component for regularly monitoring the K8s cluster. Figure 3 As shown, when the detection is triggered, the ETCD cluster node, the Master node, and the node status of the Master node are detected to see if they have collapsed. If any node status is detected to have collapsed, an alarm message is generated.

[0056] Step S102: when it is detected that the recovery interface is triggered based on the alarm information, the backup data of the node status is obtained.

[0057] Specifically, in the embodiment of the present invention, Figure 2 As shown, a backup component is developed in advance, and the ETCD data directory, the certificate and configuration information of the Master node, and the certificate and configuration information of the Worker node in the K8s cluster are copied regularly through the backup component, and the data information is stored in the backup database by calling the copy module of Ansible. Among them, Ansible is an open source automated operation and maintenance tool for configuration management, application deployment, task execution and multi-node coordination. The embodiment of the present invention uses Golang to develop a recovery program, and uses Ansible to deploy the recovery program to obtain an automated operation and maintenance tool, which can be deployed in other places in binary form or container form, even in other K8S clusters.

[0058] In some optional embodiments, when the staff finds the alarm information and manually triggers the recovery interface, the historical database data of the core database cluster node, the historical master control data of the master control node and the historical master control data of the working node in the backup database are obtained.

[0059] Step S103, restore the K8s cluster according to the backup data.

[0060] Specifically, in the embodiment of the present invention, Figure 2 As shown, a cluster recovery component is pre-built. This component needs to be manually triggered by the operation and maintenance personnel according to the actual situation to restore the K8s cluster. Moreover, the backup component is the most basic independently running component and does not depend on the other two components. The recovery component is manually triggered by the operation and maintenance personnel based on the results of the alarm component. The backup data obtained by the backup component represents the original data when the K8s cluster has not crashed. Therefore, when the operation and maintenance personnel learn that the K8s cluster has crashed based on the alarm information of the alarm component, they manually trigger the cluster recovery component. The recovery component redeploys a cluster based on Ansible, and then replaces the cluster data based on the obtained backup data to achieve cluster recovery operations and realize full recovery of the cluster, reaching the state of the last backup before the cluster crash, so that the K8S cluster can be restored to a healthy and available state.

[0061] The K8s cluster recovery method provided by the embodiment of the present invention monitors the node status of the preset nodes in the K8s cluster to determine whether a crash occurs, generates an alarm message if a crash occurs, and obtains the backup data of the node status after detecting that the recovery interface is triggered based on the alarm message, and recovers the K8s cluster according to the backup data. The present invention can automatically recover each node according to the backup data after the K8s cluster crashes by backing up and monitoring the data of the nodes of the K8s cluster, so that the K8s cluster can be restored to a healthy and available state, avoiding the situation where manual recovery is incomplete, and improving the reliability of the cluster recovery result.

[0062] In this embodiment, a K8s cluster recovery method is provided, which can be used for the above-mentioned mobile terminals, such as computers, etc. Figure 4 is a flowchart of a method for restoring a K8s cluster according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0063] Step S401, using the preset automated operation and maintenance tool, obtains the historical database data of the core database cluster nodes, the historical master control data of the master control nodes, and the historical working data of the working nodes according to the second preset period; packages the historical database data, the historical master control data, and the historical working data according to the preset format to obtain backup data, and stores the backup data in the backup database.

[0064] Specifically, in the embodiment of the present invention, the core logic of the backup component needs to maintain a global timer for triggering the backup action at a fixed time. Figure 5 As shown in the figure, after the backup action is triggered, the copy module of Ansible is called to obtain the certificates and configuration information of all nodes in the K8s cluster, such as Figure 6 As shown in the figure, it includes the data directory of the ETCD cluster nodes, the apiserver certificate of the Master node, the kubelet certificate and the configuration information of kubelet and containerd, the kubelet certificate of the Worker node and the configuration information of kubelet and containerd, etc. Then connect to the backup database, encapsulate all data in the format of time, node, backup data and other fields, and store them in the backup database. At the same time, Figure 5 As shown, in order to control the amount of data in the backup database, it is necessary to clean up the old backups, and it is also necessary to call the backup database cleanup interface and configure the cleanup strategy according to the actual situation. The cleanup operation can be performed by time, such as year, month, and day.

[0065] Step S402: monitor the node status of the preset nodes in the K8s cluster to determine whether a crash occurs. If a crash occurs, generate an alarm message. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0066] Step S403: When it is detected that the recovery interface is triggered based on the alarm information, the backup data of the node status is obtained. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0067] Step S404: Restore the K8s cluster based on the backup data. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0068] The K8s cluster recovery method provided by the embodiment of the present invention monitors the node status of the preset nodes in the K8s cluster to determine whether a crash occurs, generates an alarm message if a crash occurs, and obtains the backup data of the node status after detecting that the recovery interface is triggered based on the alarm message, and recovers the K8s cluster according to the backup data. The present invention can automatically recover each node according to the backup data after the K8s cluster crashes by backing up and monitoring the data of the nodes of the K8s cluster, so that the K8s cluster can be restored to a healthy and available state, avoiding the situation where manual recovery is incomplete, and improving the reliability of the cluster recovery result.

[0069] In this embodiment, a K8s cluster recovery method is provided, which can be used for the above-mentioned mobile terminals, such as computers, etc. Figure 7 is a flowchart of a method for restoring a K8s cluster according to an embodiment of the present invention. Figure 7 As shown, the process includes the following steps:

[0070] Step S701, monitor the node status of the preset node in the K8s cluster to determine whether a crash occurs, and generate an alarm message if a crash occurs.

[0071] Specifically, the above step S701 includes:

[0072] Step S7011: monitor the node status of the core database cluster node according to a first preset period to determine whether the core database cluster node crashes.

[0073] Step S7012: monitor the entry status, communication status, and container status of the master control node according to a first preset period to determine whether the master control node crashes.

[0074] Step S7013: monitor the communication status and container status of the working node according to the first preset period to determine whether the working node crashes.

[0075] Specifically, in the embodiment of the present invention, Figure 3 As shown in the figure, when the timer triggers the crash detection, the alarm component determines the ETCD cluster node status, Master node status and Worker node status in turn, where the Master node status includes the entry (apiserver) status, communication (kubelet) status and container (containerd) status, and the Worker node status includes the communication (kubelet) status and container (containerd) status. Crash detection is performed by obtaining relevant data from different nodes. If normal, it is written to the backup database and the next round of crash detection is performed. If abnormal, an alarm message is generated.

[0076] Step S702: When the recovery interface is detected to be triggered based on the alarm information, the backup data of the node status is obtained. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0077] Step S703, restore the K8s cluster according to the backup data.

[0078] Specifically, the above step S703 includes:

[0079] Step S7031, based on the deployment script of the K8s cluster, use the preset automated operation and maintenance tool to deploy the target cluster.

[0080] Step S7032: copy the historical database data in the core database cluster of the target cluster, and restore the core database cluster nodes of the target cluster.

[0081] Step S7033: copy the historical master control data to the master control node of the target cluster, and restore the master control node of the target cluster.

[0082] Step S7034, copy the historical working data to the directory corresponding to the working node of the target cluster, and restore the working node of the target cluster.

[0083] Step S7035, use the target cluster as the restored K8s cluster.

[0084] Specifically, in the embodiment of the present invention, Figure 8 As shown in the figure, when the operation and maintenance personnel manually trigger the cluster recovery operation after judgment, first call Ansible to redeploy a new target cluster based on the deployment script of the original K8s cluster. Then stop the newly deployed ETCD cluster and copy the backed-up ETCD data to the original ETCD node. Based on the feature that ETCD supports forced startup with backup data, let ETCD restart with the original data (backup data). After the first ETCD node starts normally, clean up the data of the remaining two ETCD nodes and add them to the cluster started by the first ETCD node in turn. After the ETCD cluster is restored, restore the configuration file to complete the recovery of the ETCD cluster, and the original cluster data can be read normally. After ETCD is restored, restore the backed-up Master data to the original directory. After the components on the Master node are successfully started, the control plane of the cluster is basically restored. After the control plane is restored, copy and restore the backup data of the Worker node to the original directory, and the Worker node can be restored normally. At this point, the entire cluster is restored, so the last target cluster is used as the restored K8s cluster.

[0085] The K8s cluster recovery method provided by the embodiment of the present invention monitors the node status of the preset nodes in the K8s cluster to determine whether a crash occurs, generates an alarm message if a crash occurs, and obtains the backup data of the node status after detecting that the recovery interface is triggered based on the alarm message, and recovers the K8s cluster according to the backup data. The present invention can automatically recover each node according to the backup data after the K8s cluster crashes by backing up and monitoring the data of the nodes of the K8s cluster, so that the K8s cluster can be restored to a healthy and available state, avoiding the situation where manual recovery is incomplete, and improving the reliability of the cluster recovery result.

[0086] In this embodiment, a recovery device for a K8s cluster is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0087] This embodiment provides a K8s cluster recovery device, such as Fig. 9 As shown, including:

[0088] The alarm module 901 is used to monitor the node status of the preset nodes in the K8s cluster, determine whether a crash occurs, and generate an alarm message if a crash occurs. The preset nodes include: core database cluster nodes, master nodes, and working nodes.

[0089] The acquisition module 902 is used to acquire the backup data of the node status when it is detected that the recovery interface is triggered. The backup data includes: historical database data, historical master control data and historical working data.

[0090] The recovery module 903 is used to restore the K8s cluster according to the backup data.

[0091] In some optional implementations, the alarm module 901 includes:

[0092] The first monitoring unit is used to monitor the node status of the core database cluster node according to a first preset period to determine whether the core database cluster node crashes.

[0093] The second monitoring unit is used to monitor the entry state, communication state and container state of the main control node according to the first preset period to determine whether the main control node crashes.

[0094] The third monitoring unit is used to monitor the communication status and container status of the working node according to the first preset period to determine whether the working node crashes.

[0095] In some optional implementations, the device further includes: a backup module, the backup module including:

[0096] The data acquisition unit is used to use the preset automatic operation and maintenance tool to acquire the historical database data of the core database cluster nodes, the historical master control data of the master control nodes, and the historical working data of the working nodes according to the second preset period.

[0097] The data storage unit is used to encapsulate the historical database data, historical master control data and historical working data according to a preset format to obtain backup data, and store the backup data in a backup database.

[0098] In some optional implementations, the backup module further includes:

[0099] The data cleaning unit is used to clean up the data in the backup database based on a preset cleaning strategy.

[0100] In some optional implementations, the recovery module 903 includes:

[0101] The cluster deployment unit is used to deploy the target cluster based on the deployment script of the K8s cluster using the preset automated operation and maintenance tools.

[0102] The first recovery unit is used to copy the historical database data in the core database cluster of the target cluster and recover the core database cluster nodes of the target cluster.

[0103] The second recovery unit is used to copy the historical master control data to the master control node of the target cluster and recover the master control node of the target cluster.

[0104] The third recovery unit is used to copy the historical working data to the directory corresponding to the working node of the target cluster, and recover the working node of the target cluster;

[0105] The cluster recovery unit is used to use the target cluster as the recovered K8s cluster.

[0106] In some optional embodiments, the device further includes: a tool construction unit, configured to construct a preset automated operation and maintenance tool based on a recovery program generated in a preset programming language.

[0107] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0108] The recovery device of the K8s cluster in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0109] The embodiment of the present invention also provides a computer device having the above Fig. 9 The recovery device of the K8s cluster is shown.

[0110] See also Fig.10 , Fig.10 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.10As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 10 is taken as an example.

[0111] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0112] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0113] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0115] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0116] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0117] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0118] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0119] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A K8s cluster recovery method, characterized in that: The method comprises: Monitor the node status of the preset nodes in the K8s cluster to determine whether a crash occurs, and generate an alarm message if a crash occurs, wherein the preset nodes include: core database cluster nodes, master nodes, and working nodes; When it is detected that the recovery interface is triggered based on the alarm information, the backup data of the node status is obtained, the backup data including: historical database data, historical master control data and historical working data; The K8s cluster is restored according to the backup data.

2. The method according to claim 1, characterized in that The node status of the preset node in the K8s cluster is monitored to determine whether a crash occurs, including: Monitoring the node status of the core database cluster node according to a first preset period to determine whether the core database cluster node crashes; Monitoring the entry state, communication state, and container state of the master control node according to the first preset period to determine whether the master control node crashes; The communication status and container status of the working node are monitored according to the first preset period to determine whether the working node crashes.

3. The method according to claim 2, characterized in that The generation of the backup data includes: Using a preset automated operation and maintenance tool, according to a second preset period, the historical database data of the core database cluster node, the historical master control data of the master control node, and the historical working data of the working node are obtained; The historical database data, the historical master control data and the historical working data are packaged according to a preset format to obtain backup data, and the backup data is stored in a backup database.

4. The method according to claim 3, characterized in that The generation of the backup data further includes: Data cleaning is performed on the backup database based on a preset cleaning strategy.

5. The method according to claim 3, characterized in that: The restoring the K8s cluster according to the backup data includes: Based on the deployment script of the K8s cluster, the target cluster is deployed using the preset automated operation and maintenance tool; Copying the historical database data in the core database cluster of the target cluster, and restoring the core database cluster nodes of the target cluster; Copying the historical master control data to the master control node of the target cluster, and restoring the master control node of the target cluster; Copy the historical working data to the directory corresponding to the working node of the target cluster, and restore the working node of the target cluster; The target cluster is used as the restored K8s cluster.

6. The method according to claim 3, characterized in that The determination of the preset automated operation and maintenance tool includes: constructing the preset automated operation and maintenance tool based on a recovery program generated in a preset programming language.

7. A K8s cluster recovery device, characterized in that: The device comprises: The alarm module is used to monitor the node status of the preset nodes in the K8s cluster, determine whether a crash occurs, and generate alarm information if a crash occurs. The preset nodes include: core database cluster nodes, master nodes, and working nodes; An acquisition module, used for acquiring backup data of the node state when detecting that the recovery interface is triggered, the backup data including: historical database data, historical master control data and historical working data; The recovery module is used to restore the K8s cluster according to the backup data.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the K8s cluster recovery method described in any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the K8s cluster recovery method described in any one of claims 1 to 6.

10. A computer program product, characterized in that It comprises computer instructions, and the computer instructions are used to enable a computer to execute the K8s cluster recovery method described in any one of claims 1 to 6.

Images