Service recovery method and apparatus, electronic device, and storage medium

Abstract

The application embodiment provides a service recovery method and device, electronic equipment and storage medium. Different service recovery instructions are generated according to different types of abnormalities of a cluster. Different service backup sets are selected according to different service recovery instructions to recover services, thereby reducing the number of service backup sets to be queried and improving the efficiency of service recovery. Meanwhile, the target service backup set is selected in combination with the backup set parameters of the service backup set, thereby improving the integrity and reliability after service recovery.

Description

Technical Field

[0001] This application relates to the field of databases, specifically to a service recovery method, apparatus, electronic device, and storage medium. Background Technology

[0002] Kubernetes (k8s) is an open-source container orchestration engine from Google for the automated deployment, scaling, and operation of container clusters. This open-source platform allows for rapid and efficient response to user needs, quick and predictable deployment of user applications, rapid scaling of user applications, and seamless integration with new application functionalities. Furthermore, it saves resources, optimizes hardware resource utilization, and provides a complete open-source solution for container orchestration and management. Based on these advantages, Kubernetes has become the de facto standard for container orchestration, and an increasing number of systems are building upon the Kubernetes platform to extend or run their applications, improving resource utilization efficiency and productivity.

[0003] While applications can be deployed using high availability methods in Kubernetes clusters, unexpected events such as earthquakes, fires, or human error in data centers can still lead to application unavailability or even data loss. Therefore, data backups are typically necessary for data recovery in case of emergencies. However, existing data recovery methods often consume significant system resources, causing system slowdowns or unresponsiveness. Summary of the Invention

[0004] This application provides a service recovery method, apparatus, electronic device, and storage medium that generates different service recovery instructions to quickly find backup sets that can be restored, thereby reducing the number of backup sets queried and improving the efficiency of service recovery.

[0005] Firstly, this application provides a service recovery method applied to a client, the method comprising:

[0006] Monitor the running status of the Kubernetes cluster in real time to determine if the Kubernetes cluster is experiencing any abnormalities;

[0007] If the Kubernetes cluster experiences an anomaly, the target anomaly is determined, and a service recovery instruction corresponding to the target anomaly is generated. Different anomalies in the Kubernetes cluster correspond to different service recovery instructions.

[0008] Based on the different service recovery instructions and the backup set parameters of multiple service backup sets, the target service backup set that can be restored for each service recovery instruction is determined. The multiple service backup sets are obtained by the Kubernetes cluster backing up according to the service backup strategy.

[0009] The service is restored based on the target service backup set to obtain the restored target service.

[0010] In some possible embodiments, before monitoring the running status of the Kubernetes cluster in real time to determine whether the Kubernetes cluster is experiencing an anomaly, the method further includes:

[0011] Obtain the service backup policy for backing up the Kubernetes cluster;

[0012] Based on the service backup strategy, determine the service to be backed up and the first storage location;

[0013] The service to be backed up is backed up to obtain multiple service backup sets, and the multiple service backup sets are stored in the first storage location.

[0014] In some possible embodiments, the step of using a cluster monitoring component to monitor the running status of the Kubernetes cluster in real time to determine whether the Kubernetes cluster is experiencing anomalies includes:

[0015] Deploy the Kubernetes cluster in the client;

[0016] A status monitoring component is deployed in the client, the status monitoring component including a cluster monitoring component and a client monitoring component;

[0017] The cluster monitoring component is used to monitor the cluster operation metrics of the Kubernetes cluster, and the client monitoring component is used to monitor the client operation metrics of the client.

[0018] Based on the cluster operation metrics and the client operation metrics, determine whether the Kubernetes cluster is experiencing anomalies.

[0019] In some possible embodiments, determining whether the Kubernetes cluster is experiencing an anomaly based on the cluster performance metrics and the client performance metrics includes:

[0020] Obtain a preset operational indicator training dataset, which includes cluster operational indicator training data, client operational indicator training data, and cluster anomaly judgment results;

[0021] The initial cluster anomaly identification model is trained using the preset operational index training dataset to determine the cluster anomaly prediction result;

[0022] Calculate the difference between the cluster anomaly judgment result and the cluster anomaly prediction result;

[0023] Adjust the model parameters of the initial cluster anomaly identification model according to the difference until the difference meets the preset training conditions, then stop adjusting the model parameters to obtain the trained cluster anomaly identification model.

[0024] Based on the cluster anomaly identification model, the cluster operation metrics, and the client operation metrics, determine whether the Kubernetes cluster is experiencing anomalies.

[0025] In some possible embodiments, the Kubernetes cluster includes a master node and slave nodes; if the Kubernetes cluster experiences an anomaly, a target anomaly is determined, and a service recovery instruction corresponding to the target anomaly is generated. Different anomalies in the Kubernetes cluster correspond to different service recovery instructions, including:

[0026] If the Kubernetes cluster experiences an anomaly, it is determined that the anomaly is either due to an abnormal cluster operation metric or an abnormal client operation metric.

[0027] If the anomaly is an abnormality in the client's operating metrics, then a first service recovery instruction is generated;

[0028] If the anomaly is an abnormality in the cluster operation metrics, then the anomaly is determined to be either an abnormality in the cluster master node or an abnormality in the cluster slave node.

[0029] If the cluster master node is abnormal, a second service recovery command will be generated;

[0030] If a cluster slave node fails, a third service recovery command will be generated.

[0031] In some possible embodiments, the step of defining the target service backup set for service recovery corresponding to different service recovery instructions and backup set parameters of multiple service backup sets includes:

[0032] Based on the different service recovery instructions, determine the backup set type to be restored, whereby the backup set type includes full backup or incremental backup;

[0033] A first service backup set is determined from the plurality of service backup sets according to the backup set type;

[0034] Obtain the backup set parameters of the first service backup set;

[0035] Based on the backup set parameters corresponding to each of the first service backup sets, the service recovery weights corresponding to each of the first service backup sets are determined, resulting in multiple service recovery weights;

[0036] Based on the multiple service recovery weights, the target service backup set is determined in the first service backup set.

[0037] In some possible embodiments, the method further includes:

[0038] The target service is verified to confirm that the target service has been restored without error.

[0039] Secondly, embodiments of this application provide a service recovery device, the service recovery device comprising:

[0040] The cluster monitoring module is used to monitor the running status of the Kubernetes cluster in real time in order to determine whether the Kubernetes cluster is experiencing any abnormalities.

[0041] The service recovery instruction generation module is used to determine the target anomaly of the Kubernetes cluster if an anomaly occurs, and generate a service recovery instruction corresponding to the target anomaly. Different anomalies of the Kubernetes cluster correspond to different service recovery instructions.

[0042] The target service backup set determination module is used to determine the target service backup set that can be restored for different service recovery instructions based on the different service recovery instructions and the backup set parameters of multiple service backup sets. The multiple service backup sets are obtained by the Kubernetes cluster backing up according to the service backup strategy.

[0043] The service recovery module is used to restore the service based on the target service backup set to obtain the restored target service.

[0044] Thirdly, this application provides an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps in the service recovery method as described in any of the first aspects.

[0045] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of the service recovery method as described in any of the first aspects.

[0046] This application provides a service recovery method, apparatus, electronic device, and storage medium, including: real-time monitoring of the running status of a Kubernetes cluster to determine whether the Kubernetes cluster has encountered an anomaly; if the Kubernetes cluster has encountered an anomaly, determining the target anomaly and generating a service recovery instruction corresponding to the target anomaly, with different anomalies corresponding to different service recovery instructions; determining the target service backup set for service recovery corresponding to different service recovery instructions based on the different service recovery instructions and backup set parameters of multiple service backup sets, where multiple service backup sets are obtained by backing up the Kubernetes cluster according to a service backup strategy; and performing service recovery based on the target service backup set to obtain the recovered target service. This application generates different service recovery instructions for different types of anomalies in the cluster and selects different service backup sets for service recovery based on different service recovery instructions, reducing the number of service backup sets that need to be queried and improving the efficiency of service recovery. Simultaneously, it combines...

[0047] Using backup set parameters to select the target service backup set can improve the integrity and reliability of the service after recovery. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram illustrating the service recovery operating system provided in an embodiment of this application;

[0050] Figure 2 This is a schematic flowchart of an embodiment of the service recovery method provided in this application;

[0051] Figure 3 A flowchart illustrating an embodiment of this application for determining whether a Kubernetes cluster is abnormal;

[0052] Figure 4 A schematic flowchart illustrating an embodiment of verifying a service recovery command provided in this application;

[0053] Figure 5 A flowchart illustrating an embodiment of selecting different service backup sets based on different service recovery instructions, provided as an example of this application.

[0054] Figure 6 A schematic diagram of an embodiment of the service recovery operation device provided in this application;

[0055] Figure 7 A schematic diagram of the structure of the electronic device involved in the embodiments of this application is shown;

[0056] Figure 8 This is a schematic diagram of the terminal device structure provided in the embodiments of this application;

[0057] Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] In the following description, specific embodiments of this application will be illustrated with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and computer execution as referred to herein includes operations by a computer processing unit representing electronic signals of data in a structured format. This operation transforms the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise alter the operation of the computer in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of this application are described in the foregoing text, which is not intended to be limiting, and those skilled in the art will understand that many of the steps and operations described below can also be implemented in hardware.

[0060] The terms "module" or "unit" as used herein can be considered as software objects executing on the computing system. The different components, modules, engines, and services described herein can be considered as implementation objects on the computing system. The apparatus and methods described herein are preferably implemented in software, but can also be implemented in hardware, both of which are within the scope of this application.

[0061] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0062] Please refer to Figure 1 This application also provides a service to restore the operating system, such as Figure 1 As shown, the service recovery operating system includes a computing device 100, which integrates the service recovery operation apparatus provided in this application. In this embodiment, the computing device 100 may be a terminal device or a server.

[0063] In this embodiment, when the computing device 100 is a server, the server can be an independent server, a server network, or a server cluster. For example, the server described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing. In this embodiment, communication between the server and the client can be achieved through any communication method, including but not limited to mobile communication based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP / IP Protocol Suite (TCP / IP) and User Datagram Protocol (UDP).

[0064] It is understood that when the computing device 100 used in the embodiments of this application is a terminal device, the terminal device can be a device that includes both receiving hardware and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a terminal device may include: cellular or other communication devices, which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. Specifically, the computing device 100 may be a desktop terminal or a mobile terminal, and may specifically be one of a mobile phone, tablet computer, laptop computer, etc.

[0065] The terminal devices involved in the embodiments of this application can also be devices that provide voice and / or data connectivity to users, handheld devices with wireless connectivity, or other processing devices connected to a wireless modem. Examples include mobile phones (or "cellular" phones) and computers with mobile terminals, such as portable, pocket-sized, handheld, computer-embedded, or vehicle-mounted mobile devices that exchange voice and / or data with a wireless access network. Examples include Personal Communication Service (PCS) phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistants (PDAs), and other devices.

[0066] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computing devices shown, or the network connectivity of computing devices, for example... Figure 1 Only one computing device is shown in the diagram. It is understood that the service to restore the operating system may also include one or more other computing devices, and / or one or more other computing devices that are network-connected to computing device 100, which is not limited here.

[0067] In addition, such as Figure 1 As shown, the service recovery system may also include a storage device 200 for storing cluster configuration data, etc.

[0068] In this embodiment of the application, the memory 200 can be a cloud memory. Cloud storage is a new concept that is extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file system functions to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) through application software or application interfaces to work together to provide data storage and business access functions to the outside world.

[0069] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0070] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0071] It should be noted that, Figure 1 The schematic diagram of the service recovery system shown is merely an example. The service recovery operating system and scenario described in this application embodiment are for the purpose of more clearly illustrating the technical solutions of this application embodiment and do not constitute a limitation on the technical solutions provided in this application embodiment. As those skilled in the art will know, with the evolution of the service recovery operating system and the emergence of new business scenarios, the technical solutions provided in this application embodiment are also applicable to similar technical problems.

[0072] The solutions provided in this application involve technologies such as Artificial Intelligence (AI), Computer Vision (CV), and Machine Learning (ML), which are specifically illustrated through the following embodiments:

[0073] AI, or Artificial Intelligence, refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, Artificial Intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. Artificial Intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0074] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0075] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include adversarial perturbation generation, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0076] The service recovery method, apparatus, electronic device, and storage medium provided in the embodiments of this application will be described in detail below. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0077] like Figure 2 The diagram shown is a flowchart of an embodiment of the service restoration method provided in this application. Specifically, the service restoration method may include the following steps:

[0078] 201. Monitor the running status of the Kubernetes cluster in real time to determine if there are any abnormalities in the Kubernetes cluster.

[0079] The service recovery method provided in this application is mainly used to recover services in a cluster. Before performing service recovery, it is necessary to back up the services so that the backup services can be used for recovery. Therefore, before determining whether the Kubernetes cluster has encountered an anomaly, this service recovery method also includes:

[0080] Obtain the service backup policy for backing up the Kubernetes cluster; determine the services to be backed up and their primary storage location based on the service backup policy; back up the services to be backed up to obtain multiple service backup sets, and store the multiple service backup sets in the primary storage location.

[0081] Specifically, in this embodiment, the Kubernetes cluster first needs to be deployed on an electronic device, and the Kubernetes cluster includes multiple different services. Next, a preset service backup strategy needs to be obtained. In this application, the service backup strategy can be read from the hardware storage of the preset electronic device, or it can be obtained from cloud storage such as a server; this is not limited here. Based on the service backup strategy, the corresponding backup content can be determined, including but not limited to: service configuration, container images, persistent data, and other key information; it also includes the service to be backed up and the first storage location. Using the service backup strategy, the service to be backed up can be backed up, resulting in multiple service backup sets, which are then stored in the first storage location. The first storage location in this application can also be hardware storage such as a local disk or cloud storage. The specific backup process can refer to existing technologies and is not limited here.

[0082] In the above embodiments, after determining the service to be backed up and the first storage location according to the service backup strategy, it is also necessary to obtain and run the namespace query command for the Kubernetes cluster to determine whether the namespace in the Kubernetes cluster is normal. Only if the namespace is normal can the namespace of the first storage location be determined, and the backed-up backup set be stored in the first storage location under the namespace.

[0083] In this embodiment, it is also necessary to continuously monitor the operation of the entire Kubernetes cluster, determine whether the cluster is abnormal, and automatically trigger a service recovery mechanism to restore service upon detection of an anomaly. In some embodiments, a status monitoring component can be deployed on the client where the Kubernetes cluster is deployed, and this component can be used to monitor the cluster's operational status in real time. In this embodiment, monitoring the Kubernetes cluster's operational status includes monitoring both the cluster's own operational status and the operational status of the client (or electronic device) where the cluster is deployed. Therefore, the status monitoring component actually includes a cluster monitoring component for monitoring the cluster's operational metrics and a client monitoring component for monitoring the client's operational metrics. Determining whether the Kubernetes cluster is abnormal requires a comprehensive assessment of both cluster and client operational metrics.

[0084] like Figure 3 The diagram shown is a flowchart illustrating an embodiment of this application for determining whether a Kubernetes cluster is abnormal. Determining whether a Kubernetes cluster is abnormal based on cluster performance metrics and client performance metrics may include the following steps:

[0085] 301. Obtain the preset training dataset for operational indicators.

[0086] 302. Use the preset operational indicators training dataset to train the initial cluster anomaly identification model and determine the cluster anomaly prediction results.

[0087] 303. Calculate the difference between the cluster anomaly judgment result and the cluster anomaly prediction result.

[0088] 304. Adjust the model parameters of the initial cluster anomaly identification model according to the difference until the difference meets the preset training conditions, then stop adjusting the model parameters to obtain the trained cluster anomaly identification model.

[0089] 305. Based on the cluster anomaly identification model, cluster operation indicators, and client operation indicators, determine whether the Kubernetes cluster is experiencing anomalies.

[0090] In some embodiments, a neural network model can be used to determine whether a Kubernetes cluster is experiencing anomalies. Specifically, an initial cluster anomaly detection model needs to be trained first to obtain a trained cluster anomaly detection model. Therefore, a training set needs to be obtained, i.e., a preset operational metric training dataset. This dataset includes cluster operational metric training data, client operational metric training data, and cluster anomaly detection results, such as whether the cluster is experiencing anomalies, and if so, whether the anomaly is a client-side or cluster-wide anomaly. Then, the initial cluster anomaly detection model is trained using the data in the operational metric training dataset to determine the cluster anomaly prediction result; and the difference between the predicted cluster anomaly result and the actual cluster anomaly detection result is calculated. Based on this difference, the model parameters of the initial cluster anomaly detection model can be continuously adjusted until the final calculated difference meets preset training conditions, such as the difference not exceeding a preset difference threshold, or the training reaching a preset number of iterations. At this point, adjusting the model parameters can be stopped, resulting in a trained cluster anomaly detection model that meets the requirements. Using the trained cluster anomaly detection model, cluster operational metrics and client operational metrics can be identified, thereby determining whether the Kubernetes cluster is experiencing anomalies and, if so, what kind of anomaly. The specific model training process and the process of using the model to determine whether the cluster is abnormal can refer to existing technologies, and no limitations are made in this application.

[0091] 202. If the Kubernetes cluster experiences an anomaly, different service recovery instructions will be generated based on the specific anomaly.

[0092] 203. Based on different service recovery instructions, determine the target service backup set that can be restored for different service recovery instructions from multiple service backup sets. The multiple service backup sets are obtained by the Kubernetes cluster backing up according to the service backup strategy.

[0093] 204. Perform service recovery based on the target service backup set to obtain the recovered target service.

[0094] In this embodiment, if an anomaly is detected in the Kubernetes cluster, different service recovery instructions can be generated based on the different anomalies. Typically, one service recovery instruction corresponds to one anomaly. Based on the different service recovery instructions, a target service backup set that can be restored can be determined from multiple service backup sets. Service restoration is then performed based on the target service backup set to obtain the restored target service.

[0095] This application provides a service recovery method, including: real-time monitoring of the running status of a Kubernetes cluster to determine if the cluster is experiencing an anomaly; if an anomaly occurs, identifying the target anomaly and generating a service recovery instruction corresponding to the target anomaly, with different anomalies corresponding to different service recovery instructions; determining the target service backup set for service recovery based on the different service recovery instructions and backup set parameters of multiple service backup sets, where the multiple service backup sets are obtained by backing up the Kubernetes cluster according to a service backup strategy; and performing service recovery based on the target service backup set to obtain the recovered target service. This application generates different service recovery instructions for different types of cluster anomalies and selects different service backup sets for service recovery based on these instructions, reducing the number of service backup sets that need to be queried and improving service recovery efficiency. Furthermore, selecting the target service backup set based on the backup set parameters of the service backup sets can improve the integrity and reliability of the recovered service.

[0096] In the embodiments of this application, a Kubernetes cluster typically consists of one master node and multiple slave nodes, with each slave node connected to the master node. If the Kubernetes cluster experiences an anomaly, different service recovery instructions are generated based on the specific anomaly, including the following aspects:

[0097] If an anomaly occurs in the Kubernetes cluster, determine whether the anomaly is due to abnormal cluster performance metrics or abnormal client performance metrics. If the anomaly is due to abnormal client performance metrics, generate a first service recovery command. If the anomaly is due to abnormal cluster performance metrics, determine whether the anomaly is due to abnormal cluster master node or abnormal cluster slave node. If the anomaly is due to abnormal cluster master node, generate a second service recovery command. If the anomaly is due to abnormal cluster slave node, generate a third service recovery command.

[0098] In the above embodiments, Kubernetes cluster anomalies include anomalies in the Kubernetes cluster itself and anomalies in the clients deploying the Kubernetes cluster. Client performance metrics include CPU utilization, memory usage, network bandwidth, service response time, and other client performance data. If these data are abnormal, a client anomaly can be determined, i.e., an abnormal client performance metric, at which point a first service recovery instruction is generated. Similarly, for the cluster master node and cluster slave nodes, the master node's performance metrics and the slave node's performance metrics can be monitored respectively to determine whether the anomaly is in the master node or a slave node. The master node and slave node performance metrics to be monitored can be adjusted according to actual conditions and are not limited here. This application generates different service recovery instructions for different anomaly types, namely a first service recovery instruction, a second service recovery instruction, and a third service recovery instruction. Since the types of anomalies differ, the services that need to be recovered also differ. Therefore, different service backup sets need to be selected for service recovery based on the different service recovery instructions.

[0099] In the above embodiments, different service recovery instructions are generated based on different anomalies. These instructions need to be verified before service recovery can proceed. In some embodiments, such as... Figure 4 As shown, verifying the service recovery command may include:

[0100] 401. Perform command format validation on the service recovery command to determine whether the service recovery command meets the command format of the Kubernetes cluster.

[0101] 402. If the service recovery command passes the command format verification, the service recovery command is then decoded to determine the abnormal identifier in the service recovery command.

[0102] 403. Determine whether the exception identifier matches the pre-stored target exception identifier; if they match, then the service recovery command has passed the verification.

[0103] Specifically, the first step is to verify that the format of the service recovery command is consistent with the format of commands in the current Kubernetes cluster. This is to avoid receiving malicious commands or commands from other clusters. Only when the formats are consistent can it be confirmed that the service canary deployment command is for service recovery of the current Kubernetes cluster. Typically, each client, cluster master node, and cluster slave node has a unique identifier. When generating a service recovery command, this unique identifier needs to be added to the command to determine whether a client or a specific node has encountered an anomaly requiring service recovery. Adding this unique identifier to the service recovery command usually generates a new anomaly identifier; different nodes or clients will have different anomaly identifiers. While generating the service recovery command including the anomaly identifier, the generated anomaly identifier also needs to be stored for subsequent service recovery command verification. After the service recovery command passes the command format verification, it needs to be decoded to determine the anomaly identifier within the command. This decoded anomaly identifier is then compared with the stored target anomaly identifier. If they match, it can be further confirmed that the current service recovery command indicates an anomaly has occurred in the current Kubernetes cluster or a client deployed on the current Kubernetes cluster, requiring service recovery. In this application, the instruction format can be set according to actual needs without any limitation; at the same time, the specific process of generating and comparing the exception identifier can also refer to the existing technology, and is not limited here.

[0104] After the service recovery command is verified, service recovery can be performed according to the command. However, before the actual service recovery, a target service backup set needs to be selected from the multiple backup service sets. The target service backup set will then be used for service recovery. Figure 5 The diagram illustrates a flowchart of an embodiment of this application, illustrating the selection of different service backup sets based on different service recovery instructions. In some embodiments, determining the target service backup set for service recovery corresponding to different service recovery instructions from multiple service backup sets may include:

[0105] 501. Based on different service recovery instructions, determine the backup set type to be restored, whereby the backup set type includes full backup or incremental backup.

[0106] 502. Determine the first service backup set from among multiple service backup sets based on the backup set type.

[0107] Specifically, if the client's performance metrics in the cluster become abnormal, generating a first data recovery instruction, it can be determined that the client has encountered an anomaly. Secondly, when a client malfunctions, the cluster deployed on that client is also highly likely to malfunction, thus requiring the restoration of the original data in the client or cluster as much as possible. Typically, full backups back up all data, while incremental backups only back up changed data; to ensure complete restoration of the client or cluster data, full backups are usually required. For nodes in the cluster, different nodes typically run different services, and these services are usually independent; if a node malfunctions, only the services on that node need to be restored, without needing to use data from a full backup, only data from an incremental backup is required. Therefore, in this embodiment, different service recovery instructions represent different anomaly types, and based on these instructions, the type of backup set to be restored can be determined; specifically, whether to use a full backup or an incremental backup for service restoration.

[0108] In this application, the backup set types include full backup and incremental backup. The backup set type of different backup sets is determined when the backup is performed according to the service backup strategy. Therefore, the first service backup set can be determined from multiple service backup sets according to the backup set type.

[0109] 503. Obtain the backup set parameters of the first service backup set.

[0110] 504. Based on the backup set parameters corresponding to each of the first service backup sets, determine the service recovery weight corresponding to each of the first service backup sets, and obtain multiple service recovery weights.

[0111] 505. Based on the recovery weights of multiple services, determine the target service backup set in the first service backup set.

[0112] In the aforementioned embodiments, determining the first service backup set already filters the backup sets once, reducing the number of backup sets needed for subsequent service recovery. However, further filtering is required to determine a unique target service backup set. Specifically, backup set parameters for each first service backup set can be obtained, including but not limited to backup time and backup data size; each first service backup set has corresponding backup set parameters. Based on the backup set parameters corresponding to each first service backup set, a service recovery weight can be determined, representing the reliability of service recovery using that service backup set. Therefore, after determining the service recovery weight corresponding to each first service backup set, a target service backup set that corresponds to the service recovery instruction can be determined from the first service backup sets based on multiple service recovery weights. In this embodiment, the higher the service recovery weight, the higher the reliability of service recovery using that service backup set; therefore, the service backup set with the highest service recovery weight can be selected as the target service backup set. The specific method for determining the service backup weight based on the backup set parameters can refer to existing technologies and is not limited here.

[0113] After selecting the target service backup set, service recovery can be performed based on it. During service recovery, the first step is to determine the second storage location and namespace of the recovered service. Subsequent steps involve either performing service recovery within the second storage location or storing the recovered service there. In some embodiments, service recovery can be performed in the second storage location. Before this, the cluster needs to be configured according to the target service backup set; that is, the configuration and state at the time of backup are reversed before service recovery to ensure that the recovered service is consistent with the backup. Simultaneously, after service recovery is complete, the target service needs to be verified to confirm that the recovery is successful. In this embodiment, a simulated real-world business scenario can be used to verify whether the recovered service can operate normally and meet business requirements.

[0114] To facilitate better implementation of the service recovery method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described service recovery method. The meanings of the terms used are the same as in the above-described service recovery method, and specific implementation details can be found in the descriptions in the method embodiments. Figure 6 The diagram shown is a schematic representation of one embodiment of the service recovery operation device provided in this application, which may include:

[0115] The cluster monitoring module 601 is used to monitor the running status of the Kubernetes cluster in real time to determine whether the Kubernetes cluster is experiencing any abnormalities.

[0116] The service recovery instruction generation module 602 is used to determine the target anomaly in the Kubernetes cluster if an anomaly occurs, and generate the corresponding service recovery instruction. Different anomalies in the Kubernetes cluster correspond to different service recovery instructions.

[0117] The target service backup set determination module 603 is used to determine the target service backup set that can be restored for different service recovery instructions based on different service recovery instructions and backup set parameters of multiple service backup sets. The multiple service backup sets are obtained by the Kubernetes cluster backing up according to the service backup strategy.

[0118] Service recovery module 604 is used to restore services based on the target service backup set to obtain the restored target service.

[0119] This application provides a service recovery device that performs the following functions: real-time monitoring of the running status of a Kubernetes cluster to determine if the cluster is experiencing an anomaly; if an anomaly occurs, identifying the target anomaly and generating a service recovery instruction corresponding to the target anomaly, with different anomalies corresponding to different service recovery instructions; determining the target service backup set for service recovery based on the different service recovery instructions and backup set parameters of multiple service backup sets, where the multiple service backup sets are obtained by backing up the Kubernetes cluster according to a service backup strategy; and performing service recovery based on the target service backup set to obtain the recovered target service. This application generates different service recovery instructions for different types of cluster anomalies and selects different service backup sets for service recovery based on these instructions, reducing the number of service backup sets that need to be queried and improving service recovery efficiency. Furthermore, selecting the target service backup set based on the backup set parameters of the service backup sets can improve the integrity and reliability of the recovered service.

[0120] In some embodiments of this application, the cluster monitoring module 601 may specifically be used for:

[0121] Deploy the Kubernetes cluster in the client;

[0122] A status monitoring component is deployed in the client, the status monitoring component including a cluster monitoring component and a client monitoring component;

[0123] The cluster monitoring component is used to monitor the cluster operation metrics of the Kubernetes cluster, and the client monitoring component is used to monitor the client operation metrics of the client.

[0124] Based on the cluster operation metrics and the client operation metrics, determine whether the Kubernetes cluster is experiencing anomalies.

[0125] In some embodiments of this application, the cluster monitoring module 601 may specifically be used for:

[0126] Obtain a preset operational indicator training dataset, which includes cluster operational indicator training data, client operational indicator training data, and cluster anomaly judgment results;

[0127] The initial cluster anomaly identification model is trained using the preset operational index training dataset to determine the cluster anomaly prediction result;

[0128] Calculate the difference between the cluster anomaly judgment result and the cluster anomaly prediction result;

[0129] Adjust the model parameters of the initial cluster anomaly identification model according to the difference until the difference meets the preset training conditions, then stop adjusting the model parameters to obtain the trained cluster anomaly identification model.

[0130] Based on the cluster anomaly identification model, the cluster operation metrics, and the client operation metrics, determine whether the Kubernetes cluster is experiencing anomalies.

[0131] In some embodiments of this application, the Kubernetes cluster includes a cluster master node and cluster slave nodes, and the service recovery instruction generation module 602 can specifically be used for:

[0132] If the Kubernetes cluster experiences an anomaly, it is determined that the anomaly is either due to an abnormal cluster operation metric or an abnormal client operation metric.

[0133] If the anomaly is an abnormality in the client's operating metrics, then a first service recovery instruction is generated;

[0134] If the anomaly is an abnormality in the cluster operation metrics, then the anomaly is determined to be either an abnormality in the cluster master node or an abnormality in the cluster slave node.

[0135] If the cluster master node is abnormal, a second service recovery command will be generated;

[0136] If a cluster slave node fails, a third service recovery command will be generated.

[0137] In some embodiments of this application, the target service backup set determination module 603 may specifically be used for:

[0138] Based on the different service recovery instructions, determine the backup set type to be restored, whereby the backup set type includes full backup or incremental backup;

[0139] A first service backup set is determined from the plurality of service backup sets according to the backup set type;

[0140] Obtain the backup set parameters of the first service backup set;

[0141] Based on the backup set parameters corresponding to each of the first service backup sets, the service recovery weights corresponding to each of the first service backup sets are determined, resulting in multiple service recovery weights;

[0142] Based on the multiple service recovery weights, the target service backup set is determined in the first service backup set.

[0143] This application also provides an electronic device, such as... Figure 7 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0144] The electronic device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will understand that... Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0145] The processor 701 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operation of the storage medium, user interface, and application programs, while the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 701.

[0146] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store applications required for operating the storage medium and at least one function (such as sound playback function, image playback function, etc.); the data storage area may store data created according to the use of the electronic device. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0147] The electronic device also includes a power supply 703 that supplies power to various components. Preferably, the power supply 703 can be logically connected to the processor 701 via a power management storage medium, thereby enabling functions such as charging, discharging, and power consumption management through the power management storage medium. The power supply 703 may also include one or more DC or AC power supplies, recharge storage media, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0148] The electronic device may also include an input unit 704, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0149] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 702 according to the following instructions, and the processor 701 runs the applications stored in the memory 702, thereby realizing the steps in the above-described service recovery method embodiment.

[0150] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0151] Therefore, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to execute the steps in any of the service recovery methods provided in embodiments of this application.

[0152] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0153] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk, or optical disk, etc. Since the computer program stored in the computer-readable storage medium can execute the steps of any of the service recovery methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the service recovery methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0154] When the computing device in this application embodiment is a terminal device, this application embodiment also provides a terminal device, such as... Figure 8 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0155] Figure 8 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 8 The mobile phone includes components such as a radio frequency (RF) circuit 810, a memory 820, an input unit 830, a display unit 840, a sensor 850, an audio circuit 860, a wireless fidelity (WiFi) module 870, a processor 880, and a power supply 890. Those skilled in the art will understand that... Figure 8 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0156] The following is combined Figure 8 A detailed introduction to each component of a mobile phone:

[0157] RF circuit 810 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 880; additionally, it transmits uplink data to the base station. Typically, RF circuit 810 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 810 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0158] The memory 820 can be used to store software programs and modules. The processor 880 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 820. The memory 820 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 820 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0159] The input unit 830 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 830 may include a touch panel 831 and other input devices 832. The touch panel 831, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 831), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 831 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 880, and can also receive and execute commands sent by the processor 880. In addition, the touch panel 831 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 831, the input unit 830 may also include other input devices 832. Specifically, other input devices 832 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0160] The display unit 840 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 840 may include a display panel 841, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Furthermore, a touch panel 831 may cover the display panel 841. When the touch panel 831 detects a touch operation on or near it, it transmits the information to the processor 880 to determine the type of touch event. Subsequently, the processor 880 provides corresponding visual output on the display panel 841 based on the type of touch event. Although in Figure 8 In this embodiment, the touch panel 831 and the display panel 841 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 831 and the display panel 841 can be integrated to realize the input and output functions of the mobile phone.

[0161] The mobile phone may also include at least one sensor 850, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 841 according to the ambient light level, and the proximity sensor can turn off the display panel 841 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, which can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0162] Audio circuit 860, speaker 861, and microphone 862 provide an audio interface between the user and the mobile phone. Audio circuit 860 converts received audio data into electrical signals and transmits them to speaker 861, where speaker 861 converts them into sound signals for output. On the other hand, microphone 862 converts collected sound signals into electrical signals, which are received by audio circuit 860, converted into audio data, and then output to processor 880 for processing. The audio data is then transmitted via RF circuit 810 to, for example, another mobile phone, or output to memory 820 for further processing.

[0163] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 870, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 8 The Wi-Fi module 870 is shown, but it is understood that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0164] The processor 880 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 820, and by calling data stored in the memory 820, thereby providing overall monitoring of the phone. Optionally, the processor 880 may include one or more processing units; optionally, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 880.

[0165] The mobile phone also includes a power supply 890 (such as a battery) to power various components. Optionally, the power supply can be logically connected to the processor 880 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Although not shown, the mobile phone may also include a camera, Bluetooth module, etc., which will not be described in detail here. In this embodiment, the processor 880 included in the mobile phone also has the function of controlling the execution of the service recovery method flow performed by the service recovery device.

[0166] This application also provides a server; please refer to [link / reference]. Figure 9 , Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 922 (e.g., one or more processors) and memory 932, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 942 or data 944. The memory 932 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 922 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the server 900.

[0167] Server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0168] The steps in the service recovery method in the above embodiments can be based on this. Figure 9 The structure of server 900 is shown.

[0169] For example, the central processing unit 922 performs the following operations by calling instructions from memory 932:

[0170] Monitor the running status of the Kubernetes cluster in real time to determine if the Kubernetes cluster is experiencing any abnormalities;

[0171] If the Kubernetes cluster experiences an anomaly, the target anomaly is determined, and a service recovery instruction corresponding to the target anomaly is generated. Different anomalies in the Kubernetes cluster correspond to different service recovery instructions.

[0172] Based on the different service recovery instructions and the backup set parameters of multiple service backup sets, the target service backup set that can be restored for each service recovery instruction is determined. The multiple service backup sets are obtained by the Kubernetes cluster backing up according to the service backup strategy.

[0173] The service is restored based on the target service backup set to obtain the restored target service.

[0174] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0175] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0176] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0177] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0179] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0180] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0181] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A service restoration method characterized by, The method applied to a client comprises: Real-time monitoring of a running state of a k8s cluster to determine whether the k8s cluster is abnormal; If the k8s cluster is abnormal, determining a target abnormality of the k8s cluster, generating a service recovery instruction corresponding to the target abnormality, different abnormality of the k8s cluster corresponding to different service recovery instructions; According to the different service recovery instructions and backup set parameters of a plurality of service backup sets, determining a target service backup set corresponding to the different service recovery instructions that can perform service recovery, the plurality of service backup sets being obtained by the k8s cluster according to a service backup strategy; According to the target service backup set, performing service recovery to obtain a target service after recovery; The real-time monitoring of the running state of the k8s cluster to determine whether the k8s cluster is abnormal comprises: Deploying the k8s cluster in the client; Deploying a state monitoring component in the client, the state monitoring component comprising a cluster monitoring component and a client monitoring component; Monitoring cluster running indicators of the k8s cluster by using the cluster monitoring component, and monitoring client running indicators of the client by using the client monitoring component; According to the cluster running indicators and the client running indicators, determining whether the k8s cluster is abnormal; According to the cluster running indicators and the client running indicators, determining whether the k8s cluster is abnormal comprises: Obtaining a preset running indicator training data set, the running indicator training data set comprising cluster running indicator training data, client running indicator training data and a cluster abnormality judgment result; Training an initial cluster abnormality identification model by using the preset running indicator training data set to determine a cluster abnormality prediction result; Calculating a difference value between the cluster abnormality judgment result and the cluster abnormality prediction result; Adjusting model parameters of the initial cluster abnormality identification model according to the difference value until the difference value meets a preset training condition, stopping adjusting the model parameters, and obtaining a trained cluster abnormality identification model; According to the cluster abnormality identification model, the cluster running indicators and the client running indicators, determining whether the k8s cluster is abnormal; According to the different service recovery instructions and backup set parameters of a plurality of service backup sets, determining a target service backup set corresponding to the different service recovery instructions that can perform service recovery, the plurality of service backup sets being obtained by the k8s cluster according to a service backup strategy; According to the different service recovery instructions, determining a backup set type to be subjected to service recovery, the backup set type comprising full backup or incremental backup; According to the backup set type, determining a first service backup set in the plurality of service backup sets; Obtaining backup set parameters of the first service backup set; According to the backup set parameters corresponding to the first service backup set respectively, determining service recovery weights corresponding to the first service backup set respectively, and obtaining a plurality of service recovery weights; According to the plurality of service recovery weights, determining the target service backup set in the first service backup set.

2. The service recovery method according to claim 1, characterized by, Before monitoring a running state of a k8s cluster in real time to determine whether the k8s cluster is abnormal, the method further comprises: obtaining a service backup strategy for backing up the k8s cluster; determining a service to be backed up and a first storage location according to the service backup strategy; backing up the service to be backed up to obtain a plurality of service backup sets and storing the plurality of service backup sets in the first storage location.

3. The service recovery method according to claim 1, characterized by, The k8s cluster comprises a cluster master node and a cluster slave node; if the k8s cluster is abnormal, a target abnormality of the k8s cluster is determined, a service recovery instruction corresponding to the target abnormality is generated, different abnormalities of the k8s cluster correspond to different service recovery instructions, comprising: if the k8s cluster is abnormal, determining whether the abnormality is a cluster running index abnormality or a client running index abnormality; if the abnormality is the client running index abnormality, generating a first service recovery instruction; if the abnormality is the cluster running index abnormality, determining whether the abnormality is a cluster master node abnormality or a cluster slave node abnormality; if it is the cluster master node abnormality, generating a second service recovery instruction; if it is the cluster slave node abnormality, generating a third service recovery instruction.

4. The service recovery method of claim 1, wherein, The method further comprises: verifying the target service to determine whether the target service is recovered correctly.

5. A service restoration apparatus characterized by comprising: The service recovery apparatus comprises: a cluster monitoring module for monitoring a running state of a k8s cluster in real time to determine whether the k8s cluster is abnormal; a service recovery instruction generation module for determining a target abnormality of the k8s cluster if the k8s cluster is abnormal, generating a service recovery instruction corresponding to the target abnormality, different abnormalities of the k8s cluster corresponding to different service recovery instructions; a target service backup set determination module for determining a target service backup set that can be recovered according to different service recovery instructions and backup set parameters of a plurality of service backup sets, the plurality of service backup sets being obtained by the k8s cluster according to a service backup strategy; a service recovery module for recovering a target service according to the target service backup set to obtain a recovered target service; wherein the monitoring of the running state of the k8s cluster in real time to determine whether the k8s cluster is abnormal comprises: deploying the k8s cluster in a client; deploying a state monitoring component in the client, the state monitoring component comprising a cluster monitoring component and a client monitoring component; monitoring a cluster running index of the k8s cluster by using the cluster monitoring component and monitoring a client running index of the client by using the client monitoring component; determining whether the k8s cluster is abnormal according to the cluster running index and the client running index; the determination of whether the k8s cluster is abnormal according to the cluster running index and the client running index comprises: Obtain a preset operation index training data set, the operation index training data set including cluster operation index training data, client operation index training data, and a cluster anomaly judgment result; Train an initial cluster anomaly identification model using the preset operation index training data set to determine a cluster anomaly prediction result; Calculate a difference between the cluster anomaly judgment result and the cluster anomaly prediction result; Adjust model parameters of the initial cluster anomaly identification model according to the difference until the difference meets a preset training condition, stop adjusting the model parameters, and obtain a trained cluster anomaly identification model; Determine whether the k8s cluster is abnormal according to the cluster anomaly identification model, the cluster operation index, and the client operation index; According to the different service recovery instructions and the backup set parameters of the plurality of service backup sets, the target service backup set that can be recovered by the different service recovery instructions, comprising: According to the different service recovery instructions, determine the backup set type to be recovered by the service, the backup set type including full backup or incremental backup; According to the backup set type, determine the first service backup set in the plurality of service backup sets; Obtain the backup set parameters of the first service backup set; According to the backup set parameters corresponding to each of the first service backup sets, determine the service recovery weight corresponding to each of the first service backup sets, and obtain a plurality of service recovery weights; According to the plurality of service recovery weights, determine the target service backup set in the first service backup set.

6. An electronic device, comprising: A processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being executed by the processor to implement the steps in the service recovery method of any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the service recovery method of any one of claims 1 to 4.

Images