Fault processing method and device, equipment, storage medium and product

Abstract

The invention discloses a fault processing method and device, equipment, a storage medium and a product, and relates to the technical field of big data, and the method comprises the steps: obtaining a service plane network card state on a node where a target component is located; when the network card state of the service plane is an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit; determining a target fault node through the open source container arrangement system according to the fault node information, and removing a basic schedulable unit on the target fault node; through the above mode, the fault node information is generated after the network card state of the service plane is determined to be the abnormal state, then the target fault node is determined through the open source container arrangement system, and the basic schedulable unit on the target fault node is removed from the responded target service, so that the fault processing efficiency and accuracy can be effectively improved, and the fault processing efficiency is improved. The service interruption time is reduced to the greatest extent, the reliability of the whole system is improved, and the high availability and stability of the cluster are ensured.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to fault handling methods, devices, equipment, storage media and products. Background Art

[0002] There will be more or less faults in the system during operation. In order to ensure the efficient and correct operation of the system, it is necessary to handle faults in a timely manner, especially node-level faults. At present, the common way to handle faults is based on health checks and automatic failover. Specifically, a health check mechanism is set up to monitor the status of each node and service. Then, when a node or replica fails, the failover mechanism is automatically triggered to remove the affected node or service from the forwarding list of the load balancer, and start the backup node or replica to take over the service to ensure that the service is not interrupted. However, the above method will take a lot of time in the failover and startup process, and the speed is slow. In addition, the detection range of the health check mechanism is small, and the detection area of ​​some planes will be ignored. Therefore, the efficiency and accuracy of the above method in handling faults are low.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a fault handling method, device, equipment, storage medium and product, aiming to solve the technical problem of low efficiency and accuracy in fault handling in the prior art.

[0005] To achieve the above objectives, the present application proposes a fault handling method, which includes:

[0006] Get the service plane network card status on the node where the target component is located;

[0007] When the service plane network card state is in an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit;

[0008] The open source container orchestration system determines the target fault node according to the fault node information, and removes the basic schedulable unit on the target fault node.

[0009] In one embodiment, the step of obtaining the service plane network card status on the node where the target component is located includes:

[0010] Obtain the identification information of the service plane network card on the node where the target component is located;

[0011] Determine the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command;

[0012] When annotation information exists on the node where the target component is located, the packet sending and receiving data of the service plane network card within a preset time period is obtained according to the name and the annotation information;

[0013] The service plane network card status on the node where the target component is located is obtained according to the received and sent packet data.

[0014] In one embodiment, after the step of determining the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command, the method further includes:

[0015] When there is no annotation information on the node where the target component is located, obtain the current state of the network;

[0016] When the current state is a preset state, determining whether the network transmission data packet and the network reception data packet have changed by using the preset basic capability;

[0017] According to the judgment result, the service plane network card status on the node where the target component is located is obtained.

[0018] In one embodiment, when the service plane network card state is in an abnormal state, the step of generating fault node information according to the probe result of the target object of the current basic schedulable unit includes:

[0019] Get the annotation data added to the resource description of the current basic schedulable unit;

[0020] When it is determined according to the annotation data that the service has declared to enable the isolation capability, determining whether the service plane network card state is an abnormal state;

[0021] When the service plane network card state is in an abnormal state, the probe result of the target object of the basic schedulable unit is set by a preset fault setting strategy to obtain fault node information.

[0022] In one embodiment, the step of determining a target faulty node according to the faulty node information by an open source container orchestration system and removing a basic schedulable unit on the target faulty node includes:

[0023] Determine the target fault node according to the fault node information through an open source container orchestration system;

[0024] Obtain the current state of the basic schedulable unit on the target fault node;

[0025] Setting the current state of the basic schedulable unit to a target state through a control plane;

[0026] Removes the basic dispatchable unit of the target state from the corresponding target service.

[0027] In one embodiment, after the step of obtaining the service plane network card status on the node where the target component is located, the step further includes:

[0028] When the service plane network card state is normal, obtaining a custom inspection script of the basic schedulable unit;

[0029] Set the execution permission of the custom check script;

[0030] A health check is performed by executing the custom check script according to the execution permission.

[0031] In addition, to achieve the above purpose, the present application also proposes a fault handling device, the fault handling device comprising:

[0032] The acquisition module is used to obtain the service plane network card status on the node where the target component is located;

[0033] A generating module, configured to generate fault node information according to a probe result of a target object of a current basic schedulable unit when the service plane network card state is in an abnormal state;

[0034] The fault processing module is used to determine the target fault node according to the fault node information through the open source container orchestration system, and remove the basic schedulable unit on the target fault node.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a fault handling device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the fault handling method described above.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the fault handling method described above are implemented.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the fault handling method described above are implemented.

[0038] One or more technical solutions proposed in the present application have at least the following technical effects: by obtaining the service plane network card status on the node where the target component is located; when the service plane network card status is in an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit; determining the target fault node according to the fault node information through the open source container orchestration system, and removing the basic schedulable unit on the target fault node; through the above method, after obtaining the service plane network card status, determine whether it is an abnormal state, if so, generate fault node information according to the probe result of the target object of the current basic schedulable unit, and then determine the node with fault, that is, the target fault node, through the open source container orchestration system, and remove the basic schedulable unit on the target fault node from the corresponding target service, thereby effectively improving the efficiency and accuracy of fault handling, so as to minimize the service interruption time, improve the reliability of the overall system, and ensure the high availability and stability of the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A flowchart of the first embodiment of the troubleshooting method of the present application is provided;

[0042] Figure 2 A flowchart of the second embodiment of the fault handling method of the present application is provided;

[0043] Figure 3 A brief flowchart of the troubleshooting method provided for this application;

[0044] Figure 4 This is a schematic diagram of the module structure of the fault handling device according to an embodiment of the present application;

[0045] Figure 5 A schematic diagram of the device structure of the hardware operating environment involved in the fault handling method in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a fault handling system, etc. The following takes the fault handling system as an example to illustrate this embodiment and the following embodiments.

[0048] Based on this, the present application embodiment provides a fault handling method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the fault handling method of the present application.

[0049] In this embodiment, the fault handling method includes steps S10 to S40:

[0050] Step S10, obtaining the service plane network card status on the node where the target component is located.

[0051] It should be noted that the target component can be a Kubelet component, which is a key component on the Kubernetes node and is responsible for tasks such as managing the container lifecycle on the node. The service plane network card status on the node where the target component is located includes abnormal status and normal status. The service plane network card status can be obtained through periodic detection of scheduled tasks, and abnormalities can be discovered in time, thereby avoiding the traditional method of waiting for a fault to occur before processing. The service plane network card status will also be cached, and the service plane network cards include but are not limited to the OM plane network cards of the management plane node and the Service plane network cards of the data plane node.

[0052] It is understandable that for an open source container orchestration system, when receiving a query request from kubelet, it will send down the IP information of the inspected network plane in the node annotation. The purpose is to enable the control platform to provide the required network information to the node so that the node can perform subsequent status detection and management.

[0053] Further, step S10 includes: obtaining identification information of the service plane network card on the node where the target component is located; determining the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command; when annotation information exists on the node where the target component is located, obtaining the packet receiving and sending data of the service plane network card within a preset time period according to the name and the annotation information; and obtaining the status of the service plane network card on the node where the target component is located according to the packet receiving and sending data.

[0054] It should be understood that identification information refers to information used to identify the service plane network card, which can be an IP. After the open source container orchestration system (Kubernetes, K8S) manages the node, it will save the IP corresponding to the service network card in the node's annotation information (annotation) and status information (status). The preset command can be a shell script execution command. At this time, the name of the service plane network card on the node where the target component is located is determined according to the identification information by executing the preset command. In addition, in the current capabilities, kubelet can already obtain the network card names of the Service and OM planes during installation, and can be used directly without parsing the Node configuration information.

[0055] It is understandable that it is also necessary to determine whether the node where the target component is located has annotation information. If so, the service plane network card status on the node where the target component is located can be obtained based on the data of the service plane network card receiving and sending packets within a preset time period. The preset time period can be 10 seconds. For example, the service plane network card receiving packet data can be viewed through the command netmonstats-i eth0-f rx_packets-t10, and the service plane network card sending packet data can be viewed through the command netmonstats-i eth0-f tx_packets-t10. Then, the service plane network card status on the node where the target component is located can be obtained based on the data of the received and sent packets. For the sake of insurance, when no changes are detected, a broadcast ping packet can be issued through ping-b{broadcast address}-c 1 to trigger the sending and receiving of packets. It should be noted that the annotation information only supports viewing physical network cards, and does not support viewing virtual network cards and bond network cards.

[0056] Furthermore, after the step of determining the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command, it also includes: when there is no annotation information on the node where the target component is located, obtaining the current state of the network; when the current state is a preset state, judging whether the network transmission data packets and the network reception data packets have changed through preset basic capabilities; and obtaining the state of the service plane network card on the node where the target component is located according to the judgment result.

[0057] It can be understood that when it is determined that there is no annotation information on the node where the target component is located, it indicates that it is necessary to obtain the service plane network card status on the node where the target component is located through the network, that is, to determine whether the current state of the network is a preset state, which can be up or unknown. If it is not a preset state, the service plane network card status is directly determined to be an abnormal state. If it is a preset state, the preset basic capabilities are used to determine whether the network transmission data packets and the network reception data packets have changed. Specifically, the network transmission data packets and the network reception data packets are compared with the data packets in the temporary file respectively. If there are changes, the service plane network card status is determined to be normal. At this time, the results will also be saved in the temporary file. It should be noted that in the initial state, the values ​​in the temporary file are all 0.

[0058] It should be noted that the network connection and data transmission rate are monitored in real time by regularly monitoring the service plane network card status on each node and using annotation information. When an abnormality is found, the status of ovs can be monitored through monit on the node. When the ovs status is abnormal, the service plane network card status is set to down, and there is no packet sending and receiving information. After ovs is restored, the service plane network card status is set to up and restored to availability.

[0059] It should be understood that kubelet persists the detection results to the host, so that kubelet's probe coroutine can obtain the status and can also be used by other services. Among them, the save path can be: / opt / paas-node-monit / node-network-status, the paas-node-monit directory permission can be: paas:paas 750, and the node-network-status file permission can be: paas:paas 640. When Kubelet is implemented, the network card detection capability is externalized and the detection capability is implemented through shell scripts. The node inspection total entry can be: / opt / kube-agent / kubernetes / kubelet / node-monit / node-monit.sh, the node network card inspection script can be: / opt / kube-agent / kubernetes / kubelet / node-monit / node-network-monit.sh, the node-monit directory permission can be: root:root700, the node-monit.sh permission can be: root:root500, and the node-network-monit.sh permission can be: root:root 500.

[0060] Furthermore, after step S10, it also includes: when the service plane network card status is normal, obtaining a custom inspection script of the basic schedulable unit; setting the execution permission of the custom inspection script; and performing a health check by executing the custom inspection script according to the execution permission.

[0061] It can be understood that the custom check script refers to a script customized by the basic schedulable unit, which can be a Pod. When it is determined that the service plane network card status is normal, it indicates that the probe result of the target object is successful. At this time, a health check can be performed by executing the custom check script according to the execution permission. The target object can be Readniess.

[0062] Step S20, when the service plane network card state is an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit.

[0063] It should be understood that before executing the probe results of the target object of the basic schedulable unit, kubelet first determines the status of the service plane network card to ensure network availability, providing more reliable protection for the health status of the container. At this time, it is necessary to determine whether the status of the service plane network card is abnormal. If so, the fault node information is generated by setting the probe results of the target object of the basic schedulable unit.

[0064] Step S30: determining a target faulty node according to the faulty node information through an open source container orchestration system, and removing a basic schedulable unit on the target faulty node.

[0065] It is understandable that the open source container orchestration system refers to an open source container orchestration system used to automate the deployment, expansion, and management of applications. After obtaining the fault node information, Kubelet reports the fault node information to the open source container orchestration system. At this time, the open source container orchestration system will determine the target fault node and remove the basic schedulable units on the target fault node to ensure that traffic will not be routed to unavailable basic schedulable units, ensuring high availability and stability of the service. On the other hand, the administrator will be notified of the abnormality, and fault isolation and removal operations will be performed to ensure that the fault is handled in a timely manner.

[0066] Further, step S30 includes: determining the target fault node according to the fault node information through the open source container orchestration system; obtaining the current state of the basic schedulable unit on the target fault node; setting the current state of the basic schedulable unit as the target state through the control plane; and removing the basic schedulable unit in the target state from the corresponding target service.

[0067] It should be understood that the current state refers to the state of the basic schedulable unit on the target fault node at the current moment. The current state can be a failed state, and the target state can be a NotReady state. For the open source container orchestration system, after receiving the fault node information, the state is set through the control plane, that is, the current state of the basic schedulable unit is set to the target state, and then the basic schedulable unit in the unready state is removed from the responding target service, which can be a Kubernetes service. That is, when it is determined that the service plane network card state is abnormal, the automated processing flow is triggered, thereby reducing the need for manual intervention or monitoring personnel access.

[0068] This embodiment obtains the service plane network card status on the node where the target component is located; when the service plane network card status is in an abnormal state, generates fault node information according to the probe result of the target object of the current basic schedulable unit; determines the target fault node according to the fault node information through the open source container orchestration system, and removes the basic schedulable unit on the target fault node; through the above method, after obtaining the service plane network card status, it is determined whether it is an abnormal state. If so, the fault node information is generated according to the probe result of the target object of the current basic schedulable unit, and then the open source container orchestration system is used to determine the node with fault, that is, the target fault node, and remove the basic schedulable unit on the target fault node from the corresponding target service, thereby effectively improving the efficiency and accuracy of fault handling, so as to minimize the service interruption time, improve the reliability of the overall system, and ensure the high availability and stability of the cluster.

[0069] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S20 includes steps S201 to S203:

[0070] Step S201, obtaining annotation data added to the resource description of the current basic schedulable unit.

[0071] It should be noted that annotation data refers to the data that the service can add to the resource description based on actual needs. The annotation data can be used to determine whether the service declares that the isolation capability has been enabled. In other words, the service can selectively enable the exception isolation function to enhance fault tolerance.

[0072] Step S202: when it is determined according to the annotation data that the service has declared to enable the isolation capability, it is determined whether the state of the service plane network card is an abnormal state.

[0073] It is understandable that after obtaining the annotation data, it is necessary to analyze the annotation data to determine whether the service has declared that the isolation capability has been enabled. For example, when the annotation Key is: gde.huawei.com / isolation-when-data-network-abnormal and the annotation Value is: true, it indicates that the service has declared that the isolation capability has been enabled. If there is no annotation or the annotation Value is: false, it indicates that the capability is not required, that is, the isolation capability has not been declared enabled.

[0074] Step S203, when the service plane network card state is abnormal, the probe result of the target object of the basic schedulable unit is set by a preset fault setting strategy to obtain fault node information.

[0075] It should be understood that the preset fault setting strategy refers to the strategy for setting probe results. When Kubelet executes the target object probe of each basic schedulable unit, if the service has declared to enable isolation capability and the service plane network card status is abnormal, the probe result of the target object of the basic schedulable unit is set directly through the preset fault setting strategy to obtain the fault node information. The fault node information can represent the failure of the probe result, indicating that the basic schedulable unit is unavailable.

[0076] This embodiment obtains annotation data added to the resource description of the current basic schedulable unit; when it is determined according to the annotation data that the service has declared the isolation capability to be enabled, it determines whether the service plane network card status is in an abnormal state; when the service plane network card status is in an abnormal state, it obtains the probe result of the target object of the basic schedulable unit; the probe result of the target object of the basic schedulable unit is set according to a preset fault setting strategy to obtain fault node information; in the above manner, it is determined according to the added annotation data whether the service has declared the isolation capability to be enabled, and if so, it is further determined whether the service plane network card status is in an abnormal state, and if so, the probe result of the target object is set according to the preset fault setting strategy, thereby effectively improving the accuracy of the obtained fault node information.

[0077] For example, to help understand the implementation process of the fault handling method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 3 , Figure 3 A brief flowchart of a troubleshooting method is provided, specifically:

[0078] The fault handling system is divided into a management plane and a data plane. The management plane includes the control plane (k8s apiserver) of the open source container orchestration system, and the data plane includes the target component (kubelet), multiple basic schedulable units (Pod), and a service plane network card (ethx). The specific process is that when the k8s apiserver receives a query request from the kubelet, it will send the IP information of the inspected network plane in the node annotation, and then periodically detect the status of the service plane network card on the node where the kubelet is located, and cache the status of the service plane network card. By adding annotation data to the current resource description of the Pod, it is determined whether the service has declared the isolation capability to be enabled. If so, it is further determined whether the status of the service plane network card is abnormal. If so, the probe result of the target object of the Pod is set by the preset fault setting strategy to obtain the fault node information, and report the fault node information to the k8s apiserver. The target fault node is determined by the k8s apiserver, and the basic schedulable units on the target fault node are removed to ensure that the request traffic is no longer sent to the target fault node.

[0079] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the fault handling method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0080] This application also provides a fault handling device, please refer to Figure 4 , the fault handling device comprises:

[0081] The acquisition module 10 is used to acquire the service plane network card status on the node where the target component is located.

[0082] The generating module 20 is used to generate fault node information according to the probe result of the target object of the current basic schedulable unit when the service plane network card state is abnormal.

[0083] The fault processing module 30 is used to determine the target fault node according to the fault node information through the open source container orchestration system, and remove the basic schedulable unit on the target fault node.

[0084] This embodiment obtains the service plane network card status on the node where the target component is located; when the service plane network card status is in an abnormal state, generates fault node information according to the probe result of the target object of the current basic schedulable unit; determines the target fault node according to the fault node information through the open source container orchestration system, and removes the basic schedulable unit on the target fault node; through the above method, after obtaining the service plane network card status, it is determined whether it is an abnormal state. If so, the fault node information is generated according to the probe result of the target object of the current basic schedulable unit, and then the open source container orchestration system is used to determine the node with fault, that is, the target fault node, and remove the basic schedulable unit on the target fault node from the corresponding target service, thereby effectively improving the efficiency and accuracy of fault handling, so as to minimize the service interruption time, improve the reliability of the overall system, and ensure the high availability and stability of the cluster.

[0085] The fault handling device provided by the present application adopts the fault handling method in the above embodiment, which can solve the technical problem of low efficiency and accuracy in handling faults in the prior art. Compared with the prior art, the beneficial effects of the fault handling device provided by the present application are the same as those of the fault handling method provided by the above embodiment, and other technical features in the fault handling device are the same as those disclosed in the above embodiment method, which will not be described in detail here.

[0086] In one embodiment, the acquisition module 10 is also used to obtain the identification information of the service plane network card on the node where the target component is located; determine the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command; when there is annotation information on the node where the target component is located, obtain the packet receiving and sending data of the service plane network card within a preset time period according to the name and the annotation information; and obtain the status of the service plane network card on the node where the target component is located according to the packet receiving and sending data.

[0087] In one embodiment, the acquisition module 10 is also used to obtain the current state of the network when there is no annotation information on the node where the target component is located; when the current state is a preset state, determine whether the network transmission data packets and the network reception data packets have changed through preset basic capabilities; and obtain the service plane network card status on the node where the target component is located based on the judgment result.

[0088] In one embodiment, the acquisition module 10 is also used to obtain a custom inspection script of the basic schedulable unit when the service plane network card status is normal; set the execution permission of the custom inspection script; and perform a health check by executing the custom inspection script according to the execution permission.

[0089] In one embodiment, the generation module 20 is also used to obtain annotation data added to the resource description of the current basic schedulable unit; when it is determined according to the annotation data that the service has declared to enable isolation capability, it is judged whether the status of the service plane network card is an abnormal state; when the status of the service plane network card is an abnormal state, the probe result of the target object of the basic schedulable unit is set by a preset fault setting strategy to obtain fault node information.

[0090] In one embodiment, the fault handling module 30 is also used to determine the target fault node according to the fault node information through the open source container orchestration system; obtain the current state of the basic schedulable unit on the target fault node; set the current state of the basic schedulable unit as the target state through the control plane; and remove the basic schedulable unit in the target state from the corresponding target service.

[0091] The present application provides a fault handling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the fault handling method in the above-mentioned embodiment one.

[0092] Reference below Figure 5 , which shows a schematic diagram of the structure of a fault handling device suitable for implementing the embodiment of the present application. The fault handling device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The fault handling device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0093] like Figure 5As shown, the fault handling device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the fault handling device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the fault handling device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a fault handling device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0094] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above functions are defined in the method of the embodiment disclosed in the present application.

[0095] The fault handling device provided by the present application adopts the fault handling method in the above embodiment, which can solve the technical problem of low efficiency and accuracy in handling faults in the prior art. Compared with the prior art, the beneficial effects of the fault handling device provided by the present application are the same as the beneficial effects of the fault handling method provided by the above embodiment, and other technical features in the fault handling device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0096] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0098] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the fault handling method in the above-mentioned embodiment.

[0099] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0100] The computer-readable storage medium may be included in the fault handling device; or may exist independently without being assembled into the fault handling device.

[0101] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0104] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned fault handling method, and can solve the technical problem that the efficiency and accuracy of the prior art in handling faults are low. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the fault handling method provided by the above-mentioned embodiment, and will not be repeated here.

[0105] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned fault handling method when executed by a processor.

[0106] The computer program product provided by the present application can solve the technical problem of low efficiency and accuracy in fault handling in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the fault handling method provided in the above embodiment, which will not be described in detail here.

[0107] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A fault handling method, characterized in that: The method comprises: Get the service plane network card status on the node where the target component is located; When the service plane network card state is in an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit; The open source container orchestration system determines the target fault node according to the fault node information, and removes the basic schedulable unit on the target fault node.

2. The method according to claim 1, characterized in that The step of obtaining the service plane network card status on the node where the target component is located includes: Obtain the identification information of the service plane network card on the node where the target component is located; Determine the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command; When annotation information exists on the node where the target component is located, the packet sending and receiving data of the service plane network card within a preset time period is obtained according to the name and the annotation information; The service plane network card status on the node where the target component is located is obtained according to the received and sent packet data.

3. The method according to claim 2, characterized in that After the step of determining the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command, the method further includes: When there is no annotation information on the node where the target component is located, obtain the current state of the network; When the current state is a preset state, determining whether the network transmission data packet and the network reception data packet have changed by using the preset basic capability; According to the judgment result, the service plane network card status on the node where the target component is located is obtained.

4. The method according to claim 1, characterized in that The step of generating fault node information according to the probe result of the target object of the current basic schedulable unit when the service plane network card state is in an abnormal state includes: Get the annotation data added to the resource description of the current basic schedulable unit; When it is determined according to the annotation data that the service has declared to enable the isolation capability, determining whether the service plane network card state is an abnormal state; When the service plane network card state is in an abnormal state, the probe result of the target object of the basic schedulable unit is set by a preset fault setting strategy to obtain fault node information.

5. The method according to any one of claims 1 to 4, characterized in that The step of determining the target fault node according to the fault node information through the open source container orchestration system and removing the basic schedulable unit on the target fault node includes: Determine the target fault node according to the fault node information through an open source container orchestration system; Obtain the current state of the basic schedulable unit on the target fault node; Setting the current state of the basic schedulable unit to a target state through a control plane; Removes the basic dispatchable unit of the target state from the corresponding target service.

6. The method according to claim 1, characterized in that After the step of obtaining the service plane network card status on the node where the target component is located, the method further includes: When the service plane network card state is normal, obtaining a custom inspection script of the basic schedulable unit; Set the execution permission of the custom check script; A health check is performed by executing the custom check script according to the execution permission.

7. A fault handling device, characterized in that: The device comprises: The acquisition module is used to obtain the service plane network card status on the node where the target component is located; A generating module, configured to generate fault node information according to a probe result of a target object of a current basic schedulable unit when the service plane network card state is in an abnormal state; The fault processing module is used to determine the target fault node according to the fault node information through the open source container orchestration system, and remove the basic schedulable unit on the target fault node.

8. A fault handling device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the fault handling method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the fault handling method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the fault handling method according to any one of claims 1 to 6 are implemented.

Images