Fault processing method, device, equipment, storage medium and product

Abstract

The application discloses a fault processing method and device, equipment, storage medium and product, and relates to the technical field of big data, and comprises the following steps: acquiring the service plane network card state on the node where the target component is located; when the service plane network card state is an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit; and determining the target fault node according to the fault node information through an open source container orchestration system, and removing the basic schedulable unit on the target fault node. In the foregoing manner, after the service plane network card state is determined to be an abnormal state, fault node information is generated, then the target fault node is determined through the open source container orchestration system, and the basic schedulable unit on the target fault node is removed from the target service in response, so that the efficiency and accuracy of processing faults can be effectively improved, the service interruption time is maximally reduced, the reliability of the overall system is improved, and the high availability and stability of the cluster are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data, and particularly relates to a fault processing method and device, equipment, a storage medium and a product. BACKGROUND

[0002] During the operation of a system, there will be more or less faults. In order to ensure the efficient and correct operation of the system, the faults need to be processed in time, especially the node-level faults. At present, the common way to process faults is based on health check and automatic failover. Specifically, a health check mechanism is set to monitor the state of each node and service, and then when a fault occurs in a node or a replica, an automatic failover mechanism is triggered to remove the affected node or service from the forwarding list of the load balancer, and at the same time, a standby node or replica is started to take over the service to ensure that the service does not interrupt. However, the above-mentioned method will consume a lot of time in the process of failover and starting, and the speed is slow, and the detection range of the health check mechanism is small, which will ignore part of the detection area of the plane. Therefore, the above-mentioned method has low efficiency and accuracy in processing faults.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a fault processing method, device, equipment, storage medium and product, which aims to solve the technical problem of low efficiency and accuracy in processing faults in the prior art.

[0005] To achieve the above-mentioned purpose, the present application provides a fault processing method, which comprises:

[0006] obtaining the service plane network card state of the node where the target component is located;

[0007] When the service plane network card state is an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit;

[0008] determining the target fault node according to the fault node information through an open source container orchestration system, and removing the basic schedulable unit on the target fault node.

[0009] In an embodiment, the step of obtaining the service plane network card state of the node where the target component is located comprises:

[0010] obtaining the identification information of the service plane network card of the node where the target component is located;

[0011] determining the name of the service plane network card of the node where the target component is located according to the identification information by executing a preset command;

[0012] When the node where the target component is located has annotation information, the transceiving packet data of the service plane network card in a preset time period is acquired according to the name and the annotation information;

[0013] The service plane network card state on the node where the target component is located is acquired according to the transceiving packet data.

[0014] In an embodiment, after the step of determining the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command, the method further comprises:

[0015] When the node where the target component is located does not have annotation information, the current state of the network is acquired;

[0016] When the current state is a preset state, whether the network transmission packet and the network receiving packet change is determined by a preset basic capability;

[0017] The service plane network card state on the node where the target component is located is acquired according to the determination result.

[0018] In an embodiment, when the service plane network card state is an abnormal state, the step of generating fault node information according to the probe result of the target object of the current basic schedulable unit comprises:

[0019] Annotation data added on the resource description of the current basic schedulable unit is acquired;

[0020] When it is determined that the service has declared to enable the isolation capability according to the annotation data, whether the service plane network card state is an abnormal state is determined;

[0021] When the service plane network card state is an abnormal state, the probe result of the target object of the basic schedulable unit is set by a preset fault setting strategy to obtain fault node information.

[0022] In an embodiment, the step of determining a target fault node according to the fault node information by the open source container orchestration system and removing the basic schedulable unit on the target fault node comprises:

[0023] A target fault node is determined according to the fault node information by the open source container orchestration system;

[0024] The current state of the basic schedulable unit on the target fault node is acquired;

[0025] The current state of the basic schedulable unit is set to a target state by a control plane;

[0026] The basic schedulable unit in the target state is removed from the target service in response.

[0027] In an embodiment, after the step of obtaining the service plane network card state of the node where the target component is located, the method further comprises:

[0028] when the service plane network card state is a normal state, obtaining a custom check script of a basic schedulable unit;

[0029] setting an execution permission of the custom check script;

[0030] performing health check according to the execution permission by executing the custom check script.

[0031] In addition, to achieve the above object, the present application further provides a fault processing device, which comprises:

[0032] an obtaining module, configured to obtain a service plane network card state of a node where a target component is located;

[0033] a generating module, configured to, when the service plane network card state is an abnormal state, generate fault node information according to a probe result of a target object of a current basic schedulable unit;

[0034] a fault processing module, configured to determine a target fault node according to the fault node information by an open source container orchestration system, and remove a basic schedulable unit on the target fault node.

[0035] In addition, to achieve the above object, the present application further provides a fault processing device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the fault processing method as described above.

[0036] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the fault processing method as described above.

[0037] In addition, to achieve the above object, the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the fault processing method as described above.

[0038] The one or more technical solutions provided in the application have at least the following technical effects: obtaining the service plane network card state on the node where the target component is located; when the service plane network card state is an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit; determining the target fault node according to the fault node information through the open source container orchestration system, and removing the basic schedulable unit on the target fault node; in the above manner, after obtaining the service plane network card state, it is judged whether it is an abnormal state, if yes, the fault node information is generated according to the probe result of the target object of the current basic schedulable unit, then the node with fault, i.e. the target fault node, is determined through the open source container orchestration system, and the basic schedulable unit on the target fault node is removed from the target service in response, so as to effectively improve the efficiency and accuracy of fault processing, minimize service interruption time, improve the reliability of the overall system, and ensure the high availability and stability of the cluster. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings from these drawings without creative labor.

[0041] Figure 1 Flowchart provided for the first embodiment of the fault processing method of the application;

[0042] Figure 2 Flowchart provided for the second embodiment of the fault processing method of the application;

[0043] Figure 3 Brief flowchart provided for the fault processing method of the application;

[0044] Figure 4 Module structure diagram of the fault processing device of the embodiment of the application;

[0045] Figure 5 Device structure diagram of the hardware running environment involved in the fault processing method in the embodiment of the application.

[0046] The purpose implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a fault processing system, etc. capable of realizing the above functions. The following takes the fault processing system as an example to describe the embodiment and the following embodiments.

[0048] Based on this, the embodiment of the application provides a fault processing method, referring to Figure 1 , Figure 1 is a flowchart of the first embodiment of the fault processing method of the application.

[0049] In the embodiment, the fault processing method comprises steps S10-S40:

[0050] Step S10, obtain the service plane network card state on the node where the target component is located.

[0051] It should be noted that the target component can be a Kubelet component, which is a key component on the Kubernetes node, responsible for managing the container life cycle and other tasks on the node. The service plane network card state on the node where the target component is located includes abnormal state and normal state, which can be periodically detected by a timing task. Abnormalities are found in time, thereby avoiding the traditional mode of waiting for faults to occur before processing. The service plane network card state is also cached. The service plane network card includes but is not limited to the OM plane network card of the management plane node and the Service plane network card of the data plane node.

[0052] It can be understood that for an open source container orchestration system, when a query request of kubelet is received, the IP information of the inspected network plane in the node annotation is issued, the purpose being to enable the control platform to provide the required network information to the node, so that the node can perform subsequent state detection and management.

[0053] Further, step S10 comprises: obtaining identification information of the service plane network card on the node where the target component is located; determining the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command; when there is annotation information on the node where the target component is located, obtaining the transceiving packet data of the service plane network card within a preset time period according to the name and the annotation information; and obtaining the service plane network card state on the node where the target component is located according to the transceiving packet data.

[0054] It should be understood that the identification information refers to information for identifying the service plane network card. After the open source container orchestration system (Kubernetes, K8S) is managed in the node, the IP corresponding to the service network card is saved in the annotation information and the status information of the node. The preset command can be a shell script execution command. At this time, the name of the service plane network card on the node where the target component is located is determined according to the identification information by executing the preset command. In addition, in the current capability, the kubelet can obtain the network card name of the Service and the OM plane during installation, and can be directly used, without the need to parse the configuration information of the node.

[0055] It can be understood that it is also necessary to determine whether the node where the target component is located exists annotation information. If yes, the service plane network card state on the node where the target component is located can be obtained according to the transmit and receive packet data of the service plane network card in a preset time period. The preset time period can be 10 seconds. For example, the received packet data of the service plane network card is viewed through the command netmon stats-i eth0-f rx_packets-t10, and the transmitted packet data of the service plane network card is viewed through the command netmon stats-i eth0-f tx_packets-t10. Then, the service plane network card state on the node where the target component is located is obtained according to the transmit and receive packet data. For safety, a broadcast ping packet can be sent through ping-b{broadcast address}-c 1 to trigger the transmit and receive packet when no change is detected. It should be noted that the annotation information only supports viewing the physical network card, and does not support viewing the virtual network card and the bond network card.

[0056] Further, after the step of determining the name of the service plane network card on the node where the target component is located according to the identification information by executing the preset command, the method further comprises: when the node where the target component is located does not exist annotation information, obtaining the current state of the network; when the current state is a preset state, determining whether the network transmission data packet and the network receive data packet change through a preset basic capability; and obtaining the service plane network card state on the node where the target component is located according to the determination result.

[0057] It can be understood that when it is determined that the node where the target component is located does not have annotation information, it indicates that the service plane network card state on the node where the target component is located needs to be obtained through the network, that is, it is determined whether the current state of the network is a preset state, which can be up or unknown. If it is not the preset state, the service plane network card state is directly determined to be an abnormal state. If it is the preset state, it is determined whether the network transmission data packet and the network received data packet change through the preset basic capability. Specifically, the network transmission data packet and the network received data packet are compared with the data packet in the temporary file. If there is a change, the service plane network card state is determined to be a normal state. At this time, the result is also saved to the temporary file. It should be noted that in the initial state, the values in the temporary file are all 0.

[0058] It should be noted that the service plane network card state on each node is monitored in real time, and the network connection and data transmission rate are monitored in real time using annotation information. When an anomaly is found, the state of the ovs on the node can be monitored through the monit, and when the ovs state is abnormal, the service plane network card state is set to down. At this time, there is no more receiving and transmitting packet information. After the ovs is restored, the service plane network card state is set to up, and the availability is restored.

[0059] It should be understood that the kubelet persists the detection result to the host for the kubelet's probe coroutine to obtain the state, and can also be used by other services. The save path can be / opt / paas-node-monit / node-network-status, the paas-node-monit directory permission can be paas: paas 750, the node-network-status file permission can be paas: paas 640. The kubelet realizes the network card detection capability externally, and the detection capability is realized through a shell script. The node check total entrance can be / opt / kube-agent / kubernetes / kubelet / node-monit / node-monit.sh, the node network card check script can be / opt / kube-agent / kubernetes / kubelet / node-monit / node-network-monit.sh, the node-monit directory permission can be root: root 700, the node-monit.sh permission can be root: root 500, and the node-network-monit.sh permission can be root: root 500.

[0060] Further, after step S10, further comprising: when the service plane network card state is a normal state, obtaining a custom check script of the basic schedulable unit; setting an execution permission of the custom check script; and performing health check by executing the custom check script according to the execution permission.

[0061] It can be understood that the custom check script refers to a script customized by the basic schedulable unit, which can be a Pod. When the service plane network card state is determined to be a normal state, it indicates that the probe result of the target object is successful. At this time, health check can be performed by executing the custom check script according to the execution permission. The target object can be Readniess.

[0062] Step S20, when the service plane network card state is an abnormal state, generating fault node information according to the probe result of the target object of the current basic schedulable unit.

[0063] It should be understood that the kubelet first judges the service plane network card state before executing the probe result of the target object of the basic schedulable unit to ensure network availability and provide more reliable guarantee for the health status of the container. At this time, it is necessary to determine whether the service plane network card state is an abnormal state. If so, the probe result of the target object of the basic schedulable unit is set to generate fault node information.

[0064] Step S30, determining a target fault node according to the fault node information by an open source container orchestration system, and removing the basic schedulable unit on the target fault node.

[0065] It can be understood that the open source container orchestration system refers to an open source container orchestration system for automatic deployment, expansion, and management of application programs. After obtaining the fault node information, the Kubelet reports the fault node information to the open source container orchestration system. At this time, the open source container orchestration system determines the target fault node and removes the basic schedulable unit on the target fault node to ensure that traffic is not routed to the unavailable basic schedulable unit, thereby guaranteeing the high availability and stability of the service. On the other hand, it also notifies the administrator of the exception, performs fault isolation and removal operation, and ensures that the fault is handled in time.

[0066] Further, step S30 comprises: determining a target fault node according to the fault node information by an open source container orchestration system; obtaining a current state of the basic schedulable unit on the target fault node; setting the current state of the basic schedulable unit to a target state through a control plane; and removing the basic schedulable unit in the target state from the target service in response.

[0067] It should be understood that the current state refers to the state of the basic schedulable unit on the target fault node at the current time, which can be a failure state, and the target state can be a NotReady state. For an open source container orchestration system, after receiving the fault node information, the state is set through the control plane, that is, the current state of the basic schedulable unit is set to the target state, and then the basic schedulable unit in the NotReady state is removed from the target service in response, which can be a Kubernetes service. When the service plane network card state is determined to be an abnormal state, an automated processing flow is triggered, thereby reducing the need for manual intervention or monitoring personnel access.

[0068] The embodiment obtains the service plane network card state of the node where the target component is located. When the service plane network card state is an abnormal state, fault node information is generated according to the probe result of the target object of the current basic schedulable unit. The open source container orchestration system determines the target fault node according to the fault node information and removes the basic schedulable unit on the target fault node. In this way, after obtaining the service plane network card state, it is determined whether it is an abnormal state. If so, fault node information is generated according to the probe result of the target object of the current basic schedulable unit. Then, the open source container orchestration system determines the node with a fault, that is, the target fault node, and removes the basic schedulable unit on the target fault node from the target service in response, thereby effectively improving the efficiency and accuracy of fault handling, minimizing service interruption time, improving the reliability of the overall system, and ensuring the high availability and stability of the cluster.

[0069] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can refer to the above introduction, and will not be described in detail. On this basis, please refer to Figure 2 , step S20 includes steps S201-S203:

[0070] Step S201, obtaining the annotation data added on the resource description of the current basic schedulable unit.

[0071] It should be noted that the annotation data refers to the data added on the resource description by the service according to the actual demand, which can be used to determine whether the service declares to enable isolation capability. That is, the service can selectively enable the abnormal isolation function to enhance fault tolerance.

[0072] Step S202, when it is determined that the service has declared to enable isolation capability according to the annotation data, it is determined whether the service plane network card state is an abnormal state.

[0073] It can be understood that after obtaining the annotation data, the annotation data needs to be analyzed to determine whether the service declares to enable the isolation capability. For example, when the annotation Key is gde.huawei.com / isolation-when-data-network-abnormal and the annotation Value is true, it indicates that the service has declared to enable the isolation capability. If there is no annotation or the annotation Value is false, it indicates that the capability is not needed, i.e., the isolation capability is not declared to be enabled.

[0074] In step S203, when the service plane network card state is an abnormal state, the probe result of the target object of the basic schedulable unit is set by a preset fault setting strategy to obtain fault node information.

[0075] It should be understood that the preset fault setting strategy refers to a strategy for setting the probe result. When the Kubelet executes the target object probe of each basic schedulable unit, if the service has declared to enable the isolation capability and the service plane network card state is an abnormal state, the probe result of the target object of the basic schedulable unit is directly set by the preset fault setting strategy to obtain fault node information. The fault node information can represent a failed probe result, indicating that the basic schedulable unit is unavailable.

[0076] In this embodiment, the annotation data added to the resource description of the current basic schedulable unit is obtained. When it is determined according to the annotation data that the service has declared to enable the isolation capability, it is determined whether the service plane network card state is an abnormal state. When the service plane network card state is an abnormal state, the probe result of the target object of the basic schedulable unit is obtained. The probe result of the target object of the basic schedulable unit is set by a preset fault setting strategy to obtain fault node information. In this way, whether the service has declared to enable the isolation capability is determined according to the added annotation data. If yes, it is further determined whether the service plane network card state is an abnormal state. If yes, the probe result of the target object is set by the preset fault setting strategy, thereby effectively improving the accuracy of obtaining fault node information.

[0077] For the purpose of facilitating understanding of the implementation process of the fault handling method obtained after the above-mentioned embodiment one, an example is provided as follows. Figure 3 , Figure 3 A brief flowchart of a fault handling method is provided, and specifically,

[0078] The fault processing system is divided into a management plane and a data plane, wherein the management plane includes a control plane (k8s apiserver) of an open source container orchestration system, and the data plane includes a target component (kubelet), a plurality of basic schedulable units (Pods), and a service plane network card (ethx). The specific process is that when the k8s apiserver receives a query request of the kubelet, the k8s apiserver issues IP information of a network plane to be detected in a node annotation, then periodically detects a service plane network card state on a node where the kubelet is located, and caches the service plane network card state. It is determined through annotation data added to a current resource description of the Pod whether the service has declared to enable isolation capability. If yes, it is further determined whether the service plane network card state is an abnormal state. If yes, a probe result of a target object of the Pod is set through a preset fault setting strategy, fault node information is obtained, and the fault node information is reported to the k8s apiserver. The target fault node is determined through the k8s apiserver, and the basic schedulable unit on the target fault node is removed, so that the request flow is no longer sent to the target fault node.

[0079] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the fault processing method of the present application. More forms of simple changes based on this technical concept are within the protection scope of the present application.

[0080] The present application also provides a fault processing device, please refer to Figure 4 , the fault processing device comprises:

[0081] The acquisition module 10 is configured to acquire a service plane network card state on a node where a target component is located.

[0082] The generation module 20 is configured to generate fault node information according to a probe result of a target object of a current basic schedulable unit when the service plane network card state is an abnormal state.

[0083] The fault processing module 30 is configured to determine a target fault node through an open source container orchestration system according to the fault node information, and remove a basic schedulable unit on the target fault node.

[0084] The embodiment obtains the service plane network card state on the node where the target component is located; when the service plane network card state is an abnormal state, generates fault node information according to the probe result of the target object of the current basic schedulable unit; and determines the target fault node according to the fault node information through an open source container orchestration system, and removes the basic schedulable unit on the target fault node. In this way, after the service plane network card state is obtained, it is determined whether it is an abnormal state, if yes, the fault node information is generated according to the probe result of the target object of the current basic schedulable unit, and then the open source container orchestration system is used to determine the node with faults, that is, the target fault node, and the basic schedulable unit on the target fault node is removed from the target service in response, so that the efficiency and accuracy of handling faults can be effectively improved, the service interruption time is minimized, the reliability of the overall system is improved, and the high availability and stability of the cluster are ensured.

[0085] The fault processing device provided in the present application adopts the fault processing method in the above embodiment, and can solve the technical problem of low efficiency and accuracy of processing faults in the prior art. Compared with the prior art, the beneficial effects of the fault processing device provided in the present application are the same as those of the fault processing method provided in the above embodiment, and other technical features in the fault processing device are the same as those disclosed in the above embodiment, which will not be repeated here.

[0086] In an embodiment, the obtaining module 10 is further configured to obtain identification information of the service plane network card on the node where the target component is located; determine the name of the service plane network card on the node where the target component is located according to the identification information by executing a preset command; when there is annotation information on the node where the target component is located, obtain the transceiving packet data of the service plane network card in a preset time period according to the name and the annotation information; and obtain the service plane network card state on the node where the target component is located according to the transceiving packet data.

[0087] In an embodiment, the obtaining module 10 is further configured to, when there is no annotation information on the node where the target component is located, obtain the current state of the network; when the current state is a preset state, determine whether the network transmission packet and the network receiving packet change by a preset basic capability; and obtain the service plane network card state on the node where the target component is located according to the determination result.

[0088] In an embodiment, the obtaining module 10 is further configured to, when the service plane network card state is a normal state, obtain a self-defined check script of a basic schedulable unit; set the execution permission of the self-defined check script; and perform health check by executing the self-defined check script according to the execution permission.

[0089] In an embodiment, the generating module 20 is further configured to acquire annotation data added on the resource description of the current basic schedulable unit; determine whether the service plane network card state is an abnormal state when it is determined according to the annotation data that the service has declared to enable the isolation capability; and set the probe result of the target object of the basic schedulable unit by a preset fault setting strategy to obtain fault node information when the service plane network card state is the abnormal state.

[0090] In an embodiment, the fault processing module 30 is further configured to determine a target fault node according to the fault node information through an open source container orchestration system; acquire a current state of the basic schedulable unit on the target fault node; set the current state of the basic schedulable unit to a target state through a control plane; and remove the basic schedulable unit in the target state from the target service in response.

[0091] The present application provides a fault processing device, which comprises at least one processor and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the fault processing method in the above-mentioned embodiment one.

[0092] Reference will be made to the following description Figure 5 which shows a structural diagram of a fault processing device suitable for implementing the embodiments of the present application. The fault processing device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The fault processing device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0093] As Figure 5As shown, the fault processing device can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 1002 or loaded from a storage apparatus 1003 into a random access memory (RAM) 1004. Various programs and data required for operation of the fault processing device are also stored in the RAM 1004. The processing apparatus 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input apparatuses 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output apparatuses 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the fault processing device to communicate with other devices wirelessly or by wire to exchange data. Although the fault processing device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0094] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication apparatus, or installed from the storage apparatus 1003, or installed from the ROM 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0095] The fault processing device provided in the present application adopts the fault processing method in the above-mentioned embodiments, and can solve the technical problem of low efficiency and accuracy in processing faults in the prior art. Compared with the prior art, the fault processing device provided in the present application has the same beneficial effects as the fault processing method provided in the above-mentioned embodiments, and other technical features in the fault processing device are the same as the features disclosed in the above-mentioned embodiments, which will not be described here.

[0096] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0097] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be made by any person skilled in the art within the spirit and scope of the application are intended to be encompassed by the application. Therefore, the scope of the application should be determined by the appended claims.

[0098] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the fault processing method in the above-described embodiments.

[0099] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any appropriate combination thereof.

[0100] The above-described computer readable storage medium can be included in the fault processing device; or can exist separately without being assembled into the fault processing device.

[0101] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0102] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0103] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0104] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the above fault processing method, and can solve the technical problem of low efficiency and accuracy of the prior art in processing faults. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the fault processing method provided by the above embodiments, and will not be described here.

[0105] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the fault processing method as described above.

[0106] The computer program product provided by the application can solve the technical problem of low efficiency and accuracy of the prior art in processing faults. Compared with the prior art, the beneficial effects of the computer program product provided by the application are the same as those of the fault processing method provided by the above-mentioned embodiments, and are not described here.

[0107] The above-mentioned is only part of the embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation, direct / indirect application in other related technical fields, or direct / indirect application in other related technical fields made by using the content of the application specification and drawings within the technical concept of the application is included in the patent protection scope of the application.

Claims

1. A fault handling method, characterized in that, The method includes: Get the service plane network interface card status on the node where the target component is located; When the service plane network card is in an abnormal state, fault node information is generated based on the probe results of the target object of the current basic schedulable unit. The target fault node is determined based on the fault node information using an open-source container orchestration system, and the basic schedulable unit on the target fault node is removed. The step of obtaining the service plane network interface card status on the node where the target component is located includes: Obtain the identification information of the service plane network interface card on the node where the target component is located; The name of the service plane network interface card on the node where the target component is located is determined by executing a preset command based on the identification information. When annotation information exists on the node where the target component is located, the packet sending and receiving data of the service plane network card within a preset time period is obtained according to the name and the annotation information; The status of the service plane network interface card on the node where the target component is located is obtained based on the received and transmitted packet data. The step of generating fault node information based on the probe results of the target object of the current basic schedulable unit when the service plane network card is in an abnormal state includes: Retrieve annotation data added to the resource description of the current basic schedulable unit; When it is determined from the annotation data that the service has declared that isolation capability has been enabled, it is determined whether the status of the service plane network card is abnormal. When the service plane network card is in an abnormal state, the probe results of the target object of the basic schedulable unit are set according to the preset fault setting strategy to obtain the fault node information.

2. The method as described in claim 1, characterized in that, After the step of determining the name of the service plane network interface card on the node where the target component is located based on the identification information by executing a preset command, the method further includes: If the node containing the target component does not contain annotation information, obtain the current state of the network; When the current state is a preset state, the network transmission data packets and network reception data packets are determined by the preset basic capabilities to see if they have changed. Based on the judgment result, obtain the service plane network card status on the node where the target component is located.

3. The method as described in claim 1 or 2, characterized in that, The step of determining the target faulty node based on the faulty node information using an open-source container orchestration system and removing the basic schedulable units on the target faulty node includes: The target fault node is determined based on the fault node information using an open-source container orchestration system. Obtain the current state of the basic schedulable units on the target faulty node; The current state of the basic schedulable unit is set to the target state through the control plane; Remove the basic schedulable unit of the target state from the target service in response.

4. The method as described in claim 1, characterized in that, Following the step of obtaining the service plane network interface card status on the node where the target component is located, the method further includes: When the service plane network card is in a normal state, obtain the custom check script of the basic schedulable unit; Configure the execution permissions for the custom inspection script; A health check is performed by executing the custom check script based on the specified execution permissions.

5. A fault handling device, characterized in that, The device includes: The acquisition module is used to obtain the status of the service plane network interface card on the node where the target component is located; The generation module is used to generate fault node information based on the probe results of the target object of the current basic schedulable unit when the service plane network card is in an abnormal state. The fault handling module is used to determine the target fault node based on the fault node information through an open-source container orchestration system, and remove the basic schedulable unit on the target fault node; The acquisition module is also used to acquire the identification information of the service plane network interface card on the node where the target component is located; The name of the service plane network interface card on the node where the target component is located is determined by executing a preset command based on the identification information. When annotation information exists on the node where the target component is located, the packet sending and receiving data of the service plane network card within a preset time period is obtained according to the name and the annotation information; The status of the service plane network interface card on the node where the target component is located is obtained based on the received and transmitted packet data. The fault handling module is also used to obtain annotation data added to the resource description of the current basic schedulable unit; When it is determined from the annotation data that the service has declared that isolation capability has been enabled, it is determined whether the status of the service plane network card is abnormal. When the service plane network card is in an abnormal state, the probe results of the target object of the basic schedulable unit are set according to the preset fault setting strategy to obtain the fault node information.

6. A fault handling device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the fault handling method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the fault handling method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the fault handling method as described in any one of claims 1 to 4.

Images