Process self-healing method and device

Abstract

The embodiments of the present application relate to the technical field of servers. Provided are a process self-healing method and a device. The method comprises: when a target object causes an interruption to an upgrade process of a software system, acquiring a first instruction, wherein the target object is a subsystem having an upgrade exception or a component of the subsystem; on the basis of the first instruction, detecting whether the target object has been successfully upgraded; when it is detected that the target object has been successfully upgraded, determining an architecture type of the target object, wherein the architecture type is used for characterizing the relationship between related nodes, and the related nodes include nodes that run the target object or nodes affected by the upgrade exception of the target object; on the basis of the architecture type, adjusting the related nodes to a normal running state, wherein the state is a state that supports the resumed execution of the upgrade process; and skipping the target object and resuming the execution of the upgrade process of the software system, thereby resuming the upgrade process and avoiding adverse impacts on system security caused by manual modification of metadata.

Description

A process self-healing method and device

[0001] The present application claims priority to the Chinese patent application No. 202410721271.X, filed on June 4, 2024, and entitled "A process self-healing method and device", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of servers, in particular to a process self-healing method and device. BACKGROUND

[0003] The one-key upgrade feature of the system of the device cluster can provide the user with a page-guided whole-system upgrade capability, especially for the cluster of the hyper-converged infrastructure. The whole-system upgrade of the cluster involves multiple subsystems, so the upgrade task and the surrounding business can be extremely complex. The one-key upgrade feature can provide the user with a more fast and efficient, safe and reliable upgrade experience.

[0004] However, in such a large-scale cluster upgrade scenario, there may be a situation where only individual steps or partial functions of the subsystem are repeatedly rolled back and upgraded, but the upgrade is still unsuccessful. At this time, the subsequent upgrade process will be interrupted, and the user needs to manually analyze and process the upgrade task of the subsystem and the upgrade task of the whole system of the cluster, and manually restart the upgrade process to complete the upgrade after manually repairing the upgrade exception of the subsystem. This not only requires the operator to have strong professional ability, but also has a large amount of manual processing work, takes a long time, and is prone to human-induced task omission, operation errors, and other situations, which cannot guarantee the smooth progress of the upgrade operation, and may even adversely affect the safety of the system. SUMMARY

[0005] The embodiments of the present application provide a process self-healing method, device, computing device, device cluster, computer storage medium and computer program product, which can realize upgrade process self-healing when the cluster system upgrade is abnormal, and reduce the dependence on manual work.

[0006] In a first aspect, the embodiments of the present application provide a process self-healing method, which comprises: in the case that a target object in a software system causes the upgrade process of the software system to be interrupted, obtaining a first instruction, the target object being an upgrade-abnormal subsystem or a component of the subsystem; detecting whether the target object is upgraded successfully according to the first instruction; in the case that the target object is detected to be upgraded successfully, determining an architecture type of the target object, the architecture type being used to represent the relationship between related nodes, the related nodes including a node running the target object or a node affected by the upgrade abnormality of the target object; adjusting the related nodes to a normal running state according to the architecture type, the normal running state being a state supporting the continuous execution of the upgrade process; and continuing to execute the upgrade process of the software system by skipping the target object.

[0007] In the embodiment, if the software system deployed in the server cluster is interrupted due to an abnormal upgrade of a target object, which is a subsystem or a component of a subsystem, the related nodes of the target object will also run abnormally, and the metadata will also change accordingly. After manual repair of the abnormality of the target object and manual upgrade, the upgrade process itself cannot perceive the success of the upgrade of the target object. In the embodiment, the first instruction is used to trigger detection of whether the upgrade of the target object is successful, and after determining that the upgrade of the target object is successful, the related nodes are found according to the architecture type of the target object, and abnormal records in the corresponding metadata are automatically corrected, so that the related nodes are in a normal running state. In this way, after the upgrade process is started, if the upgrade process detects that the upgrade of the target object is successful and the related nodes are in a normal running state, the next object of the target object (for example, a component of a subsystem to which the target object belongs or a next subsystem) can continue to be upgraded. Thus, manual data modification does not harm the system security, and the usability is high.

[0008] In some possible implementation manners, in a case where it is detected that the upgrade of the target object is successful, the architecture type of the target object is determined, including: in a case where it is detected that the upgrade of the target object is successful, a configuration item of the target object is acquired, the configuration item being used to describe an architecture type to which the target object belongs; and the architecture type of the target object is determined according to the configuration item.

[0009] In this way, the architecture type of a subsystem is described by a pre-set configuration item, and the architecture type of any subsystem is also the architecture type of a component of the subsystem. After the target object is found, the architecture type of the target object can be determined by reading the configuration item, thereby facilitating rapid determination of a metadata processing manner of the related nodes and facilitating improvement of the automation degree of the self-recovery operation.

[0010] In some possible implementation manners, the architecture type of the target object is a cluster architecture or a high-availability architecture, and the related nodes are adjusted to be in a normal running state according to the architecture type, including: according to the architecture type, abnormal records in the metadata of all the related nodes caused by the target object are corrected, so that the related nodes are all in a normal running state, and the target object deployed on the related nodes is all in an upgraded successful state, to support continuous execution of the upgrade process.

[0011] In this way, because the related nodes of the cluster architecture or the high-availability architecture affect each other, the metadata of these nodes needs to be uniformly processed, so as to restore the interrupted upgrade process.

[0012] In some possible implementation manners, the target object is a component of a subsystem, and the architecture type of the target object is a tree structure upgrade sequence architecture; and according to the architecture type, adjusting the related nodes to a normal operation state comprises: according to the architecture type, taking the target object as a leaf node, and correcting abnormal records in metadata of a node where the target object is located, so that the node is in a state of supporting the upgrade process to continue execution; and according to a tree structure upgrade sequence of a subsystem to which the target object belongs, recursively traversing all components as parent nodes in the subsystem from the leaf node; and determining that the components as parent nodes and the nodes where the components are located are in the state of supporting the upgrade process to continue execution.

[0013] In this way, because the components in the subsystem are in the tree structure upgrade sequence architecture, and the upgrade success between the target object and other components does not have a certain relationship, the metadata processing manners of the node running the target object and the node running other components are different, and the recursive traversal of the parent node is used to facilitate the recovery of the interrupted upgrade process.

[0014] In some possible implementation manners, in a case where the target object in the software system causes the upgrade process of the software system to be interrupted, the first instruction is acquired, comprising: in a case where the target object in the software system causes the upgrade process of the software system to be interrupted, providing an access interface for a user; and acquiring, from the access interface, the first instruction initiated by the user for the target object.

[0015] In this way, the process self-healing operation can be started in a manual triggering manner according to user demand.

[0016] In some possible implementation manners, after the first instruction is acquired, the method comprises: outputting a risk prompt of the process self-healing;

[0017] acquiring confirmation information input by the user for the risk prompt; in a case where the confirmation information comprises a risk prompt read identifier and identity information of the user, verifying whether the identity information of the user matches preset identity information; and if the identity information of the user matches the preset identity information, performing detection on whether the target object is upgraded successfully according to the first instruction.

[0018] In this way, through the risk prompt and identity confirmation, the security of the process self-healing operation is improved.

[0019] In some possible implementation manners, in a case where the target object in the software system causes the upgrade process of the software system to be interrupted, the first instruction is acquired, comprising: in a case where the target object in the software system causes the upgrade process of the software system to be interrupted, starting a timing polling task; and according to the timing polling task, generating the first instruction when each preset polling interval is reached.

[0020] In this way, the process self-recovery operation is started in a manner of automatic polling triggering, and the operation correctness is improved.

[0021] In some possible implementation manners, after the target object is skipped to continue the upgrade process of the software system, the method further includes: if the target object is the last upgraded subsystem in the upgrade process, executing a preset script to start an upgrade post-operation of the software system.

[0022] In this way, the upgrade post-operation can be performed by automatically executing the script without interface pulling, and the integrity of the upgrade operation is ensured.

[0023] In some possible implementation manners, before the first instruction is acquired, the method includes: in a case where the target object causes the upgrade process of the software system to be interrupted, determining an upgrade abnormality reason of the target object; and according to a preset repair strategy, outputting corresponding processing suggestion information for the upgrade abnormality reason.

[0024] In this way, the operation personnel is guided to perform standardized repair processing on the upgrade abnormality of the target object, and the operation failure rate is reduced.

[0025] In a second aspect, an embodiment of the present application provides a process self-recovery device, and the device includes: an acquisition module, configured to: in a case where a target object in a software system causes an upgrade process of the software system to be interrupted, acquire a first instruction, the target object being an upgrade-abnormal subsystem or a component of a subsystem; and a processing module, configured to: according to the first instruction, detect whether the target object is upgraded successfully; the processing module is further configured to: in a case where it is detected that the target object is upgraded successfully, determine an architecture type of the target object, the architecture type being used to represent a relationship between related nodes, the related nodes including a node running the target object or a node affected by the upgrade abnormality of the target object; the processing module is further configured to: according to the architecture type, adjust the related nodes to a normal running state, the normal running state being a state supporting the upgrade process to continue to be executed; and the processing module is further configured to: skip the target object to continue the upgrade process of the software system.

[0026] In some possible implementation manners, the acquisition module is specifically configured to: in a case where it is detected that the target object is upgraded successfully, acquire a configuration item of the target object, the configuration item being used to describe the architecture type to which the target object belongs; and the processing module is specifically configured to: according to the configuration item, determine the architecture type of the target object.

[0027] In some possible implementation manners, the architecture type of the target object is a cluster architecture or a high-availability architecture, and the processing module is specifically configured to: according to the architecture type, correct abnormal records in metadata of all the related nodes caused by the target object, so that the related nodes are all in the normal running state, and the target objects deployed on the related nodes are all in an upgraded-success state, to support the upgrade process to continue to be executed.

[0028] In some possible implementation manners, the target object is a component of the subsystem, the architecture type of the target object is an architecture in a tree structure upgrade order between the target object and the remaining components in the subsystem to which the target object belongs, and the processing module is specifically configured to: according to the architecture type, take the target object as a leaf node, and correct an abnormal record in metadata of a node where the target object is located, so that the node is in a state of supporting continuous execution of the upgrade process; according to the tree structure upgrade order of the subsystem to which the target object belongs, recursively traverse all components as parent nodes from the leaf node; and determine that the components as the parent nodes and nodes where the components are located are in the state of supporting the continuous execution of the upgrade process.

[0029] In some possible implementation manners, the obtaining module is specifically configured to: in a case where the target object in the software system causes interruption of an upgrade process of the software system, provide an access interface to a user; and obtain, from the access interface, a first instruction initiated by the user for the target object.

[0030] In some possible implementation manners, the processing module is further configured to: output a risk prompt of the process self-recovery; the obtaining module is further configured to obtain confirmation information input by the user for the risk prompt; and the processing module is further configured to: in a case where the confirmation information includes a read identifier of the risk prompt and identity information of the user, verify whether the identity information of the user matches preset identity information; and if the identity information of the user matches the preset identity information, perform detection on whether the target object is successfully upgraded according to the first instruction.

[0031] In some possible implementation manners, the obtaining module is specifically configured to: in a case where the target object in the software system causes interruption of an upgrade process of the software system, start a timing polling task; and according to the timing polling task, generate the first instruction when each preset polling interval is reached.

[0032] In some possible implementation manners, the processing module is further configured to: if the target object is a last upgraded subsystem in the upgrade process, execute a preset script to start an upgrade post-operation on the software system.

[0033] In some possible implementation manners, the processing module is further configured to: in a case where the target object causes interruption of an upgrade process of the software system, determine an upgrade abnormal reason of the target object; and according to a preset repair strategy, output corresponding processing suggestion information for the upgrade abnormal reason.

[0034] In a third aspect, an embodiment of the present application provides a computing device, including: at least one memory for storing a program; and at least one processor for executing the program stored in the memory; wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0035] In a fourth aspect, an embodiment of the present application provides a device cluster, comprising at least one computing device, the computing device comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory; wherein when the program stored in the memory is executed, the processor is configured to perform the method described in the first aspect or any possible implementation manner of the first aspect.

[0036] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is run on a processor, the processor executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0037] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when run on a processor, causes the processor to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0038] In a seventh aspect, an embodiment of the present application provides a chip, comprising at least one processor and an interface; the at least one processor acquires program instructions or data through the interface; and the at least one processor is configured to execute the program instructions to implement the method described in the first aspect or any possible implementation manner of the first aspect.

[0039] It can be understood that the beneficial effects of the above-mentioned second aspect to seventh aspect can refer to the related description in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0041] FIG. 1 is an architecture schematic diagram of a computing device cluster provided by an embodiment of the present application;

[0042] FIG. 2 is a self-recovery schematic diagram of a software system upgrade process in an embodiment of the present application;

[0043] FIG. 3A is a subsystem architecture schematic diagram in an example of the present application;

[0044] FIG. 3B is a subsystem architecture schematic diagram in an example of the present application;

[0045] FIG. 3C is a subsystem architecture schematic diagram in an example of the present application;

[0046] FIG. 4 is a flowchart illustrating a flow self-recovery method according to an embodiment of the present application;

[0047] FIG. 5 is a flowchart illustrating a flow self-recovery method according to an embodiment of the present application;

[0048] FIG. 6A is a flowchart illustrating a flow self-recovery method according to an embodiment of the present application;

[0049] FIG. 6B is a schematic diagram of an upgrade interface according to an embodiment of the present application;

[0050] FIG. 7A is a schematic diagram of a confirmation pop-up window according to an embodiment of the present application;

[0051] FIG. 7B is a schematic diagram of a confirmation pop-up window according to an embodiment of the present application;

[0052] FIG. 8 is a flowchart illustrating a flow self-recovery method according to an embodiment of the present application;

[0053] FIG. 9 is a schematic diagram illustrating a flow self-recovery device according to an embodiment of the present application;

[0054] FIG. 10 is a schematic diagram illustrating a chip according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] The term "and / or" used in this document is used to describe an association relationship between associated objects, which means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The symbol " / " in this document means an or relationship between associated objects, for example, A / B means A or B.

[0056] The terms "first" and "second" and the like in the description and claims of this document are used to distinguish different objects, and are not used to describe a specific order of the objects.

[0057] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of the words "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0058] In the description of the embodiments of the present application, unless otherwise specified, "a plurality of" means two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like.

[0059] In order to reduce the dependence on manual repair of upgrade process exceptions in the device cluster system upgrade scenario, an embodiment of the present application provides a process self-healing method. The method mainly executes corresponding process self-healing operations according to the architecture type of the system when the subsystem of the device cluster occurs upgrade exception, guarantees the smooth execution of the entire cluster system upgrade, reduces the manual participation, and improves the efficiency of the upgrade operation and the security of the cluster system.

[0060] In order to facilitate understanding of the technical solutions of the present application, a computing device cluster provided by an embodiment of the present application is introduced below.

[0061] For example, FIG. 1 shows an architecture schematic diagram of a computing device cluster provided by an embodiment of the present application. As shown in FIG. 1, the computing device cluster 1 can include several computing devices 10, and each computing device 10 can communicate and interact with each other through an internal network. Among them, the computing device 10 can be a physical server, such as a rack server, a blade server, a high-density server, a GPU server, etc. Each computing device 10 can include a processor (central processing unit, CPU), a memory (including an internal memory and a hard disk, etc.), and a network card and other hardware components to provide corresponding computing resources, storage resources and network resources.

[0062] For example, each of the computing devices 10 can run multiple virtual instances, which can be virtual machines (virtual machine, VM) or containers (Container), but are not limited thereto. The computing resources, storage resources and network resources of the computing device 10 can be allocated to each virtual instance for use, and each computing device 10 or each virtual instance running thereon can be regarded as a node (Node). In this way, all nodes of the entire computing device cluster 1 can cooperate together to realize the services provided by the computing device cluster 1.

[0063] Specifically, in the present example, the computing device cluster 1 can deploy and run a software system 100 through the nodes, where the software system 100 can be a cluster environment including multiple subsystems 1, 2, …, can support running of various scales and types of application programs, and can provide a high-performance and high-reliability computing environment to provide corresponding various services to customers. By way of illustration and not limitation, the software system 100 can include an operating system (OS) running on each node, and can also include various application programs, software or components for cluster management or communication (such as cluster management software, etc.), a distributed storage system, a monitoring and logging system, and the like. It should be understood that the cluster management software is software for managing the entire cluster, including resource scheduling, node monitoring, task management, and the like, for example, the cluster management software can be Kubernetes; the distributed storage system is a system for storing and managing data in the entire cluster, for example, the HDFS distributed storage system of Hadoop; the monitoring and logging system is a system for monitoring the running state of each node and application program in the cluster, and recording log information of the system. As a specific example, the software system 100 can be based on Hyperconverged Infrastructure (HCI), integrating all computing resources, storage resources, and network resources of the cluster 1, realizing modular seamless scale-out, and forming a unified resource pool.

[0064] Also, in the present embodiment, at least one node (also referred to herein as a "first node") of the computing device cluster 1 also deploys an upgrade module 110, which can be used by the first node runtime to provide upgrade services for the software system 100. By way of example, when the upgrade module 110 is running, it can perform upgrade processing on each subsystem 1, 2, … of the software system 100 in sequence according to a preset upgrade process, and in the case of repeated rollback and upgrade failure of any subsystem 1, 2, … (or a component or functional module, etc.) (i.e., upgrade exception, same below), determine the failure cause according to a preset repair strategy and output a processing suggestion for the upgrade exception cause, so that an operator can perform repair operations on the upgrade exception subsystem (or component) according to the upgrade exception cause and processing suggestion, and manually perform upgrade processing.

[0065] By way of example, the upgrade module 110 of the computing device cluster 1 can provide a corresponding interface (such as an interface UI or API), and a user (such as an operator) can access the interface through a terminal 2 to perform management operations on the computing device cluster 1, including but not limited to triggering the entire system of the software system 100 to perform an upgrade process, and performing corresponding repair operations when any subsystem 1, 2, … is upgraded abnormally.

[0066] In some possible embodiments, in the concurrent upgrade scenario of the software system 100, a complex pre-operation step is usually required, and then the upgrade module 110 upgrades each subsystem 1, 2, … in a predetermined order, and after each subsystem 1, 2, … is successfully upgraded, a corresponding post-operation step is performed. If a target object, i.e. a subsystem or a component of a subsystem, is abnormally upgraded during the upgrade process, the first node will interrupt the upgrade process of the entire software system 100, and at this time the target object may have high availability or resource sharing, etc. In order to not affect the normal service supply, the root cause of the abnormal upgrade must be solved in time, and the upgrade process is restarted to continue execution to avoid service unavailability caused by too many failed upgrades of the computing devices 10.

[0067] For example, if there are one hundred computing devices 10 in the current computing device cluster 1, at least ninety computing devices 10 are required to cooperate to run the software system 100, and the remaining ten computing devices 10 can be used as backup servers. The upgrade module 110 can first upgrade the ten backup servers. In this way, after the backup servers complete the system upgrade, a part (such as ten) of the ninety computing devices 10 executing the workloads can be selected and the services on these computing devices 10 can be migrated to the backup servers before upgrading these computing devices 10. In this way, the system upgrade of the entire cluster 1 can be completed one by one without interrupting the service. Then, if the upgrade process is interrupted due to the target object, the target object is in an abnormal upgrade state, and all computing devices 10 running the target object are also in an abnormal running state.

[0068] In order to avoid the adverse impact of too many computing devices 10 running abnormally in the cluster 1 on customer service, in this example, the first node detects the cause of the abnormal upgrade of the target object, and according to the predetermined repair strategy, a corresponding processing suggestion is matched to guide manual repair of the abnormality. After repair, the target object can be upgraded successfully by executing a script or the like, or it can be rolled back first and then upgraded successfully. However, after the abnormal upgrade of the target object is repaired, the upgrade process of the software system 100 (or the first node) cannot perceive that the current abnormality has been repaired, and the nodes running the target object or the nodes affected by the abnormal upgrade of the target object (referred to as “related nodes” herein) are still in an abnormal running state, so the entire upgrade process is still interrupted.

[0069] To this end, in the embodiment, as shown in FIG. 1, at least one node (also referred to as a "second node" herein) in the computing device cluster 1 is further deployed with an upgrade flow self-healing module 120. When the upgrade flow self-healing module 120 is run by the second node, in the case of interruption of the upgrade flow, the upgrade flow self-healing module 120 can process the abnormal record in the metadata of the related node of the target object caused by the target object to realize self-healing of the upgrade flow.

[0070] It should be noted that in the embodiment, the upgrade module 110 can have a certain upgrade flow fault tolerance capability, that is, the upgrade module 110 can pre-set a corresponding repair strategy for upgrade failure caused by some specified reasons (such as software package damage or configuration error, etc.), so that the upgrade module 110 can match the corresponding repair strategy for automatic repair after the subsystem or its component fails to upgrade, and then back off to try to upgrade again. If the upgrade is successful, the upgrade flow continues to execute downward. If repeated back-off upgrade is still unsuccessful, that is, it exceeds the upgrade flow fault tolerance capability of the upgrade module 110, the upgrade flow is interrupted. The upgrade self-healing module 120 can perform upgrade flow self-healing when an upgrade exception beyond the upgrade flow fault tolerance capability of the upgrade module 110 occurs, so as to reduce manual intervention and facilitate efficient and safe completion of upgrade flow self-healing.

[0071] For example, FIG. 2 shows an upgrade flow self-healing schematic diagram in an embodiment of the present application. As shown in FIG. 2, the first node executes step 1 through the upgrade module 110 to execute the upgrade flow of the entire software system 100, which can specifically start from the first subsystem (such as subsystem 1) for upgrade. When it is determined that the subsystem 1 is successfully upgraded (i.e., step 2), the next subsystem is executed for upgrade through step 3, until all the subsystems are successfully upgraded. If an upgrade exception occurs (i.e., step 4) when upgrading a certain subsystem (such as subsystem 2), the entire upgrade flow is interrupted. At this time, the upgrade module 110 can detect the cause of the upgrade exception of the subsystem 2 and give a processing suggestion to guide the operator to perform repair operation and manual upgrade. After the subsystem 2 is successfully upgraded after the exception repair, the second node can execute step 5 through the upgrade flow self-healing module 120 in an automatic or passive triggering manner to query the metadata of all related nodes of the subsystem 2 and correct the abnormal record in the metadata caused by the upgrade exception of the subsystem 2. In this way, the subsystem 2 is also successfully upgraded, and the related nodes are in a normal operation state after the abnormal record is corrected, that is, in a state capable of supporting the upgrade flow to continue to execute. Then, the upgrade flow self-healing module 120 forcibly pulls up the upgrade flow, so that the upgrade flow detects that the subsystem 2 and its related nodes are all normal, and then continues to execute downward from the current interrupted position, such as continuing to upgrade the subsystems 3, 4, …, until the upgrade of the entire software system 100 is completed.

[0072] Next, the principle of the upgrade flow self-recovery on the computing device cluster 1 is described in detail in combination with the drawings.

[0073] In the present embodiment, the target object belongs to different architectures, and the metadata processing strategy performed by the upgrade flow self-recovery module 120 in step 5 is also different. Therefore, the first node detects the architecture type of the target object in step 5 performed by the upgrade flow self-recovery module 120, and executes step 5 according to the architecture type. As an example but not limitation, the architecture type of the target object can be a cluster architecture, a high-availability architecture, or a tree-shaped upgrade sequence of a subsystem or each object inside the subsystem. Specifically:

[0074] In some possible examples, the architecture type of the target object is a cluster architecture, for example, as shown in FIG. 3A, the plurality of nodes (such as nodes 00, 01, and 02) of the computing device cluster 1 cooperatively run the target object (still taking the subsystem 2 as an example), and the nodes 00, 01, and 02 share storage resources, and the nodes 00, 01, and 02 affect each other during running. After the upgrade of the subsystem 2 is abnormally interrupted and the repair operation is performed, the upgrade flow self-recovery module 120 determines that the subsystem 2 is a cluster architecture through step A31, and then regards the nodes 00, 01, and 02 as the nodes related to the subsystem 2, and performs step A32 to uniformly modify the metadata of the nodes 00, 01, and 02, so as to switch the running abnormal state of the nodes 00, 01, and 02 to a normal running state (i.e., a state capable of supporting the continuous execution of the upgrade flow). Then, the upgrade flow self-recovery module 120 forcibly triggers the first node to start the upgrade flow, so that the first node performs detection on the subsystem 2 and the nodes 00, 01, and 02. Since the subsystem 2 has been repaired to the state of upgrade success, and the nodes 00, 01, and 02 are also in the normal running state, the first node determines that the current state of the subsystem 2 and the nodes 00, 01, and 02 can normally provide services, and can continue to perform the upgrade on the next subsystem, that is, the upgrade flow is resumed to continue from the current interrupted position, until all the software systems 100 are upgraded successfully.

[0075] In some possible examples, the target object is a component of a subsystem, and the architecture type of the target object is an architecture in which the target object and the remaining components of the subsystem have a tree-shaped upgrade sequence, for example, as shown in FIG. 3C. In the software system 100 deployed on the computing device cluster 1, the target object, for example, the functional module 23 of the subsystem 2, has a tree-shaped upgrade sequence relationship with the functional modules 20, 21, 22, and 24 of the subsystem 2. The upgrade abnormality of any functional module does not necessarily affect the upgrade success of other nodes running the module. After the functional module 23 is upgraded abnormally and repaired successfully, the upgrade process self-healing module 120 determines, through step A31, that the internal architecture of the subsystem 2 is a tree-shaped upgrade sequence, and then regards the functional module 23 as a leaf node. The metadata of the node 01 on which the functional module 23 is located is processed to a normal running state. Then, the upgrade process is started to execute the upgrade of the next subsystem from the leaf node to the parent node (for example, the functional module 21 and the functional module 20 as the parent node of the functional module 21), and the upgrade of all software systems 100 is successfully completed.

[0076] In some possible examples, the target object is a component of a subsystem, and the architecture type of the target object is an architecture in which the target object and the remaining components of the subsystem have a tree-shaped upgrade sequence, for example, as shown in FIG. 3C. In the software system 100 deployed on the computing device cluster 1, the target object, for example, the functional module 23 of the subsystem 2, has a tree-shaped upgrade sequence relationship with the functional modules 20, 21, 22, and 24 of the subsystem 2. The upgrade abnormality of any functional module does not necessarily affect the upgrade success of other nodes running the module. After the functional module 23 is upgraded abnormally and repaired successfully, the upgrade process self-healing module 120 determines, through step A31, that the internal architecture of the subsystem 2 is a tree-shaped upgrade sequence, and then regards the functional module 23 as a leaf node. The metadata of the node 01 on which the functional module 23 is located is processed to a normal running state. Then, the upgrade process is started to execute the upgrade of the next subsystem from the leaf node to the parent node (for example, the functional module 21 and the functional module 20 as the parent node of the functional module 21), and the upgrade of all software systems 100 is successfully completed.

[0077] Thus, in the present embodiment, according to the architecture type of different subsystems, corresponding node metadata processing strategies can be performed, thereby realizing self-healing of large-scale system upgrade process, reducing manual participation, and facilitating improvement of security and ease of use of upgrade service.

[0078] In some possible examples, if the target object belongs to the last upgraded subsystem in the upgrade process, that is, the position of the upgrade process interruption is close to the end point of the process, the upgrade process self-healing module 120 can trigger the execution of subsequent post-operation according to the position of the upgrade process interruption. In this way, even if the upgrade module 110 does not provide a corresponding entry for the user to manually start the post-operation of the system upgrade, the upgrade process self-healing module 120 can automatically determine whether to trigger the post-operation step according to the position of the upgrade process interruption, and it is not necessary to manually start the post-operation by means of table lookup, data modification, script or SQL execution, thereby avoiding manual omission or error processing, and especially avoiding manual data modification affecting the security of the entire cluster system.

[0079] It should be understood that the cluster system upgrade process self-healing principle of the general server similar to the above can also be used for the upgrade service in a more complex multi-layer virtualization implementation scenario of the overall system to facilitate enhancement of remote operation and maintenance capability, and will not be described in detail.

[0080] Next, based on the above description, a process self-healing method provided by the embodiments of the present application is introduced. It can be understood that the method is based on the above description, and part or all of the contents in the method can be referred to the description above.

[0081] Referring to FIG. 4, FIG. 4 is a flowchart of a process self-healing method provided by the embodiments of the present application. It can be understood that the method can be executed by any device, equipment, platform or equipment cluster having computing and processing capability, and the following is described by taking the execution on the second node of the computing equipment cluster 1 shown in FIG. 1 (which can be a computing equipment 10 or a virtual instance on the computing equipment 10) as an example. As shown in FIG. 4, the method can include the following steps.

[0082] In S410, in the case that a target object in a software system causes interruption of an upgrade process of the software system, a first instruction is acquired, the target object is an upgraded abnormal subsystem or a component of the subsystem, and the first instruction is used to instruct self-healing operation on the upgrade process.

[0083] In this embodiment, as shown in FIG. 1, at least one node of the computing device cluster 1 is deployed with the software system 100, which can include a plurality of subsystems 1, 2, …, and at least one node (e.g., the first node) of the computing device cluster 1 can perform an upgrade operation on the software system 100, and sequentially upgrade the subsystems 1, 2, … according to a preset upgrade procedure.

[0084] During the upgrade process, when a certain subsystem or component thereof, i.e., a target object, is upgraded abnormally (e.g., a state in which it cannot continue to upgrade or rollback) and causes the entire upgrade procedure of the software system 100 to be interrupted, the second node of the computing device cluster 1 can automatically generate an instruction (i.e., a first instruction) or passively receive a first instruction input by a human, which is used to instruct a self-healing operation on the upgrade procedure.

[0085] It should be understood that the second node and the first node can be the same node or different nodes.

[0086] S420, according to the first instruction, detecting whether the target object is upgraded successfully;

[0087] In this embodiment, the target object is manually repaired, and after the repair, the target object is manually upgraded, but the upgrade procedure itself cannot perceive whether the target object is repaired and whether the upgrade is successful. Therefore, the second node can monitor the upgrade state of the target object according to the first instruction to detect whether the target object is upgraded successfully. If it is determined that the target object is upgraded successfully, S430 can be executed, and if it is detected that the target object is still upgraded abnormally (or in a rollback state or an upgrading state), corresponding prompt information can be output.

[0088] S430, in the case where it is detected that the target object is upgraded successfully, determining the architecture type of the target object.

[0089] In this embodiment, the second node can determine the architecture type of the target object when it is detected that the target object is upgraded successfully. The architecture type is used to represent the relationship between related nodes, and the related nodes of the target object include nodes running the target object or nodes affected by the upgrade abnormality of the target object.

[0090] For example, the computing device cluster 1 can store configuration items of each subsystem 1, 2, …, which are used to describe the architecture type of the corresponding subsystem. It should be understood that the architecture type of any subsystem is also the architecture type of the components in the subsystem.

[0091] By way of example but not limitation, the architecture type of the target object can be a cluster architecture, a high-availability architecture, or an architecture in which components within a subsystem are arranged in a tree-shaped upgrade sequence. Thus, the second node can determine the architecture type of the target object by querying the configuration items of the target object.

[0092] When the target object is upgraded abnormally, the metadata of the computing device cluster 1 can change in various ways, depending on the current upgrade scope of the target object (e.g., an abnormal upgrade step or an abnormal upgrade of a component or module) and the architecture type of the target object.

[0093] If the target object is of a cluster architecture, all nodes running the target object affect each other, i.e., an abnormal upgrade of the target object (or a part of the target object) deployed on any of the nodes causes the system of other nodes on which the target object (or a part of the target object) is deployed to fail to upgrade, and all the nodes running the target object are abnormal.

[0094] If the target object is of a high-availability architecture, the master node and the standby node running the target object affect each other, i.e., an abnormal upgrade of the target object on the master node causes the target object on the standby node synchronized with the master node to fail to upgrade, and both the master node and the standby node are abnormal.

[0095] If the target object is of an architecture in which components within a subsystem are arranged in a tree-shaped upgrade sequence, an abnormal upgrade of the target object as a component or module causes all nodes running the target object to be abnormal.

[0096] Therefore, the second node can handle different target objects of different architecture types differently.

[0097] S440, according to the architecture type, adjusting the related nodes to a normal running state, the normal running state being a state that supports the upgrade process to continue.

[0098] In this embodiment, the second node can uniformly correct the metadata of all related nodes of the target object according to the architecture type of the target object, so that these nodes are in a normal running state. Since the entire target object is also manually processed to be in an upgrade success state at this time, the states of these related nodes and the target object both support the upgrade process of the entire software system 100 to continue. If the first node detects that the current target object is upgraded abnormally or that the related nodes are in an abnormal running state, the upgrade process will not continue.

[0099] S450, skipping the target object to continue the upgrade process of the software system.

[0100] In this embodiment, when the second node corrects the abnormal record of the metadata of the related node of the target object, the first node triggers to perform a detection operation in the upgrade process, which is used to detect the state of the current target object and its related nodes. If the first node determines that the current target object is upgraded successfully and its related nodes are in a normal running state, it can be determined that the upgrade of the target object is completed, so as to skip the upgrade of the target object and continue to upgrade the next object of the target object in the software system 100.

[0101] For example, if the target object is a component that is not the last one to be upgraded in a subsystem, the upgrade process continues to upgrade the next component of the subsystem; if the target object is a component that is the last one to be upgraded in a subsystem, the upgrade process continues to upgrade the next subsystem of the subsystem; if the target object is a subsystem, the upgrade process continues to upgrade the next subsystem of the subsystem, until the upgrade of the entire software system 100 is completed.

[0102] In this way, in this embodiment, if the upgrade process is interrupted due to the abnormal upgrade of a subsystem, the corresponding metadata processing can be performed according to the architecture type of the subsystem, so as to reduce manual intervention, avoid missing detection or errors caused by manual modification of data, improve the recovery efficiency of the upgrade process, and improve the security of the cluster system.

[0103] The flow self-recovery method of one specific embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0104] For example, FIG. 5 shows a flowchart of the flow self-recovery method provided by one specific embodiment of the present application.

[0105] As shown in FIG. 5, the method can specifically include:

[0106] S510, performing a pre-upgrade operation on the software system.

[0107] In this embodiment, before the first node of the computing device cluster 1 performs the upgrade process on the software system 100, a corresponding pre-upgrade operation can be performed on the software system 100 and the nodes running the software system 100. The pre-upgrade operation can include n (n>0) steps, for example, the pre-upgrade operation can include backing up data to avoid data loss due to unexpected situations during the upgrade process; it can also include a business migration step, which migrates the business on some nodes in the cluster to other nodes before upgrading these nodes, and the like, so as to ensure that the services provided by the software system 100 are not interrupted during the upgrade process.

[0108] S520, performing an upgrade on each subsystem of the software system in sequence according to the upgrade process.

[0109] In this embodiment, the first node of the computing device cluster 1 can perform an upgrade procedure for the software system 100. When the upgrade procedure is performed, each subsystem 1, 2, … in the software system 100 can be upgraded in a preset order, and when the first node upgrades each subsystem 1, 2, …, a plurality of upgrade steps can be performed on each subsystem 1, 2, … to complete the upgrade of all components (including all components or functional modules, etc.) in each subsystem 1, 2, ….

[0110] S530, in the case where the target object causes the upgrade procedure of the software system to be interrupted, obtaining a first instruction.

[0111] In this embodiment, if an upgrade step of any subsystem or a component of the subsystem has an upgrade abnormality during the execution of the upgrade procedure, the entire upgrade procedure is interrupted, and the upgrade state abnormality of the subsystem or the component (also referred to as a “target object” herein) is recorded in the log of the computing device cluster 1. The metadata of the related node of the target object also changes accordingly, thereby describing the running state abnormality of the related node caused by the upgrade abnormality of the target object.

[0112] For example, the metadata of the node in the computing device cluster 1 can include but is not limited to the following information:

[0113] Node information: used to describe the basic attributes and characteristics of the node belonging to the cluster, which can specifically include the name, IP address, hardware configuration, operating system version, etc. of the node; configuration information: used to describe the configuration and parameter settings of the subsystem on the node, which can specifically include configuration files, environment variables, etc.; service information: used to describe the basic attributes and state of each service on the node, which can specifically include the name, version, state, etc. of the service; log information: used to record the running situation and events of the subsystem on the node, which can specifically include subsystem logs, application program logs, etc.

[0114] In this example, the upgrade abnormality of the target object can cause the change of the configuration information to fail to be successfully applied or fail to be correctly rolled back to the previous state; can also cause the software components on the related node of the target object to be in different versions or states; and can also cause the event log record and monitoring information on the related node to be inaccurate (for example, the record of the running state of the node is inaccurate, or the record of the subsystem upgrade and rollback is inaccurate), etc.

[0115] For example, if the software system 100 has a target object during the upgrade, this step can specifically include:

[0116] S531, in the case that the target object causes the upgrade process of the software system to be interrupted, determining an upgrade abnormality cause of the target object, and matching a corresponding processing suggestion according to a preset repair strategy.

[0117] In the present example, in the case that the first node determines that the target object has an upgrade abnormality, the cause of the upgrade abnormality of the target object can be located, and a corresponding processing suggestion can be matched according to a preset repair strategy to guide an operator to perform a corresponding repair operation on the target object. In this way, the target object upgrade abnormality caused by system environment abnormality, network abnormality, and the like can be repaired by a human being, and after the repair, the target object can be manually upgraded by a human being running a script.

[0118] S532, providing an access interface for a user;

[0119] S533, obtaining, from the access interface, the first instruction initiated by the user for the target object.

[0120] In the present example, after the operator repairs the target object, the operator can send a first instruction to the second node to instruct the second node to perform a self-healing operation on the interrupted upgrade process.

[0121] In some possible implementation manners, the second node and / or the first node of the computing device cluster 1 can provide an interface for a user to access to perform an upgrade operation or a self-healing operation.

[0122] For example, referring to FIG. 6A, when the first node is upgraded to part 2 of the subsystem 2 through steps 1 to 3, an upgrade abnormality occurs, the part 2 repeatedly rolls back but still fails to be upgraded, and the part 2 is in a state of being unable to be successfully upgraded or rolled back, a processing suggestion is given to guide an operator to perform step 4 to repair the upgrade abnormality of the part 2 and perform a manual upgrade. Then, the operator can trigger a manual processing abnormality through step 5, that is, send a first instruction to the second node to trigger the execution of a self-healing operation.

[0123] In this example, in combination with FIG. 6B, the subsystem 2 can be a subsystem deployed on the nodes 02, 03 in the computing device cluster 1, and the upgrade progress of the subsystem 2 can be displayed to the user through the interface 60 as shown in FIG. 6B. In this example, the first node sequentially performs the upgrade steps of checking the upgrade package configuration, checking the upgrade package version, and checking the service state on the part 2 of the subsystem 2 in the node 02 according to the upgrade flow, and an upgrade exception occurs after the step of checking the service state. At this time, the checking system that triggers the execution of the upgrade flow alarms, and determines the cause of the upgrade exception of the subsystem 2. In this example, the possible causes are that the current node is not powered on or that there is a hard disk exception, and corresponding processing suggestions are given to guide the operator to solve the problems of node not powered on and hard disk exception, so as to repair the upgrade exception of the subsystem 2. In addition, the control 610 can be provided in the subsystem 2 upgrade display area of the interface 60, and the operator can initiate the first instruction for the subsystem 2 through the control 610 to instruct the second node to perform the self-healing operation of the upgrade flow.

[0124] Similarly, with continued reference to FIG. 6B, if an upgrade exception occurs when upgrading the subsystem 3, and the upgrade exception occurs when upgrading the second part of the subsystem 3 (the subsystem 3 also includes a first part, etc.), in addition to displaying the cause of the upgrade exception and the processing suggestion, etc., a corresponding control 620 can also be provided in the second part upgrade display area on the interface 60, so that the operator can initiate the first instruction for the second part (i.e., the target object) through the control 620 to instruct the second node to perform the self-healing operation of the upgrade flow, i.e., to perform the steps of S540 to S570, etc.

[0125] Optionally, in this example, after obtaining the first instruction through S530, the following steps can also be performed:

[0126] S534, outputting a risk prompt of the flow self-healing;

[0127] S535, obtaining the confirmation information input by the user for the risk prompt.

[0128] In this step, since the upgrade flow self-healing operation involves high-risk operations such as modifying node metadata, in order to ensure the security of the cluster system, after the second node obtains the first instruction, the operator can also be provided with a confirmation interface to display the risk prompt of the flow self-healing and to submit the confirmation information to perform a second confirmation on the self-healing operation initiated by the operator.

[0129] As a specific example, the confirmation interface can be a confirmation pop-up window, which can be used to prompt the operator of the risk of self-healing operation and to confirm the operator's identity. For example, referring to FIG. 7A, after the operator triggers the control 610 shown in FIG. 6B, the operator can receive a pop-up window 70, which displays a risk prompt describing that the current self-healing operation will skip the upgrade of all nodes under the subsystem, and the operator needs to confirm (which can be done by checking the checkbox) that the operation is performed after the current upgrade is rolled back and re-upgraded but still fails, and the pop-up window 70 also provides the operator to input a username and a password to confirm that the operator has the authority to initiate this high-risk operation.

[0130] Similarly, referring to FIG. 7B, after the operator triggers the control 620 shown in FIG. 6B, the operator can receive a pop-up window 71, which displays a risk prompt describing that the current self-healing operation will skip the upgrade of the object, and the operator needs to confirm that the operation is performed after the current upgrade is rolled back and re-upgraded but still fails, and the pop-up window 71 also provides the operator to input a username and a password to confirm that the operator has the authority to initiate this high-risk operation.

[0131] In this way, the operator submits corresponding confirmation information in the confirmation pop-up window 70 or 71, which can include a username, a password, and a checked checkbox (also referred to herein as a "risk prompt read identifier") to the second node.

[0132] S536, in the case where the confirmation information includes a risk prompt read identifier and user identity information, verifying whether the user identity information matches preset identity information;

[0133] S537, if the user identity information matches the preset identity information, performing the following step S540.

[0134] In the present example, after the second node verifies that the username and the password match the pre-stored user identity information, the self-healing operation on the upgrade process can be performed, i.e., the following steps S540 to S570 are performed.

[0135] Therefore, in the present embodiment, after the first instruction is obtained, or after the first instruction is obtained and it is confirmed that the user has read the related risk prompt and the user identity information matches the preset identity information, the following S540 can be performed:

[0136] S540, according to the first instruction, detecting whether the target object is upgraded successfully.

[0137] In the present embodiment, after the second node starts the self-healing operation on the upgrade process, the step 6 shown in FIG. 6A can be performed first, i.e., verifying the upgrade status of the target object.

[0138] S550, in the case of detecting that the target object is upgraded successfully, determining the architecture type of the target object.

[0139] Specifically, the step S550 can include:

[0140] S551, in the case of detecting that the target object is upgraded successfully, obtaining the configuration item of the target object.

[0141] S552, according to the configuration item, determining the architecture type of the target object.

[0142] For example, the computing device cluster 1 can set the corresponding configuration item for each subsystem 1, 2, …, and the configuration item of any subsystem 1, 2, … is used to describe the architecture type of the subsystem (also the architecture type of the components in the subsystem). Therefore, the second node can obtain the configuration item of the target object to determine the architecture type of the target object, wherein the architecture type can include cluster architecture, high-availability architecture, and architecture of tree-shaped upgrade order between internal components of the subsystem, but is not limited thereto.

[0143] S560, according to the architecture type, adjusting the related nodes to the normal running state.

[0144] In this step, the second node can uniformly process the metadata of the related nodes according to the architecture type of the target object, for example, modify the service information of the related nodes so that they are all in the normal running state, clear the records about the upgrade exception of the target object in the log information of the related nodes, etc., but is not limited thereto. In this way, after correcting the metadata, the related nodes are all in the normal running state, and since the target object is also manually processed to the upgraded successfully state, the upgrade process can continue to be executed.

[0145] In some possible examples, if the target object is similar to the cluster architecture shown in FIG. 3A or similar to the high-availability architecture shown in FIG. 3B, the related nodes of the target object will affect each other, and therefore, when performing S560, it can specifically include:

[0146] S561, according to the architecture type, correcting the abnormal records in the metadata of the related nodes due to the target object, so that the related nodes are all in the normal running state.

[0147] In the present example, the second node can uniformly process the metadata of all related nodes of the target object, so that these related nodes are in the same state, including but not limited to: uniformly modifying the service information of each related node, so that these related nodes are in a normal running state, and uniformly clearing the records about the upgrade exception of the target object on each related node. Since the target object deployed on the current related nodes (only a part of the target object may be deployed on each node) is manually processed to be in an upgrade success state, and the metadata of the related nodes has no abnormal records about this, the upgrade process can be supported to resume execution.

[0148] In some possible examples, if the target object is a component of a subsystem and is in a tree structure upgrade order architecture of the subsystem as shown in FIG. 3C, the upgrade success or failure of the target object has no necessary relationship with other components of the subsystem, and therefore, when S560 is executed, it can specifically include:

[0149] S562, according to the architecture type, taking the target object as a leaf node, and correcting the abnormal records in the metadata of the node where the target object is located due to the target object, so that the node is in a state supporting the continuous execution of the upgrade process.

[0150] In the present example, the second node can first take the target object as a leaf node in the tree structure upgrade order of the subsystem, and first modify and clear the abnormal records in the metadata of the node where the target object is located due to the target object, so that the node is changed to a normal running state to support the continuous execution of the upgrade process.

[0151] S563, according to the tree structure upgrade order of the subsystem to which the target object belongs, recursively traversing all components as parent nodes from the leaf node;

[0152] S564, determining that these components as parent nodes and the nodes where they are located are in a state supporting the continuous execution of the upgrade process.

[0153] In the present example, after processing the metadata of the node where the target object is located, according to the tree structure upgrade order of the subsystem to which the target object belongs, recursively traversing upwards to find all components (components or modules) as parent nodes, and according to the metadata of the nodes where these components are located, confirming that these components are all upgraded successfully and the nodes where they are located are in a normal running state, so as to ensure that the entire subsystem of the target object is in a state without upgrade exception, to support the continuous execution of the upgrade process.

[0154] Still taking the subsystem 3 shown in FIG. 6B as an example, the upgrade exception of the second part does not affect the success of the upgrade of the first part, but the success of the upgrade of the entire subsystem 3 is determined by the upgrade states of all parts thereof. If an upgrade step of the second part is abnormal, the abnormality of the upgrade step needs to be manually repaired, the second part is manually upgraded successfully, the metadata of the node where the second part is located is processed, the abnormal record of the second part and the node where the second part is located is eliminated (i.e., step 7 in FIG. 6A), and the metadata of the nodes where the other parts of the entire subsystem 3 are located is processed, so that the entire subsystem 3 is in a normal upgrade state (i.e., step 8 in FIG. 6A). In this way, the second node automatically arranges the rollback order and rollback steps of the target object and the related nodes of the subsystem, that is, the abnormal upgrade steps, objects, and subsystems are processed according to the operation order arranged by the second node, so that the subsequent upgrade process can be smoothly pulled up and executed (i.e., steps 8 to 11 in FIG. 6A).

[0155] In the embodiment, after the metadata of all the nodes related to the target object is processed through the above-mentioned S560 step, the second node further executes:

[0156] S570, skipping the target object to continue executing the upgrade process of the software system.

[0157] In the embodiment, the second node can trigger the first node to start the upgrade process to continue execution from the current interrupted position. Specifically, the second node can make the first node forcibly start the detection step (such as the system alarm detection step shown in FIG. 6A) of the upgrade process to re-detect the state of the target object and the nodes related thereto. Since the target object has been manually repaired and upgraded successfully, and there is no upgrade abnormality information in the metadata of the related nodes, the upgrade process can continue to execute downward based on the fault tolerance capability thereof, skip the target object to continue upgrading the software system, and thus complete the upgrade of the entire software system 100.

[0158] In some possible examples, during the execution of S570, if the target object is the last subsystem to be upgraded in the software system 100, that is, there is no next subsystem to be upgraded after the upgrade process is resumed, the second node can execute:

[0159] If the target object is the last subsystem to be upgraded in the upgrade process, a preset script is executed to start the post-operation of the software system.

[0160] In this step, the first node is awakened to continue performing the post-operation of the software system 100, such as system testing and verification, business migration, etc., but is not limited thereto, by executing the preset script and the like.

[0161] In this way, by identifying the architecture types of different subsystems in a large-scale cluster, when an exception occurs in the entire system that upgrades the business fault tolerance capability, after repairing the problem of the upgraded business itself, the upgrade process self-recovery is triggered to complete the recovery of the upgrade workflow. Moreover, even if the causes of the problems in the upgrade process are different, the normalized self-recovery operation can be realized by the method of the embodiment of the application, the manual participation is reduced, the usability is high, and the security of the cluster system is ensured.

[0162] In addition, in the embodiment, the processing order of the target object in the entire upgrade process can be identified to automatically determine whether to trigger the post-operation, improve the automation degree of the self-recovery operation, and ensure the complete implementation of the entire upgrade business.

[0163] In a possible implementation, the self-recovery operation of the upgrade process of the second node can also be automatically triggered. Specifically, referring to FIG. 8, the difference between the implementation and the embodiment shown in FIG. 5 is that the polling interval of the upgrade process self-recovery operation can be set in advance. After the upgrade process is interrupted and the processing suggestion is given, the second node can perform the following steps:

[0164] S810 starts a timing polling task.

[0165] S820, according to the timing polling task, generates a first instruction when each preset polling interval is reached.

[0166] In the implementation, after the upgrade process is interrupted, the second node automatically starts the timing polling task to automatically generate the first instruction at each polling interval, triggers the second node to perform the upgrade process self-recovery operation, and listens to the upgrade state of the target object to determine whether to process the related node metadata according to the architecture type of the target object according to whether the target object is upgraded successfully. In this way, by waiting for the timing task to complete the upgrade process self-recovery, manual repair operations can be reduced, and the accuracy of the process self-recovery operation can be improved.

[0167] Based on the method in the above embodiment, the embodiment of the application provides a process self-recovery device. Please refer to FIG. 9, which is a structural schematic diagram of a process self-recovery device provided by the embodiment of the application.

[0168] As shown in FIG. 9, the apparatus 900 can include an obtaining module 901 and a processing module 902. The obtaining module 901 can be configured to obtain a first instruction in a case where a target object in a software system causes an upgrade process of the software system to be interrupted, the target object being an abnormal-upgrade subsystem or a component of the abnormal-upgrade subsystem. The processing module 902 can be configured to detect whether the target object is successfully upgraded according to the first instruction, and determine an architecture type of the target object in a case where it is detected that the target object is successfully upgraded, the architecture type being used to represent relationships between related nodes, the related nodes including a node running the target object or a node affected by the abnormal upgrade of the target object. The processing module 902 can also be configured to adjust the related nodes to a normal running state according to the architecture type, the normal running state being a state that supports the upgrade process to continue, and continue the upgrade process of the software system by skipping the target object.

[0169] It should be understood that the apparatus 900 is configured to execute the self-healing method of the flow in the above-described embodiments, and the implementation principle and technical effects of the corresponding program modules in the apparatus 900 are similar to those described in the above-described method. The working process of the apparatus 900 can refer to the corresponding process in the above-described method, and details are not described herein.

[0170] Based on the method in the above-described embodiments, an embodiment of the present application provides a computing device. The computing device can include at least one memory configured to store a program, and at least one processor configured to execute the program stored in the memory. When the program stored in the memory is executed, the processor is configured to execute the method in the above-described embodiments. As a specific example, the computing device can be the computing device 10 shown in FIG. 1, which is capable of running the upgrade process self-healing module 120.

[0171] Based on the method in the above-described embodiments, an embodiment of the present application provides a computing device cluster. The computing device cluster can include at least one computing device, and the computing device can include at least one memory configured to store a program, and at least one processor configured to execute the program stored in the memory. When the program stored in the memory is executed, the processor is configured to execute the method in the above-described embodiments. As a specific example, the computing device cluster can be the computing device cluster 1 shown in FIG. 1, but is not limited thereto.

[0172] Based on the method in the above-described embodiments, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is run on a processor, the processor executes the method in the above-described embodiments.

[0173] Based on the method in the above embodiment, the embodiment of the application provides a computer program product, characterized in that when the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0174] Based on the method in the above embodiment, the embodiment of the application further provides a chip. Please refer to FIG. 10, which is a structural schematic diagram of a chip provided by the embodiment of the application. As shown in FIG. 10, the chip 1000 comprises one or more processors 1001 and interface circuits 1002. Optionally, the chip 1000 can further comprise a bus 1003. Wherein:

[0175] The processor 1001 can be an integrated circuit chip with processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 1001. The processor 1001 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step disclosed in the embodiment of the application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor.

[0176] The interface circuit 1002 can be used for sending or receiving data, instructions or information. The processor 1001 can process the data, instructions or other information received by the interface circuit 1002, and can send the processed information out through the interface circuit 1002.

[0177] Optionally, the chip 1000 further comprises a memory, which can include a read-only memory and a random access memory, and provides operation instructions and data for the processor. Part of the memory can also include a non-volatile random access memory (NVRAM).

[0178] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling operation instructions stored in the memory (the operation instructions can be stored in an operating system).

[0179] Optionally, the interface circuit 1002 can be used for outputting the execution result of the processor 1001.

[0180] It should be noted that the functions of the processor 1001 and the interface circuit 1002 respectively can be realized by hardware design, software design or combination of hardware and software, which is not limited here.

[0181] It should be understood that each step of the above method embodiments can be completed by a logic circuit in the form of hardware in the processor or instructions in the form of software.

[0182] It can be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in some possible implementations, each step in the above embodiments can be selectively executed, partially executed or fully executed according to actual conditions, which is not limited here.

[0183] It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0184] The method steps in the embodiments of the present application can be implemented in the form of hardware or by the processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

[0185] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0186] It can be understood that various numerical numbers involved in the embodiments of the present application are only distinguished for convenience of description, and are not used to limit the scope of the embodiments of the present application.

Claims

1. A method for process self-healing, the method comprising: The method comprises: In the case that a target object in a software system causes interruption of an upgrade process of the software system, obtaining a first instruction, the target object being a subsystem or a component of a subsystem that is upgrade-exceptional; According to the first instruction, detecting whether the target object is upgraded successfully; In the case that the target object is detected to be upgraded successfully, determining an architecture type of the target object, the architecture type being used to represent a relationship between related nodes, the related nodes including a node running the target object or a node affected by the upgrade exception of the target object; According to the architecture type, adjusting the related nodes to a normal operation state, the normal operation state being a state supporting continuous execution of the upgrade process; Continuing execution of the upgrade process of the software system by skipping the target object.

2. The method of claim 1, wherein, The case that, in the case that the target object is detected to be upgraded successfully, the architecture type of the target object is determined, comprises: In the case that the target object is detected to be upgraded successfully, obtaining a configuration item of the target object, the configuration item being used to describe the architecture type to which the target object belongs; According to the configuration item, determining the architecture type of the target object.

3. The method according to claim 1 or 2, characterized in that, The architecture type of the target object is a cluster architecture or a high-availability architecture, The case that, according to the architecture type, the related nodes are adjusted to the normal operation state, comprises: According to the architecture type, correcting abnormal records in metadata of all the related nodes caused by the target object, so that the related nodes are all in the normal operation state and target objects deployed on the related nodes are all in the upgraded-successful state, to support continuous execution of the upgrade process.

4. The method according to claim 1 or 2, characterized in that, The target object is a component of a subsystem, and the architecture type of the target object is a tree-structure upgrade sequence architecture; The case that, according to the architecture type, the related nodes are adjusted to the normal operation state, comprises: According to the architecture type, taking the target object as a leaf node, and correcting abnormal records in metadata of a node where the target object is located caused by the target object, so that the node is in a state supporting continuous execution of the upgrade process; According to a tree-structure upgrade sequence of a subsystem to which the target object belongs, recursively traversing all components that are parent nodes in the subsystem from the leaf node; Determining that the components that are the parent nodes and nodes where the components are located are in the state supporting continuous execution of the upgrade process.

5. The method according to any one of claims 1 to 4, characterized in that, The case that, in the case that a target object in a software system causes interruption of an upgrade process of the software system, obtaining a first instruction, comprises: In the case that a target object in a software system causes interruption of an upgrade process of the software system, providing an access interface to a user; Obtaining, from the access interface, the first instruction initiated by the user for the target object.

6. The method of claim 5, wherein, After the first instruction is obtained, the method comprises: Outputting a risk prompt of process self-recovery; Obtaining confirmation information input by the user for the risk prompt; In the case that the confirmation information includes a risk prompt read identifier and identity information of the user, verifying whether the identity information of the user matches preset identity information; If the identity information of the user matches preset identity information, the first instruction is executed to detect whether the target object is successfully upgraded.

7. The method according to any one of claims 1 to 4, characterized in that, In the case that the target object in the software system causes the upgrade process of the software system to be interrupted, a first instruction is acquired, including: In the case that the target object in the software system causes the upgrade process of the software system to be interrupted, a timing polling task is started; According to the timing polling task, the first instruction is generated when each preset polling interval is reached.

8. The method according to any one of claims 1 to 7, characterized in that, After the target object is skipped to continue the upgrade process of the software system, the method further includes: If the target object is the last upgraded subsystem in the upgrade process, a preset script is executed to start an upgrade post-operation of the software system.

9. The method according to any one of claims 1 to 8, characterized in that, Before the first instruction is acquired, the method includes: In the case that the target object causes the upgrade process of the software system to be interrupted, an upgrade abnormal reason of the target object is determined; According to a preset repair strategy, corresponding processing suggestion information is output for the upgrade abnormal reason.

10. A computing device, comprising: Comprise: At least one memory for storing a program; At least one processor for executing the program stored in the memory; When the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1-9.

Images