Application process recovery method and device

Abstract

The present invention discloses an application process recovery method and device, wherein the method is as follows: if the communication process of an application node receives a preset event of the application process of the application node, the caller of the application process is notified that the application process has been abnormally closed, so that the caller switches traffic to a node outside the application node; the preset event is an isolation event, a recovery event, or a failure event; the daemon process of the application node monitors the survival status of the communication process and the application process according to a first preset period; if the daemon process determines that the communication process and / or the application process are not alive, the non-surviving process in the communication process and / or the application process is recovered; the communication process monitors the survival status of the daemon process according to a second preset period; if the communication process determines that the daemon process is not alive, the daemon process is recovered.

Description

Technical Field

[0001] The present invention relates to the technical field of process management, and in particular to an application process recovery method and device. Background Art

[0002] The application process of a financial institution needs to process a large number of transactions every day. Financial institutions have a low tolerance for transaction errors and need to control the error rate of transactions at a very low level. Therefore, for financial institutions, the normal operation of the application process is an important guarantee for the transaction processing quality of financial institutions. However, the application process may be abnormally closed for some reason. If transactions are still sent to the application process after the abnormal closure, it will cause transaction losses, so the application process needs to be restored in time.

[0003] At present, in order to restore the application process in time, a daemon process is often configured for the application process. The specific recovery method is that the caller of the application process in application node one performs a heartbeat test on the application process through a heartbeat test. After the heartbeat test times out, it is determined that the application process is abnormally closed, and the transaction is sent to the application process of application node two outside application node one, and the daemon process is notified to restore the application process. However, the heartbeat test cannot be set to a shorter period, otherwise the heartbeat test alone will occupy a large amount of traffic, and the meaning of the heartbeat test will be lost. The cycle of the heartbeat test is a long time for transactions. If the application process is discovered by relying on the heartbeat test, then the transactions during the heartbeat test timeout period will be lost. Moreover, the daemon process may also be abnormally closed, resulting in the inability to restore the application process in time. Therefore, the current method of restoring the application process will cause more transaction losses, which is a problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides an application process recovery method and device, which solves the problem that the application process recovery method in the prior art may cause more transaction losses.

[0005] In a first aspect, the present invention provides an application process recovery method, comprising: if a communication process of an application node receives a preset event of an application process of the application node, notifying a caller of the application process that the application process has been abnormally closed, so that the caller switches traffic to a node outside the application node; the preset event is an isolation event, a recovery event, or a failure event;

[0006] The daemon process of the application node monitors the survival status of the communication process and the application process according to a first preset period; if the daemon process determines that the communication process and / or the application process are not alive, the daemon process recovers the non-surviving process in the communication process and / or the application process;

[0007] The communication process monitors the survival status of the daemon process according to a second preset period; if the communication process determines that the daemon process is not alive, the daemon process is restored.

[0008] In the above manner, a socket connection is established between the communication process and the application process. Since the communication process and the application process are both processes of the application node, under the socket mechanism, the communication between processes is faster. The communication process can quickly receive a preset event after the application process is abnormally closed, and then notify the caller of the application process that the application process is abnormally closed, so that the caller can switch the traffic in time. Moreover, the daemon process of the application node monitors the survival status of the communication process and the application process according to the first preset period, and promptly restores the communication process and\or the non-surviving process in the application process. Moreover, the communication process monitors the survival status of the daemon process according to the second preset period, and promptly restores the daemon process, so that the communication process and the daemon process keep each other alive, further ensuring the survival of the application process. Since process recovery and sending of disconnection notification messages are executed asynchronously, the traffic of the caller will not be affected by process recovery.

[0009] Optionally, if the daemon process obtains a management instruction, after executing the management instruction, a first storage file of the state information is generated according to the management instruction; the management instruction is an instruction to perform a preset operation on the application process and / or the communication process;

[0010] The first storage file of the state information is stored in a persistent storage space.

[0011] In the above method, the first storage file of the state information is stored in the persistent storage space, so as to back up the state information and record the preset operations performed on the application process and / or the communication process.

[0012] Optionally, the recovering the non-surviving process in the communication process and / or the application process includes:

[0013] Acquire a second storage file in the persistent storage space; the state information stored in the second storage file is the latest state information of the non-surviving process stored in the persistent storage space;

[0014] The non-surviving process is restored according to the status information of the second storage file.

[0015] In the above method, the status information stored in the second storage file is the latest status information of the non-surviving process stored in the persistent storage space. By restoring the non-surviving process according to the status information of the second storage file, the non-surviving process can be restored to the latest status.

[0016] Optionally, if the communication process receives a state change event and the communication process determines that the application process has not stopped, the caller is notified to isolate or restore the application node and / or the application process; the state change event instructs the application process to stop processing the caller's transaction, or instructs the application process to resume processing the caller's transaction.

[0017] In the above manner, the state change event is used to promptly stop processing the caller's transaction, thereby preventing further transaction losses.

[0018] Optionally, if the daemon process receives a manual process stop instruction, the process indicated by the manual process stop instruction is stopped; the process indicated by the manual process stop instruction includes at least the application process.

[0019] In the above manner, a method for manually stopping the application process is provided.

[0020] Optionally, the manual process stop instruction further indicates the following information: stop the daemon process and the communication process, or stop the communication process.

[0021] Optionally, the daemon process of the application node monitors the survival status of the communication process and the application process according to a first preset period, including:

[0022] The daemon process monitors the survival status of the communication process and the application process by executing a preset status query command line according to the first preset period;

[0023] The communication process monitors the survival status of the daemon process according to a second preset period, including:

[0024] The communication process monitors the survival status of the daemon process by executing the preset status query command line according to the second preset period.

[0025] In the above manner, the command line can be directly executed through the operating system kernel, and unified monitoring of the process can be achieved through a unified command line.

[0026] In a second aspect, the present invention provides an application process recovery device, comprising: a notification module, which is used for notifying a caller of the application process that the application process has been abnormally closed if a communication process receives a preset event of an application process of an application node, so that the caller switches traffic to a node outside the application node; the preset event is an isolation event, a recovery event, or a failure event;

[0027] A processing module, used for the daemon process to monitor the survival status of the communication process and the application process according to a first preset period; if the daemon process determines that the communication process and / or the application process are not alive, then restore the non-surviving process in the communication process and / or the application process; and used for the communication process to monitor the survival status of the daemon process according to a second preset period; if the communication process determines that the daemon process is not alive, then restore the daemon process.

[0028] Optionally, the processing module is further used for:

[0029] If the daemon process obtains a management instruction, then after executing the management instruction, a first storage file of status information is generated according to the management instruction; the management instruction is an instruction to perform a preset operation on the application process and / or the communication process; the first storage file of the status information is stored in a persistent storage space.

[0030] Optionally, the processing module is specifically used for:

[0031] Acquire a second storage file in the persistent storage space; the state information stored in the second storage file is the latest state information of the non-surviving process stored in the persistent storage space;

[0032] The non-surviving process is restored according to the status information of the second storage file.

[0033] Optionally, the processing module is further used for:

[0034] If the communication process receives a state change event and determines that the application process has not stopped, the communication process notifies the caller to isolate or restore the application node and / or the application process; the state change event instructs the application process to stop processing the caller's transaction, or instructs the application process to resume processing the caller's transaction.

[0035] Optionally, the processing module is further used for:

[0036] If the daemon process receives a manual process stop instruction, the daemon state of the process indicated by the manual process stop instruction in the daemon process is stopped; the process indicated by the manual process stop instruction at least includes the application process.

[0037] Optionally, the processing module is further used for: the manual process stop instruction also indicates the following information: stop the daemon process and the communication process, or stop the communication process.

[0038] Optionally, the processing module is specifically used for:

[0039] The daemon process monitors the survival status of the communication process and the application process by executing a preset status query command line according to the first preset period;

[0040] The communication process monitors the survival status of the daemon process by executing the preset status query command line according to the second preset period.

[0041] The beneficial effects of the above-mentioned second aspect and each optional device of the second aspect can refer to the beneficial effects of the above-mentioned first aspect and each optional method of the first aspect, and will not be repeated here.

[0042] In a third aspect, the present invention provides a computer device, including a program or an instruction, which, when executed, is used to execute the above-mentioned first aspect and each optional method of the first aspect.

[0043] In a fourth aspect, the present invention provides a storage medium, comprising a program or an instruction, which, when executed, is used to execute the above-mentioned first aspect and each optional method of the first aspect.

[0044] These and other aspects of the present invention will become more apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 A flowchart corresponding to an application process recovery method provided by an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the system architecture of an application process recovery system provided by an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of the structure of an application process recovery device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] like Figure 1 As shown, the present application provides an application process recovery method.

[0051] Step 101: If the communication process of the application node receives a preset event of the application process of the application node, it notifies the caller of the application process that the application process has been abnormally closed, so that the caller switches traffic to a node outside the application node.

[0052] The preset event is an isolation event, a recovery event, or a failure event;.

[0053] Step 102: The daemon process of the application node monitors the survival status of the communication process and the application process according to a first preset period; if the daemon process determines that the communication process and / or the application process is not alive, the non-surviving process in the communication process and / or the application process is restored.

[0054] Step 103: the communication process monitors the survival status of the daemon process according to a second preset period; if the communication process determines that the daemon process is not alive, the daemon process is restored.

[0055] In step 101, since the communication process and the application process establish a socket connection, when the application process runs normally, the socket connection will always exist. If the application process is abnormally closed, the socket connection will be disconnected, which will trigger a preset event.

[0056] It should be noted that the method of notifying the caller of the application process that the application process has been abnormally closed can be that the communication process directly notifies the caller of the application process, or that the caller is notified through an intermediate infrastructure such as a console, etcd, etc. that can notify all callers.

[0057] In an optional implementation manner, in step 102, the process in which the daemon process of the application node monitors the survival status of the communication process and the application process according to the first preset period may be as follows:

[0058] The daemon process monitors the survival status of the communication process and the application process by executing a preset status query command line according to the first preset period.

[0059] In step 103, the communication process monitors the survival status of the daemon process according to a second preset period, including:

[0060] The communication process monitors the survival status of the daemon process by executing the preset status query command line according to the second preset period.

[0061] It should be noted that the first preset period and the second preset period can be set to the same duration and different starting times, which can reduce the abnormal closure of the daemon process and the communication process at the same time, but cannot be automatically restored.

[0062] In an optional implementation, if the daemon process obtains a management instruction, after executing the management instruction, a first storage file of the state information is generated according to the management instruction; and the first storage file of the state information is stored in a persistent storage space.

[0063] The management instruction is an instruction for executing a preset operation on the application process and / or the communication process.

[0064] Among them, the persistent storage space includes local disk, remote disk, redis database and other storage spaces.

[0065] For example, the status information is one of start, stop, isolate, and release.

[0066] Based on the above implementation, the execution process of step 102 is as follows:

[0067] Acquire a second storage file in the persistent storage space; and restore the non-surviving process according to the status information of the second storage file.

[0068] The state information stored in the second storage file is the latest state information of the non-surviving process stored in the persistent storage space.

[0069] That is to say, through the above method, the non-surviving process can be restored to the latest state information stored in the persistent storage space.

[0070] It should be noted that the non-surviving process may also be restored to a preset state, such as an isolation state.

[0071] In an optional implementation, if the communication process receives a state change event and the communication process determines that the application process has not stopped, the caller is notified to isolate or restore the application node and / or the application process.

[0072] It should be noted that downtime or active isolation are both isolation states.

[0073] The state change event instructs the application process to stop processing the caller's transaction, or instructs the application process to resume processing the caller's transaction.

[0074] In an optional implementation, if the daemon process receives a manual process stop instruction, it stops the indicated process.

[0075] The process indicated by the manual process stop instruction includes at least the application process.

[0076] Combine the following Figure 2 , through a detailed description of the specific guard system Figure 1 The method shown.

[0077] Figure 2 The overall design idea is as follows:

[0078] The process daemon adopts a dual-process (process A, process B) daemon mode. Process A is Figure 1 The daemon process of the method shown, process 2 is Figure 1 The communication process of the method shown.

[0079] Process A is responsible for keeping process B and other application processes alive. Process A is responsible for synchronizing information such as process daemon status, isolation / deisolation status, etc. Process A is responsible for persisting process status information and restoring it after restart; Process B is responsible for keeping process A alive; Process B is responsible for application process health checks and actively notifies the caller when an exception occurs.

[0080] This daemon system mainly consists of two processes: process 1 (daemon process) and process 2 (communication process). The specific process is as follows:

[0081] Step (1): When process 1 is started, process 2 is started. The two form a dual daemon, responsible for keeping each other alive and synchronizing their status. Process 1 and process 2 periodically check each other's survival status through a preset status query command line (such as kill -0). If a process does not exist, the process is restored. If process 1 does not exist, process 2 will restore process 1.

[0082] Step (2): After the abnormal closure of process 1 and process 2 is restored, they can be restored to the state before the abnormal closure. In particular, when process 1 accepts a manual stop instruction of a process (such as an application process), even if process 1 is illegally restarted, the process will not be restarted. Other states are similar.

[0083] Step (3): When a management command is sent to process 1 through a command line tool or console to start, stop, isolate, or de-isolate the entire node or an application process, process 1 will synchronize the status information to process 2 and persist it to a local file. If process 2 or an application process is closed abnormally, it will be restored to the persistent state after the last management command was executed.

[0084] Step (4): Process 1 serves as a unified management command entry point, accepts management commands such as start, stop, isolation, and de-isolation issued by the command line or console, manages the process life cycle, checks the process survival status, notifies process 2 of status change events, modifies the local status, and persists it.

[0085] Step (5): When process 1 starts, process 2 and the application process are started and restored according to the status information persisted in the local file. Of course, if the status information shows that an application process does not need to be started and guarded, process 1 will ignore it.

[0086] Step (6): Once the process is started, periodically use the preset status query command line (kill -0) to monitor the process's survival. If it fails, restart and recover. If the process already exists, only the process PID is mounted for subsequent inspection.

[0087] Step (7): Process 2 serves as a unified traffic entry and exit, manages all cross-machine / local caller connections, performs local process status health checks, accepts status change events such as process 1 or active health checks or application active notifications, and notifies the caller of status change events. If the process is not stopped, it only isolates or restores the node or process. After normally processing the in-transit request response, it will no longer accept new requests. After process 2 is started, it periodically uses the preset status query command line (kill-0) to monitor the survival of process 1. If it fails, it will be restarted and restored. If process 1 already exists, only the PID of process 1 is mounted for subsequent inspections.

[0088] Step (8): After process 2 is started, the health check of the application process is performed periodically. When there are health check events such as process restart events, link disconnection, isolation / de-isolation, etc., all callers connected to process 2 are actively notified. If the caller receives notification of the state change event, the node or a service of the node is isolated or restored, and the online traffic is transferred to the normal node, thereby reducing the transaction loss interval from seconds to milliseconds. The benefits are obvious in high TPS financial scenarios, with only dozens of failures in tens of thousands of QPS. Process state changes such as isolation and de-isolation are similar, but there will be no transaction failures. For example: without isolating the nodes, there are 50,000 QPS on two nodes, and kill-9 service processes (forced shutdown) on one of the nodes. Synchronous calls lose approximately 49 transactions, asynchronous calls lose approximately 66 transactions, and the service process resumes transaction processing in about 6 seconds.

[0089] like Figure 3 As shown, the present invention provides an application process recovery device, comprising: a notification module 301, which is used for notifying the caller of the application process that the application process has been abnormally closed if the communication process receives a preset event of the application process of the application node, so that the caller switches the traffic to a node outside the application node; the preset event is an isolation event, a recovery event, or a failure event;

[0090] The processing module 302 is used for the daemon process to monitor the survival status of the communication process and the application process according to a first preset period; if the daemon process determines that the communication process and / or the application process is not alive, then restore the non-surviving process in the communication process and / or the application process; and for the communication process to monitor the survival status of the daemon process according to a second preset period; if the communication process determines that the daemon process is not alive, then restore the daemon process.

[0091] Optionally, the processing module 302 is further used for:

[0092] If the daemon process obtains a management instruction, then after executing the management instruction, a first storage file of status information is generated according to the management instruction; the management instruction is an instruction to perform a preset operation on the application process and / or the communication process; the first storage file of the status information is stored in a persistent storage space.

[0093] Optionally, the processing module 302 is specifically used for:

[0094] Acquire a second storage file in the persistent storage space; the state information stored in the second storage file is the latest state information of the non-surviving process stored in the persistent storage space;

[0095] The non-surviving process is restored according to the status information of the second storage file.

[0096] Optionally, the processing module 302 is further used for:

[0097] If the communication process receives a state change event and determines that the application process has not stopped, the communication process notifies the caller to isolate or restore the application node and / or the application process; the state change event instructs the application process to stop processing the caller's transaction, or instructs the application process to resume processing the caller's transaction.

[0098] Optionally, the processing module 302 is further used for:

[0099] If the daemon process receives a manual process stop instruction, the process indicated by the manual process stop instruction is stopped; the process indicated by the manual process stop instruction at least includes the application process.

[0100] Optionally, the manual process stop instruction further indicates the following information: stop the daemon process and the communication process, or stop the communication process.

[0101] Optionally, the processing module 302 is specifically used for:

[0102] The daemon process monitors the survival status of the communication process and the application process by executing a preset status query command line according to the first preset period;

[0103] The communication process monitors the survival status of the daemon process by executing the preset status query command line according to the second preset period.

[0104] Based on the same inventive concept, an embodiment of the present invention further provides a computer device, including a program or an instruction. When the program or the instruction is executed, the application process recovery method and any optional method provided in the embodiment of the present invention are executed.

[0105] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, including a program or instructions. When the program or instructions are executed, the application process recovery method and any optional method provided in the embodiment of the present invention are executed.

[0106] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0109] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0110] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for recovering an application process, characterized in that: include: If the communication process of the application node receives a preset event of the application process of the application node, it notifies the caller of the application process that the application process has been abnormally closed, so that the caller switches traffic to a node outside the application node; the preset event is an isolation event, a recovery event, or a failure event; wherein a socket connection is established between the communication process and the application process, and the preset event received by the communication process is triggered when the socket connection is disconnected; The daemon process of the application node monitors the survival status of the communication process and the application process according to a first preset period; if the daemon process determines that the communication process and / or the application process are not alive, the daemon process recovers the non-surviving process in the communication process and / or the application process; The communication process monitors the survival status of the daemon process according to a second preset period; if the communication process determines that the daemon process is not alive, the daemon process is restored; The daemon process and the communication process are responsible for keeping each other alive and synchronizing their status; the daemon process will synchronize the status information to the communication process and persist it to a local file; If the daemon process obtains a management instruction, then after executing the management instruction, a first storage file of status information is generated according to the management instruction, and the management instruction is an instruction to perform a preset operation on the application process and / or the communication process; the first storage file of the status information is stored in a persistent storage space.

2. The method according to claim 1, characterized in that The recovering of the non-surviving process in the communication process and / or the application process comprises: Acquire a second storage file in the persistent storage space; the state information stored in the second storage file is the latest state information of the non-surviving process stored in the persistent storage space; The non-surviving process is restored according to the status information of the second storage file.

3. The method according to claim 1, characterized in that Also includes: If the communication process receives a state change event and determines that the application process has not stopped, the communication process notifies the caller to isolate or restore the application node and / or the application process; the state change event instructs the application process to stop processing the caller's transaction, or instructs the application process to resume processing the caller's transaction.

4. The method according to claim 1, characterized in that Also includes: If the daemon process receives a manual process stop instruction, the process indicated by the manual process stop instruction is stopped; the process indicated by the manual process stop instruction at least includes the application process.

5. The method according to claim 4, characterized in that The manual process stop instruction also indicates the following information: stop the daemon process and the communication process, or stop the communication process.

6. The method according to any one of claims 1 to 5, characterized in that: The daemon process of the application node monitors the survival status of the communication process and the application process according to a first preset period, including: The daemon process monitors the survival status of the communication process and the application process by executing a preset status query command line according to the first preset period; The communication process monitors the survival status of the daemon process according to a second preset period, including: The communication process monitors the survival status of the daemon process by executing the preset status query command line according to the second preset period.

7. An application process recovery device, characterized in that: include: A notification module, for notifying the caller of the application process that the application process has been abnormally closed if the communication process receives a preset event of the application process of the application node, so that the caller switches the traffic to a node outside the application node; the preset event is an isolation event, a recovery event, or a failure event; wherein a socket connection is established between the communication process and the application process, and the preset event received by the communication process is triggered when the socket connection is disconnected; A processing module, used for the daemon process to monitor the survival status of the communication process and the application process according to a first preset period; if the daemon process determines that the communication process and / or the application process are not alive, then restore the non-surviving process in the communication process and / or the application process; and used for the communication process to monitor the survival status of the daemon process according to a second preset period; if the communication process determines that the daemon process is not alive, then restore the daemon process; wherein the daemon process and the communication process are responsible for keeping each other alive and synchronizing the status; the daemon process will synchronize the status information to the communication process and persist it to a local file; The processing module is also used for: if the daemon process obtains a management instruction, then after executing the management instruction, generating a first storage file of status information according to the management instruction; the management instruction is an instruction to perform a preset operation on the application process and / or the communication process; and storing the first storage file of the status information in a persistent storage space.

8. A computer device, characterized in that: The method comprises a program or an instruction, and when the program or the instruction is executed, the method according to any one of claims 1 to 6 is performed.

9. A computer-readable storage medium, characterized in that: The method comprises a program or an instruction, and when the program or the instruction is executed, the method according to any one of claims 1 to 6 is performed.

Images