State storage and recovery method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a state storage and recovery method and device, electronic equipment and a storage medium. The method comprises the steps that when a snapshot task of state data runs, detailed information of the state data is collected, and in response to completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system, according to the method, the state data of the distributed system is stored in the storage system, so that a management platform of the distributed system reads detailed information of the state data from the storage system, the distributed system is controlled to recover the previous state according to the detailed information, the detailed information of the state data is collected in real time after a snapshot task when a program in the distributed system runs, the real-time performance is high, and the reliability is high. Therefore, frequent sending of requests to the distributed system is avoided, network broadband and system resources are saved, and performance and stability of the distributed system are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a state storage and recovery method, a state storage and recovery device, an electronic device and a computer-readable storage medium. Background Art

[0002] Flink is a distributed system that requires efficient allocation and management of computing resources to execute streaming applications. It integrates with common cluster resource managers, but can also be set up to run as a standalone cluster or even a library.

[0003] In distributed computing Flink applications, due to resource limitations and node failures, Flink's state data needs to be saved to disk so that the system state can be restored in the event of a system crash or node failure. This state storage and recovery technology is usually called Checkpoint / Savepoint technology.

[0004] Currently, Flink's Checkpoint and Savepoint both store state data directly in the storage system, such as the common HDFS (Hadoop Distributed File System), Huawei Cloud OBS (Object Storage Service), Amazon Simple Storage Service (AmazonS3), etc.

[0005] When Flink restarts and wants to restore the previous running state, you need to specify the path to restore it. Most real-time platforms now require scheduled requests to the Flink RestFul API (an interface for information interaction provided by Flink) to obtain Flink's Checkpoint / Savepoint records. This scheduled request solution has low performance. If the computing tasks in Flink often need to record the correspondence between tasks and Checkpoint / Savepoint records, requests will be sent frequently to the Flink cluster, which will occupy a large amount of network bandwidth and system resources, thereby affecting the performance and stability of the distributed system. When all operators in Flink complete the Checkpoint / Savepoint and the application in Flink stops, the request to the Flink RestFul API has not been triggered, which will cause inconsistencies between the actual records and the data collected by the Flink RestFul API. In addition, there will be a certain delay in the scheduled trigger request to obtain the Checkpoint / Savepoint records. Summary of the invention

[0006] In view of the above problems, embodiments of the present invention are proposed to provide a state storage and recovery method, a corresponding state storage and recovery device, an electronic device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0007] In order to solve the above problems, an embodiment of the present invention discloses a state storage and recovery method, which is applied to a distributed system. The method includes:

[0008] When the snapshot task of the state data is running, collecting detailed information of the state data;

[0009] In response to the completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system so that the management platform of the distributed system reads the detailed information of the status data from the storage system and controls the distributed system to restore the previous state according to the detailed information.

[0010] Optionally, the storage system is a relational database, and storing the detailed information in a storage system outside the distributed system includes:

[0011] The detailed information is sent to a message queue so that the detailed information is written into the relational database through the message queue.

[0012] Optionally, the distributed system includes a work management node and multiple task management nodes, and when the snapshot task of the state data is running, collecting detailed information of the state data includes:

[0013] The work management node triggers the snapshot task and assigns the snapshot task to the task management node for execution;

[0014] When the snapshot task is running, the detailed information storage plug-in running on the work management node collects detailed information of the status data;

[0015] The sending the detailed information to the message queue includes:

[0016] In response to the completion of the snapshot task, the detailed information storage plug-in running on the work management node sends the detailed information to a message queue.

[0017] Optionally, the detailed information includes a storage path of the state data, a size of the state data, and a completion time of the snapshot task.

[0018] Optionally, the snapshot task includes at least one of a checkpoint snapshot task and a backup point snapshot task.

[0019] The embodiment of the present invention further discloses a state storage and recovery device, which is applied to a distributed system. The device includes:

[0020] An information collection module, used to collect detailed information of the state data when the snapshot task of the state data is running;

[0021] An information storage module is used to store the detailed information in a storage system outside the distributed system in response to the completion of the snapshot task, so that the management platform of the distributed system can read the detailed information of the status data from the storage system and control the distributed system to restore the previous state according to the detailed information.

[0022] Optionally, the storage system is a relational database, and the information storage module includes:

[0023] The information sending submodule is used to send the detailed information to a message queue so that the detailed information can be written into the relational database through the message queue.

[0024] Optionally, the distributed system includes a work management node and multiple task management nodes, and the information collection module includes:

[0025] A task execution module, configured to trigger the snapshot task by the work management node and assign the snapshot task to the task management node for execution;

[0026] An information collection module, configured to collect detailed information of the status data by means of a detailed information storage plug-in running on the work management node when the snapshot task is running;

[0027] The information sending submodule includes:

[0028] The information sending unit is used to send the detailed information to the message queue by the detailed information storage plug-in running on the work management node in response to the completion of the snapshot task.

[0029] Optionally, the detailed information includes a storage path of the state data, a size of the state data, and a completion time of the snapshot task.

[0030] Optionally, the snapshot task includes at least one of a checkpoint snapshot task and a backup point snapshot task.

[0031] The embodiment of the present invention further discloses an electronic device, including:

[0032] one or more processors; and

[0033] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the apparatus to perform any of the state storage and recovery methods described above.

[0034] An embodiment of the present invention further discloses a computer-readable storage medium, wherein a computer program stored in the storage medium enables a processor to execute any of the state storage and recovery methods described above.

[0035] According to an embodiment of the present invention, detailed information of the status data is collected when a snapshot task of the status data is running, and in response to the completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system, so that a management platform of the distributed system can read the detailed information of the status data from the storage system and control the distributed system to restore the previous state according to the detailed information, so that when a program in the distributed system is running, the detailed information of the status data can be collected in real time after the snapshot task, with high real-time performance, avoiding frequent requests to the distributed system, saving network bandwidth and system resources, and thus improving the performance and stability of the distributed system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart of a state storage and recovery method embodiment of the present invention;

[0037] Figure 2 is a flowchart of steps of another embodiment of a state storage and recovery method of the present invention;

[0038] Figure 3 This is a schematic diagram of the architecture of the detailed information storage plug-in and its workflow;

[0039] Figure 4 is a structural block diagram of an embodiment of a state storage and recovery device of the present invention;

[0040] Figure 5 The invention is a structural block diagram of an electronic device for state storage and restoration according to an exemplary embodiment. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Reference Figure 1 , shows a flowchart of a state storage and recovery method embodiment of the present invention, which is applied to a distributed system. The method may specifically include the following steps:

[0043] Step 101: When a snapshot task of state data is running, detailed information of the state data is collected.

[0044] In the embodiment of the present invention, a distributed system refers to a distributed computing system, such as a Flink cluster, or any other applicable distributed system, which is not limited in the embodiment of the present invention.

[0045] In the embodiment of the present invention, the state data refers to the state data of the computing task obtained based on the distributed snapshot technology during the execution of the computing task in the distributed system. The snapshot task is the task of obtaining the state data.

[0046] For example, a Checkpoint Barrier is inserted into the data stream, which divides the data stream into segments. Flink's Checkpoint logic is that when a new piece of data flows in, the state changes. After Flink's operator receives the Checkpoint Barrier, it takes a snapshot of the state. Each Checkpoint Barrier has an ID, indicating which Checkpoint the data segment belongs to. When a Checkpoint Barrier with an ID of n reaches each operator, it means that a snapshot of the state update between n-1 and n is to be taken. The Checkpoint Barrier is inserted into the data stream, but it does not affect the original processing order of the data stream.

[0047] In the embodiment of the present invention, detailed information of the state data refers to information describing the state data so as to select and locate the state data.

[0048] In an optional embodiment of the present invention, the detailed information includes a storage path of the state data, a size of the state data, and a completion time of the snapshot task.

[0049] In the embodiment of the present invention, during the running of the snapshot task, detailed information of the state data is collected.

[0050] In an optional embodiment of the present invention, the snapshot task includes at least one of a checkpoint snapshot task and a backup point snapshot task.

[0051] A checkpoint snapshot task is a snapshot task that is triggered periodically.

[0052] For example, Flink can run distributedly on thousands of nodes, breaking down the process of a large computing task into small computing processes, and then distributing the tasks to parallel nodes for processing. During the execution of the task, it can automatically discover data inconsistencies caused by errors in the event processing process. In this case, the state data of the execution process is persistently stored through Checkpoint based on distributed snapshot technology. Once the task is terminated abnormally, Flink can automatically recover the task from the Checkpoint to ensure the consistency of the data during the processing.

[0053] A backup point snapshot task is a manually triggered snapshot task.

[0054] For example, Flink can run streaming applications 24 / 7 with almost no downtime. When dynamic updates or quick recovery are required, Flink uses Savepoint technology to save snapshots of task execution on storage media. When the task is restarted, the original computing state can be restored directly from the previously saved Savepoint, allowing the task to continue running in the state before the shutdown.

[0055] Step 102, in response to the completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system, so that the management platform of the distributed system reads the detailed information of the status data from the storage system and controls the distributed system to restore the previous state according to the detailed information.

[0056] In the embodiment of the present invention, when the snapshot task is completed, the collected detailed information is automatically sent to the storage system and saved by the storage system. The storage system refers to a system outside the distributed system that can be used to store detailed information.

[0057] In the embodiment of the present invention, the management platform of the distributed system is a real-time platform that can manage the distributed system, and the user can manage the distributed system and the applications running on it in real time through the management platform. The management platform can read detailed information of the state data stored previously from the storage system.

[0058] In an embodiment of the present invention, the management platform can control the distributed system to restore the previous state according to the detailed information. Specifically, after reading the detailed information of the previous multiple state data, the management platform displays the detailed information to the user, and the user can select from the previous multiple state data on the management platform. The management platform sends a recovery instruction to the distributed system according to the detailed information corresponding to the user's selection, and the instruction can carry the storage path of the state data, so that the distributed system can obtain the corresponding state data according to the storage path, thereby restoring to the previous state.

[0059] According to an embodiment of the present invention, detailed information of the status data is collected when a snapshot task of the status data is running, and in response to the completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system, so that a management platform of the distributed system can read the detailed information of the status data from the storage system and control the distributed system to restore the previous state according to the detailed information, so that when a program in the distributed system is running, the detailed information of the status data can be collected in real time after the snapshot task, with high real-time performance, avoiding frequent requests to the distributed system, saving network bandwidth and system resources, and thus improving the performance and stability of the distributed system.

[0060] In an optional embodiment of the present invention, the storage system is a relational database, and a specific implementation method of storing the detailed information in a storage system outside the distributed system may include: sending the detailed information to a message queue so that the detailed information is written into the relational database through the message queue.

[0061] The storage system may be a relational database. In response to the completion of the snapshot task, the detailed information may be sent to a message queue first, so that the detailed information is written to the relational database through the message queue, thereby achieving decoupling of the sender and the receiver, and facilitating expansion.

[0062] Reference Figure 2 , shows a flowchart of another embodiment of a state storage and recovery method of the present invention, which is applied to a distributed system. The method may specifically include the following steps:

[0063] Step 201: The work management node triggers the snapshot task and assigns the snapshot task to the task management node for execution.

[0064] In an embodiment of the present invention, the distributed system includes a work management node and multiple task management nodes.

[0065] Among them, the work management node has many responsibilities related to coordinating the distributed execution of applications in a distributed system. It decides when to schedule the next task (or a group of tasks), responds to completed tasks or execution failures, coordinates snapshot tasks, and coordinates recovery from failures, etc. The task management node executes the tasks of the job flow and caches and exchanges data flows. There must always be at least one task management node.

[0066] In the embodiment of the present invention, the work management node triggers the snapshot task and assigns the snapshot task to the task management node for execution.

[0067] Step 202: When the snapshot task is running, the detailed information storage plug-in running on the work management node collects detailed information of the status data.

[0068] In an embodiment of the present invention, when a snapshot task is running, a detailed information storage plug-in running on a work management node collects detailed information. The detailed information storage plug-in is a plug-in for collecting and storing detailed information. The detailed information storage plug-in can collect detailed information of status data.

[0069] Step 203: In response to the completion of the snapshot task, the detailed information storage plug-in running on the work management node sends the detailed information to a message queue, so that the detailed information is written into the relational database through the message queue.

[0070] In the embodiment of the present invention, the detailed information storage plug-in sends the collected detailed information to the message queue in response to the completion of the snapshot task, so that the detailed information is written into the relational database through the message queue.

[0071] For example, Figure 3 The architecture diagram of the detailed information storage plug-in and its workflow shown in the figure, Flink cluster refers to the distributed system, the real-time platform refers to the management platform of the Flink cluster, the database refers to the storage system, and the storage-plugin plug-in refers to the detailed information storage plug-in. The Flink cluster includes JobManager (work management node) and TaskManager (task management node), and the storage-plugin plug-in is deployed on JobManager. The real-time platform can specify checkpoints for JobManager and start tasks. JobManager triggers checkpoints regularly. The storage-plugin plug-in can collect detailed information about the status data obtained by the checkpoint. In response to the completion of the checkpoint, the storage-plugin plug-in writes the collected detailed information to the message queue outside the distributed system. Then the consumer consumes the data in the message queue and writes the detailed information to the database. The real-time platform can read the checkpoint records (i.e., detailed information) in the database.

[0072] Among them, in Flink, the storage-plugin plug-in is implemented based on FileSystemCheckpointStorage (a file system checkpoint storage component, a component inside the Flink cluster, used to save state data to the file system), so the checkpoint storage type of the real-time platform should select file storage. First, one of HDFS, OBS, and Minio must be integrated, and the state data of the checkpoint generated will be stored in these file systems. The enhancement of FileSystemCheckpointStorage in the present invention is actually to rewrite the closeAndFinalizeCheckpoint (checkpoint closing and completion) method of the CheckpointMetadataOutputStream (checkpoint metadata output stream) object.

[0073] Specifically, in Flink, storage is controlled by the parameter state.checkpoint-storage. According to Flink official information, the value of state.checkpoint-storage must be the implementation class of CheckpointStorageFactory. Therefore, the custom EnnCheckpointStorageFactory class implements the CheckpointStorageFactory interface and overrides the createFromConfig method (used to create CheckpointStorage objects). The EnnCheckpoint Storage object is created in the createFromConfig method. In the EnnCheckpointStorage class, the ConfigurableCheckpointStorage interface and the methods of the interface are implemented, and the FileSystemCheckpointStorage object is referenced, which is the so-called static proxy. Create a FileSystemCheckpointStorage object in the construction method of EnnCheckpointStorage.In the overridden createCheckpointStorage method, create an EnnCheckpointStorageAccess object and pass the FsCheckpointStorageAccess object. In the EnnCheckpointStorageAccess class, implement the CheckpointStorageAccess interface and reference the FsCheckpointStorageAccess object. When overriding the initializeLocationForCheckpoint method, get the FsCheckpointStorage Location object through the EnnCheckpointStorageAccess object. Then create the EnnCheckpointStorageLocation object, inherit the EnnCheckpointStreamFactory and implement the CheckpointStorageLocation class in the EnnCheck pointStorageLocation class, and reference the FsCheckpointStorageLocation object. In the overridden createMetadataOutputStream method, create the EnnCheckpointMetadataOutputStream object and initialize the relevant property values, including the FsCheckpointMetadataOutputStream object. The EnnCheckpointMetadataOutputStream class inherits the CheckpointMetadataOutputStream class and references the FsCheckpointMetadataOutputStream object. In the overridden closeAndFinalizeCheckpoint method, the closeAndFinalizeCheckpoint method is called through the FsCheckpointMetadataOutputStream object. At this point, the checkpoint / savepoint process is complete. At this point, the detailed information of the checkpoint can be collected and written to the message queue, waiting for subsequent consumption and writing to the relational database. At this point, the storage-plugin plug-in is completed, and then this jar package is placed under the Classpath path of the Flink runtime, waiting to be loaded when used, so that the detailed information of each checkpoint / savepoint is collected.

[0074] According to an embodiment of the present invention, the snapshot task is triggered by the work management node and the snapshot task is assigned to the task management node for execution. When the snapshot task is running, the detailed information storage plug-in running on the work management node collects detailed information of the status data. In response to the completion of the snapshot task, the detailed information storage plug-in running on the work management node sends the detailed information to the message queue so that the detailed information is written to the relational database through the message queue, so that when the program in the distributed system is running, the detailed information of the status data can be collected in real time after the snapshot task, with high real-time performance, avoiding frequent requests to the distributed system, saving network bandwidth and system resources, and thus improving the performance and stability of the distributed system.

[0075] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0076] Reference Figure 4 The following is a structural block diagram of an embodiment of a state storage and recovery device of the present invention, which is applied to a distributed system and may specifically include the following modules:

[0077] The information collection module 301 is used to collect detailed information of the state data when the snapshot task of the state data is running;

[0078] The information storage module 302 is used to store the detailed information in a storage system outside the distributed system in response to the completion of the snapshot task, so that the management platform of the distributed system can read the detailed information of the status data from the storage system and control the distributed system to restore the previous state according to the detailed information.

[0079] Optionally, the storage system is a relational database, and the information storage module includes:

[0080] The information sending submodule is used to send the detailed information to a message queue so that the detailed information can be written into the relational database through the message queue.

[0081] Optionally, the distributed system includes a work management node and multiple task management nodes, and the information collection module includes:

[0082] A task execution module, configured to trigger the snapshot task by the work management node and assign the snapshot task to the task management node for execution;

[0083] An information collection module, configured to collect detailed information of the status data by means of a detailed information storage plug-in running on the work management node when the snapshot task is running;

[0084] The information sending submodule includes:

[0085] The information sending unit is used to send the detailed information to the message queue by the detailed information storage plug-in running on the work management node in response to the completion of the snapshot task.

[0086] Optionally, the detailed information includes a storage path of the state data, a size of the state data, and a completion time of the snapshot task.

[0087] Optionally, the snapshot task includes at least one of a checkpoint snapshot task and a backup point snapshot task.

[0088] According to an embodiment of the present invention, detailed information of the status data is collected when a snapshot task of the status data is running, and in response to the completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system, so that a management platform of the distributed system can read the detailed information of the status data from the storage system and control the distributed system to restore the previous state according to the detailed information, so that when a program in the distributed system is running, the detailed information of the status data can be collected in real time after the snapshot task, with high real-time performance, avoiding frequent requests to the distributed system, saving network bandwidth and system resources, and thus improving the performance and stability of the distributed system.

[0089] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0090] Figure 5 1 is a block diagram of an electronic device 600 for state storage and recovery according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0091] Reference Figure 5 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0092] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned state storage and recovery method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0093] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0094] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0095] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0096] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0097] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0098] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of contact between the user and the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0099] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0100] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned shutdown control method.

[0101] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of an electronic device 600 to complete the above-mentioned shutdown control method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0102] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a server, enables the server to perform a state storage and recovery method, applied to a distributed system, the method comprising:

[0103] When the snapshot task of the state data is running, collecting detailed information of the state data;

[0104] In response to the completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system so that the management platform of the distributed system reads the detailed information of the status data from the storage system and controls the distributed system to restore the previous state according to the detailed information.

[0105] Optionally, the storage system is a relational database, and storing the detailed information in a storage system outside the distributed system includes:

[0106] The detailed information is sent to a message queue so that the detailed information is written into the relational database through the message queue.

[0107] Optionally, the distributed system includes a work management node and multiple task management nodes, and when the snapshot task of the state data is running, collecting detailed information of the state data includes:

[0108] The work management node triggers the snapshot task and assigns the snapshot task to the task management node for execution;

[0109] When the snapshot task is running, the detailed information storage plug-in running on the work management node collects detailed information of the status data;

[0110] The sending the detailed information to the message queue includes:

[0111] In response to the completion of the snapshot task, the detailed information storage plug-in running on the work management node sends the detailed information to a message queue.

[0112] Optionally, the detailed information includes a storage path of the state data, a size of the state data, and a completion time of the snapshot task.

[0113] Optionally, the snapshot task includes at least one of a checkpoint snapshot task and a backup point snapshot task.

[0114] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0115] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0116] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0119] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0120] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0121] The state storage and recovery method and the state storage and recovery device provided by the present invention are introduced in detail above. The principle and implementation mode of the present invention are explained by using specific examples in this article. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and the scope of application. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A state storage and recovery method, characterized in that: Applied to a distributed system, the method comprises: When the snapshot task of the state data is running, collecting detailed information of the state data; In response to the completion of the snapshot task, the detailed information is stored in a storage system outside the distributed system so that the management platform of the distributed system reads the detailed information of the status data from the storage system and controls the distributed system to restore the previous state according to the detailed information.

2. The method according to claim 1, characterized in that The storage system is a relational database, and storing the detailed information in a storage system outside the distributed system includes: The detailed information is sent to a message queue so that the detailed information is written into the relational database through the message queue.

3. The method according to claim 2, characterized in that The distributed system includes a work management node and multiple task management nodes. When the snapshot task of the state data is running, the detailed information of the state data is collected, including: The work management node triggers the snapshot task and assigns the snapshot task to the task management node for execution; When the snapshot task is running, the detailed information storage plug-in running on the work management node collects detailed information of the status data; The sending the detailed information to the message queue includes: In response to the completion of the snapshot task, the detailed information storage plug-in running on the work management node sends the detailed information to a message queue.

4. The method according to any one of claims 1 to 3, characterized in that: The detailed information includes the storage path of the state data, the size of the state data, and the completion time of the snapshot task.

5. The method according to any one of claims 1 to 3, characterized in that: The snapshot task includes at least one of a checkpoint snapshot task and a backup point snapshot task.

6. A state storage and recovery device, characterized in that: Applied to a distributed system, the device comprises: An information collection module, used to collect detailed information of the state data when the snapshot task of the state data is running; An information storage module is used to store the detailed information in a storage system outside the distributed system in response to the completion of the snapshot task, so that the management platform of the distributed system can read the detailed information of the status data from the storage system and control the distributed system to restore the previous state according to the detailed information.

7. The device according to claim 6, characterized in that The storage system is a relational database, and the information storage module includes: The information sending submodule is used to send the detailed information to a message queue so that the detailed information can be written into the relational database through the message queue.

8. The device according to claim 7, characterized in that The distributed system includes a work management node and multiple task management nodes, and the information collection module includes: A task execution module, configured to trigger the snapshot task by the work management node and assign the snapshot task to the task management node for execution; An information collection module, configured to collect detailed information of the status data by means of a detailed information storage plug-in running on the work management node when the snapshot task is running; The information sending submodule includes: The information sending unit is used to send the detailed information to the message queue by the detailed information storage plug-in running on the work management node in response to the completion of the snapshot task.

9. An electronic device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the apparatus to perform the method of any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the method according to any one of claims 1 to 5.

Images