The invention relates to the technical field of distributed
data stream processing, in particular to a distributed
data stream asynchronous snapshot method and
system based on flink, and the method comprises the following steps: designing and executing a snapshot
algorithm which is completed in limited time under the condition that all processes are in an
active state, thereby meeting the termination requirement; when the snapshots are executed, the causal sequence of the data streams is maintained, and it is ensured that the delivery sequence recorded in the snapshots is consistent with the sequence in actual task execution, so that the feasibility requirement is met; the flink-based distributed
data stream asynchronous snapshot method and
system provided by the invention have the beneficial effects that an asynchronous obstacle snapshot
algorithm is introduced, and the flink-based distributed data
stream asynchronous snapshot method and
system are a new snapshot technology for realizing good
throughput. The asynchronous obstacle snapshot
algorithm is the first algorithm which considers the minimum state required by the loop-free execution topology. In addition, an asynchronous barrier snapshot algorithm is extended to accommodate a loop execution graph by storing only records that need to be reprocessed when
recovery is required.