Data snapshot method, device, electronic device, and computer-readable storage medium

Abstract

The embodiment of the present invention discloses a data snapshot method, device, electronic equipment, and computer-readable storage medium. The method includes: in response to receiving a data snapshot instruction, the first thread creates a data snapshot thread, and updates the data to be snapshotted. Identification information; in response to detecting that the data to be snapshotted is modified, the modified identification information of the data to be snapshotted is set as the latest identification information; the data snapshot thread performs snapshot processing on the data according to the identification information, and for those with non-latest identification A data snapshot is executed for the data to be snapshotted of the information, and a delayed data snapshot is executed for the data to be snapshotted with the latest identification information. This technical solution can realize the snapshot operation of the full amount of data without affecting the service of the main thread, eliminates the delay jitter caused by the Fork operation in the prior art, and can also greatly reduce memory consumption, improve memory utilization, and reduce Resource costs, thus effectively ensuring the stability of Redis products and services.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of data processing, and in particular to a data snapshot method, device, electronic equipment, and computer-readable storage medium. Background technique [0002] With the development of data technology, the application of high-performance Redis database is more and more widely. Redis is an open source, written in ANSI C language, supports network, can be based on memory and can be persistent log-type Key-Value database , in order to ensure efficiency, the data in Redis is first cached in memory, and then periodically writes updated data to disk or writes modification operations to additional record files, also known as persistence or data snapshot operations. In the current Redis internal mechanism, persistence, data snapshot, and master-slave synchronization all rely on the operating system's Fork and COW (copy on write, copy-on-write) mechanisms, where both Fork and COW generate new ...

AI Technical Summary

Problems solved by technology

However, this mechanism has the following defects: 1. The Fork system call will consume a lot of time, resulting in second-level jitter in the main thread service; 2. The COW copy is based on a physical memory page and will be released when the child process exits. During the snapshot, a large amount of additional memory is copied, causing memory consumption to double

The above defects will have a great impact on the stability of Redis products and services, and also limit the memory utilization of Redis in the virtual machine environment, and also increase resource costs.

Images