Long-term and short-term data backup storage method and device based on disk CDP

Abstract

The application relates to a long-term and short-term data backup and storage method and device based on a disk CDP, and belongs to the field of disaster recovery backup. The method comprises the following steps: creating and generating data mirror images, bitmaps, I / O logs and metadata under a current directory, then recording real-time change data by using the files, modifying the directory name of the current directory, deleting the I / O log, obtaining T0 directory and files, then repeating the above steps, and obtaining T1 to TN directories and files according to subsequent triggered generation of incremental data instructions, and rolling back long-term data through the data mirror images under the T0 to TN directories. The device comprises a short-term data directory generation module, a short-term data recording module, a long-term data generation module, a subsequent long-term data generation module and a real-time data storage module. The application realizes rolling back long-term and short-term data through a set of systems, the strategy configuration is unified, the storage cost is low, and the application provides one-stop and low-TCO data protection for high-compliance and high-continuity scenes.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of disaster recovery backup, and relates to a long-term and short-term data backup and storage method and device based on a disk CDP, a computer device, a computer storage medium and a computer program product. BACKGROUND

[0002] Continuous Data Protection (CDP) is a storage technology that realizes recovery at any time point by capturing data changes in real time. The technology is divided into two types: true CDP and quasi-CDP. The former realizes recovery at any time point, and the latter performs backup at fixed intervals.

[0003] Generally, in order to achieve the effect of rollback at any time point, the true CDP technology adopts a real-time backup mode to record any I / O of the production end. For a system with heavy I / O load, in order to save the I / O change record, the backup storage space will be quickly filled, which is doomed to be unable to save the backup data for a long time. However, for many users, it is necessary to save part of the backup data for a long time for archiving and auditing work.

[0004] To this end, in order to make up for the "long-term storage short board" of true CDP, the prior art generally adopts a "double-track parallel" scheme: one side uses true CDP to be responsible for "short-term real-time recovery", such as seconds-level rollback to the state before failure after the business system crashes, and the other side uses "timed backup", such as tape library and cloud storage, to be responsible for "long-term compliance retention", but the scheme also has the following defects:

[0005] 1. The CDP and the timed backup need to be configured with strategies, monitored with storages, and troubleshot, and the cost of operation and maintenance is doubled;

[0006] 2. The data change logs and the independent backup set are largely repeated, so that the storage cost is increased in vain;

[0007] 3. When a fault occurs, the short-term rollback through CDP is needed first, and then the long-term data is retrieved by switching the backup system, so that the process is long and the recovery and rehearsal efficiency is low.

[0008] Therefore, in the face of some scenes that need both "core system real-time protection" and "compliance long-term retention", especially in banks, hospitals and large enterprises, how to solve the fundamental contradiction between the CDP real-time backup and the long-term storage demand in architecture, cost and process is a technical problem that needs to be solved urgently. SUMMARY

[0009] The application provides a long-term and short-term data backup and storage method and device based on a disk CDP, a computer device, a computer storage medium and a computer program product.

[0010] The technical solution of the application for solving the above technical problems is as follows:

[0011] In a first aspect, the application provides a long-term and short-term data backup and storage method based on a disk CDP, which comprises the following steps:

[0012] A short-term data directory generation step is used to create a temporary current directory, and initial state data images, bitmaps, I / O logs and metadata are created under the current directory.

[0013] A short-term data recording step is used to receive captured real-time change I / O, and before new data is landed, write-time copying is performed on original blocks of the data images under the current directory and the same level I / O logs are written, then the target blocks of the data images are overwritten, and subsequently the bitmap of the same level is used to mark the storage data situation of the target blocks of the data images.

[0014] A long-term data generation step is used to modify the directory name of the current directory and delete the I / O logs when a T0 full data generation instruction is triggered, and T0 directories and files are obtained.

[0015] A subsequent long-term data generation step is used to sequentially repeat the above three steps, and according to a subsequent triggered incremental data generation instruction, T1 to TN directories and files are sequentially obtained, wherein the data images under the T0 to TN directories are mutually dependent and jointly constitute a mirror image completely consistent with the original disk.

[0016] A real-time data storage step is used to sequentially execute the short-term data directory generation step and the short-term data recording step, and the current directory and files are obtained.

[0017] In a second aspect, the application provides a long-term and short-term data backup and storage device based on a disk CDP, which comprises the following steps:

[0018] A short-term data directory generation module is used to create a temporary current directory, and initial state data images, bitmaps, I / O logs and metadata are created under the current directory.

[0019] A short-term data recording module is used to receive captured real-time change I / O, and before new data is landed, write-time copying is performed on original blocks of the data images under the current directory and the same level I / O logs are written, then the target blocks of the data images are overwritten, and subsequently the bitmap of the same level is used to mark the storage data situation of the target blocks of the data images.

[0020] The long-term data generation module is used to modify the directory name of the current directory and delete the I / O log when the command to generate full data at time T0 is triggered, thereby obtaining the T0 directory and its files.

[0021] The subsequent long-term data generation module is used to repeat the above three modules in sequence, and obtain the directories and files of T1 to TN in sequence according to the subsequent triggered instructions to generate incremental data. The data images under the directories of T0 to TN are interdependent and together form an image that is completely consistent with the original disk.

[0022] The real-time data storage module is used to sequentially execute the short-term data directory generation module and the short-term data recording module to obtain the current directory and its files.

[0023] Thirdly, a computer device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the disk-based CDP-based long-term and short-term data backup and preservation method as described in any of the preceding claims.

[0024] Fourthly, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the long-term and short-term data backup and preservation method based on disk CDP as described above.

[0025] Fifthly, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the disk-based CDP-based long-term and short-term data backup and preservation method described in any of the preceding claims.

[0026] The beneficial effects of this invention are:

[0027] (1) This invention, through the design of a long and short data backup and storage organization structure, deeply embeds the real-time protection capability and long-term retention capability of CDP into the same system. The short-term rollback cycle and the long-term image retention cycle can be configured through a single policy template, and the policy configuration is unified. Storage monitoring and fault diagnosis are completed in a single system, effectively reducing the manpower and time costs of operation and maintenance.

[0028] (2) This invention combines copy-on-write technology to generate independent derived mirror data for long-term storage from the I / O change stream recorded in real time by CDP, which meets the requirements of compliance audit and disaster recovery. At the same time, through the native I / O log of CDP, it supports data traceability at any point in time at the millisecond level, realizing "one data source, dual function support", effectively reducing storage overhead.

[0029] (3) The derivative images used for long-term data storage are designed to be interdependent chain relationships. After the first "full" data storage, only the "incremental" data is stored. Lightweight metadata is also used. All of these avoid the redundant data of traditional CDP logs and timed backups, and further reduce storage overhead.

[0030] (4) This invention achieves a seamless process of "short-term real-time rollback + long-term image retrieval" within a single system: In case of business failure, the data rollback of the most recent time point can be completed in seconds through the CDP log within a short period; when historical data needs to be traced, the independent derived image generated by the long-term strategy can be directly mounted and restored without switching the backup system, mounting tapes or decompressing files; it lays the foundation for compressing the rollback process from "dual system relay" to "closed loop within a single system", and provides a "one-stop, low TCO" data protection paradigm for high compliance and high continuity scenarios such as finance, healthcare, and government and enterprise. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the long-term and short-term data backup and storage method for generating T0 and T1 directories and their files provided in Embodiment 1 of the present invention.

[0033] Figure 2 This is a schematic diagram of the data mirror and bitmap correspondence structure in Embodiment 1 of the present invention.

[0034] Figure 3 This is a schematic diagram of the T0 time directory and its file generation process in Embodiment 1 of the present invention.

[0035] Figure 4 This is a schematic diagram of the long-term and short-term data backup organization structure of the T0 to T1 directories and their files generated in Embodiment 1 of the present invention.

[0036] Figure 5 This is a schematic diagram of the long-term and short-term data backup and storage method for the T0 to T2 directories and their files generated in Embodiment 2 of the present invention.

[0037] Figure 6 This is a schematic diagram of the data flow after rewinding to time T1 from the T0 and T1 directories and their files generated in Embodiment 3 of the present invention.

[0038] Figure 7This is a schematic diagram of the structure of the long-term and short-term data backup and storage device based on disk CDP provided in Embodiment 4 of the present invention.

[0039] Figure 8 This is a schematic diagram of the data rollback module structure provided in Embodiment 4 of the present invention.

[0040] Figure 9 This is a schematic diagram of the structure of the electronic device provided in Embodiment 5 of the present invention.

[0041] The attached diagram lists the components represented by each number as follows:

[0042] 4001 Short-term data catalog generation module; 4002 Short-term data recording module; 4003 Long-term data generation module; 4004 Subsequent long-term data generation module; 4005 Real-time data storage module; 4006 Data rollback module; 40061 First rollback unit; 40062 Second rollback unit; 510 Processor; 520 Communication interface; 530 Memory; 540 Communication bus. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0044] Terminology Explanation

[0045] Rollback: Rollback refers to an "undoing action" with a clear intention and explicit instruction, the purpose of which is to eliminate the impact of "incomplete / error state", that is, there must be an "actively triggered undoing action";

[0046] Short-term data rollback: For data changes within a single operation / transaction, the "undo command" can be used to roll back to the state before the operation. This is to address the need for rapid rollback in cases of accidental deletion, ransomware, or logical errors, and generally does not exceed 14 days.

[0047] Long-term data rollback: For data changes across multiple versions / historical periods, rollback to a historical version or milestone state to meet compliance audits, monthly / grade recovery, disaster recovery, etc. Long-term mirror points are generally ≥30 days, with the upper limit set according to compliance requirements. It is also common to be 1 to 5 years, and the longest can be "permanent".

[0048] Current directory: is the "base path that the program or command is currently operating on". Its function is to specify the "relative path reference system" of files / commands. The current directory is actively switched by the user / program, and the stored content is not fixed and will not be actively cleared by the system.

[0049] Copy-on-write: A performance optimization technique that delays or avoids unnecessary data copying. Its core principle is that when multiple objects share the same data, the data is only copied when an object attempts to modify the data; otherwise, the original data is continuously shared.

[0050] Derived image: refers to a new data image built on the basis of an existing data image (parent image) by modifying or adding files / configurations / applications. In this embodiment, it is also a data image generated according to incremental instructions.

[0051] Example 1

[0052] Currently, true CDP has a "shortcoming in long-term data retention." Existing technologies generally adopt a "dual-track parallel" approach: on one hand, true CDP is used for "short-term real-time recovery," such as rolling back to the pre-failure state within seconds after a business system crash; on the other hand, "scheduled backups," such as tape libraries and cloud storage, are used for "long-term compliant data retention." However, this approach also has the following technical problems:

[0053] 1. CDP and scheduled backups require separate policy configuration, storage monitoring, and troubleshooting, doubling the cost of operation and maintenance.

[0054] 2. The data change logs and independent backup sets have a large amount of duplication, which increases storage costs out of nowhere;

[0055] 3. In case of failure, a short rollback via CDP is required before switching to the backup system to retrieve long-term data. This process is lengthy and results in low efficiency for recovery and drills.

[0056] To address the aforementioned issues, this invention provides a method for long-term and short-term data backup and storage based on disk CDP. Figure 1 This is a schematic diagram of the long-term and short-term data backup and preservation method based on disk CDP provided in an embodiment of the present invention, as shown below. Figure 1 As shown, this method includes:

[0057] Step S101: Create a temporary current directory and create the initial state data image, bitmap, I / O log and metadata in the current directory.

[0058] In this embodiment, the data mirror is a file used to store I / O changes, i.e., real-time backup data; the bitmap is a file used to mark whether data exists in the corresponding data block in the data mirror; the I / O log is a file used to record rollback data; and the metadata is a file used to record and retain the image metadata for a long time.

[0059] It's worth noting that data mirroring, in practical applications, needs to be precisely tailored to different data types and scenarios: database mirroring focuses on transaction-level disaster recovery for structured databases, ensuring zero data loss and uninterrupted business operations through real-time transaction log synchronization and automatic failover; storage mirroring targets hardware-level risks, addressing hardware fault tolerance issues such as single-disk failures through physical block redundancy and hot-swappable reconstruction; file-level mirroring adapts to unstructured files, achieving lightweight cross-environment backups through incremental transmission and metadata retention. Furthermore, in the hybrid data asset scenarios of banks and hospitals described in this embodiment, a single mirroring technology may not meet all needs, but a "layered strategy" can be used to combine specific mirrors, such as using database mirroring to protect the core database, file / storage mirroring to protect unstructured files, and storage mirroring to cover hardware risks.

[0060] It's also worth noting that the metadata mentioned above is used to record long-term retention image metadata; that is, metadata and data images are used together. Specifically: In terms of long-term retention and incremental backup, metadata builds a "change index" for data images by recording changes such as file modification time and inode number, enabling efficient incremental backups. For example, file-level images (such as rsync) rely on the metadata's "timestamp + inode number" to identify differences, significantly saving storage and transmission costs. This aligns with the design of "derived images for long-term data retention adopting a chain relationship," ensuring that each derived image's change point is accurately marked by metadata, forming a traceable version chain. In terms of operation history tracing, metadata records "who modified the configuration / data and when," giving image operations "audit-level transparency." In high-audit scenarios such as banks and hospitals, this not only proves data compliance but also reconstructs the entire operation chain, directly supporting the technical goals of archiving and auditing. Ultimately, this combination upgrades data mirroring from a "storage tool" to a "data asset intelligent management center," improving backup efficiency and compliance while enabling manageable, searchable, and traceable data assets throughout their entire lifecycle through chain-like relationship design and operational history tracing.

[0061] Preferably, step S101 further includes:

[0062] Step S1011: Create a temporary current directory;

[0063] Step S1012: Create a sparse data mirror in the current directory;

[0064] Step S1013: Divide the data mirror into logical blocks according to a preset size;

[0065] Step S1014: Create metadata and establish the correspondence between the data mirror logical blocks and the original disk physical blocks through the metadata;

[0066] Step S1015: Create a bitmap with all bits initially set to 0. The bitmap marks whether there is data in the corresponding data block in the data mirror in the initial state.

[0067] Step S1016: Create an empty I / O log.

[0068] It is worth noting that sparse files are a special file storage format that allocates disk space only for the actual data written, while data mirroring is a hierarchical file system template. This embodiment uses sparse data mirror files to allow the mirror file to store only the actual data, skipping a large number of unused empty areas, and further optimizing the storage and transmission efficiency of data mirroring.

[0069] It's also worth noting that, in terms of specific operation, metadata is the core translation layer connecting data mirror logical blocks and disk physical blocks. Metadata marks valid / empty blocks, traverses all logical blocks, determines whether actual data has been written, confirms and records valid data and null fill areas, and reserves a mapping table of "logical block number → physical block number" for each valid logical block to fill during subsequent physical block allocation. Therefore, the underlying storage logic of "data mirror logical block → metadata → physical block mapping," overlaid with a sparse mechanism as a "space optimization layer"—changing the "block mapping" of the mirror from "forcibly allocating all physical blocks" to "allocating only physical blocks with valid data, and marking empty blocks with metadata"—allows data mirroring to perform better in scenarios with high requirements for storage space, transmission speed, and lightweight operation.

[0070] Step S102: Receive the captured real-time change I / O. Before the new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0071] It is understood that the method in this embodiment runs on a single disaster recovery backup system at the backup end. The backup end's job is to receive and process the captured real-time changes in I / O. This embodiment does not specifically limit the method or source of capturing real-time changes in I / O.

[0072] It is also understandable that "same-level I / O log" and "same-level bitmap" both refer to I / O logs and bitmaps in the same directory. For example, in this embodiment, "same-level I / O log" represents the I / O logs in the current directory, and "same-level bitmap" represents the bitmaps in the current directory.

[0073] Preferably, step S102 further includes:

[0074] Step S1021: Receive the captured real-time change I / O;

[0075] Step S1022: Before overwriting the target data block in the current directory, the original data of the target block is copied using the copy-on-write mechanism, and the copy is immediately written as a reverse record to the same level I / O log. The reverse record data includes: data block size, block number, timestamp, offset within the mirror and checksum.

[0076] Step S1023: After writing the I / O log to disk, overwrite the real-time changed I / O into the data mirror target block.

[0077] Step S1024: Mark whether the target block of the data mirror contains data by using the same level bitmap. If the target block contains data that has been written and is valid at this level, mark it as 1; if the target block does not contain data, mark it as 0.

[0078] To better understand the relationship between bitmap-marked data mirroring target blocks, such as Figure 2 As shown, specifically: the data image is divided into several 8 MB data blocks. The system uses a bitmap to record the usage or status of these blocks. Each bit corresponds to a data block in the data image. Bit=1 indicates that the block has been allocated or written with data, and bit=0 indicates that the block is unused, unwritten, or empty. The bitmap and the data image are in one-to-one correspondence. The target block in the data image can be quickly located through the bit index. This allows for quick determination of which blocks need to be rolled back, skipped, or compressed. It supports sparse files that only record valid blocks, saving space and improving rollback efficiency.

[0079] As a preferred approach, for the rollback scenario in this embodiment, the data mirror can be divided into 1MB-16MB segments. Below is a brief explanation of the selection based on different detailed requirements: A 1MB segment transfers only 1MB per rollback, resulting in low network / disk pressure, a large total number of blocks, and 8 times more metadata than 8MB, which is acceptable and suitable for scenarios with frequent rollbacks, limited bandwidth, or a large number of small files; a 4MB segment offers a 5-15% higher deduplication rate than 8MB, moderate rollback traffic, and metadata size that is an order of magnitude smaller than 64KB but half that of 8MB, making it the preferred choice for general scenarios; an 8MB segment balances deduplication rate and latency, suitable for existing systems to reuse; a 16MB segment has higher network / disk read / write latency, halves the metadata size, and doubles the rollback size, suitable for high-speed bandwidth scenarios and scenarios highly sensitive to metadata. Therefore, the data mirror segmentation should be flexibly chosen according to actual needs, and this embodiment does not impose specific limitations.

[0080] Step S103: When the instruction to generate full data at time T0 is triggered, modify the directory name of the current directory and delete the I / O log to obtain the T0 directory and its files.

[0081] It's worth noting that step S101 mentioned the current directory as temporary because it will be fixed in step S103. Therefore, the current directory cannot be considered a temporary directory. Furthermore, the current directory is the "base path currently being operated on" by the program or command, specifying the "relative path reference system" for files / commands. It is actively switched by the user / program, such as the `cd` command, and is a "dynamically changing path reference" that can persistently store fixed content. A temporary directory, on the other hand, is a "place to store temporary data," and its attributes are non-persistent. Temporary directories are automatically managed by the system / program. Even if the I / O log is only a few KB, such as log fragments or configuration cache, as long as it is marked as "temporary," the system / program may clean up the I / O log to prioritize space reuse. Therefore, "rollback" itself is a process of performing undo / restore operations using rollback resources, which can only be achieved using the current directory in this example. It's important to distinguish between these two.

[0082] It is also worth noting that, as mentioned in steps S101 and S102 above, when new I / O data is written to the corresponding block in the data mirror of the current directory, information such as the block, timestamp, and block number will be stored in the I / O log as a record order. Therefore, the entire I / O log is equivalent to the reverse I / O change data array on the storage domain disk. Furthermore, the I / O log does not simply store a "log of I / O operations," but rather a "reverse undo instruction set" specifically designed for rollback operations. When the data mirror is written to an 8MB data block, the I / O log not only records the "block content, timestamp, and block number," but also implicitly contains the logic of "how to undo this write operation during rollback," thus achieving "forward write" → "reverse undo." As a "reverse array," the I / O log needs to be traversed from the latest record to the oldest record during rollback. This traversal order prioritizes the latest records, which helps improve rollback efficiency. Additionally, in this field, the "current state" of the data mirror is the sum of all historical I / O operations. The "reverse" design of the I / O log allows the rollback operation to accurately separate "all changes after a specified point in time," thereby achieving accurate restoration of historical versions. Therefore, those skilled in the art should understand that through steps S101 and S102, "data mirror + I / O log" can achieve short-term data rollback.

[0083] In step S103, this embodiment will perform the initial generation of long-term data, namely, the generation of the directory and its files at time T0. To better understand this process, as follows... Figure 3As shown, let me elaborate: Above the horizontal dotted line is only the current directory, which means that the system only has the current directory at this time. The current directory contains the data image, bitmap, I / O log and metadata that have been recorded (which can be understood as being in the process of step S102). When the system receives the full data instruction at time T0, it will delete the I / O log and change the directory name of the current directory to T0. After the above operations, the T0 directory below the horizontal dotted line is obtained. The T0 directory contains the data image, bitmap and metadata, which completes step S103. Since the T0 directory is renamed from the current directory, the entire directory and its files can be persistently saved for archiving, auditing and other uses.

[0084] Of course, and so on Figure 3 As shown, there is a vertical dotted line to the right of the T0 directory. To the right of this line is the new current directory (i.e., step S104). The new current directory still contains a data mirror, bitmap, I / O log, and metadata. The data mirror is a sparse file, the bitmap has all bits set to 0 for the corresponding block number, and the I / O log is empty. This is used for subsequent short-term data rollback. How to rollback is explained in detail above and will not be repeated here. However, it is important to emphasize that, based on the requirements of true CDP, those skilled in the art should understand that the current directory in the system is unique and does not need to have multiple copies occupying storage space. Therefore, the subsequent generation of the T1 directory should not result in multiple copies of the current directory causing the system to fail.

[0085] Step S104: Create a temporary current directory again, and create the initial state data image, bitmap, I / O log and metadata in the current directory.

[0086] Step S105: Continue to receive captured real-time changes in I / O. Before new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0087] Step S106: When the instruction to generate incremental data at time T1 is triggered, the directory name of the current directory is modified and the I / O log is deleted to obtain the T1 directory and its files. During the process, the mutual dependency relationship between the T0 data mirror and the T1 data mirror is set, and the two together form an image that is completely consistent with the original disk.

[0088] As a preferred embodiment, in step S106, the specific operation of setting the mutual dependency relationship between the T0 data mirror and the T1 data mirror during the process is as follows: the incremental data mirror stored in the T1 directory takes the data mirror at time T0 as the backup mirror to form a parent-child dependency chain; the bitmap stored in the T1 directory is set by block, bit=1 indicates that the block has been written and is valid at this level, bit=0 indicates that it is a hole and needs to be traced back to the parent mirror to T0; the metadata records stored in the T1 directory include: parent and child UUID, data block size and checksum.

[0089] It is important to note that T0 is the full root node and has no parent image, so its bitmap does not need to be set to trace back. Those skilled in the art should understand this after understanding the chained logic of "full + incremental". Specifically, an explicit assignment of `layer.backing_file == NULL` can be added to determine the trace endpoint, but this embodiment does not specifically limit this. Meanwhile, the backup image (i.e., the backing file) is relative to the current image layer; the backup image is its parent image, providing a fallback function. Creating a backup image is a familiar operation for those skilled in the art. For example, the command to create a `qcow2` is `qemu-img create -f qcow2 -b .. / T0 / image.qcow2 T1 / image.qcow2`. `T1 / image.qcow2` only has a file header and a small amount of metadata. The file header records `backing_file_path = .. / T0 / image.qcow2`. This embodiment also does not specifically limit this.

[0090] Step S107: Then execute steps S104 and S105 in sequence to obtain the current directory and its files.

[0091] After completing steps S101 to S107, the following is obtained: Figure 4The generated T0 to T1 directories and their files are organized into long-term and short-term data backups. Specifically, the long-term retention data on the left consists of the T0 and T1 directories. The T0 directory contains the data image, bitmap, and metadata at time T0, and the T1 directory contains the data image, bitmap, and metadata at time T1. The data images at time T0 and T1 are linked. The short-term rollback data on the right is located in the current directory. This directory contains the data image, bitmap, I / O log, and metadata. Unlike the T0 directory, the short-term rollback data consists of real-time data. This is because the local I / O change receiving module writes real-time I / O changes to the data image, and the original blocks of the data image are copied on write and written to the corresponding I / O log. Therefore, this data organization structure deeply embeds CDP's real-time protection capabilities and long-term retention capabilities into the same system, and the components form a good rollback loop within a single system.

[0092] In this embodiment, the data generated during the CDP backup process is combined with copy-on-write technology to generate independent point data for long-term storage from the real-time backup data. The former can be used for short-term rollback, while the latter can be used for long-term recovery and backup data storage. This can efficiently meet the needs of short-term data rollback at any point in time and long-term retention of backup data at some critical points in the CDP backup scenario. The strategy configuration is unified, the storage overhead is low, and it lays the foundation for compressing the rollback process from "dual-system relay" to "single-system closed loop".

[0093] Example 2

[0094] like Figure 5 As shown, in one embodiment, a method for long-term and short-term data backup and storage based on disk CDP is provided. This method includes:

[0095] Step S201: Create a temporary current directory, and create the initial state data image, bitmap, I / O log and metadata in the current directory;

[0096] Step S202: Receive the captured real-time change I / O. Before the new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0097] Step S203: When the instruction to generate full data at time T0 is triggered, modify the directory name of the current directory and delete the I / O log to obtain the T0 directory and its files;

[0098] Step S204: Create a temporary current directory again, and create the initial state data image, bitmap, I / O log and metadata in the current directory;

[0099] Step S205: Continue to receive captured real-time changes in I / O. Before new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0100] Step S206: When the instruction to generate incremental data at time T1 is triggered, the directory name of the current directory is modified and the I / O log is deleted to obtain the T1 directory and its files. During the process, the mutual dependency relationship between the T0 data mirror and the T1 data mirror is set, and the two together form an image that is completely consistent with the original disk.

[0101] Step S207: Create a temporary current directory again, and create the initial state data image, bitmap, I / O log and metadata in the current directory;

[0102] In step S208, continue to receive captured real-time changes in I / O. Before the new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0103] Step S209: When the instruction to generate incremental data at time T2 is triggered, the directory name of the current directory is modified and the I / O log is deleted to obtain the T2 directory and its files. During the process, the mutual dependency relationship between the T1 data mirror and the T2 data mirror is set. The T0 data mirror, T1 data mirror and T2 data mirror together form an image that is completely consistent with the original disk.

[0104] Step S210, then execute steps S207 and S208 in sequence to obtain the current directory and its files.

[0105] Unlike the above embodiments, this embodiment further clarifies the generation process of the organizational structure for long-term and short-term data backup by generating T0, T1, and T2 directories and their files. After the first "full" T0 backup, T1 and T2 only store "incremental" backups. Combined with lightweight metadata, this avoids redundant data from traditional CDP logs and scheduled backups, effectively reducing storage overhead.

[0106] Example 3

[0107] like Figure 6 As shown, in one embodiment, a method for long-term and short-term data backup and storage based on disk CDP is provided. This method includes:

[0108] Step S301: Create a temporary current directory, and create the initial state data image, bitmap, I / O log and metadata in the current directory;

[0109] Step S302: Receive the captured real-time change I / O. Before the new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0110] Step S303: When the instruction to generate full data at time T0 is triggered, modify the directory name of the current directory and delete the I / O log to obtain the T0 directory and its files;

[0111] Step S304: Create a temporary current directory again, and create the initial state data image, bitmap, I / O log and metadata in the current directory;

[0112] Step S305: Continue to receive captured real-time changes in I / O. Before new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0113] Step S306: When the instruction to generate incremental data at time T1 is triggered, the directory name of the current directory is modified and the I / O log is deleted to obtain the T1 directory and its files. During the process, the mutual dependency relationship between the T0 data mirror and the T1 data mirror is set, and the two together form an image that is completely consistent with the original disk.

[0114] Step S307: Create a temporary current directory again, and create the initial state data image, bitmap, I / O log and metadata in the current directory;

[0115] In step S308, continue to receive captured real-time changes in I / O. Before the new data is written to disk, perform a copy-on-write operation on the original block of the data mirror in the current directory and write it to the same-level I / O log. Then, overwrite the target block of the data mirror and mark the data storage status of the target block of the data mirror using the same-level bitmap.

[0116] Step S309: When the rollback TM time mirroring command is triggered, if the TM time is in a short period time window, short-term data is rolled back through the current directory and files in the directories from T0 to TN; if the TM time is in a long period time window, long-term data is rolled back through files in the directories from T0 to TN.

[0117] Preferably, step S309 further includes:

[0118] Step S3091: When the rollback command for the TM time mirror is triggered and the TM time is equal to or earlier than the T1 time, first restore the T1 time data mirror from the bitmap in the T1 directory, and then trace back to the T0 time data mirror according to the interdependence between the data mirrors, restore the T0 time data mirror, and thus complete the task rollback.

[0119] Step S3092: When the rollback command for the TM time mirror is triggered and the TM time is later than the T1 time, first write the old data back to the same level data mirror through the current directory I / O log, then restore the T1 time data mirror from the bitmap under the T1 directory which is closest to the TM time, and then trace back to the T0 time data mirror according to the mutual dependency relationship between the data mirrors, restore the T0 time data mirror, and thus complete the task rollback.

[0120] It is worth noting that if the user only needs to roll back short-term real-time data, the old data can be written back to the same-level data mirror directly through the current directory I / O log.

[0121] Unlike the above embodiments, this embodiment takes the generation of T0 and T1 directories and their files as an example to further clarify how to achieve the entire process of "short-term real-time rollback + long-term image retrieval" within a single system using organizational structure: In the event of a business failure, the CDP logs within the short period and the derived data image within the long period are directly invoked, and the data rollback of the most recent time point is completed in seconds; when historical data needs to be traced, the independent derived image generated through the long-term strategy is directly mounted and restored without switching the backup system, mounting tapes, or decompressing files, providing a "one-stop, low-cost" data protection solution for high compliance and high continuity scenarios such as finance, healthcare, and government and enterprise.

[0122] Example 4

[0123] like Figure 7 As shown, in one embodiment, a long-term and short-term data backup and storage device based on disk CDP is provided. This device includes:

[0124] The short-term data directory generation module 4001 is used to create a temporary current directory and create an initial state data image, bitmap, I / O log and metadata in the current directory;

[0125] The short-term data logging module 4002 is used to receive captured real-time changes in I / O. Before new data is written to disk, it performs a copy-on-write operation on the original block of the data mirror in the current directory and writes it to the same-level I / O log. Then it overwrites the target block of the data mirror and marks the data storage status of the target block of the data mirror through the same-level bitmap.

[0126] The long-term data generation module 4003 is used to modify the directory name of the current directory and delete the I / O log when the command to generate full data at time T0 is triggered, so as to obtain the T0 directory and its files.

[0127] The subsequent long-term data generation module 4004 is used to repeat the above short-term data directory generation module 4001, short-term data recording module 4002 and long-term data generation module 4003 in sequence, and obtain the directories and files from T1 to TN in sequence according to the instruction to generate incremental data triggered later. The data images under the directories from T0 to TN are interdependent and together form an image that is completely consistent with the original disk.

[0128] The real-time data storage module 4005 is used to sequentially execute the short-term data directory generation module 4001 and the short-term data recording module 4002 to obtain the current directory and its files.

[0129] The data rollback module 4006 is used to roll back long-term data through files in the directories from T0 to TN, or short-term data through the current directory and files in the directories from T0 to TN, based on the overlap between the TM and TN times when the rollback mirror command is triggered.

[0130] As a preferred option, such as Figure 8 As shown, the data rollback module 4006 further includes:

[0131] The first rollback unit 40061 is used to, when the rollback TM time mirror command is triggered and the TM time is earlier than or equal to the TN time, first restore the TN time data mirror from the bitmap under the TN directory, and then trace back to T0 according to the interdependence between the data mirrors to restore all data mirrors before the TM time.

[0132] The second rollback unit 40062 is used to, when the rollback command for the TM time mirror is triggered and the TM time is later than the TN time, first write the old data back to the same level data mirror through the current directory I / O log, then restore the TN time data mirror from the bitmap under the TN directory that is closest to the TM time, and then trace back to T0 according to the interdependence between the data mirrors to restore all data mirrors before the TN time.

[0133] It should be noted that although the real-time data storage module 4005 is the main working module for rolling back short-term data in this schematic structure, if it is necessary to roll back short-term data before generating the T0 directory or during the time period between generating the T0 and T1 directories, the short-term data recording module 4002 can also be used directly to roll back the short-term data. The real-time data storage module 4005 and the short-term data recording module 4002 are not different in terms of their constituent files and functions, as has been described above, and those skilled in the art should understand. Therefore, the entire process of this embodiment can meet the needs of short-term data rollback.

[0134] The device in this embodiment belongs to "one system, unified timeline". During rollback, the data range is determined by the protection time window strategy. If the target time is within a short-cycle protection window, the native I / O log of CDP is directly invoked for rollback, and long-term data images are automatically merged, providing seamless coverage from short-term to long-term. The rollback includes the full cycle data and supports second-level recovery at any point in time. If the target time is within a long-cycle image window, the independent derived data image generated by the real copy is invoked for rollback. The rollback includes all historical data, greatly simplifying the complexity and risk of data protection.

[0135] Example 5

[0136] exist Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 9 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call logical instructions in the memory 530 to execute the long-term and short-term data backup and preservation method based on disk CDP as described in any one of embodiments 1 to 3.

[0137] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] Example 6

[0139] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it implements the long-term and short-term data backup and preservation method based on disk CDP as described in any one of embodiments 1 to 3.

[0140] Example 7

[0141] In this embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method for long-term and short-term data backup and preservation based on disk CDP as described in the embodiment.

[0142] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0143] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0144] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0145] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, Ruby, and Go, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0146] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for long-term and short-term data backup and preservation based on disk CDP, characterized in that, The method includes: The steps for generating a short-term data directory are as follows: create a temporary current directory, and then create an initial state data image, bitmap, I / O log, and metadata within the current directory; The short-term data logging step involves receiving captured real-time I / O changes. Before writing the new data to disk, a copy-on-write mechanism is used to copy the original block of the data mirror in the current directory and write it to the sibling I / O log. This copy then overwrites the target block of the data mirror. Finally, a sibling bitmap is used to mark the data storage status of the target block. More specifically, this step involves: receiving captured real-time I / O changes; copying the original data of the target block using a copy-on-write mechanism before overwriting it, and immediately writing the copy as a reverse record to the sibling I / O log. The reverse record includes: data block size, block number, timestamp, offset within the mirror, and checksum; after writing the I / O log to disk, the real-time I / O changes are overwritten to the target block of the data mirror; and a sibling bitmap is used to mark whether the target block stores data. If the target block contains valid data written to it at this level, it is marked as 1; otherwise, it is marked as 0. The long-term data generation process involves triggering the command to generate full data at time T0, modifying the directory name of the current directory, deleting the I / O log, and obtaining the T0 directory and its files. The subsequent long-term data generation steps repeat the above three steps in sequence, and according to the subsequently triggered incremental data generation instructions, obtain the directories and their files from T1 to TN in sequence. The data images under the T0 to TN directories are interdependent and together form an image completely identical to the original disk. More specifically, this step repeats the short-term data directory generation step, the short-term data recording step, and the long-term data generation step in sequence. In the long-term data generation step, according to the subsequently triggered incremental data generation instructions from time T1 to time TN, obtain the directories and their files from T1 to TN in sequence. Each of the T1 to TN directories contains only one incremental data image. After the incremental data image is generated, the data image at time T(N-1) is used as the backup image to form a parent-child dependency chain. The bitmap is set by block. bit=1 indicates that the block has been written and is valid at this level. bit=0 indicates that it is an empty hole and needs to be traced back to the parent image until T0. The metadata records include: parent and child UUIDs, data block size, and checksum. The real-time data saving process involves sequentially executing the short-term data directory generation step and the short-term data recording step to obtain the current directory and its files.

2. The method for long-term and short-term data backup and preservation based on disk CDP according to claim 1, characterized in that, The short-term data catalog generation step further includes: Create a temporary current directory; Create a sparse data mirror in the current directory; The data mirror is divided into logical blocks according to a preset size; Create metadata to establish the correspondence between data mirror logical blocks and original disk physical blocks; Create a bitmap with all bits initially set to 0. The bitmap marks whether there is data in the corresponding data block in the data mirror in the initial state. Create an empty I / O log.

3. The method for long-term and short-term data backup and preservation based on disk CDP according to claim 1 or 2, characterized in that, The method further includes: The data rollback process involves triggering a rollback command for the TM time frame. If the TM time frame is within a short-cycle time window, short-term data is rolled back using files in the current directory and directories from T0 to TN. If the TM time frame is within a long-cycle time window, long-term data is rolled back using files in directories from T0 to TN.

4. The method for long-term and short-term data backup and preservation based on disk CDP according to claim 3, characterized in that, The data rollback step also includes: When the command to roll back the TM time mirror is triggered and the TM time is earlier than or equal to the TN time, the data mirror of the TN time is first restored from the bitmap under the TN directory, and then the data mirror is traced back to T0 according to the interdependence between the data mirrors to restore all data mirrors before the TM time. When the command to roll back the TM time mirror is triggered and the TM time is later than the TN time, the old data is first written back to the same level data mirror through the current directory I / O log. Then, the TN time data mirror is restored from the bitmap under the TN directory that is closest to the TM time. Then, according to the interdependence between the data mirrors, the data mirrors are traced back to T0 in sequence to restore all data mirrors before the TN time.

5. A long-term and short-term data backup and storage device based on disk CDP, characterized in that, The device includes: The short-term data directory generation module is used to create a temporary current directory, and to create an initial state data image, bitmap, I / O log and metadata in the current directory; The short-term data logging module receives captured real-time I / O changes. Before writing new data to disk, it performs a copy-on-write operation on the original block of the data mirror in the current directory and writes it to the sibling I / O log. Then, it overwrites the target block of the data mirror and marks the data storage status of the target block using a sibling bitmap. More specifically, this module receives captured real-time I / O changes; before overwriting the target block of the data mirror in the current directory, it copies the original data of the target block using a copy-on-write mechanism and immediately writes the copy as a reverse record to the sibling I / O log. The reverse record data includes: data block size, block number, timestamp, offset within the mirror, and checksum; after writing the I / O log to disk, it overwrites the real-time I / O changes to the target block of the data mirror; and it marks whether the target block stores data using a sibling bitmap. If the target block contains valid data written to this level, it is marked as 1; otherwise, it is marked as 0. The long-term data generation module is used to modify the directory name of the current directory and delete the I / O log when the command to generate full data at time T0 is triggered, so as to obtain the T0 directory and its files. The subsequent long-term data generation module repeats the above three modules sequentially, and obtains directories and files from T1 to TN according to the subsequently triggered incremental data generation instructions. The data images under directories T0 to TN are interdependent and together form an image completely identical to the original disk. More specifically, this module repeats the short-term data directory generation step, the short-term data recording step, and the long-term data generation step sequentially. In the long-term data generation step, according to the subsequently triggered incremental data generation instructions from time T1 to time TN, directories and files from T1 to TN are obtained sequentially. Each of the T1 to TN directories contains only one incremental data image. After the incremental data image is generated, the data image at time T(N-1) is used as the backup image to form a parent-child dependency chain. The bitmap is set by block, bit=1 indicates that the block has been written and is valid at this level; bit=0 indicates a hole, which needs to be traced back to the parent image until T0. The metadata records include: parent and child UUIDs, data block size, and checksum. The real-time data storage module is used to sequentially execute the short-term data directory generation module and the short-term data recording module to obtain the current directory and its files.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the long-term and short-term data backup and preservation method based on disk CDP as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the long-term and short-term data backup and preservation method based on disk CDP as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the long-term and short-term data backup and preservation method based on disk CDP as described in any one of claims 1 to 4.

Images