Method, apparatus and computer program product for data replication

Abstract

The invention relates to a method, equipment and a product for data replication. The method comprises the following steps: in response to a received data replication request, obtaining compressibility information of a data block of a source end; the method further includes determining a compressibility flag for the data block based on the compressibility information. In addition, the method also includes copying the data block and a compressibility flag to the target end through the transmission path, where the compressibility flag indicates whether the data block can be compressed at the target end. In this way, the transmission delay in the data copying process can be reduced, and the throughput of the data can be improved. Meanwhile, the risk that data cannot be recovered due to long-time transmission delay can be reduced. Besides, the method that compression attempts are no longer carried out on incompressible data blocks can reduce unnecessary calculation in the copying process, system resources can be efficiently utilized, and the user experience feeling is improved.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more specifically to methods, apparatus and computer program products for data replication. Background Technology

[0002] Data replication technology aims to ensure data reliability and availability by copying data from one storage location to one or more other locations. Data replication can be synchronous or asynchronous. Data compression technology is a technique that transforms raw data into a more compact, smaller format. Data compression technology is widely used in data storage, transmission, and processing.

[0003] Data replication and data compression technologies are related in data processing and storage. In some cases, data replication may need to be combined with data compression to reduce the amount of data transferred during the replication process. Simultaneously, data compression can reduce storage space usage, thereby providing more storage resources and backup space for data replication. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, and computer program product for data replication.

[0005] In a first aspect of the embodiments of this disclosure, a method for data copying is provided. The method includes, in response to receiving a data copying request, acquiring compressibility information of a data block at a source end. The method further includes, based on the compressibility information, determining a compressibility flag for the data block. Furthermore, the method includes, copying the data block and the compressibility flag to a target end via a transmission path, wherein the compressibility flag indicates whether the data block is compressible at the target end.

[0006] In a second aspect of embodiments of this disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement a method for data copying. The method includes, in response to receiving a data copying request, acquiring compressibility information of a data block at a source end. The method further includes determining a compressibility flag of the data block based on the compressibility information. Furthermore, the method includes copying the data block and the compressibility flag to a target end via a transmission path, wherein the compressibility flag indicates whether the data block is compressible at the target end.

[0007] In a third aspect of embodiments of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements a method for data copying. The method includes, in response to receiving a data copying request, acquiring compressibility information of a data block at a source end. The method further includes determining a compressibility flag of the data block based on the compressibility information. Furthermore, the method includes copying the data block and the compressibility flag to a target end via a transmission path, wherein the compressibility flag indicates whether the data block is compressible at the target end.

[0008] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which several embodiments of the present disclosure may be implemented is shown;

[0011] Figure 2 A flowchart of a method for data copying according to some embodiments of the present disclosure is shown;

[0012] Figures 3A-3C A schematic diagram illustrating a scenario for bypassing online compression functionality in a write storage medium, according to some embodiments of the present disclosure;

[0013] Figures 4A-4B A schematic diagram illustrating a method for synchronously copying data according to some embodiments of the present disclosure is shown;

[0014] Figures 5A-5B A schematic diagram of asynchronous data replication according to some embodiments of the present disclosure is shown;

[0015] Figure 5C Schematic diagrams are shown for transmitting bitmaps and data blocks of compressible tags during copying, according to some embodiments of the present disclosure; and

[0016] Figure 6 A block diagram of a device that can implement several embodiments of the present disclosure is shown. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] In storage systems, data compression is a key technology for improving data replication and storage efficiency. By applying compression algorithms, the size of data blocks can be reduced, thereby saving storage space and improving transmission efficiency. However, for already compressed data blocks or when the data block source is a "high-entropy file" (high entropy means that the data content has high randomness or unpredictability, and therefore may not be suitable for compression), applying compression techniques again is often ineffective, because these data blocks can hardly save any more space when compressed further. In related technologies, when data blocks are copied from the source to the destination, these data blocks pass through the data input / output (I / O) path of the destination. If compression algorithms are blindly applied without prior checks on the data blocks, the central processing unit (CPU) and memory resources of the destination may be wasted on invalid compression attempts, because these already compressed or encrypted data blocks cannot be significantly compressed further. In addition, invalid compression attempts not only waste computing resources but may also lead to increased I / O response time.

[0020] Therefore, embodiments of this disclosure propose a scheme for data replication. In these embodiments, compressibility tags are attached to data blocks to be replicated at the source end, and the data blocks and corresponding compressibility tags are passed to the target end during the data replication process. This allows the target end to avoid unnecessary data compression attempts as much as possible based on the compressibility tags, thereby reducing IO latency and improving IO throughput. Simultaneously, it avoids invalid compression of incompressible data blocks, reducing computational workload and allowing for more optimized allocation of system resources. Furthermore, for storage systems that enable advanced replication (such as fan-out or cascaded topologies), compressibility tags are passed to the target end whenever data is read from the source end. The target end can recognize these tags, thus skipping compression attempts on incompressible data blocks. This method ensures efficient utilization of resources throughout the replication chain, improving the user experience.

[0021] Figure 1 A schematic diagram of an example environment 100 in which various embodiments of this disclosure may be implemented is shown. For example... Figure 1 As shown, host 110 connects to the Logical Unit Number (LUN) pool 121 via a network to the storage system 120 to enable data addressing and access. Host 110 can be a physical server or host, or a hardware device running virtual machines and other network services. The Logical Unit Number pool 121 manages storage logical units, provides data storage space, and can contain a collection of multiple physical storage devices (such as hard drives). A Logical Unit Number (LUN) is used to identify a logical unit within a storage device. In a Storage Area Network (SAN) environment, host 110 connects to the storage system 120 via the network and uses the LUN as an identifier to access specific logical units on the storage device.

[0022] refer to Figure 1 Through a network connection, the host 110 can send requests to the storage system 120 to access the LUN, thereby reading and writing the data stored therein. This connection method enables the host 110 to flexibly access and manage the data in the storage system 120, improving the efficiency and reliability of data access.

[0023] like Figure 1As shown, a storage system controller 127 is also configured in the storage system 120. The controller 127 is the core component of the storage system 120, responsible for managing and controlling the operation of the entire storage system. Internally, the tiered driver / replicator 122 serves as the entry point for the controller 127, allowing it to read actual data blocks from the source and send them to the target. The tiered driver / replicator 122 enables tiered management to optimize data storage performance and can also be used for data backup and disaster recovery, thereby ensuring data integrity and recoverability. The controller 127 is internally configured with a file / snapshot 123, used to capture the state or image of data at a specific point in time. The file / snapshot 123 can also periodically create snapshots of the data for data recovery when necessary.

[0024] Continue to refer to Figure 1 In the storage system 120, an inline compression (ILC) 126 is configured, which can perform compression calculations in real time. When a write I / O request from the host 110 enters the general block file system container layer 124, the data block is kept in the data log of the persistent file data cache 125 so that compression calculations can be performed before being flushed to the backend. In some embodiments, the inline compression 126 can be implemented in real time via hardware or software.

[0025] Continue to refer to Figure 1 In the storage system 120, a storage pool 128 for storing data blocks is also configured. The storage pool 128 can be composed of HDDs, SSDs, or a mixture of different types of storage media. The storage pool 128 is used to store and manage a large number of data blocks, which may contain various types of information, such as documents, images, videos, and database records. In some embodiments, the storage system 120 can be located at the source end or the destination end. In some embodiments, multiple storage systems 120 can be constructed into a large storage system.

[0026] Figure 2 A flowchart of a method 200 for data copying according to some embodiments of the present disclosure is shown. (Reference) Figure 1The source storage system 120 shown can serve as the execution entity of method 200. In block 202, in response to receiving a data copy request, compressibility information of the data blocks from the source is obtained. In some embodiments, the compressibility information of the data blocks may be the file extension information or the compression ratio information of the data blocks. In some embodiments, the compressibility information of the data blocks can be obtained from the host or application layer at the source, for example, the file extension of the data block file, such as .rar, .zip, .tgz, etc. In some embodiments, if some data blocks are encrypted data blocks, then these data blocks can be considered uncompressible. In some embodiments, if some data blocks are random data blocks, then these data blocks can be considered uncompressible. In some embodiments, after compression ratio detection, it is found that these data blocks cannot be compressed again. In some embodiments, the compressibility information of the data blocks can be obtained from a compression mapping table storing the compressibility information of the data blocks.

[0027] In box 204, a compressibility flag for a data block is determined based on compressibility information. In some embodiments, if the data block's extension is .rar, .zip, .tgz, etc., then the data block's compressibility flag is considered incompressible. In some embodiments, if a data block returns incompressible after a compression ratio check, then the data block's compressibility flag is also incompressible. In some embodiments, random data blocks can be marked as incompressible. In some embodiments, if some data blocks are compressible, then the compressibility flags for these data blocks are compressible. In some embodiments, compressible data blocks can be marked with binary symbols, such as 1 indicating incompressible and 0 indicating compressible. In some embodiments, these compressibility flags can be recorded in a bitmap.

[0028] In box 206, a data block and a compressibility marker are copied to the target end via a transmission path, wherein the compressibility marker indicates whether the data block is compressible at the target end. In some embodiments, the data block to be copied and the compressibility marker can be copied to the target end. In some embodiments, data blocks marked as incompressible will not be re-attempted for compression at the target end.

[0029] By attaching compressibility tags to data blocks in this way, the target end can be effectively reminded not to perform invalid compression on incompressible data blocks, reducing redundant calculations during the copying process, thereby reducing transmission latency during data copying and improving data processing capabilities. At the same time, it also reduces the risk of data loss due to long transmission delays, achieving efficient allocation of system resources and thus optimizing the user experience.

[0030] Figures 3A-3CA schematic diagram is shown illustrating a scenario for bypassing online compression functionality in a write storage medium, according to some embodiments of the present disclosure. Figure 3A The illustration shows schematic diagrams of some embodiments of this disclosure that bypass the online compression function 300A based on the type of data block obtained from the data application layer. For example... Figure 3A As shown, the storage system determines from the data application layer that the data block is an incompressible file (a data file with high randomness or unpredictability), such as a .zip file. The data application layer can mark the IO request packet as incompressible. The storage system can use this incompressible mark to determine that the file is unlikely to be further compressed. Therefore, the storage system can bypass online compression 312A and directly store the data 314A in its original format on the storage medium. This saves CPU and memory resources, as attempting to compress an already compressed file is usually ineffective. In some embodiments, the information that the data block is incompressible can be updated and recorded in the compression mapping table 316A.

[0031] The following will combine Figure 3B This is a schematic diagram illustrating some embodiments of the present disclosure where online compression is not performed after an invalid online compression attempt. For example... Figure 3B As shown, if the storage system does not obtain a flag indicating whether a data block is compressible from the data application layer, the data block will be sent to online compression 312B for compression efficiency checking. Online compression 312B attempts to compress these data blocks, but finds that the size of the compressed data blocks has not been significantly reduced, and may even have increased. In this case, the storage system will determine that these data blocks are incompressible, thus skipping online compression 312B, directly writing the data 314B to the storage medium, and updating the compression mapping table 316B.

[0032] Continue to refer to Figure 3B In some embodiments, if the compressed data size (e.g., including the compressed zip header and compressed data blocks) can be filled within 16 sectors, the compressed data will be flushed to the storage medium. Otherwise, the original raw data blocks will be flushed, and all resources allocated for the compression check operation will be released. In some embodiments, before the data blocks are transmitted to the storage layer such as the storage medium, a preliminary compression check can be performed at the protocol layer. If the protocol layer determines that the data is already in a compressed format or is incompressible, the storage system will mark the data as incompressible, thereby bypassing online compression 312B. For example, when transmitting an encrypted data file over a network, the protocol layer can detect the high randomness of the data file, thereby determining that the encrypted data file is not suitable for further compression.

[0033] The following will combine Figure 3CThis diagram illustrates some embodiments of the present disclosure that bypass online compression 300C under resource constraints. (See reference...) Figure 3C In some embodiments, the online compression function 312C is temporarily disabled for some reason (such as system resource constraints or manual pause). During this period, all data entering the storage system, regardless of its compressibility, will not be processed by online compression 312C. This data 314C will be written directly to the storage medium in its original format. Even if some of this data could be compressed to save space, this compression process will not occur because online compression 312C is paused. In some embodiments, after data 314C is written to the storage medium, the compressibility information of that data 314C can be updated in the compression mapping table 316C.

[0034] By bypassing incompressible data blocks through online compression and distinguishing between compressible and incompressible data blocks, data processing efficiency can be improved while compressing truly compressible data blocks, saving storage space. Furthermore, by differentiating between compressible and incompressible data, the system can optimize resource allocation, thereby enhancing the user experience.

[0035] Figures 4A-4B A schematic diagram of a method for synchronously copying data according to some embodiments of the present disclosure is shown. The following will be combined with... Figure 4A Schematic diagrams illustrating some embodiments of this disclosure for synchronously copying data 400A. (See reference...) Figure 4A The storage system's synchronous replication uses a mirror view to copy data from the primary node 420A to the secondary node 430A. When a write request occurs, the mirror view splits the request to ensure that data is written to both the local primary node 420A and the remote secondary node 430A, thus ensuring data consistency. In some embodiments, the mirror view and the remote mirror driver (RMD) have the same functionality. In some embodiments, the remote mirror driver can use a fixed bitmap to track any changes to the mirror during a broken state and then transmit them to the remote site during resynchronization. In some embodiments, the mirror view or the remote mirror driver does not directly interact with the underlying components of the storage system (such as logs, caches, etc.). Instead, these underlying components are encapsulated as a logical device, providing a contiguous address space for the mirror view or remote mirror.

[0036] like Figure 4AAs shown, the application layer 410A first issues a write request via 441A, marking the beginning of data writing to the master node 420A. Upon receiving the write request, the local master mirror view 422A can correctly write the data to the master node 420A's storage medium via 442-1A, and then replicate the data to the slave mirror view 432A on the slave node 430A via 442-2A. The slave mirror view 432A also writes the data to the slave node 430A's storage medium via 442-3A. Only after both the master node 420A and the slave node 430A have successfully written the data will a confirmation message be returned via 443A. This method ensures data synchronization between the local and remote sites.

[0037] The following will combine Figure 4B This diagram illustrates how, in some embodiments of the present disclosure, compressibility tags and 400B data blocks are sent from the source to the target during synchronous replication. (See reference...) Figure 4B At the source of synchronous replication, during the data writing process, the upper-layer application 410B can distinguish the source of the I / O data packet 430B. If it detects that the I / O request comes from a highly random file or a file that has already been compressed, it can add the DO_NOT_COMPRESS flag to the request of the I / O data packet 430B. This flag will be passed to the lower layer of the storage system along with the I / O data packet 430B. This flag can instruct the storage system to bypass the online compression 414B and write the data packet directly to the storage medium without performing real-time online compression.

[0038] Continue to refer to Figure 4B In the case of synchronous data replication, the remote mirror driver 412B at the source end can split the I / O data packet 430B into I / O data packets 430-1B and I / O data packets 430-2B, etc., and add a DO_NOT_COMPRESS flag to the I / O block header. When this tagged I / O block is transmitted to the target end through a network such as Fibre Channel, the remote mirror driver 422B at the target end will read this flag and generate an I / O data packet 430B with the same flag, and then pass it to the lower-level processing at the target end. The target end storage system determines whether to perform online compression 424B based on the flag.

[0039] In this way, the system can skip the on-the-fly online compression step for certain types of I / O requests, thereby improving processing efficiency. This is particularly useful for highly random files, where further compression may not yield significant storage savings but could instead increase processing time and computational resource consumption.

[0040] Figures 5A-5B A schematic diagram of asynchronous data replication according to some embodiments of the present disclosure is shown. Figure 5A Schematic diagrams of some embodiments of this disclosure for asynchronous data replication 500A are shown. Asynchronous replication utilizes snapshots to replicate point-in-time consistent copies of multiple volumes. Asynchronous replication does not require the target to be a mirror image of the source on every I / O, but only that the target periodically synchronizes with the source on a Recovery Point Objective (RPO). The Recovery Point Objective is the acceptable amount of data that can be lost due to interruption, measured in time. This time difference affects the amount of data that must be replicated during the next synchronization.

[0041] like Figure 5A As shown, some data has accumulated in storage cell 510A of storage device A, and snapshot 1530A also records the data state of storage device A at the time of initialization. Storage device B can refresh its data to the same state as the data in storage device A based on snapshot 1530A. At this time, the data between storage device A and storage device B has been successfully synchronized (which can be called the "IN_SYNC" state), and their data is completely consistent. Storage device A and storage device B have a common base snapshot pair 530A. In other words, the "IN_SYNC" state means that the data replication or synchronization process between the source end, such as storage device A, and the target end, such as storage device B, has been completed, and the data content on both sides matches without data loss or difference. However, due to the characteristics of asynchronous replication, this state may only be temporary, because the data on the source end storage device A may continue to change, while the data on the target end storage device B will be updated in the next synchronization cycle.

[0042] Continue to refer to Figure 5A Storage devices A and B can use snapshots to achieve asynchronous replication. After the initial synchronization is complete, after a certain period, some data accumulates on storage device A. Storage device A can create a data snapshot 2 540A after a certain period. The source storage device A will determine the snapshot difference (the data difference between snapshots 530 and 540) at 512A and send it to storage device B. Storage device B can apply the snapshot difference at 522A to incrementally import the difference into the storage device. Specifically, combined with Figure 5B The asynchronous replication engine 510B in storage device A, the source end, can read the actual data from storage device A based on the snapshot difference and send it to the target end, storage device B. After the data in storage unit 520A in storage device B is updated, snapshot 2 540A in storage device B will also be consistent with snapshot 2 540A in storage device A. In some embodiments, during the process of determining the snapshot difference in 512A, the difference can be periodically updated based on the snapshot by enumerating the differences in metadata between the common base snapshot and the latest snapshot. Then, the asynchronous replication engine 510B will read the actual data blocks from the source end based on the result and send them to the target end.

[0043] In this way, after initial synchronization, the storage system can update data by comparing the differences between the base snapshot and the latest snapshot, which improves the efficiency of data transfer.

[0044] The following will combine Figure 5B This describes the process of sending compressibility tags and data blocks from the source to the target during asynchronous replication in some embodiments of this disclosure. Figure 5B A schematic diagram illustrating the transmission of compressibility tags and data blocks 500B from a source to a target during asynchronous replication according to some embodiments of the present disclosure is shown. The source replication engine 510B reads data 514B from the underlying layer and transmits it to the target replication engine 520B. The target replication engine 520B then stores the read data in the target storage medium. During this process, the target replication engine at the source can simultaneously obtain information about the compression tags in the compression mapping table 516B while reading data 514B. In some embodiments, the compressibility tags of the data blocks have already been recorded in the compression mapping table 516B through online compression 512B when the data is written to the source storage medium. The asynchronous replication engine 510B at the source can fill the I / O block header with the information from the compression mapping table in the I / O request and send it to the target replication engine 520B. The target-side replication engine 520B can read these tags and determine whether to bypass online compression 522B, then store the data in the target-side storage medium 524B, and also update the target-side compression mapping table 526B. In some embodiments, descriptive information about whether a data block can be compressed is stored in the compression mapping table, and the compressibility of the data block can be determined by consulting the compression mapping table.

[0045] Combination Figure 5C , Figure 5C A schematic diagram of a data copying process 500C involving the transfer of compressible tags via bitmaps is shown in some embodiments of this disclosure. Reference Figure 5B and 5C In 510C, the source-side copy 510B reads data block 514B from the underlying layer and makes corresponding markings in the bitmap in 520C based on the compressibility information of the data block in the compression mapping table 516B. Combined with... Figure 5BIn the storage engine, the bitmap is presented in the form of 520C. Then, in 530C, the list of data blocks is combined into an I / O request, and a new bitmap (e.g., an 8-byte bitmap) is attached to indicate the compressibility information of each data block. This is then sent to the Fibre Channel 530B in 540C. Although adding the bitmap incurs some overhead (each I / O request header requires an additional 8 bytes to store this bitmap), the benefit is that it avoids unnecessary compression efficiency checks at the target end, thus improving overall data processing efficiency. For the backend storage, it only needs 1 bit to represent the compressibility of each data block, so this overhead is relatively small.

[0046] Continue to refer to Figure 5B and Figure 5C In 550C, after the target-side replication engine 520B receives an I / O request, in 560C, it will calculate the compressibility flag of each data block based on the bitmap in the I / O request header, and should remap it to an "incompressible" flag ("incompressible" represents an instruction) for use by the lower-level online compression 522B. These data blocks 524B will be stored in the target-side storage medium.

[0047] Continue to refer to Figure 5B When incompressible data blocks enter the target storage system, online compression (522B) is bypassed. No compression efficiency checks are performed to avoid invalid compression attempts and improve performance, because if a data block is determined to be incompressible at the source, it is also considered incompressible at the target. Figure 5C In 570C, compressibility information can also be stored in the compression mapping table 526B at the target end so that future replication sessions can continue to use this information. For example, this compressibility information can also be used for resynchronization after a replication session failover, and for cascading replication to the target end of the target.

[0048] Continue to refer to Figure 5C In some embodiments, the data copying process 500C, which transmits compressible tags via bitmaps, is suitable for the initialization phase of synchronous copying or for re-copying after an interruption. In some embodiments, the data copying process 500C, which transmits compressible tags via bitmaps, is suitable for the initialization phase of asynchronous copying and for cascading asynchronous copying processes. In some embodiments, synchronous copying operates through a mirror view or a remote mirror driver, while asynchronous copying operates through a copying engine. Figure 4AIn some embodiments, the initialization transfer phase of synchronous replication (unlike the IN_SYNC phase, the data source in this phase is not the application layer 410A, but the storage unit on the source end) can transmit compressibility information through the same working principle as asynchronous replication, thereby avoiding invalid compression attempts on the target end and thus improving system performance.

[0049] This approach reduces transmission latency and increases data throughput during data replication. It also reduces the risk of data loss due to prolonged transmission delays. Furthermore, by eliminating compression attempts on incompressible data blocks at the target end, unnecessary computation during replication is reduced, leading to more efficient use of system resources and an improved user experience.

[0050] Figure 6 A schematic block diagram of an example device 600 that can be used to implement embodiments of the present disclosure is shown. As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 602 or loaded from storage unit 608 into random access memory (RAM) 603. Various programs and data required for the operation of device 600 may also be stored in RAM 603. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0051] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0052] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as method 300. For example, in some embodiments, method 300 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of method 300 described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform method 300 by any other suitable means (e.g., by means of firmware).

[0053] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0054] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0055] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0056] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for data replication, comprising: In response to receiving a data copy request, obtain the compressibility information of the data blocks from the source end; Based on the compressibility information, the compressibility marker of the data block is determined; as well as The data block and the compressibility flag are copied to the target end via a transmission path, the compressibility flag indicating whether the data block can be compressed at the target end.

2. The method according to claim 1, wherein obtaining compressibility information of the source data block in response to receiving a data copy request includes: The compressibility information of the data block is obtained from the first layer of the data application layer at the source end; or The compressibility information of the data block is obtained from the compression mapping table of the second layer at the source end.

3. The method according to claim 2, wherein obtaining the compressibility information of the data block from the first layer of the data application layer at the source end includes: The source type of the data block is obtained from the data application layer of the first layer at the source end; as well as Based on the source type of the data block, the compressibility information of the data block is determined, wherein the source type indicates whether the data block can be compressed.

4. The method of claim 3, wherein determining the compressibility marker of the data block based on the compressibility information comprises: In response to the source type indicating that the data block is incompressible, an incompressible flag indicating that the data block is incompressible at the destination is added to the first-layer transport request packet.

5. The method of claim 1, wherein copying the data block and the compressibility tag to the target end via a transmission path comprises: Based on the remote mirror driver of the source end, the transmission request data packet is sent to the target end, and the transmission request data packet includes at least the data block and the compressibility tag.

6. The method of claim 2, wherein obtaining the compressibility information of the data block from the compression mapping table of the second layer at the source end comprises: The compressibility information of the data block is recorded in the compression mapping table of the second layer at the source end.

7. The method of claim 6, wherein recording the compressibility information of the data block into the compression mapping table of the second layer at the source end comprises: Based on the compression ratio of the data block, the compressibility information of the data block is determined; Based on the fact that the online compression function has been stopped, the compressibility information of the data block is determined; or Based on protocol-level compression, the compressibility information of the data block is determined, wherein the protocol compression is used to compress the data block before sending the data block to the network.

8. The method of claim 7, wherein determining the compressibility marker of the data block based on the compressibility information comprises: In response to the data block being compressible, the data block is marked as compressible in the bitmap corresponding to the data block; as well as In response to the fact that the data block is incompressible, the data block is marked as incompressible in the bitmap.

9. The method of claim 8, wherein copying the data block and the compressibility tag to the target end via a transmission path comprises: The bitmap corresponding to the data block at the source end is converted into the compressible tag of the data block; as well as Based on the replication engine at the source end, the transmission request data packet is copied to the target end through the transmission path, and the transmission request includes at least the compressible tag and the data block.

10. The method of claim 9, further comprising: In response to the asynchronous replication method used for the data replication, the data block and the compressibility tag are copied to the target end based on the compression mapping table.

11. An electronic device, comprising: At least one processor; as well as Coupled to the at least one processor and having instructions stored thereon, the instructions, when executed by the at least one processor, cause the electronic device to perform actions, the actions including: In response to receiving a data copy request, obtain the compressibility information of the data blocks from the source end; Based on the compressibility information, determine the compressibility flag of the data block; and The data block and the compressibility flag are copied to the target end via a transmission path, the compressibility flag indicating whether the data block can be compressed at the target end.

12. The electronic device of claim 11, wherein obtaining compressibility information of the data block from the source end in response to receiving a data copy request includes: The compressibility information of the data block is obtained from the first layer of the data application layer at the source end; or The compressibility information of the data block is obtained from the compression mapping table of the second layer at the source end.

13. The electronic device of claim 12, wherein obtaining the compressibility information of the data block from the first layer of the data application layer at the source end comprises: The source type of the data block is obtained from the data application layer of the first layer at the source end; as well as Based on the source type of the data block, the compressibility information of the data block is determined, wherein the source type indicates whether the data block can be compressed.

14. The electronic device of claim 13, wherein determining the compressibility marker of the data block based on the compressibility information comprises: In response to the source type indicating that the data block is incompressible, an incompressible flag indicating that the data block is incompressible at the destination is added to the first-layer transport request packet.

15. The electronic device of claim 11, wherein copying the data block and the compressibility marker to the target end via a transmission path comprises: Based on the remote mirror driver of the source end, the transmission request data packet is sent to the target end, and the transmission request data packet includes at least the data block and the compressibility tag.

16. The electronic device of claim 12, wherein obtaining the compressibility information of the data block from the compression mapping table of the second layer at the source end comprises: The compressibility information of the data block is recorded in the compression mapping table of the second layer at the source end.

17. The electronic device of claim 16, wherein recording the compressibility information of the data block in the compression mapping table of the second layer at the source end comprises: Based on the compression ratio of the data block, the compressibility information of the data block is determined; Based on the fact that the online compression function has been stopped, the compressibility information of the data block is determined; or Based on protocol-level compression, the compressibility information of the data block is determined, wherein the protocol compression is used to compress the data block before sending the data block to the network.

18. The electronic device of claim 17, wherein determining the compressibility marker of the data block based on the compressibility information comprises: In response to the data block being compressible, the data block is marked as compressible in the bitmap corresponding to the data block; as well as In response to the fact that the data block is incompressible, the data block is marked as incompressible in the bitmap.

19. The electronic device of claim 18, wherein copying the data block and the compressibility tag to the target end via a transmission path comprises: The bitmap corresponding to the data block at the source end is converted into the compressible tag of the data block; as well as Based on the replication engine at the source end, the transmission request data packet is copied to the target end through the transmission path, and the transmission request includes at least the compressible tag and the data block.

20. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform actions, the actions comprising: In response to receiving a data copy request, obtain the compressibility information of the data blocks from the source end; Based on the compressibility information, the compressibility marker of the data block is determined; as well as The data block and the compressibility flag are copied to the target end via a transmission path, the compressibility flag indicating whether the data block can be compressed at the target end.

Images