Data synchronization method and device, electronic equipment, storage medium and computer program product

Abstract

The invention relates to a data synchronization method and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps that target data generated by a target application program is sent to a first virtual disk and a first transmission module through a first data synchronization module; sending the target data to a second device through a first transmission module; writing the target data into a first specified file block contained in the first physical disk through the first virtual disk by utilizing the first loopback equipment; and determining the synchronization state of the target data through the first data synchronization module. Therefore, by using the loopback equipment, the whole physical disk does not need to be mounted, but a part of data space in the partitioned physical disk can be dynamically appointed as the file block to perform data synchronization, so that the data synchronization is not limited by the physical partition any more, and the data synchronization efficiency is improved. The data synchronization method provided by the invention can be suitable for various types of storage environments, so that dynamic and flexible data synchronization requirements can be met.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data synchronization, and more particularly, to a data synchronization method, apparatus, electronic device, storage medium, and computer program product. Background Art

[0002] In the power industry, critical systems such as SCADA and energy management systems often require dual-node operation to improve reliability. Specifically, if software or hardware failures occur on the primary node, the backup node can continue providing services, ensuring uninterrupted business operations. In this case, data synchronization between the primary and backup nodes is necessary.

[0003] In the related art, there are mainly two methods of data synchronization. The first method is to synchronize data through a write-ahead log. For example, when writing data to the database, the data record is also written to the write-ahead log. After the log is synchronized from machine A to machine B, the application on machine B needs to reverse parse the write-ahead log and then write the parsing results to the database of machine B to achieve data synchronization. However, this data synchronization method cannot make the application whose data is synchronized unaware. In other words, the application whose data is synchronized must participate in it, which will result in a high degree of intrusion into the application.

[0004] Another data synchronization method is block-level synchronization, but this requires the use of a separate disk partition. Specifically, a separate disk partition must be created for the application whose data is being synchronized. However, since the operating system may have been in use for a while, the dedicated disk partition may no longer exist. In this case, creating a dedicated disk partition for the application to synchronize data requires reinstalling the operating system or adding a new disk, a time-consuming and labor-intensive process that also incurs additional costs. It can be seen that the data synchronization methods in related technologies are either highly invasive to applications or have usage restrictions due to disk partitioning, and cannot meet the dynamic and flexible data synchronization requirements. Summary of the Invention

[0005] The present disclosure provides a data synchronization method, apparatus, electronic device, storage medium, and computer program product to at least address the problems in the above-mentioned related technologies, namely, that the data synchronization methods are either highly invasive to applications or are limited in use due to disk partitioning, and cannot meet the dynamic and flexible data synchronization requirements.

[0006] According to the first aspect of the embodiments of the present disclosure, a data synchronization method is provided, which is applied to a first device. The first device includes a first data synchronization module, a first virtual disk, a first transmission module, a first loop device, and a first physical disk. The data synchronization method includes: in response to a data writing request of a target application program, sending, by the first data synchronization module, target data generated by the target application program to the first virtual disk and the first transmission module; sending, by the first transmission module, the target data to a second device; writing, by the first virtual disk using the first loop device, the target data into a first specified file block included in the first physical disk, where the first specified file block is a file block with a specified size pre-created in the first physical disk, and the file block with the specified size is mapped to the first loop device; determining, by the first data synchronization module, a synchronization state of the target data.

[0007] Optionally, the sending, by the first data synchronization module, the target data generated by the target application program to the first virtual disk and the first transmission module includes: calling, by the first data synchronization module, a preset function to split the target data into multiple data blocks, and marking a state of each data block in the multiple data blocks as an unsynchronized state; sending, by the first data synchronization module, the multiple data blocks to the first transmission module, and sending the multiple data blocks and their states to the first virtual disk; the determining, by the first data synchronization module, the synchronization state of the target data includes: determining, by the first data synchronization module, a synchronization state of the multiple data blocks.

[0008] Optionally, the first loop device includes a metadata partition and a real data partition; the writing, by the first virtual disk using the first loop device, the target data into the specified file block included in the first physical disk includes: writing, by the first virtual disk, the multiple data blocks into the real data partition, and writing a state of each data block into the metadata partition; writing the multiple data blocks to be written into the real data partition into the specified file block.

[0009] Optionally, after determining, by the first data synchronization module, the synchronization state of the multiple data blocks, the data synchronization method further includes: for each data block in the multiple data blocks, when determining that the data block is successfully synchronized, sending a synchronization success indication to the first loop device; modifying, by the first loop device based on the synchronization success indication, a state of the data block from the unsynchronized state to a synchronized state.

[0010] Optionally, determining the synchronization status of the multiple data blocks by the first data synchronization module includes: for each of the multiple data blocks, when the first data synchronization module receives a first write confirmation and a second write confirmation for the data block, determining that the data block is successfully synchronized, where the first write confirmation is the write confirmation for the data block by the first transmission module, and the second write confirmation is the write confirmation for the data block by the first physical disk.

[0011] Optionally, determining the synchronization status of the multiple data blocks by the first data synchronization module includes: for each of the multiple data blocks, when the first data synchronization module receives a third write confirmation for the data block, determining that the data block is successfully synchronized, where the third write confirmation is the write confirmation for the data block by the first physical disk.

[0012] Optionally, the data synchronization method further includes: registering the first virtual disk to the first data synchronization module, and mounting the first data synchronization module on the kernel of the first device to register the first virtual disk to the operating system of the first device; mounting the first virtual disk under the target directory of the operating system; when it is detected that the target application writes data into the target directory, determining that the data write request is detected.

[0013] Optionally, the second device includes a second data synchronization module, a second virtual disk, a second transmission module, a second loop device, and a second physical disk; the second transmission module is configured to receive the target data sent by the first transmission module and send the target data to the second data synchronization module; the second data synchronization module is configured to send the target data to the second virtual disk; the second virtual disk is configured to use the second loop device to write the target data into a second specified file block included in the second physical disk, where the second specified file block is a file block with a specified size pre-created in the second physical disk, and the file block with the specified size is mapped to the second loop device.

[0014] Optionally, sending the target data to the second device by the first transmission module includes: sending the target data to the second device by the first transmission module using the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP).

[0015] According to a second aspect of the embodiments of the present disclosure, a data synchronization device is provided, which is applied to a first device. The first device includes a first data synchronization module, a first virtual disk, a first transmission module, a first loop device, and a first physical disk. The data synchronization device includes: a data sending module configured to, in response to a data writing request of a target application program, send target data generated by the target application program to the first virtual disk and the first transmission module through the first data synchronization module; a data transmission module configured to send the target data to a second device through the first transmission module; a data writing module configured to write the target data into a first specified file block included in the first physical disk by using the first loop device through the first virtual disk, where the first specified file block is a file block with a specified size pre-created in the first physical disk, and the file block with the specified size is mapped to the first loop device; a data status determination module configured to determine the synchronization status of the target data through the first data synchronization module.

[0016] Optionally, the data sending module is configured to: split the target data into multiple data blocks by calling a preset function through the first data synchronization module, and mark the status of each data block in the multiple data blocks as an unsynchronized status; send the multiple data blocks to the first transmission module through the first data synchronization module, and send the multiple data blocks and their statuses to the first virtual disk; the data status determination module is configured to: determine the synchronization status of the multiple data blocks through the first data synchronization module. Optionally, the first loop device includes a metadata partition and a real data partition; the data writing module is configured to: write the multiple data blocks into the real data partition through the first virtual disk, and write the status of each data block into the metadata partition; write the multiple data blocks written into the real data partition into the specified file block.

[0017] Optionally, the data synchronization device further includes: an indication sending module configured to, for each data block in the multiple data blocks, send a synchronization success indication to the first loop device when it is determined that the data block is successfully synchronized; a status modification module configured to modify the status of the data block from the unsynchronized status to the synchronized status based on the synchronization success indication through the first loop device.

[0018] Optionally, the data status determination module is configured to: for each of the multiple data blocks, determine that the data block is successfully synchronized when the first data synchronization module receives a first write confirmation and a second write confirmation for the data block, where the first write confirmation is the write confirmation of the first transmission module for the data block, and the second write confirmation is the write confirmation of the first physical disk for the data block.

[0019] Optionally, the data status determination module is configured to: for each of the multiple data blocks, determine that the data block is successfully synchronized when the first data synchronization module receives a third write confirmation for the data block, where the third write confirmation is the write confirmation of the first physical disk for the data block.

[0020] Optionally, the data synchronization device further includes: a registration module configured to register the first virtual disk to the first data synchronization module and mount the first data synchronization module on the kernel of the first device to register the first virtual disk to the operating system of the first device; a mounting module configured to mount the first virtual disk under a target directory of the operating system; and a request detection module configured to determine that a data write request is detected when it is detected that the target application writes data into the target directory.

[0021] Optionally, the second device includes a second data synchronization module, a second virtual disk, a second transmission module, a second loop device, and a second physical disk; the second transmission module is configured to receive the target data sent by the first transmission module and send the target data to the second data synchronization module; the second data synchronization module is configured to send the target data to the second virtual disk; the second virtual disk is configured to write the target data into a second specified file block included in the second physical disk by using the second loop device, where the second specified file block is a file block with a specified size pre-created in the second physical disk, and the file block with the specified size is mapped to the second loop device.

[0022] Optionally, the data transmission module is configured to: send the target data to the second device through the first transmission module by using the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP).

[0023] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including: a processor; and a memory for storing executable instructions of the processor; where the processor is configured to execute the instructions to implement the data synchronization method according to the present disclosure.

[0024] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data synchronization method according to the present disclosure.

[0025] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, including a computer program which, when executed by a processor, implements the data synchronization method according to the present disclosure.

[0026] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects: In the present disclosure, when performing data synchronization, by using a loop device, it is possible to achieve not mounting the entire physical disk, but rather dynamically specifying a part of the data space in a physically partitioned disk as a file block for data synchronization, so that data synchronization is no longer restricted by physical partitions. That is, the data synchronization method provided by the present disclosure can be applied to various types of storage environments, thereby meeting dynamic and flexible data synchronization requirements. Moreover, by performing data synchronization between two machines through file block mapping, the application programs of the data to be synchronized can be unaware. That is, the data synchronization method provided by the present disclosure has good transparency to upper-layer application programs, thereby avoiding excessive intrusion of data synchronization into application programs.

[0027] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0029] Figure 1 is a flowchart showing a data synchronization method according to an exemplary embodiment of the present disclosure; Figure 2 is a schematic diagram showing the correspondence between a first virtual disk and a first specified file block according to an exemplary embodiment of the present disclosure; Figure 3 is a specific implementation flowchart showing a data synchronization method according to an exemplary embodiment of the present disclosure; Figure 4 is a block diagram showing a data synchronization device according to an exemplary embodiment of the present disclosure; Figure 5 is a block diagram showing an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] To enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0032] It should be noted here that "at least one of several items" in the present disclosure all represents the inclusion of the following three parallel situations: "any one of the several items", "a combination of any multiple of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example, "performing at least one of step one and step two" means the following three parallel situations: (1) performing step one; (2) performing step two; (3) performing step one and step two.

[0033] Figure 1 is a flowchart showing a data synchronization method according to an exemplary embodiment of the present disclosure, which is applied to a first device. In the present disclosure, data synchronization is mainly performed by mounting code to the Linux kernel at the block device driver layer of the Linux kernel. Specifically, the data of the block device in the first device can be copied to the second device in real time by using an imaging mechanism. In addition, in the present disclosure, the first device can be referred to as machine A, and the second device can be referred to as machine B.

[0034] The above-mentioned first device may include a first data synchronization module, a first virtual disk, a first transmission module, a first loop device, and a first physical disk. Each module included in the first device will be described in detail below.

[0035] First data synchronization module: It can serve as the driver of the first virtual disk, and it can handle all IO requests, including but not limited to: operations such as opening and closing of the device, reading data, writing data, and flushing the buffer. In addition, while forwarding data to the first virtual disk, the first data synchronization module can also copy the data to the first transmission module.

[0036] The first virtual disk: It can also be referred to as the first virtual block device. It is mainly used to simulate a block device through software and write an adapted block device driver. Additionally, the calling process when it is invoked is similar to that of a real physical disk, i.e., a real physical block device.

[0037] The first transmission module: It is mainly used to manage the network communication status between two machines, including but not limited to: establishing a connection, communication negotiation, master-slave relationship confirmation, network data transmission, etc. For example, the first transmission module can establish a TCP long connection channel between two machines, and in order to improve data synchronization efficiency, it can also directly connect the network cards of the two machines in a point-to-point manner to isolate from the service network.

[0038] In addition, in addition to the above modules, the first device can also include a first block device driver. This first block device driver can provide an interface for communication between the operating system and the block device (real physical disk), and can perform read and write operations on the block device.

[0039] Refer to Figure 1 , in step 101, in response to a data write request of the target application, the target data generated by the target application can be sent to the first virtual disk and the first transmission module through the first data synchronization module.

[0040] According to an exemplary embodiment of the present disclosure, the target data can be sliced into multiple data blocks by calling a preset function through the first data synchronization module, and the status of each data block in the multiple data blocks can be marked as an unsynchronized state. Then, the multiple data blocks can be sent to the first transmission module through the first data synchronization module, and the multiple data blocks and their status can be sent to the first virtual disk. Next, the synchronization status of the multiple data blocks can also be determined through the first data synchronization module.

[0041] It should be noted that the core of the foregoing first data synchronization module is to create a first virtual disk driver to the operating system using the device driver framework provided by the Linux kernel. And this first virtual disk driver can implement standard block device operation interfaces. For example, these operation interfaces can include but not be limited to: submit_bio, open, release, etc.

[0042] submit_bio is mainly used to process read and write requests of all block devices. Specifically, the IO requests initiated by the application layer can be encapsulated into bio requests by the file system and submitted to the first virtual disk. Further, this submit_bio method can split the request into multiple 4K-sized data blocks and update the metadata of each data block.

[0043] open is mainly called when the device is opened, and it is mainly used to manage the access count and status of the device. For example, when it is called, it can increase the reference count and update the status to the writable state.

[0044] release is mainly called when the device is released, and it is mainly used to reduce the reference count, flush the outstanding IO requests to the downstream, and persist the metadata. Additionally, if the reference count has been cleared, release can update the status to the closed state and can close the first loop device.

[0045] In step 102, the target data can be sent to the second device through the first transmission module.

[0046] According to an exemplary embodiment of the present disclosure, the target data can be sent to the second device through the first transmission module using the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP). Specifically, in a scenario where data reliability is not sensitive, the first transmission module can use UDP instead of TCP, that is, it can sacrifice some reliability in exchange for higher real-time performance.

[0047] In step 103, the target data can be written into the first specified file block included in the first physical disk through the first virtual disk using the first loop device. The first specified file block can be a file block with a specified size pre-created in the first physical disk, and the file block with the specified size can be mapped to the first loop device.

[0048] It should be noted that when creating the correspondence between the first virtual disk and the first specified file block, a file block with a specified size can be created in the physical disk first: fallocate -l 1024M / data / sync.img. Then, the file block with the specified size can be mapped to the first loop device: losetup / dev / loop0 / data / sync.img. Next, the first loop device can be pointed to the first virtual disk / dev / sync / dev / loop0, and metadata can be created on the first loop device to record the device status and synchronization information. Figure 2 is a schematic diagram showing the correspondence between the first virtual disk and the first specified file block according to an exemplary embodiment of the present disclosure.

[0049] In the present disclosure, the first virtual disk is mainly used to connect the first data synchronization module and the real block device driver, and a part of the data space in the disk with partitions already divided can be dynamically specified as file blocks, and then a mapping relationship can be established between the first virtual disk and the file blocks, so that it is not necessary to mount the entire physical disk partition. Its advantages are mainly the following two points: 1. There is no need to reformat the disk for file synchronization, which can save costs; 2. When deploying multiple programs that need to synchronize data on dual machines, multiple virtual disks can be flexibly divided to separately achieve data synchronization for each program.

[0050] According to an exemplary embodiment of the present disclosure, the first loop device may include a metadata partition and a real data partition. Multiple data blocks can be written into the real data partition through the first virtual disk, and the status of each data block can be written into the metadata partition. Then, the multiple data blocks written into the real data partition can be written into the specified file blocks of the real physical disk.

[0051] It should be noted that the core function of the first virtual disk is metadata management. The metadata can be a small part of the space divided at the head of the file block, and the size of this space can be dynamically set according to the file block size. The advantage of directly using the file block to store metadata is that it no longer depends on an additional block device.

[0052] In addition, in addition to storing information such as metadata area identifier, size, version, CRC check bits, etc. in the metadata partition, most of the space is mainly used to store a bitmap with a length of the number of file blocks / data block size. Each bit in this bitmap represents a data block, and the function of this bitmap is to track which data blocks are modified on machine A but have not been synchronized to machine B. Specifically, when a data block on machine A is written, the corresponding bit in the bitmap can be marked as 1, indicating that the data block has not been synchronized; when machine B successfully receives and writes this data block, the bit corresponding to this data block in the bitmap on machine A can be marked as 0, indicating that the data block has been successfully synchronized. In this way, once a data transmission failure occurs between machine A and machine B and the failure is recovered, only the data blocks marked as 1 need to be synchronized, which can improve the data synchronization efficiency.

[0053] In step 104, the synchronization status of the target data can be determined through the first data synchronization module.

[0054] According to an exemplary embodiment of the present disclosure, for each of a plurality of data blocks, when the first data synchronization module receives a first write confirmation and a second write confirmation for the data block, it can be determined that the data block has been successfully synchronized. The first write confirmation may be a write confirmation of the first transmission module for the data block, and the second write confirmation may be a write confirmation of the first physical disk for the data block. That is, in the present disclosure, the first data synchronization module considers that the corresponding data block has been successfully synchronized only after receiving the write confirmation of the first transmission module and the write confirmation of the first physical disk at the same time. This data synchronization mode is the synchronous replication mode, and the advantage of this mode is that it can effectively ensure data consistency between the first device and the second device.

[0055] According to an exemplary embodiment of the present disclosure, for each of a plurality of data blocks, when the first data synchronization module receives a third write confirmation for the data block, it can be determined that the data block has been successfully synchronized. The third write confirmation may be a write confirmation of the first physical disk for the data block. This data synchronization mode is the asynchronous replication mode. Since this mode allows the data block to be considered successfully synchronized as long as the write confirmation of the first physical disk is received, this mode can ensure the continuous progress of the data writing process, that is, it can ensure the data writing efficiency. However, the asynchronous replication mode performs worse than the synchronous replication mode in terms of data consistency. In an actual scenario, which data synchronization mode to use can be selected based on a comprehensive consideration of the data consistency requirement and the data writing efficiency.

[0056] According to an exemplary embodiment of the present disclosure, after determining the synchronization status of a plurality of data blocks through the first data synchronization module, for each of the plurality of data blocks, when it is determined that the data block has been successfully synchronized, a synchronization success indication may also be sent to the first loopback device. Then, the first loopback device can modify the status of the data block from the unsynchronized state to the synchronized state based on the received synchronization success indication.

[0057] In this way, by modifying the status of the corresponding data block from the unsynchronized state to the synchronized state based on the synchronization success indication, it can be ensured that the status of each data block is updated in a timely manner, and thus it can be quickly distinguished which data blocks have been successfully synchronized and which data blocks have not been synchronized, facilitating effective monitoring and management of the data blocks.

[0058] According to an exemplary embodiment of the present disclosure, a first virtual disk can be registered to a first data synchronization module, and the first data synchronization module can be mounted on the kernel of a first device to register the first virtual disk to the operating system of the first device. That is, in the present disclosure, the first virtual disk can be handed over to the first data synchronization module, and the first data synchronization module can be mounted on the Linux kernel of the first device, and then the registration of the first virtual disk / dev / sync to the operating system of the first device can be achieved. Then, the first virtual disk can be mounted under the target directory of the operating system. For example, the first virtual disk can be mounted to the data directory of the operating system mount / dev / sync / data. Next, in the case where it is detected that a target application writes data into the target directory, it can be determined that a data write request is detected.

[0059] According to an exemplary embodiment of the present disclosure, a second device may include a second data synchronization module, a second virtual disk, a second transmission module, a second loop device, and a second physical disk.

[0060] The second transmission module can be used to receive the target data sent by the first transmission module and can send the target data to the second data synchronization module; the second data synchronization module can be used to send the target data to the second virtual disk.

[0061] The second virtual disk can be used to write the target data into a second specified file block included in the second physical disk by using the second loop device. The second specified file block can be a file block with a specified size pre-created in the second physical disk, and the file block with the specified size can be mapped to the second loop device.

[0062] Figure 3 It is a specific implementation flowchart showing a data synchronization method according to an exemplary embodiment of the present disclosure.

[0063] Referring to Figure 3 , when an application data write request arrives at the file system layer of machine A, no intervention is required, and the first data synchronization module can intercept all IO requests as the driver of the first virtual disk. Next, while the first data synchronization module forwards the data to the first virtual disk, it can also copy the data to the first transmission module. Then, the first virtual disk can forward the data to the driver of the real physical disk, that is, to the driver of the first physical disk, to write the data into the first physical disk.

[0064] The first transmission module can forward the data to the second transmission module of Machine B. Then, the second transmission module can send the data to the second data synchronization module of Machine B, and the second data synchronization module can forward the data to the second virtual disk of Machine B. Next, the second virtual disk of Machine B can forward the data to the driver of the real physical disk, that is, to the driver of the second physical disk, to write the data to the second physical disk of Machine B.

[0065] Figure 4 FIG. is a block diagram showing a data synchronization device 400 according to an exemplary embodiment of the present disclosure. The data synchronization device 400 can be applied to a first device, which can include a first data synchronization module, a first virtual disk, a first transmission module, a first loopback device, and a first physical disk.

[0066] Referring to Figure 4 , the data synchronization device 400 may include a data sending module 401, a data transmission module 402, a data writing module 403, and a data status determination module 404.

[0067] In response to a data writing request of a target application, the data sending module 401 can send the target data generated by the target application to the first virtual disk and the first transmission module through the first data synchronization module.

[0068] According to an exemplary embodiment of the present disclosure, the data sending module 401 can call a preset function through the first data synchronization module to split the target data into multiple data blocks, and can mark the status of each data block in the multiple data blocks as an unsynchronized status. Then, the data sending module 401 can send the multiple data blocks to the first transmission module through the first data synchronization module, and can send the multiple data blocks and their statuses to the first virtual disk. Next, the data status determination module 404 can also determine the synchronization status of the multiple data blocks through the first data synchronization module.

[0069] The data transmission module 402 can send the target data to the second device through the first transmission module.

[0070] According to an exemplary embodiment of the present disclosure, the data transmission module 402 can send the target data to the second device through the first transmission module using the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP). Specifically, in a scenario where data reliability is not sensitive, the first transmission module can use UDP instead of TCP, that is, it can sacrifice some reliability in exchange for higher real-time performance.

[0071] The data writing module 403 can write the target data into the first specified file block included in the first physical disk by using the first loop device through the first virtual disk. The first specified file block can be a file block with a specified size pre-created in the first physical disk, and this specified-size file block can be mapped to the first loop device.

[0072] According to an exemplary embodiment of the present disclosure, the first loop device may include a metadata partition and a real data partition. The data writing module 403 can write multiple data blocks into the real data partition through the first virtual disk, and can write the status of each data block into the metadata partition. Then, the data writing module 403 can write the multiple data blocks written into the real data partition into the specified file block of the real physical disk.

[0073] The data status determination module 404 can determine the synchronization status of the target data through the first data synchronization module.

[0074] According to an exemplary embodiment of the present disclosure, for each of the multiple data blocks, when the first data synchronization module receives the first write confirmation and the second write confirmation for this data block, the data status determination module 404 can determine that this data block is successfully synchronized. The first write confirmation can be the write confirmation of the first transmission module for this data block, and the second write confirmation can be the write confirmation of the first physical disk for this data block. That is, in the present disclosure, the first data synchronization module only considers that the corresponding data block is successfully synchronized after receiving the write confirmation of the first transmission module and the write confirmation of the first physical disk at the same time. This data synchronization mode is the synchronous replication mode, and the advantage of this mode is that it can effectively ensure data consistency between the first device and the second device.

[0075] According to an exemplary embodiment of the present disclosure, for each of the multiple data blocks, when the first data synchronization module receives the third write confirmation for this data block, the data status determination module 404 can determine that this data block is successfully synchronized. The third write confirmation can be the write confirmation of the first physical disk for this data block. This data synchronization mode is the asynchronous replication mode. Since this mode allows the data block to be considered successfully synchronized as long as the write confirmation of the first physical disk is received, this mode can ensure the continuous progress of the data writing process, that is, it can ensure the data writing efficiency. However, the asynchronous replication mode performs worse than the synchronous replication mode in terms of data consistency. In actual scenarios, which data synchronization mode to use can be selected based on a comprehensive consideration of data consistency requirements and data writing efficiency.

[0076] According to an exemplary embodiment of the present disclosure, the above data synchronization device 400 may further include an indication sending module and a status modification module.

[0077] After determining the synchronization status of multiple data blocks through the first data synchronization module, for each of the multiple data blocks, when it is determined that the data block has been successfully synchronized, the sending module may be instructed to further send a synchronization success indication to the first loopback device. Then, the status modification module may, based on the received synchronization success indication through the first loopback device, modify the status of the data block from the unsynchronized state to the synchronized state.

[0078] In this way, by modifying the status of the corresponding data block from the unsynchronized state to the synchronized state based on the synchronization success indication, it can be ensured that the status of each data block is updated in a timely manner, and thus it can be quickly distinguished which data blocks have been successfully synchronized and which data blocks have not been synchronized, facilitating the effective monitoring and management of the data blocks.

[0079] According to an exemplary embodiment of the present disclosure, the above data synchronization apparatus 400 may further include a registration module, a mounting module, and a request detection module.

[0080] The registration module may register the first virtual disk to the first data synchronization module, and may mount the first data synchronization module to the kernel of the first device, so as to register the first virtual disk to the operating system of the first device. That is, in the present disclosure, the first virtual disk may be handed over to the first data synchronization module, and the first data synchronization module may be mounted to the linux kernel of the first device, thereby realizing the registration of the first virtual disk / dev / sync to the operating system of the first device. Then, the mounting module may mount the first virtual disk to the target directory of the operating system. For example, the first virtual disk may be mounted to the data directory of the operating system mount / dev / sync / data. Next, when the request detection module detects that the target application writes data into the target directory, it may determine that a data write request is detected.

[0081] According to an exemplary embodiment of the present disclosure, the second device may include a second data synchronization module, a second virtual disk, a second transmission module, a second loopback device, and a second physical disk.

[0082] The second transmission module may be used to receive the target data sent by the first transmission module, and may send the target data to the second data synchronization module; the second data synchronization module may be used to send the target data to the second virtual disk.

[0083] The second virtual disk may be used to write the target data into the second specified file block included in the second physical disk by using the second loopback device. The second specified file block may be a file block with a specified size pre-created in the second physical disk, and the file block with the specified size may be mapped to the second loopback device.

[0084] Figure 5 is a block diagram showing an electronic device 500 according to an exemplary embodiment of the present disclosure.

[0085] Referring Figure 5 , the electronic device 500 includes at least one memory 501 and at least one processor 502. Instructions are stored in the at least one memory 501, and when the instructions are executed by the at least one processor 502, a data synchronization method according to an exemplary embodiment of the present disclosure is executed.

[0086] As an example, the electronic device 500 may be a PC computer, a tablet device, a personal digital assistant, a smart phone, or other devices capable of executing the above instructions. Here, the electronic device 500 does not have to be a single electronic device, and may also be any assembly of devices or circuits that can execute the above instructions (or instruction sets) alone or jointly. The electronic device 500 may also be a part of an integrated control system or a system manager, or may be a portable electronic device configured to interface with a local or remote (e.g., via wireless transmission).

[0087] In the electronic device 500, the processor 502 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0088] The processor 502 may run instructions or code stored in the memory 501, where the memory 501 may also store data. The instructions and data may also be sent and received via a network interface device over a network, where the network interface device may employ any known transmission protocol.

[0089] The memory 501 may be integrated with the processor 502, for example, by arranging RAM or flash memory within an integrated circuit microprocessor, etc. In addition, the memory 501 may include a separate device, such as an external disk drive, a storage array, or other storage devices that may be used by any database system. The memory 501 and the processor 502 may be operatively coupled or may communicate with each other, for example, via an I / O port, a network connection, etc., such that the processor 502 can read files stored in the memory.

[0090] In addition, the electronic device 500 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.). All components of the electronic device 500 may be connected to each other via a bus and / or a network.

[0091] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above data synchronization method. Examples of the computer-readable storage medium here include: read-only memory (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc memory, hard disk drive (HDD), solid state drive (SSD), cartridge memory (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and provide the computer program and any associated data, data files, and data structures to a processor or computer such that the processor or computer can execute the computer program. The computer program in the above computer-readable storage medium may run in an environment deployed in computer devices such as a client, host, proxy device, server, etc. In addition, in one example, the computer program and any associated data, data files, and data structures are distributed on a networked computer system such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0092] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including a computer program that, when executed by a processor, implements the data synchronization method according to the present disclosure.

[0093] According to the data synchronization method, device, electronic device, storage medium, and computer program product of the present disclosure, when performing data synchronization, by using a loop device, it is possible to achieve not having to mount the entire physical disk, but rather dynamically specifying a part of the data space in a physically partitioned disk that has already been partitioned as a file block for data synchronization, so that data synchronization is no longer restricted by physical partitions. That is, the data synchronization method provided by the present disclosure can be applied to various types of storage environments, thereby meeting dynamic and flexible data synchronization requirements. Moreover, by performing data synchronization between two machines through file block mapping, it is also possible to make the application programs of the data to be synchronized unaware. That is, the data synchronization method provided by the present disclosure has good transparency to upper-layer application programs, thereby avoiding excessive intrusion of data synchronization into application programs.

[0094] According to an exemplary embodiment of the present disclosure, by setting the data synchronization mode to the synchronous replication mode, it is possible to effectively ensure data consistency between the first device and the second device.

[0095] According to an exemplary embodiment of the present disclosure, by setting the data synchronization mode to the asynchronous replication mode, it is possible to ensure the continuous progress of the data writing process, that is, to ensure the data writing efficiency.

[0096] According to an exemplary embodiment of the present disclosure, by modifying the status of the corresponding data block from the unsynchronized state to the synchronized state based on the synchronization success indication, it is possible to ensure that the status of each data block is updated in a timely manner, and further quickly distinguish which data blocks have been successfully synchronized and which data blocks have not been synchronized, facilitating effective monitoring and management of the data blocks.

[0097] Those skilled in the art will readily conceive of other implementations of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and embodiments are only to be considered exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0098] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A data synchronization method, applied to a first device, characterized in that, The first device includes a first data synchronization module, a first virtual disk, a first transmission module, a first loop device, and a first physical disk. The data synchronization method includes: In response to a data write request of a target application, the first data synchronization module sends target data generated by the target application to the first virtual disk and the first transmission module; The first transmission module sends the target data to a second device; The first virtual disk uses the first loop device to write the target data into a first specified file block included in the first physical disk. The first specified file block is a file block with a specified size pre-created in the first physical disk, and the file block with the specified size is mapped to the first loop device; The first data synchronization module determines the synchronization status of the target data.

2. The data synchronization method according to claim 1, wherein The step of the first data synchronization module sending the target data generated by the target application to the first virtual disk and the first transmission module includes: The first data synchronization module calls a preset function to split the target data into multiple data blocks, and marks the status of each data block in the multiple data blocks as unsynchronized; The first data synchronization module sends the multiple data blocks to the first transmission module, and sends the multiple data blocks and their statuses to the first virtual disk; The step of the first data synchronization module determining the synchronization status of the target data includes: The first data synchronization module determines the synchronization status of the multiple data blocks.

3. The data synchronization method according to claim 2, wherein The first loop device includes a metadata partition and a real data partition; The step of the first virtual disk using the first loop device to write the target data into the specified file block included in the first physical disk includes: The first virtual disk writes the multiple data blocks into the real data partition, and writes the status of each data block into the metadata partition; Write the multiple data blocks written into the real data partition into the specified file block.

4. The data synchronization method according to claim 3, wherein After the first data synchronization module determines the synchronization status of the multiple data blocks, the data synchronization method further includes: For each data block in the multiple data blocks, when it is determined that the data block is successfully synchronized, a synchronization success indication is sent to the first loop device; Based on the synchronization success indication, the first loop device modifies the status of the data block from the unsynchronized state to the synchronized state.

5. The data synchronization method according to claim 2, wherein The step of the first data synchronization module determining the synchronization status of the multiple data blocks includes: For each data block in the multiple data blocks, when the first data synchronization module receives a first write confirmation and a second write confirmation for the data block, it is determined that the data block is successfully synchronized. The first write confirmation is the write confirmation of the first transmission module for the data block, and the second write confirmation is the write confirmation of the first physical disk for the data block.

6. The data synchronization method according to claim 2, wherein Determining the synchronization status of the multiple data blocks by the first data synchronization module includes: For each of the multiple data blocks, when the first data synchronization module receives a third write confirmation for the data block, it is determined that the data block is successfully synchronized, where the third write confirmation is a write confirmation of the first physical disk for the data block.

7. The data synchronization method according to claim 1, wherein The data synchronization method further includes: Registering the first virtual disk to the first data synchronization module and mounting the first data synchronization module on the kernel of the first device to register the first virtual disk to the operating system of the first device; Mounting the first virtual disk under the target directory of the operating system; When it is detected that the target application writes data into the target directory, it is determined that the data write request is detected.

8. The data synchronization method according to claim 1, wherein The second device includes a second data synchronization module, a second virtual disk, a second transmission module, a second loop device, and a second physical disk; The second transmission module is configured to receive the target data sent by the first transmission module and send the target data to the second data synchronization module; The second data synchronization module is configured to send the target data to the second virtual disk; The second virtual disk is configured to use the second loop device to write the target data into a second specified file block included in the second physical disk, where the second specified file block is a file block with a specified size pre-created in the second physical disk, and the file block with the specified size is mapped to the second loop device.

9. The data synchronization method according to claim 1, characterized in that, Sending the target data to the second device by the first transmission module includes: Sending the target data to the second device by the first transmission module using the Transmission Control Protocol (TCP) or the User Datagram Protocol (UDP).

10. A data synchronization device, applied to a first device, characterized in that, The first device includes a first data synchronization module, a first virtual disk, a first transmission module, a first loop device, and a first physical disk. The data synchronization device includes: A data sending module, configured to, in response to a data write request of a target application, send the target data generated by the target application to the first virtual disk and the first transmission module through the first data synchronization module; A data transmission module, configured to send the target data to a second device through the first transmission module; A data writing module, configured to write the target data into a first specified file block included in the first physical disk by using the first loop device through the first virtual disk, where the first specified file block is a file block with a specified size pre-created in the first physical disk, and the file block with the specified size is mapped to the first loop device; A data status determination module, configured to determine the synchronization status of the target data through the first data synchronization module.

11. An electronic device, characterized in that, Includes: A processor; A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the instructions to implement the data synchronization method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data synchronization method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the data synchronization method according to any one of claims 1 to 9.

Images