First data synchronization method, system, device and storage medium without using snapshot

By creating first synchronization tables in memory and setting the synchronization status of IO thread management data blocks, the problem of lack of first data synchronization without using snapshots in real-time synchronization is solved, and data consistency and storage space savings are achieved. It is suitable for disks that do not support snapshot functions. .

CN118885537BActive Publication Date: 2024-12-03CHENGDU VINCHIN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411320328.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-23
Publication Date
2024-12-03
Estimated Expiration
2044-09-23

AI Technical Summary

Technical Problem

The prior art lacks a first data synchronization method that does not use snapshots in real-time synchronization, resulting in an increase in storage space usage and does not support all disk types, and the first data synchronization cannot be completed without snapshot functions.

Method used

By creating the first synchronization table in memory, setting the first synchronization thread, IO blocking thread and IO dispatch thread in the IO filter driver, managing the synchronization status of data blocks, blocking and issuing IO requests, ensuring data consistency and reducing storage space usage .

Benefits of technology

The first data synchronization is achieved without using snapshots, ensuring data consistency, reducing storage space usage on the production side, and is suitable for disk types that do not support snapshot functions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118885537B_ABST
    Figure CN118885537B_ABST
Patent Text Reader

Abstract

The present invention relates to a method, system, device and storage medium for first data synchronization without using snapshots, and belongs to the technical field of data disaster recovery and backup. The method comprises the steps of: creating a first synchronization table of a disk; setting a first synchronization thread, an IO blocking thread and an IO sending thread in an IO filter driver; when the first synchronization thread is executed and the system generates a disk write request, the IO blocking thread intercepts the write request, reads the target data block from the disk, and updates the progress status of the first synchronization table at the same time; the IO sending thread checks the first synchronization table, and releases the write request once it is confirmed that the data block synchronization is completed. The system comprises: a module for creating a first synchronization table; a module for setting a thread; a module for blocking write IO requests; a module for sending write target data; and a module for polling and sending write IO requests. The present invention realizes the first synchronization without using snapshots, ensures data consistency during the first synchronization, and reduces the storage space occupied by the production end.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of data disaster recovery and backup, and relates to a first data synchronization method, system, device and storage medium without using snapshots. Background Art

[0002] Real-time synchronization refers to the process by which data is copied to another location or system immediately after it is created or modified. This synchronization can be done within the same network, across networks, or across geographical locations. The purpose of real-time synchronization is to ensure data consistency and availability, and to quickly recover even if the original data source fails.

[0003] The first data synchronization is the first step in the real-time synchronization solution, and its purpose is to synchronize the initial data of the production end to the server end completely. During the first data synchronization process, the production end is still constantly generating IO, and the data will be constantly updated. In order to ensure the consistency of the synchronized data, snapshot technology is usually used for the first data synchronization. However, the snapshot file will inevitably occupy a certain amount of storage space on the production end, and not all disk types in all scenarios support the snapshot function. In the existing technology, there is almost no first data synchronization technology that does not use snapshots in real-time synchronization.

[0004] Therefore, how to help customers complete the first data synchronization without the snapshot function and save storage space on the production side is a technical problem that urgently needs to be solved. Summary of the invention

[0005] In order to solve the technical problems in the above-mentioned background technology, the present invention provides a method, system, device and storage medium for initial data synchronization without using snapshots.

[0006] The technical solution of the present invention to solve the above technical problems is as follows:

[0007] In a first aspect, a method for initial data synchronization without using a snapshot is provided, the method comprising the steps of:

[0008] Create a first synchronization table for the disk in memory. The elements of the first synchronization table correspond to the data blocks on the disk one by one. The elements record the first synchronization status of the data blocks.

[0009] Set the first synchronization thread, IO blocking thread and IO sending thread in the IO filter driver;

[0010] When the first synchronization thread is executing and generates a write IO request to the disk, it is determined whether the write target data block data has been synchronized. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent;

[0011] According to the write IO request, the IO blocking thread reads the write target data block data from the disk and then sends the read data. During the execution, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time and the write IO request is transferred to the IO sending thread;

[0012] The IO sending thread polls the first synchronization table. If it is found that the first synchronization of the write target data block has been completed, a write IO request is sent and sent.

[0013] In a second aspect, a first data synchronization system without using snapshots is provided, the system comprising:

[0014] Create a first synchronization table module, which is used to create a first synchronization table of the disk in the memory. The elements of the first synchronization table correspond to the data blocks of the disk one by one, and the elements record the first synchronization status of the data blocks;

[0015] Set up a thread module to set up the first synchronization thread, IO blocking thread and IO sending thread in the IO filter driver;

[0016] The blocking write IO request module is used to determine whether the write target data block data has been synchronized when the first synchronization thread is executing and a write IO request to the disk is generated. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent;

[0017] The write target data sending module is used to read the write target data block data from the disk according to the write IO request, and then send the read data. During the execution process, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time and the write IO request is transferred to the IO sending thread;

[0018] The polling and sending write IO request module is used for the IO sending thread to poll the first synchronization table. If it is found that the first synchronization of the write target data block is completed, the write IO request is sent and sent.

[0019] According to a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for first data synchronization without using a snapshot as described in any one of the above items is implemented.

[0020] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the first data synchronization method without using a snapshot is implemented as described above.

[0021] The beneficial effects of the present invention are:

[0022] Once the first synchronization task is started, the first synchronization thread will read each data block on the disk in turn. However, since the first synchronization is a time-consuming process, new IO requests will inevitably be generated on the production side during this period to modify the data on the disk. At this time, by introducing IO blocking threads and IO sending threads, new write IO requests are processed in advance, and records are made in the first synchronization table, and the processed write IO requests are sent on time. The first synchronization is achieved without using snapshots, ensuring data consistency during the first synchronization and reducing the storage space occupied by the production side. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 The figure is a flowchart of a method for initial data synchronization without using a snapshot in an embodiment of the present invention.

[0025] Figure 2 The figure is a working principle diagram of the IO blocking thread and the IO sending thread in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of the structure of the first data synchronization system without using snapshots in an embodiment of the present invention.

[0027] Figure 4 Schematic diagram of the structure of a module for sending and writing target data in an embodiment of the present invention.

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0029] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0030] 310. Create the first synchronization table module; 320. Set the thread module; 330. Block the write IO request module; 340. Send the write target data module; 350. Poll and send the write IO request module; 360. Get the read IO request module; 370. Determine the read IO request sending module; 3401. Calculate the write target data block position unit; 3402. Read the write target data unit; 3403. Send the write target data unit; 410. Processor; 420. Communication interface; 430. Memory; 440. Communication bus. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0032] A snapshot is a fully available copy of the specified data. The copy contains the image of the data at a certain point in time. A snapshot can quickly capture and save the state of data at a specific point in time without interrupting the operation of the system, and generate a consistent data copy of the data at that point in time. The existing real-time synchronization solution is to create a snapshot of the disk on the production side when the first synchronization is started, and then read the snapshot data and write it to the server to complete the first synchronization. Since the data for the first synchronization is the data when the task is started, rather than the updated data, this ensures the consistency of the synchronized data. However, the snapshot is first stored on the production side and then sent to the server. The snapshot will definitely occupy the storage space of the production side. As the subsequent IO requests change, it will occupy more and more, thus affecting the performance of the production side. In addition, the snapshot function usually relies on the support of the underlying storage system, such as some old file systems, specific storage hardware, and restricted virtualization environments. These may not support the snapshot function, which may cause the failure to smoothly achieve real-time synchronization. Therefore, how to help customers complete the first data synchronization without the snapshot function and save storage space on the production side is a technical problem that needs to be solved urgently.

[0033] In view of the above problems, an embodiment of the present invention provides a method for initial data synchronization without using snapshots. Figure 1 A schematic diagram of a method for initial data synchronization without using a snapshot provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0034] Step S101, creating a first synchronization table of the disk in the memory, the elements of the first synchronization table correspond to the data blocks of the disk one by one, and the elements record the first synchronization status of the data blocks.

[0035] As shown in Table 1, the present embodiment divides the disk into several data blocks of equal size, such as Block 0, Block 1, Block 2, Block 3, and Block N. The first synchronization and subsequent real-time synchronization are operated with data blocks as the minimum unit. During the first synchronization, a first synchronization table is maintained in the memory, as shown in Table 2. The table is used to identify the first synchronization status of each data block on the disk, and the number of its elements is equal to the number of data blocks and corresponds one to one. Generally, the first synchronization status is set as follows: first synchronization has not started, first synchronization is in progress, and first synchronization is completed. However, by introducing some additional states, various situations in the first synchronization process can be more comprehensively monitored and managed, and the reliability and flexibility of data synchronization can be improved. For example, the first synchronization status can also include: if the first synchronization operation exceeds the predetermined time and is still not completed, the status of the data block can be marked as timeout, which may trigger a retry mechanism or error handling; the synchronization operation of the data block has been completed, but some confirmation steps are still required. The embodiments of the present invention do not specifically limit this.

[0036] Table 1. Schematic diagram of the disk structure

[0037] Block 0 Block 1 Block 2 Block 3 .... .... .... .... .... .... .... Block N

[0038] Table 2. Structure of the first synchronization table

[0039] 2 2 2 1 0 .... .... 0 1 .... 0 0

[0040] In addition, as shown in Table 2, the present embodiment sets the value of each element to 0, 1 or 2, where 0 represents that the first synchronization of the corresponding data block has not started; 1 represents that the first synchronization of the corresponding data block is in progress; and 2 represents that the first synchronization of the corresponding data block is completed. However, the first synchronization status can also be marked in English, such as X, Y and Z. The embodiments of the present invention are not specifically limited to this.

[0041] It is worth noting that the first synchronization table of this embodiment is set in the memory, which has the following advantages: (1) Improved performance: The access speed of the memory is much higher than that of the disk, which can significantly reduce the access delay, speed up the query and update speed of the data block status, and improve the overall synchronization efficiency; (2) Reduced disk IO: Avoided frequent disk read and write operations, reduced the pressure on disk IO, which helps to reduce disk wear and improve the response speed of the system.

[0042] Step S102, setting the first synchronization thread, IO blocking thread and IO issuing thread in the IO filter driver.

[0043] The IO filter driver is a software component located between the application and the storage system, and its main functions include monitoring and filtering IO requests, etc. In this embodiment, the IO filter driver helps coordinate the data consistency during the first synchronization process.

[0044] First synchronization thread: responsible for performing the first synchronization task, reading the data blocks on the disk one by one, and synchronizing them to the server. The main goal of the first synchronization thread is to ensure that all data blocks are completely copied once.

[0045] IO blocking thread: When new IO requests (write operations) arrive, this thread is responsible for temporarily suspending or delaying these requests to avoid modifying the data blocks being synchronized during the first synchronization, thereby ensuring data consistency; it is also responsible for pre-processing the data of the write target data block.

[0046] IO sending thread: After the first synchronization thread or IO blocking thread completes the synchronization of a specific data block, this thread is responsible for releasing the previously suspended IO requests and allowing them to continue execution.

[0047] It should be noted that the execution mode of the above three threads depends on the CPU architecture of the system and the scheduling policy of the operating system. The operating system will decide how to schedule these threads based on factors such as the current system load, the number of CPU cores, and thread priority. For example, on a single-core processor, all threads are executed concurrently, and the operating system simulates multitasking through time slicing. Even if they appear to be running at the same time, they are actually switching execution quickly; on a multi-core processor, threads can be executed in parallel. Each core can be assigned one or more threads to achieve true simultaneous operation; when the system load is high, the operating system may switch between threads more frequently to ensure that all threads can be processed. The embodiments of the present invention do not specifically limit this.

[0048] Step S103, when the first synchronization thread is executing and generates a write IO request to the disk, it is determined whether the write target data block data has completed synchronization. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent.

[0049] Step S104, according to the write IO request, the IO blocking thread reads the write target data block data from the disk, and then sends the read data. During the execution process, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time and the write IO request is transferred to the IO sending thread.

[0050] When the first synchronization thread is executing, it is responsible for reading the data blocks on the disk and synchronizing them to the server. This process needs to ensure data consistency, that is, the synchronized data should be the data state at the beginning of the first synchronization. If the data on the disk is modified during the first synchronization, the synchronized data may be inconsistent with the original data and cannot meet the requirements.

[0051] The introduction of IO blocking threads can temporarily block write IO requests to the disk during the first synchronization and synchronize the write target data block data in advance. Once the IO blocking thread completes the data synchronization of the write target data block, the IO blocking thread transfers the write IO request to the IO issuing thread for release, allowing them to continue execution. This ensures data consistency and guarantees the responsiveness and performance of the system to a certain extent.

[0052] At the same time, with the help of the first synchronization table, the first synchronization thread, IO blocking thread and IO sending thread can all know the first synchronization status of the target data block in time, so as to ensure that the three of them can coordinate their work and avoid duplication of data synchronization.

[0053] Step S105, the IO issuing thread polls the first synchronization table, and if it is found that the first synchronization of the write target data block is completed, a write IO request is sent and issued.

[0054] It is understandable that polling is an active checking method, in which the IO delivery thread periodically checks the first synchronization table to determine the synchronization status of the data block. Polling can determine when the first synchronization of the data block has been completed, so that the previously blocked write IO request can be safely delivered. Polling is usually performed in a loop, and the IO delivery thread will query the first synchronization table at regular intervals to check whether the status of any data block has been updated to "completed" or "2".

[0055] It is also understood that the frequency of polling can be adjusted according to the performance requirements and load conditions of the system. A higher polling frequency can respond to changes in synchronization status more quickly, but may increase CPU usage. The polling interval is the time between two polls. A shorter interval can process write IO requests more quickly, but may increase the load on the system; a longer interval can reduce the system load, but the response time for processing requests will be longer. The embodiments of the present invention do not specifically limit this.

[0056] However, in order to avoid directly accessing the first synchronization table for each poll, a cache can be used to store recent status changes, which can reduce the number of accesses to the first synchronization table and improve efficiency.

[0057] Compared with the prior art, this embodiment realizes the first synchronization without using snapshots, thereby ensuring data consistency during the first synchronization and reducing storage space occupied by the production end.

[0058] Based on the above embodiment, in the method, step S103 further includes:

[0059] Step S1031, receiving the first synchronization task start instruction;

[0060] Step S1032, according to the start instruction, the first synchronization thread sequentially reads the data block data from the disk, and then writes the read data into the sending queue, and during the execution, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time;

[0061] Step S1033, when a write IO request for modifying the disk is generated, it is determined whether the data of the write target data block is synchronized. If so, the IO sending thread sends the write IO request; if not, the IO filter driver captures the write IO request and transfers the write IO request to the IO blocking thread, and executes step S1034;

[0062] Step S1034: the IO blocking thread prevents the write IO request from being sent.

[0063] Based on the above embodiment, in the method, step S104 further includes:

[0064] Step S1041, the IO blocking thread obtains the offset field and the data length field of the write IO request, and calculates the start block number and the end block number of the write target data block;

[0065] Step S1042: According to the settlement result, the IO blocking thread starts to read the write target data block data from the disk, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to the first synchronization in progress, and transfers the write IO request to the IO issuing thread;

[0066] Step S1043, when the IO blocking thread completes reading and writing the target data block, the IO blocking thread encapsulates the read data and inserts it into the sending queue, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to first synchronization completed.

[0067] It is worth noting that each IO request structure has two important fields, the offset field (offset) and the data length field (length). An IO request may span multiple data blocks. Therefore, in order to read the target data block more accurately, a better way is to calculate the target starting block number and the target ending block number. The specific calculation method is: starting block number = offset / block_size, ending block number = (offset+length) / block_size, and the calculation results are rounded down. Block_size refers to the data block size.

[0068] like Figure 2 As shown in the figure, the IO blocking thread blocks the write IO request. After calculating the target block number of the write IO request, the IO blocking thread initiates read requests for the data from the starting block number to the end block number in sequence. After starting to read the data of the write IO target data block (BlockK), it is necessary to set the corresponding element in the first synchronization table from 0 to 1, and hand over the blocked write IO request to the IO issuing thread.

[0069] Likewise Figure 2 As shown, the IO blocking thread encapsulates the read data into a first synchronization data packet and inserts it into the sending queue, and sets the corresponding element in the first synchronization table to 2.

[0070] Also like Figure 2 As shown in the figure, after receiving the write IO request, the IO issuing thread polls the first synchronization table to check whether the target data block of the current write IO request has completed the first synchronization (that is, whether the corresponding element value in the table is 2). If the target block has completed the first synchronization, a copy of the current write IO request is encapsulated into a real-time synchronization data packet and inserted into the sending queue, and finally the current write IO request is issued.

[0071] It should be noted that the preferred method of sending in this embodiment is to store the sending queue first and then send it to the server. This method uses the sending queue as a buffer to temporarily store real-time synchronization data packets, which can cope with network fluctuations or changes in the processing capacity of the backup server, and can also ensure that the data packets are sent in a specific order to maintain data consistency and integrity; by controlling the size of the sending queue and the sending rate, overload can be avoided. Other sending methods can also be sent directly, sent through memory cache, or sent through a callback mechanism, and the embodiments of the present invention do not specifically limit this.

[0072] Different from the previous embodiment method, this embodiment can more accurately identify the location of the target data block, arrange data transmission in an orderly manner, manage the load of the server, and improve synchronization efficiency.

[0073] In addition, preferably, the step S1043 further includes:

[0074] Step S10431, setting a data sending thread in the IO filter driver;

[0075] Step S10432, when the IO blocking thread finishes reading and writing the target data block, the IO blocking thread stores the read data into the local cache queue, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to first synchronization completed;

[0076] Step S10433, receiving a sending instruction, the data sending thread caches the encapsulated data locally, and then inserts the encapsulated data into a sending queue.

[0077] It is worth noting that, in the case of determining whether the write target data block data has been synchronized, it can be transferred to the IO sending thread for sending (transferring to a dedicated IO thread can reduce the burden of the main thread, improve the responsiveness and throughput of the system, and avoid disorder), and a copy of the write cache can be copied and then transferred to the IO sending thread for sending (through the cache mechanism, the write request can be temporarily stored, reducing the impact on the main thread, and the write operation can be processed in batches through the IO thread to improve efficiency). The embodiments of the present invention do not specifically limit this.

[0078] This embodiment takes into account that the bandwidth of the sending network may fluctuate, and first stores the write target data block data in the local cache queue, and then the data sending thread inserts the encapsulated data into the sending queue according to the sending instruction, which can reduce the impact of insufficient network bandwidth on the business system.

[0079] Based on the above embodiment, the method includes:

[0080] Step S201, creating a first synchronization table of the disk in the memory, the elements of the first synchronization table correspond to the data blocks of the disk one by one, and the elements record the first synchronization status of the data blocks;

[0081] Step S202, setting the first synchronization thread, IO blocking thread and IO issuing thread in the IO filter driver;

[0082] Step S203: when the first synchronization thread is executing and generates a write IO request to the disk, it is determined whether the write target data block data is synchronized. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent.

[0083] Step S204: According to the write IO request, the IO blocking thread reads the write target data block data from the disk, and then sends the read data. During the execution, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time, and the write IO request is transferred to the IO sending thread;

[0084] Step S205, the IO issuing thread polls the first synchronization table, and if it is found that the first synchronization of the write target data block is completed, a write IO request is sent and issued;

[0085] Step S206, when a read IO request for reading the disk is generated, the first synchronization thread obtains the read IO request;

[0086] Step S207, according to the read IO request, the first synchronization line queries the first synchronization table to determine the current first synchronization status of the read target data block. If the first synchronization is in progress, wait for the first synchronization to be completed before issuing the read IO request to perform the read operation; if the first synchronization has not started or the first synchronization is completed, directly issue the read IO request to perform the read operation.

[0087] Different from the previous implementation method, if the target data block is performing synchronization, directly placing a read IO request may cause data consistency and integrity problems during the read operation. In this embodiment, the synchronization status of the target data block is checked before the read operation. If the synchronization is in progress, the read operation is performed after the synchronization is completed. This method of processing read IO is simpler and more intuitive, easy to implement and maintain, and also reduces additional synchronization logic. For example, there is no need to introduce complex locking mechanisms or version control, and reasonable performance can be provided without sacrificing data consistency.

[0088] like Figure 3 As shown, a first data synchronization system without using snapshots, the system also includes:

[0089] A first synchronization table creation module 310 is used to create a first synchronization table of the disk in the memory, wherein the elements of the first synchronization table correspond to the data blocks of the disk one by one, and the elements record the first synchronization status of the data blocks;

[0090] Setting a thread module 320, which is used to set a first synchronization thread, an IO blocking thread and an IO sending thread in the IO filter driver;

[0091] The blocking write IO request module 330 is used to determine whether the write target data block data has completed synchronization when the first synchronization thread is executing and generates a write IO request to the disk. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent;

[0092] The write target data sending module 340 is used to read the write target data block data from the disk according to the write IO request, and then send the read data. During the execution process, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time and the write IO request is transferred to the IO sending thread;

[0093] The polling and issuing write IO request module 350 is used for the IO issuing thread to poll the first synchronization table. If the first synchronization of the write target data block is found to be completed, the write IO request is sent and issued.

[0094] The read IO request acquisition module 360 ​​is used to synchronize the thread to acquire the read IO request for the first time when a read IO request for reading the disk is generated;

[0095] The module 370 for determining whether a read IO request is being sent is used to query the first synchronization table on the first synchronization line according to the read IO request, and determine the current first synchronization status of the read target data block. If the first synchronization is in progress, the read IO request is sent after the first synchronization is completed to perform the read operation; if the first synchronization has not started or the first synchronization is completed, the read IO request is directly sent to perform the read operation.

[0096] Based on the above embodiments, Figure 4 As shown, the sending write target data module 340 further includes:

[0097] The write target data block location calculation unit 3401 is used for the IO blocking thread to obtain the offset field and the data length field of the write IO request, and calculate the start block number and the end block number of the write target data block;

[0098] The write target data reading unit 3402 is used to read the write target data block data from the disk according to the settlement result, and then adjust the first synchronization progress state of the corresponding element on the first synchronization table to the first synchronization in progress, and transfer the write IO request to the IO issuing thread;

[0099] The write target data sending unit 3403 is used for, when the IO blocking thread completes reading the write target data block, the IO blocking thread encapsulates the read data and inserts it into the sending queue, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to first synchronization completed.

[0100] Compared with the prior art, the first synchronization thread, IO blocking thread, and IO sending thread in this embodiment can coordinate with each other with the help of the first synchronization table, even if a new write IO request is generated, to ensure the consistency of synchronized data and the stability of the system, without storing a copy of the entire disk.

[0101] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 5As shown, the electronic device may include: a processor 410 , a communication interface 420 , a memory 430 and a communication bus 440 , wherein the processor 410 , the communication interface 420 , and the memory 430 communicate with each other via the communication bus 440 . The processor 410 can call the logic instructions in the memory 430 to execute the following method: create a first synchronization table of the disk in the memory, the elements of the first synchronization table correspond to the data blocks of the disk one by one, and the elements record the first synchronization status of the data blocks; set the first synchronization thread, IO blocking thread and IO sending thread in the IO filter driver; when the first synchronization thread is executing and a write IO request to the disk is generated, it is determined whether the write target data block data has completed synchronization. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent; according to the write IO request, the IO blocking thread reads the write target data block data from the disk, and then sends the read data. During the execution process, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time and the write IO request is transferred to the IO sending thread; the IO sending thread polls the first synchronization table, and if it is found that the first synchronization of the write target data block is completed, the write IO request is sent and sent.

[0102] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0103] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the multi-version control method under the API framework described in the above embodiment is implemented.

[0104] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0105] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0106] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0107] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, Ruby, Go, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for first data synchronization without using snapshots, characterized in that: The method comprises the steps of: Create a first synchronization table for the disk in memory. The elements of the first synchronization table correspond to the data blocks on the disk one by one. The elements record the first synchronization status of the data blocks. Set the first synchronization thread, IO blocking thread and IO sending thread in the IO filter driver; When the first synchronization thread is executing and generates a write IO request to the disk, it is determined whether the write target data block data has been synchronized. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent; According to the write IO request, the IO blocking thread reads the write target data block data from the disk and then sends the read data. During the execution, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time and the write IO request is transferred to the IO sending thread; The IO sending thread polls the first synchronization table. If it is found that the first synchronization of the write target data block has been completed, a write IO request is sent and sent.

2. The method for first data synchronization without using snapshots according to claim 1, characterized in that: When the first synchronization thread is being executed and a write IO request to the disk is generated, it is determined whether the synchronization step of writing the target data block data is completed, and further includes: Receive the first synchronization task start instruction; According to the startup instruction, the first synchronization thread reads the data block data from the disk sequentially, and then sends the read data. During the execution process, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time; When a write IO request to modify the disk is generated, it is determined whether the data of the write target data block is synchronized. If so, the IO sending thread sends the write IO request; if not, the IO filter driver captures the write IO request and transfers the write IO request to the IO blocking thread to execute the next step; The IO blocking thread prevents write IO requests from being sent.

3. The method for initial data synchronization without using snapshots according to claim 1, characterized in that: According to the write IO request, the IO blocking thread reads the write target data block data from the disk, and then sends the read data. During the execution, the first synchronization progress status of the corresponding element on the first synchronization table is updated immediately and the write IO request is transferred to the IO sending thread step, which also includes: The IO blocking thread obtains the offset field and the data length field of the write IO request, and calculates the start block number and the end block number of the write target data block; According to the calculation results, the IO blocking thread starts to read the write target data block data from the disk, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to the first synchronization in progress, and transfers the write IO request to the IO issuing thread; When the IO blocking thread finishes reading and writing the target data block, the IO blocking thread encapsulates the read data and inserts it into the sending queue, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to first synchronization completed.

4. The method for first data synchronization without using snapshots according to claim 3, characterized in that: When the IO blocking thread completes reading and writing the target data block, the IO blocking thread encapsulates the read data and inserts it into the sending queue, and then immediately adjusts the first synchronization progress state of the corresponding element on the first synchronization table to the first synchronization completion step, further comprising: Set up the data sending thread in the IO filter driver; When the IO blocking thread finishes reading and writing the target data block, the IO blocking thread stores the read data into the local cache queue, and then immediately adjusts the first synchronization progress status of the corresponding element on the first synchronization table to first synchronization completed; After receiving the sending instruction, the data sending thread caches the encapsulated data locally and then inserts the encapsulated data into the sending queue.

5. The method for initial data synchronization without using snapshots according to claim 1, characterized in that: Also includes: When a read IO request to read the disk is generated, the first synchronization thread obtains the read IO request; According to the read IO request, the first synchronization thread queries the first synchronization table to determine the current first synchronization status of the read target data block. If the first synchronization is in progress, the read IO request is issued after the first synchronization is completed to perform the read operation; If the first synchronization has not started or is completed, a read IO request is directly issued to perform the read operation.

6. A first data synchronization system without using snapshots, characterized in that: The system comprises: Create a first synchronization table module, which is used to create a first synchronization table of the disk in the memory. The elements of the first synchronization table correspond to the data blocks of the disk one by one, and the elements record the first synchronization status of the data blocks; Set up a thread module to set up the first synchronization thread, IO blocking thread and IO sending thread in the IO filter driver; The blocking write IO request module is used to determine whether the write target data block data has been synchronized when the first synchronization thread is executing and a write IO request to the disk is generated. If so, the IO sending thread sends the write IO request; if not, the IO blocking thread obtains the write IO request and prevents the write IO request from being sent; The write target data sending module is used to read the write target data block data from the disk according to the write IO request, and then send the read data. During the execution process, the first synchronization progress status of the corresponding element on the first synchronization table is updated in real time and the write IO request is transferred to the IO sending thread; The polling and sending write IO request module is used for the IO sending thread to poll the first synchronization table. If it is found that the first synchronization of the write target data block is completed, the write IO request is sent and sent.

7. The first data synchronization system without using snapshots according to claim 6, characterized in that: The sending and writing target data module also includes: Calculate the write target data block location unit, which is used for the IO blocking thread to obtain the offset field and data length field of the write IO request, and calculate the start block number and the end block number of the write target data block; Read the write target data unit, which is used for the IO blocking thread to start reading the write target data block data from the disk according to the calculation result, and then immediately adjust the first synchronization progress status of the corresponding element on the first synchronization table to the first synchronization in progress, and transfer the write IO request to the IO issuing thread; Send write target data unit, used for when the IO blocking thread finishes reading the write target data block, the IO blocking thread will encapsulate the read data and insert it into the sending queue, and then immediately adjust the first synchronization progress status of the corresponding element on the first synchronization table to the first synchronization completed.

8. The system for first-time data synchronization without using snapshots according to claim 6, characterized in that: The system further comprises: The module for obtaining a read IO request is used to synchronize the thread to obtain the read IO request for the first time when a read IO request for reading the disk is generated; The module for judging the sending of read IO requests is used to query the first synchronization table according to the read IO request, and judge the current first synchronization status of the read target data block. If the first synchronization is in progress, the read IO request is sent after the first synchronization is completed to perform the read operation; if the first synchronization has not started or the first synchronization is completed, the read IO request is directly sent to perform the read operation.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the first data synchronization method without using a snapshot according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for first data synchronization without using a snapshot as described in any one of claims 1 to 5 is implemented.