Asynchronous replication method, apparatus, electronic device, and program product

Abstract

The application relates to the technical field of data storage, in particular to an asynchronous replication method and device, electronic equipment and a program product. The method is applied to a master terminal device connected with a host and a slave terminal device, address change information is extracted in real time when a host write service is processed, and the change information is written into a log buffer or jointly written into the log buffer and a data buffer; a consistency point label is generated in the log buffer according to a preset time interval, and then the to-be-replicated data is organized according to the buffer record and the consistency point label and sent to the slave terminal device. Through the write change tracking combined with the consistency boundary identification mode, the application can reduce snapshot and difference analysis overhead, reduce synchronization time delay and system resource occupation, improve replication transmission efficiency, and enhance the completeness and consistency of the slave terminal data version.

Description

Technical Field

[0001] This application relates to the technical field of data storage, and more particularly to an asynchronous copying method, apparatus, electronic device, and program product. Background Technology

[0002] In relevant enterprise-level storage disaster recovery, remote asynchronous replication typically employs a data synchronization scheme based on periodic snapshots and differential data transfer.

[0003] However, this approach relies on snapshot creation and analysis of differences between versions, and the processing latency increases with the write load, making it difficult to meet the low recovery point objective (RPO) requirement. At the same time, frequent snapshots and difference comparisons will continuously consume the central processing unit (CPU), input / output operations per second (IOPS), and storage space, and the transmission efficiency and version consistency guarantee capability are limited when the difference data is discrete.

[0004] Therefore, how to reduce synchronization latency and resource consumption, and improve transmission efficiency and data version consistency in remote asynchronous replication scenarios has become a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides an asynchronous replication method, apparatus, electronic device, and program product to solve the technical problems in related technologies of how to reduce synchronization latency and resource consumption, and improve transmission efficiency and data version consistency in remote asynchronous replication scenarios.

[0006] In a first aspect, this application provides an asynchronous replication method applied to a master device, which is connected to both a host device and a slave device. The method includes:

[0007] When processing a write request sent by the host, determine the current address change information corresponding to the write request;

[0008] Update the target buffer based on the current address change information. The target buffer may include the log buffer or the target buffer may include both the log buffer and the data buffer.

[0009] Consistency point labels are generated in the log buffer at preset intervals.

[0010] Based on the target buffer and consistency point label, determine the data to be replicated and transmit the data to be replicated to the slave device.

[0011] By moving the difference identification forward to the write processing stage, the master device can still quickly organize and replicate data according to the log order under high write load, and use consistency point tags to clarify the version boundary of each replication round, thereby meeting the requirements of synchronization latency control and replication version consistency in cross-regional data protection scenarios.

[0012] Optionally, the method described above determines the data to be replicated based on the target buffer and consistency point labels, including:

[0013] In response to the start of the scheduled copy task, a start message is sent to the slave device. The start message includes a consistency point label.

[0014] Receive the first confirmation message corresponding to the start message sent by the slave device;

[0015] Based on the consistency point labels, identify multiple target address change information in the log buffer;

[0016] Based on multiple target address change information, determine the data to be copied.

[0017] In this way, the process of determining the data to be replicated is directly based on the consistency point label to locate the log record, reducing the reliance on full snapshot differential analysis. The replication boundary can also be kept consistent with the trigger time of the scheduled copy task, thus making the remote replication process more stable in terms of data consistency and lower in terms of processing overhead.

[0018] Optionally, the method described above determines the data to be copied based on multiple target address change information, including:

[0019] For the first address change information, determine whether there is changed data reference information in the first address change information. If so, obtain the first copied data from the data buffer. If not, obtain the first copied data from the log buffer. The first address change information is one of multiple target address change information.

[0020] Based on the first copy data corresponding to each target address change information, determine the data to be copied.

[0021] In this way, since the change information of each target address can be parsed item by item and aggregated into the data to be replicated, the integrity and accuracy of the replicated data can be improved, meeting the requirements for consistency and reliability in asynchronous replication scenarios.

[0022] Optionally, in the method described above, the current address change information includes the logical block address. Based on the current address change information, the target buffer is updated, including:

[0023] Determine if the log buffer contains a logical block address that is identical to the current address change information;

[0024] If so, copy the second address change information corresponding to the logical block address in the log buffer to the data buffer, update the changed data reference information corresponding to the second address change information in the log buffer, and write the current address change information into the log buffer;

[0025] If not, the current address change information is written to the log buffer.

[0026] In this way, duplicate logical block addresses can be quickly identified and existing change records can be reused, and replicable data in the data buffer can be maintained by reference, thereby reducing the storage overhead caused by repeated writes and making the retrieval and sending of subsequent data to be copied more direct, thus improving the processing efficiency and record consistency in asynchronous copying scenarios.

[0027] Optionally, the method described above, transmitting the data to be copied to the slave device, includes:

[0028] The data to be copied is compressed to obtain compressed data.

[0029] Send compressed data to the slave device;

[0030] After receiving the second confirmation message from the slave device, clear the data to be copied.

[0031] In this way, the transmission overhead and local data usage during the remote replication process are controlled, and the timing of clearing the data to be replicated is consistent with the receipt confirmation, which can maintain the consistency of the replicated data.

[0032] Secondly, this application provides an asynchronous replication method applied to a slave device, which is connected to a master device. The method includes:

[0033] The system receives the start message sent by the master device through the synchronization queue and sends the first confirmation message to the master device. The start message includes a consistency point label.

[0034] The synchronization queue receives the data to be replicated sent by the master device. The data to be replicated is determined by the master device through the target buffer and the consistency point label. The target buffer includes the log buffer, or the target buffer includes the log buffer and the data buffer.

[0035] Based on the consistency point label, data verification is performed on the data to be replicated through the synchronization queue;

[0036] After the data verification is successful, the data to be copied is written to the storage medium through the production queue.

[0037] In this way, when the master device uses a log buffer or a combination of a log buffer and a data buffer to organize the replicated data, the slave device can form a verifiable and disk-persistent complete version of the replicated data for each round.

[0038] Optionally, the method described above performs data verification on the data to be replicated through a synchronization queue based on the consistency point label, including:

[0039] Based on the consistency point label, determine the sequence number and logical block address corresponding to multiple target address change information respectively;

[0040] Based on the sequence number and logical block address corresponding to multiple target address change information, the integrity and consistency checks of the data to be copied are performed through a synchronization queue.

[0041] In this way, the validity of the data to be replicated in the remote asynchronous replication chain can be confirmed in a timely manner, thereby improving the correct correspondence and version consistency of the replicated data.

[0042] Thirdly, this application provides an asynchronous replication apparatus applied to a master device, which is connected to both a host device and a slave device. The apparatus includes:

[0043] The logging module is used to determine the current address change information corresponding to the write request when processing the write request sent by the host;

[0044] The update module is used to update the target buffer based on the current address change information. The target buffer may include the log buffer or the target buffer may include both the log buffer and the data buffer.

[0045] The generation module is used to generate consistency point labels in the log buffer at preset intervals;

[0046] The determination module is used to determine the data to be copied based on the target buffer and consistency point labels, and to transmit the data to be copied to the slave device.

[0047] Optionally, the above-described device determines that the module is specifically used for:

[0048] In response to the start of the scheduled copy task, a start message is sent to the slave device. The start message includes a consistency point label.

[0049] Receive the first confirmation message corresponding to the start message sent by the slave device;

[0050] Based on the consistency point labels, identify multiple target address change information in the log buffer;

[0051] Based on multiple target address change information, determine the data to be copied.

[0052] Optionally, the above-described device determines that the module is specifically used for:

[0053] For the first address change information, determine whether there is changed data reference information in the first address change information. If so, obtain the first copied data from the data buffer. If not, obtain the first copied data from the log buffer. The first address change information is one of multiple target address change information.

[0054] Based on the first copy data corresponding to each target address change information, determine the data to be copied.

[0055] Optionally, in the above-described device, the update module is specifically used for:

[0056] Determine if the log buffer contains a logical block address that is identical to the current address change information;

[0057] If so, copy the second address change information corresponding to the logical block address in the log buffer to the data buffer, update the changed data reference information corresponding to the second address change information in the log buffer, and write the current address change information into the log buffer;

[0058] If not, the current address change information is written to the log buffer.

[0059] Optionally, the above-described device determines that the module is specifically used for:

[0060] The data to be copied is compressed to obtain compressed data.

[0061] Send compressed data to the slave device;

[0062] After receiving the second confirmation message from the slave device, clear the data to be copied.

[0063] Fourthly, this application provides an asynchronous replication apparatus applied to a slave device, the slave device being connected to a master device, the apparatus comprising:

[0064] The transceiver module is used to receive the start message sent by the master device through the synchronization queue and send the first confirmation message to the master device. The start message includes a consistency point label.

[0065] The transceiver module is also used to receive data to be replicated sent by the master device through a synchronization queue. The data to be replicated is determined by the master device through a target buffer and a consistency point label. The target buffer includes a log buffer, or the target buffer includes a log buffer and a data buffer.

[0066] The verification module is used to verify the data to be replicated through the synchronization queue based on the consistency point label.

[0067] The write module is used to write the data to be copied to the storage medium through the production queue after the data verification is passed.

[0068] Optionally, in the above-described apparatus, the verification module is specifically used for:

[0069] Based on the consistency point label, determine the sequence number and logical block address corresponding to multiple target address change information respectively;

[0070] Based on the sequence number and logical block address corresponding to multiple target address change information, the integrity and consistency checks of the data to be copied are performed through a synchronization queue.

[0071] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0072] The memory stores instructions that the computer executes;

[0073] The processor executes computer-executable instructions stored in memory to implement any of the methods of the first aspect.

[0074] Sixthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0075] The memory stores instructions that the computer executes;

[0076] The processor executes computer-executable instructions stored in memory to implement any of the methods in the second aspect.

[0077] In a seventh aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0078] Eighthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the second aspects.

[0079] Ninthly, this application provides a computer program product, including a computer program that, when executed by a computer, implements the method as described in any of the first aspects.

[0080] In a tenth aspect, this application provides a computer program product, including a computer program that, when executed by a computer, implements the method as described in any of the second aspects.

[0081] The asynchronous replication method, apparatus, electronic device, and program product provided in this application determine the current address change information corresponding to the write request when processing the write request sent by the host, and update the target buffer, including the log buffer or including the log buffer and data buffer, according to the current address change information. This enables timely collection and orderly recording of data changes caused by writing, reducing reliance on periodic snapshot creation and comparison of version differences, and reducing synchronization latency and system resource consumption. Furthermore, by generating consistency point tags in the log buffer at preset intervals, and determining the data to be replicated based on the target buffer and consistency point tags and transmitting it to the slave device, the efficiency of determining the data to be replicated and the targeting of transmission are improved, ensuring the clarity and consistency of the version boundaries of the transmitted data, thereby improving the data transmission efficiency and disaster recovery synchronization quality in remote asynchronous replication scenarios. Attached Figure Description

[0082] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0083] Figure 1 A schematic diagram of an application scenario provided in this application embodiment;

[0084] Figure 2 A flowchart illustrating an asynchronous replication method provided in an embodiment of this application;

[0085] Figure 3 A schematic diagram of the structure of a log buffer and a data buffer provided in an embodiment of this application;

[0086] Figure 4 A flowchart illustrating another asynchronous copying method provided in an embodiment of this application;

[0087] Figure 5 A flowchart illustrating yet another asynchronous copying method provided in an embodiment of this application;

[0088] Figure 6 An interactive schematic diagram of an asynchronous copying method provided in an embodiment of this application;

[0089] Figure 7 This is a schematic diagram of an asynchronous replication device provided in an embodiment of this application;

[0090] Figure 8 A schematic diagram of another asynchronous replication device provided in the embodiments of this application;

[0091] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0092] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0093] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0094] It should be noted that although the terms "first," "second," etc., are used to describe various types of information in the embodiments of this application, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. Optionally, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0095] It should be understood that the terms "comprising" or "including" indicate the presence of the previously mentioned features, steps, or operations, but do not preclude the presence, occurrence, or addition of one or more other features, steps, or operations. The terms "and / or," etc., used in this application can be interpreted as inclusive, or mean any one or any combination thereof. Optionally, "A and / or B" means "any one of the following: A; B; A and B." Additionally, the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0096] Enterprise-grade storage disaster recovery technology is widely used in cross-regional data protection scenarios such as financial transactions, medical image archiving, and cloud computing platforms. Its core requirement is to asynchronously replicate changed data to remote sites while the main site continues to process host write operations. Such scenarios typically include a master device connected to the host and a slave device that synchronizes data with the master device, requiring remote data protection to be achieved without significantly impacting the performance of online services.

[0097] Existing remote asynchronous replication solutions typically employ periodic snapshots and differential data transmission to achieve data synchronization. The master device collects business writes over a period of time, generates a snapshot, performs difference analysis on the snapshots before and after, extracts the changed address ranges and corresponding data, and then sends the difference data to the slave device so that the slave can reconstruct the corresponding version of the data state.

[0098] This approach has significant limitations under high write loads. Firstly, snapshot creation and difference analysis continuously consume CPU, IOPS, and storage space, and processing latency increases with write volume, lengthening the remote replication cycle and making it difficult to meet the fast synchronization requirements of low RPO scenarios. Secondly, difference data is often discretely distributed, making it difficult to maintain stability and efficiency during transmission. Furthermore, the lack of fine-grained synchronization boundary markers during version switching can easily lead to incomplete or inconsistent slave data versions, thus affecting the reliability of disaster recovery.

[0099] How to reduce synchronization latency and resource consumption, and improve transmission efficiency and data version consistency in remote asynchronous replication scenarios has become an urgent technical problem to be solved.

[0100] To address the aforementioned technical issues, this application provides an asynchronous replication method. When processing a write request sent by the host, the method determines the current address change information corresponding to the write request and updates the target buffer based on this information. Subsequently, consistency point tags are generated in the log buffer at preset intervals. Then, the data to be replicated is determined based on the target buffer and the consistency point tags and transmitted to the slave device. This method is applicable to remote disaster recovery architectures where the master device is connected to both the host and slave devices, maintaining asynchronous replication continuity while balancing synchronization efficiency and version consistency.

[0101] Below, in conjunction with Figure 1 This section provides examples illustrating the application scenarios in which asynchronous copying methods are used.

[0102] Figure 1 This is a schematic diagram illustrating the architecture of an application scenario provided in an embodiment of this application. Please refer to [link / reference]. Figure 1 , Figure 1 It can include a host, a master device, and a slave device.

[0103] The host establishes a communication connection with the master device, and the master device and the slave device build an asynchronous replication link, which can be applied to data backup and disaster recovery scenarios such as distributed storage, data disaster recovery, and cloud data synchronization.

[0104] The host can serve as the entity responsible for generating and distributing business data. The host can carry front-end business read / write requests, generate various data write commands and corresponding address change data, and distribute data read / write requests to the master device.

[0105] The master device can serve as the core data processing and replication initiator. It can receive business data from the host, perform local data writing, change log recording, data caching, and other operations, and synchronize incremental data changes to slave devices via a pre-defined asynchronous replication mechanism.

[0106] Slave devices can serve as data backup nodes. They can receive incremental data change information synchronized from master devices in real time, perform data parsing and storage, and achieve asynchronous data synchronization with the master device. When the master device fails, crashes, or experiences data anomalies, the backup data from the slave device can be used to quickly restore business operations, ensuring the integrity and availability of business data.

[0107] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0108] Figure 2 This is a flowchart illustrating an asynchronous replication method provided in an embodiment of this application. The asynchronous replication method is executed by a master device, which is connected to both a host device and a slave device. The host device, acting as the initiator of write requests, continuously sends write commands to the master device targeting logical storage space. The slave device, acting as the remote replication receiver, receives the data to be replicated from the master device. Please refer to... Figure 2 The method includes:

[0109] S201. When processing a write request sent by the host, determine the current address change information corresponding to the write request.

[0110] A write request can be used to trigger the address change logging process.

[0111] A write request may include at least the target logical block address, write length, write data, and arrival order information related to this write.

[0112] Current address change information can be used to characterize the change in the data location corresponding to the write request.

[0113] Current address change information may include at least one piece of information that characterizes a change in the write location.

[0114] Current address change information may include sequence number, logical block address, data size, and changed data reference information.

[0115] The sequence number can be used to mark the order in which data is written.

[0116] For each data write request generated by the master, an incrementing sequence number is assigned. This sequence number is a globally incrementing sequence number.

[0117] The logical block address can be the block addressing number on the storage medium corresponding to the data to be written.

[0118] Logical block addresses can be used to locate the storage block where data updates have occurred.

[0119] Data size can be used to characterize the block length of the business data corresponding to this address change information.

[0120] The changed data reference information can be the information of the corresponding change record.

[0121] In practice, after receiving a write request from the host, the master device first completes the normal write process for the host's service path, and simultaneously extracts the address range and data length of this request during this process. To ensure stable order boundaries for subsequent replication processes, the master device can associate a sequence number with the write request and associate this sequence number with the address range of the logical block being written. For write requests spanning multiple consecutive logical blocks, the master device can generate an address change record covering the entire consecutive range; for write requests containing multiple discrete address segments, the master device can split and generate multiple current address change records to identify the change set within the corresponding time window when filtering based on consistency point labels later.

[0122] In one possible embodiment, the master device can receive block write commands from the host through the front-end interface module, obtain data pages from the write load through the cache management module, and then the replication record module can generate current address change information based on the starting logical block address and the number of blocks in the write command. For example, if the host writes 128KB of data to the logical block address (LBA) range 1000 to LBA1031, the master device generates a current address change message containing the starting address LBA1000 and an address length of 32 logical blocks; if the host initiates an overwrite write to LBA1008 to LBA1015 at the same time, the master device generates new current address change information to reflect the subsequent overwrite relationship in the timing.

[0123] Based on the above analysis, it can be seen that by determining the current address change information in real time when the host write arrives, instead of extracting differences based on snapshot comparison after the replication cycle ends, the basis for incremental records oriented towards replication can be directly formed. This gives subsequent log updates, consistency point tag generation, and data organization to be replicated a clear data source and order basis, thereby moving the difference identification action forward to the write processing time and reducing the burden of subsequent batch analysis.

[0124] S202. Update the target buffer based on the current address change information.

[0125] The target buffer includes the log buffer, or the target buffer includes both the log buffer and the data buffer.

[0126] The target buffer can be used to carry current address change information and data related to replication, forming a work area on the master device side for temporarily storing content to be replicated.

[0127] A log buffer can be used to record address change information and store consistency point tags. It can be implemented using a memory circular buffer. Each log record in the buffer can contain address change information and metadata related to the data length.

[0128] The data buffer is used to store data that will be used in subsequent copying processes.

[0129] When the target buffer includes a log buffer and a data buffer, the master device can adjust the corresponding buffer contents according to the current address change information to ensure that subsequent replication processing can determine the data to be replicated based on the target buffer.

[0130] In practice, when only a log buffer is used, the master device writes the current address change information and the relevant storage information of the data written this time into the log buffer. The records in the log buffer can be arranged in the order of writing arrival and are continuously appended as the writing arrives.

[0131] In an implementation where the target buffer includes both a log buffer and a data buffer, the master device can retrieve the log buffer based on the current address change information; and based on the retrieval results, write the corresponding address change record into the log buffer. Simultaneously, for the data required for replication, it performs a data copy operation from the log buffer to the data buffer and updates the corresponding associated metadata.

[0132] In one possible implementation, the log buffer adopts a fixed-length circular structure, and the master device maintains write pointers, read pointers, and boundary pointers for generated consistency point tags. When the write pointer advances to the end of the buffer, writing continues from the beginning of the buffer; if the relevant buffer space does not yet meet the current write record requirements, the master device can perform buffer reorganization or trigger subsequent transmission processing before releasing the original space.

[0133] Data buffers can be managed using a sequential append method.

[0134] For example, when there is a correlation between historical changes and current writes, the master device can coordinate and update the relevant content in the target buffer to ensure the correctness of subsequent replication.

[0135] After this processing, the target buffer can save the address change trajectory organized in the writing order as well as the data content required for subsequent replication, so that when the data to be replicated is extracted from the time window defined by a certain consistency point label, the corresponding address change information and data content can be obtained.

[0136] In one possible implementation, the current address change information includes a logical block address. Based on the current address change information, the target buffer is updated, including: determining whether there is a logical block address in the log buffer that is the same as the current address change information; if so, copying the second address change information corresponding to the logical block address in the log buffer to the data buffer, updating the changed data reference information corresponding to the second address change information in the log buffer, and writing the current address change information into the log buffer; if not, writing the current address change information into the log buffer.

[0137] After receiving a write request from the host, the control unit parses the logical block address in the write request and generates current address change information. Then, it performs an address matching query in the log buffer. The log buffer can consist of sequentially written log pages, and the data buffer can consist of a storage area for retaining reusable change records. Both can be located in the host device's memory or non-volatile cache media.

[0138] When the same logical block address is detected in the log buffer, the control unit copies the second address change information corresponding to the logical block address to the data buffer, updates the change data reference information of the original record, and then appends the current address change information to the log buffer in the writing order to maintain log continuity.

[0139] If no identical logical block address is detected, the current address change information is directly written to the log buffer.

[0140] This processing can be achieved by using an address index table in conjunction with a hash lookup. The address index table records the mapping relationship between logical block addresses and log record locations. In practical applications, other models of this component can also be selected, and this application does not limit this.

[0141] In this implementation, when the same logical block address appears repeatedly, the historical change record is synchronously copied to the data buffer and its validity is maintained by the change data reference information, so that the reusable data can be directly located when the data to be copied is extracted based on the consistency point label.

[0142] For address change information that does not repeat, only log records are retained to reduce redundant copying overhead. In this way, the target buffer can distinguish and manage frequently repeated address change records from ordinary change records while maintaining the continuity of log writing.

[0143] By adopting this specific implementation method, duplicate logical block addresses can be quickly identified and existing change records can be reused. Copyable data in the data buffer can be maintained by reference, thereby reducing the storage overhead caused by repeated writing and making the extraction and sending of subsequent data to be copied more direct, thus improving the processing efficiency and record consistency in asynchronous copying scenarios.

[0144] Based on the above analysis, this step reduces the subsequent processing path that relies on snapshot comparison to deduce the difference data by updating the target buffer in real time when the write occurs, and provides a directly readable data foundation for log range filtering and streaming.

[0145] S203. Generate consistency point labels in the log buffer at preset intervals.

[0146] Consistency point labels can be used to divide replication synchronization units and serve as the boundary for subsequently determining the data to be replicated.

[0147] Consistency point labels can be generated in the log buffer.

[0148] A consistency point label may contain at least a timestamp of the time of generation, a start order identifier, and an end order identifier. The start order identifier corresponds to the first valid address change record after the previous consistency point label, and the end order identifier corresponds to the current address change information of the last completed record before the time the current label was generated.

[0149] The preset duration can be used to control the generation cycle of consistency point tags.

[0150] The preset duration can be configured as a fixed millisecond or second-level time window, such as 50 milliseconds, 100 milliseconds, or 500 milliseconds, based on the replication latency target of the master device and the remote transmission capability, so as to divide the continuous write stream into manageable data version boundaries.

[0151] In practice, the master device is equipped with a periodic trigger or a timed task. After a preset time period, it accesses the current record status of the log buffer and obtains the range of address change records added since the generation of the last consistency point label.

[0152] If a new log record is added within the time window, the master device reads the beginning and end sequence identifier information of the window, generates a consistency point tag record, and writes the tag directly into the log buffer or into the tag index table associated with the log buffer, so that the tag forms a mapping relationship with the corresponding log interval.

[0153] If no new write request is generated within the time window, the master device may not generate an empty tag, or it may generate an empty boundary record containing only a timestamp and not corresponding to the data range, in order to meet the slave device's need for time progression recognition.

[0154] To ensure clear consistency boundaries, the master device uses address change records that have been stably written to the log buffer as the standard when generating tags, and does not include semi-finished records that have not yet been written and encapsulated into the current tag range, so that the sequence identification information range corresponding to a consistency point tag is complete and continuous.

[0155] In one possible embodiment, in addition to timestamps and sequence identification information ranges, consistency point tags may also include auxiliary metadata such as the tag number, the previous tag number, and the number of log entries within the time window. The master device establishes a replication task queue based on this metadata.

[0156] For example, if a batch of address change records is added to the log buffer within 0 to 100 milliseconds, a corresponding consistency point label is generated at 100 milliseconds; if another batch of address change records is added within 100 to 200 milliseconds, the next label is generated at 200 milliseconds.

[0157] In this way, the master device divides the continuously arriving write stream into multiple time-progressing replication units, each with a clear start and end sequence boundary.

[0158] When receiving data later, the slave device can reconstruct the version evolution relationship according to the tag order.

[0159] Based on the above analysis, it can be seen that by generating consistency point labels at preset intervals, the master device no longer relies on snapshot creation as the synchronization boundary, but directly defines the remote replication version based on the log range, so that the replication boundary has the characteristics of fine granularity, continuous advancement and explicit identification, thereby supporting fast asynchronous synchronization in low recovery point target scenarios.

[0160] S204. Based on the target buffer and consistency point label, determine the data to be copied and transmit the data to be copied to the slave device.

[0161] The data to be replicated can be the set of data extracted from the target buffer by the master device based on the log range defined by the consistency point label and sent to the slave device.

[0162] The data to be replicated includes at least the set of address change records corresponding to the consistency point label and the data content corresponding to each address change record.

[0163] When the target buffer only includes the log buffer, the data to be replicated can be directly composed of the log records and their associated data in the log buffer; when the target buffer includes both the log buffer and the data buffer, the master device can extract the corresponding data to be replicated from the log buffer and the data buffer respectively, and combine them into a data stream to be sent in the order defined by the consistency point labels.

[0164] Below, in conjunction with Figure 3 This section provides an explanation of the log buffer and data buffer.

[0165] Figure 3 This is a schematic diagram illustrating the structure of a log buffer and a data buffer provided in an embodiment of this application. See also... Figure 3 , Figure 3 It can include a log buffer and a data buffer.

[0166] The log buffer can include multiple address change messages and consistency point labels.

[0167] Address change information can carry the sequence number SEQ, logical block address LBA, data size SIZE, and changed data reference information REF.

[0168] The consistency point label contains the consistency point identifier LCP_ID, timestamp, and sequence number set.

[0169] For example, the sequence number set corresponding to LCP_ID:1 is {1,2,3}, which means that this consistency point covers all address change information corresponding to sequence number 1, sequence number 2, and sequence number 3.

[0170] The sequence number set corresponding to LCP_ID:2 is {4,5}, which means that this consistency point covers the address change information corresponding to sequence number 4 and sequence number 5.

[0171] The data buffer maintains a reference mapping table, which stores the correspondence between REF and data memory addresses.

[0172] like Figure 3 As shown, the data buffer includes:

[0173] SEQ:1 records REF1, associated with the REF1 entry in the data buffer, i.e., address 0x80;

[0174] SEQ:3 records REF2, associated with the REF2 entry in the data buffer, i.e., address 0x11;

[0175] SEQ:5 records carry REF3, associated with the REF3 entry in the data buffer, i.e., address 0x3f.

[0176] In practice, after detecting that a consistency point label has been generated and the timed copy task has started, the master device reads the record range corresponding to the consistency point label and filters out all target address change information falling within that record range from the target buffer. Subsequently, the master device sequentially traverses all target address change information, determining the logical block address range, data length, and corresponding data content for each change. The master device encapsulates each target address change information and its corresponding data into a copy entry, and can perform sequential concatenation, length alignment, and checksum padding before sending to form a copy message suitable for network transmission.

[0177] For records containing multiple consecutive logical block addresses within the same consistency point label interval, the master device can merge adjacent intervals without changing the order of occurrence, thereby reducing message header overhead; for discretely distributed records, the original entry boundaries are maintained during transmission.

[0178] When transmitting to slave devices, master devices can stream output through the replication link established with slave devices. During streaming output, the master device first sends boundary metadata corresponding to the consistency point label, and then continuously sends each replication entry within the range of that label, enabling the slave device to identify the complete range of a round of synchronization data based on the label boundary.

[0179] For example, when sending data corresponding to a certain tag, the master device first sends the timestamp of the tag and the corresponding record range, and then sends the address change records and data content within that range in sequence. After sending is complete, the master device can mark the tag as sent and reclaim the corresponding buffer space when the release conditions are met.

[0180] In one possible implementation, transmitting the data to be copied to the slave device includes: compressing the data to be copied to obtain compressed data; sending the compressed data to the slave device; and clearing the data to be copied after receiving a second confirmation message from the slave device.

[0181] In this embodiment, after the master device completes the organization of the data to be copied, it first sends the data to the compression module for encoding processing. The compressed data output by the compression module is transmitted to the slave device via the network sending module. After receiving the data, the slave device can decompress the compressed data and restore the corresponding data content.

[0182] The second confirmation message is returned by the slave device after confirming that the compressed data has been successfully received. After receiving this message, the master device will remove the data to be copied that has been confirmed for transmission in this round of copying from the local cache, the target buffer, or the copying queue associated with it, so that it will no longer participate in subsequent synchronization.

[0183] This processing method compresses the data to be replicated before sending it, reducing the transmission load on the replication link. Furthermore, by using the second confirmation message as the basis for clearing, the master device ensures that it only releases the corresponding data resources after confirming that the slave device has received them, thus completing the replication loop after transmission confirmation.

[0184] Therefore, the transmission overhead and local data usage during the remote replication process are controlled, and the timing of clearing the data to be replicated is consistent with the receipt confirmation, which can maintain the consistency of the replicated data.

[0185] Based on the above analysis, it can be seen that the master device directly filters the log range and extracts the corresponding data from the target buffer according to the consistency point label, omitting the process of comparing the snapshots of the previous and next versions; at the same time, transmitting according to the consistency point label can enable the slave device to obtain a clear version boundary, ensuring that the data sent in each round during the asynchronous replication process has identifiable integrity.

[0186] This embodiment provides an asynchronous replication method that, when processing a write request sent by the host, determines the current address change information corresponding to the write request; updates the target buffer based on the current address change information, the target buffer including a log buffer, or the target buffer including both a log buffer and a data buffer; generates consistency point tags in the log buffer at preset intervals; and determines the data to be replicated based on the target buffer and the consistency point tags, and transmits the data to be replicated to the slave device. In this way, compared to processing methods that rely on periodic snapshots and difference analysis, difference identification is moved forward to the write processing stage, allowing the master device to quickly organize replicated data according to log order even under high write loads, and clearly defining the version boundaries of each replication round using consistency point tags. This simultaneously meets the requirements of synchronization latency control and replication version consistency in cross-regional data protection scenarios.

[0187] Below, in conjunction with Figure 4 The process of determining the data to be copied based on the target buffer and the consistency point label (S204) is explained.

[0188] Figure 4 This is a flowchart illustrating another asynchronous replication method provided in an embodiment of this application. Based on the above embodiments, see [link to relevant documentation]. Figure 4 The method includes:

[0189] S401. In response to the start of the timed copy task, a start message is sent to the slave device.

[0190] The startup message includes a consistency point label.

[0191] A timed copy mechanism can be set up to enable periodic asynchronous data replication between the master and slave devices, ensuring version consistency between the master and slave data.

[0192] When the preset timed copy task is triggered, the master device can generate a start message to start this batch data copy process, and encapsulate the consistency point tag in the start message and send it to the slave device.

[0193] Among them, the consistency point label can be used to mark the log consumption start point corresponding to this copy task. As a unique identifier for the master device to filter the data to be synchronized and the slave device to align the data recovery point, it can accurately define the data synchronization range of this asynchronous copy.

[0194] S402. Receive the first confirmation message corresponding to the start message sent by the slave device.

[0195] The first confirmation message can be used to inform the master device that it has successfully received the start command and is ready to receive subsequent copied data.

[0196] After receiving the start message and consistency point tag sent by the master device, the slave device can complete message parsing and task readiness verification, and return the corresponding first confirmation message to the master device. By receiving the first confirmation message, the master device confirms that the data replication link between the master and slave ends is in normal condition and the synchronization task handshake is completed, thereby triggering the subsequent log filtering and data copying process to ensure the orderly progress of the asynchronous replication process.

[0197] S403. Based on the consistency point label, determine multiple target address change information in the log buffer.

[0198] The master device can use the consistency point label as the retrieval benchmark to filter out all address change information that is located after the consistency point and has not been synchronized in the log buffer, and determine the multiple address change information obtained by the filter as the target address change information.

[0199] This method can accurately pinpoint the incremental changes that need to be synchronized in this scheduled copy task, achieving precise incremental synchronization based on log points. Unlike the traditional full snapshot comparison method, this effectively reduces system resource overhead.

[0200] S404. Based on multiple target address change information, determine the data to be copied.

[0201] Each target address change can record core parameters such as the logical block address, data size, sequence number, and reference information of the changed data.

[0202] The master device can traverse all target address change information, and based on the storage addressing information and data content corresponding to each target address change information, aggregate and organize all incremental data that needs to be synchronized to the slave device this time, and finally determine the data to be copied.

[0203] The data to be replicated fully covers all business data changes after the consistency point, ensuring the integrity of the incremental data in this asynchronous replication and providing a complete data foundation for subsequent data synchronization and version consistency recovery of slave devices.

[0204] Further, based on multiple target address change information, the data to be copied is determined, including: for the first address change information, determining whether there is change data reference information in the first address change information; if so, the first copy data is obtained from the data buffer; if not, the first copy data is obtained from the log buffer. The first address change information is one of multiple target address change information. Based on the first copy data corresponding to each target address change information, the data to be copied is determined.

[0205] In practice, after determining multiple target address change information, the master device can sequentially read the change data reference information identifier and address mapping information carried by each address change information, and determine the corresponding data source location accordingly.

[0206] When the changed data reference information corresponds to a valid reference in the data buffer, the control logic indexes the storage location in the data buffer according to the changed data reference information and reads the historical data or current data corresponding to the address change information; when no changed data reference information is detected, the corresponding log entry is located from the log buffer according to the logical block address, timestamp or global incrementing sequence number in the address change information, and the first replicated data is obtained by parsing.

[0207] After each first copy of data is acquired, it can be aggregated according to the log record order or address order to form a set of data to be copied, which will then be used as the data content to be transmitted to the slave device.

[0208] In this embodiment, the data buffer and the log buffer can be composed of a memory ring buffer. The log buffer records address change information and its data reference relationship, and the data buffer stores a copy of the data after it has been updated by the copy-on-write mechanism.

[0209] When the master device reads the target address change information, it uses the change data reference information to determine the storage location of the data copy and extracts the first copy data from the corresponding buffer, thereby completing the aggregation of the data to be copied.

[0210] Using the above method, the master device can accurately select the data source between the data buffer and the log buffer based on the changed data reference information, and uniformly organize the data content corresponding to multiple target address change information into the data to be replicated. This ensures that the replicated object is consistent with the address change record and maintains the correspondence between log records and data copies. Since each target address change information can be parsed item by item and aggregated into the data to be replicated, the integrity and accuracy of the replicated data can be improved, meeting the consistency and reliability requirements in asynchronous replication scenarios.

[0211] The implementation details of each step in this application embodiment can be found in the description of the corresponding steps or operations in the above method embodiments; repeated content will not be repeated.

[0212] This embodiment provides an asynchronous replication method that, in response to the initiation of a scheduled copy task, sends a start message to the slave device. The start message includes a consistency point tag. The method then receives a first confirmation message corresponding to the start message from the slave device. Based on the consistency point tag, it determines multiple target address change information in the log buffer. Finally, based on the multiple target address change information, it determines the data to be replicated. By adopting this method, the determination of the data to be replicated is directly based on locating log records using the consistency point tag, reducing reliance on full snapshot differential analysis. The replication boundary also remains consistent with the trigger time of the scheduled copy task, thus enabling the remote replication process to have more stable data consistency and lower processing overhead.

[0213] Figure 5 This is a flowchart illustrating another asynchronous replication method provided in an embodiment of this application. See also... Figure 5 This method is applied to a slave device, which is connected to a master device. The method includes:

[0214] S501: Receive the start message sent by the master device through the synchronization queue, and send the first confirmation message to the master device.

[0215] The startup message includes a consistency point label.

[0216] The slave device is equipped with a receive control module, a synchronization queue management module, and a message feedback module.

[0217] The master device and the slave device can maintain a communication connection through a replication link.

[0218] The replication link can be an Ethernet-based Transmission Control Protocol (TCP) connection, a low-latency connection based on Remote Direct Memory Access (RDMA), or other reliable communication links capable of carrying replication message transmission.

[0219] After the slave device enters the replication receiving state, the receiving control module can first listen to the session port corresponding to the master device. When it detects the replication start message initiated by the master device, it writes the start message into the message receiving area of ​​the synchronization queue.

[0220] The synchronization queue can be used to receive the start message and data to be copied sent by the master device, and to carry out the subsequent data verification process.

[0221] Synchronization queues can be configured as ordered queue structures maintained at the session level.

[0222] A queue item in a synchronization queue includes at least a message type field, a session identifier field, a consistency point label field, a length field, and a status field.

[0223] The initiation message can be used by the master device to initiate the current replication process to the slave device. Its function is to establish the receiving context of the current replication before the data body is transmitted.

[0224] Consistency point tags can be used to identify units of data replication synchronization.

[0225] In this application, a combination of monotonically increasing tag number, timestamp and sequence number, or a logical consistency boundary number generated by the master end can be used, as long as it can uniquely identify the synchronization range in this round of replication.

[0226] In practice, after receiving the start message in the synchronization queue, the slave device first parses the consistency point tag in the message header and creates a receive session context corresponding to the consistency point tag in local memory. The session context records at least the tag value, the receive start time, the expected data status, the initial value of the verification status, and the subsequent write status.

[0227] If the same consistency point label already exists in the synchronization queue and the corresponding session is in an incomplete state, the slave device will identify the startup message as a duplicate startup message and, in accordance with the idempotent strategy, only refresh the session lifetime without creating the context again.

[0228] If the consistency point label is less than the upper bound of the currently completed consistency point labels, the slave device can mark it as an expired message and return an abnormal status.

[0229] In a normal receiving scenario, after receiving a valid start message, the slave device immediately sends a first confirmation message to the master device through the message feedback module.

[0230] The first confirmation message can carry at least the consistency point tag sent by the master device and the reception confirmation identifier.

[0231] The first confirmation message can be used to notify the master device and slave device that the receiving session context for this round of replication has been successfully established, and to allow the master device to continue sending the data to be replicated corresponding to the consistency point tag.

[0232] In one possible embodiment, the first confirmation message may also carry receive window information generated by the slave device. The first confirmation message can be used to indicate the amount of data that the synchronization queue can currently accept.

[0233] In another possible implementation, the first confirmation message only contains the consistency point label and the confirmation status code, and the master device continues to send the data to be replicated according to the predetermined fragment length.

[0234] Regardless of the message content format used, this step follows the same processing order: the slave device first receives the start message containing the consistency point tag, and then sends back the first confirmation message.

[0235] Based on the above analysis, this step enables the slave device to establish a one-to-one receiving boundary and receiving session context corresponding to the consistency point label before the data to be replicated arrives. This allows subsequent data entering the synchronization queue to be directly attached to the corresponding consistency point label for processing, ensuring that the master device and the slave device coordinate around the same synchronization unit.

[0236] S502: Receive the data to be copied sent by the master device through the synchronization queue.

[0237] The data to be replicated is determined by the master device through the target buffer and consistency point label. The target buffer includes the log buffer, or the target buffer includes both the log buffer and the data buffer.

[0238] After receiving the first confirmation message, the master device can send the data to be replicated corresponding to the consistency point tag to the slave device.

[0239] The slave device can continue to receive the data to be copied through the synchronization queue and complete data attachment, caching, and status advancement according to the established receive session context.

[0240] The data to be copied can be data that needs to be received, verified and written to the storage medium by the slave device, and its source is determined by the master device before sending.

[0241] The target buffer can be used by the master device to determine the source of the data to be copied.

[0242] One implementation includes only a log buffer, while another includes both a log buffer and a data buffer.

[0243] The log buffer can be used to store log data related to replication. The log data records at least the logical address, data length, write order, and association with consistency point labels written by the host.

[0244] The data buffer can be used to store the actual data content to be copied. The master device can extract the corresponding data payload from the data buffer according to the index relationship in the log buffer, and combine it with the log information to form the data to be copied that can be sent.

[0245] The slave device does not participate in the master device's extraction of the target buffer, but the slave device needs to be compatible with both types of data organization methods on the receiving side.

[0246] When the target buffer of the master device only includes the log buffer, the master device can directly filter the log records before the consistency point label or within the label boundary, and organize the log records and their corresponding data content into data to be replicated and sent.

[0247] When the target buffer includes a log buffer and a data buffer, the master device can first locate the actual data block in the data buffer based on the address mapping entry in the log buffer, and then encapsulate the log description information and the data block together as the data to be copied and send it.

[0248] After receiving the data to be replicated in the synchronization queue, the slave device writes each data unit into the data receiving area of ​​the synchronization queue and registers the corresponding consistency point label, fragment number, starting logical address, data length, and intra-fragment verification information in the corresponding queue entry.

[0249] To accommodate the packet-splitting mechanism during network transmission, the data to be replicated in this application can arrive in fragments. The slave device treats each fragment as a receiving element in the synchronization queue and merges multiple fragments into the same replication session based on the consistency point label in the start message.

[0250] In practice, the synchronization queue can include a head pointer, a tail pointer, and a status bitmap. After receiving a data packet, the slave device executes the write pointer advance, copies the data body to a circular buffer or a linked buffer, and then writes the metadata to the queue descriptor.

[0251] If the data packet sent by the master device contains the total number of fragments and the current fragment sequence number, the slave device marks the reception progress in the receiving session context accordingly; if the master device only sends data streams continuously, the slave device determines the data boundary corresponding to the current consistency point label based on the packet header length field and the end flag.

[0252] Once the data to be copied enters the synchronization queue, it remains in a pending verification state and is not directly pushed to disk. This state design separates the receiving and writing actions in time. The synchronization queue only undertakes the functions of ordered receiving and pending verification buffering, and does not modify the final data pages in the storage medium during this stage.

[0253] In one possible embodiment, the data to be replicated may include two parts: log records and corresponding data blocks. The slave device records the location mappings of the log segments and data segments in the synchronization queue so that they can be checked for matching during subsequent verification.

[0254] In another possible implementation, the data to be replicated is only represented as data update entries sorted by logical address, while log information is embedded in the data packet in the form of header metadata. The slave device can still complete the aggregation of the same replication unit based on the consistency point label and sequence number field.

[0255] Based on the above analysis, this step enables the slave device to receive the complete set of data to be copied organized by the master device according to the target buffer around the same consistency point label, and temporarily store the data set in a controlled state in the synchronization queue, providing the basic input for subsequent data verification.

[0256] S503. Based on the consistency point label, perform data verification on the data to be replicated through the synchronization queue.

[0257] After receiving the data to be copied in the synchronization queue, the slave device can trigger the data verification process.

[0258] Data verification can be used to verify whether the data to be copied meets the replication requirements before it is written to the storage medium. In this application, it is performed by the verification processing module inside the synchronization queue and does not depend on the production queue.

[0259] The verification processing module can read the consistency point tag from the receiving session context and retrieve all received data fragments to be replicated under that tag. Based on the consistency point tag, the module performs data verification on the data to be replicated to determine whether the data to be replicated meets the subsequent write requirements.

[0260] The consistency point labels here are not only used to characterize the synchronization range, but also serve as the basis for identifying the synchronization boundary, enabling the slave device to clearly determine whether all the data currently participating in the verification belongs to the same replication unit.

[0261] In practice, the slave device can verify the data to be replicated belonging to the current session in the synchronization queue based on the consistency point label, and update the corresponding state information of the session according to the verification result. If the verification fails, the session state is set to verification failure, and subsequent write phases are prevented. In this phase, the synchronization queue not only serves as a cache, but also manages the state machine of the data to be replicated.

[0262] Each queue item can move between the states of "pending receipt", "received", "pending verification", "verification passed", and "verification failed".

[0263] After verification is completed, the session context can be updated to a writable state or a failed state. If it is a failed state, the slave device can send an error message back to the master device and wait for retransmission, but this error handling does not change the basic process of verification before writing in this claim.

[0264] Optionally, based on the consistency point label, data verification is performed on the data to be replicated through the synchronization queue, including: determining the sequence number and logical block address corresponding to multiple target address change information according to the consistency point label; and performing integrity verification and consistency verification on the data to be replicated through the synchronization queue based on the sequence number and logical block address corresponding to multiple target address change information.

[0265] In practice, after generating a consistency point label, the master device can retrieve multiple target address change information corresponding to the consistency point label from the log buffer, extract the sequence number corresponding to each target address change information according to the log generation order, and then determine the corresponding logical block address by combining the address mapping table.

[0266] After receiving the data to be copied, the synchronization queue first checks whether the data fragments are consistent with the preset log order based on the sequence number, and then checks whether the content of the data block completely covers the target address range based on the logical block address. If the data to be copied has missing blocks, duplicate blocks, or inconsistent address offsets, the integrity check or consistency check is determined to have failed.

[0267] The combined use of sequence number and logical block address allows the verification process to simultaneously constrain data order and address correspondence. The verification result is output as a pass flag or an exception flag and is sent back to the subsequent replication control logic.

[0268] This implementation completes the verification and association of the data to be copied within the synchronization queue, ensuring that the data verification and the queue's sending and receiving process maintain a consistent timing relationship. It can uniformly include the address change range, log order, and target block location corresponding to the consistency point label into the verification scope, thereby ensuring that the data entering the subsequent writing stage has a complete address coverage relationship and a consistent version boundary.

[0269] In this way, the validity of the data to be replicated in the remote asynchronous replication chain can be confirmed in a timely manner, thereby improving the correct correspondence and version consistency of the replicated data.

[0270] Based on the above analysis, this step is crucial for resolving the issue of incomplete or inconsistent data versions on the slave end. By using consistency point tags as the boundaries of synchronization units and performing data verification of the data to be replicated within the synchronization queue, the slave device can confirm that the replicated data meets the subsequent write conditions before entering the storage medium. Thus, subsequent write operations correspond to data versions that have already been verified, rather than incomplete data fragments in the receiving process. This ensures that the asynchronous replication link maintains version consistency even in scenarios involving network packet fragmentation, discrete address updates, and batch transmissions.

[0271] S504. After the data verification is passed, the data to be copied is written to the storage medium through the production queue.

[0272] After the data verification is successful, the slave device can transfer the data to be replicated under the corresponding consistency point tag from the synchronization queue to the production queue, and then the production queue will execute the writing to the storage medium.

[0273] The production queue is a queue structure that is actually written to the storage medium after the data has been verified. It is separate from the synchronization queue, so that the receiving verification path and the disk writing path are independent of each other.

[0274] Storage media can be used to carry the copied data that is eventually written.

[0275] The storage medium can be a hard disk drive, a solid-state drive, a disk array logical volume, a non-volatile memory, or a storage pool composed of these media.

[0276] After the production module in the slave device reads the "verification passed" flag from the receiving session context, it generates write tasks according to the data set corresponding to the consistency point label and pushes the write tasks into the production queue in sequence.

[0277] In practice, each task item in the production queue includes at least a consistency point label, a target logical address, a write length, a data cache pointer, and a write completion status.

[0278] If the data to be copied is stored in the synchronization queue buffer in segments according to logical addresses, the slave device first generates a set of write descriptors before entering the production queue, with each descriptor pointing to a continuous data segment; if the data to be copied has been reassembled into a continuous buffer in the synchronization queue in sequence, the production queue can directly reference the continuous buffer and initiate a write operation.

[0279] The production module writes data to the backend volume, file system image region, or raw device address space of the slave device according to the task items in the production queue. After the write is completed, the status of the task item is updated to "completed", and the corresponding consistency point tag is registered as a "written to disk" tag so that subsequent replication rounds can maintain the recovery point sequence based on this tag.

[0280] In one possible embodiment, the production queue is dequeued serially in the order of receipt, so that data within the same consistency point tag is written to the storage medium in the order verified and confirmed by the synchronization queue.

[0281] In another possible implementation, the production queue can merge multiple write descriptions of adjacent logical addresses into a batch write request to reduce the number of backend writes, but the merge operation is only performed without changing the content and boundaries of the same consistency point label data.

[0282] If the storage medium supports cache flush control, the slave device can perform a flush confirmation after writing all the data corresponding to the current consistency point tag to ensure that the data version corresponding to the consistency point tag has been stably recorded in the non-volatile medium.

[0283] The slave device can release the corresponding cache space and receive session context resources in the synchronization queue, and mark the task item with that tag in the production queue as completed.

[0284] Based on the above analysis, this step decouples the receiving, verification, and disk writing stages by switching the verified data from the synchronization queue to the production queue before writing it to the storage medium. Since the production queue only processes data sets that have passed consistency point tag verification, the data written to the storage medium is always a consistent version with well-defined boundaries, a fixed order, and complete content. For multi-round replication data continuously output by the master device in remote asynchronous replication scenarios, this processing method allows the slave device to solidify recovery points round by round using consistency point tags, reducing the number of incomplete versions entering the media.

[0285] The implementation details of each step in this application embodiment can be found in the description of the corresponding steps or operations in the above method embodiments; repeated content will not be repeated.

[0286] This embodiment provides an asynchronous replication method that receives a start message from a master device via a synchronization queue and sends a first confirmation message to the master device. The start message includes a consistency point tag. The method also receives data to be replicated from the master device via the synchronization queue. This data is determined by the master device using a target buffer and the consistency point tag. The target buffer may include a log buffer or a combination of a log buffer and a data buffer. Based on the consistency point tag, the data to be replicated is validated via the synchronization queue. After successful validation, the data to be replicated is written to the storage medium via a production queue. In this application, the slave device first establishes a receiving session context corresponding to the consistency point tag based on the start message. Then, it receives the data to be replicated organized by the master device around the consistency point tag and performs data validation within the synchronization queue. Finally, the validated data is handed over to the production queue for writing to the storage medium. This allows for reception and disk write control during asynchronous replication using the consistency point tag as a stable synchronization boundary. This enables the slave device to form a verifiable and disk-writeable complete version of the data for each round of replication, even when the master device uses a log buffer or a combination of a log buffer and a data buffer to organize the replicated data.

[0287] Below, in conjunction with Figure 6 This section provides an example to illustrate the process of an asynchronous copying method.

[0288] Figure 6 This is an interactive schematic diagram illustrating an asynchronous replication method provided in an embodiment of this application. See also... Figure 6 , Figure 6 This includes master devices and slave devices.

[0289] After establishing connections with both the master and slave devices, the master device processes write requests sent by the master. When processing a write request, the master device obtains the current address change information corresponding to that request. This information includes the logical block address and may also correspond to the data size and a globally incrementing sequence number. The master device then writes the current address change information to the target buffer. The target buffer may consist only of the log buffer, or it may include both the log buffer and the data buffer.

[0290] When the master device writes the current address change information to the target buffer, it first checks whether the same logical block address already exists in the log buffer. The log buffer can be configured as a memory circular buffer to hold consecutively arriving write request records.

[0291] When the same logical block address exists in the log buffer, the second address change information corresponding to that logical block address is copied to the data buffer. For logical block addresses that have already been recorded, when a subsequent write request overwrites them, the original corresponding data is copied to the reference position in the data buffer, the changed data reference information corresponding to the second address change information is updated in the log buffer, and then the current address change information is written to the log buffer; when the same logical block address does not exist in the log buffer, the current address change information is directly written to the log buffer.

[0292] The task initiator in the master device generates consistency point labels in the log buffer at preset time intervals, so that the address change information in the log buffer forms identifiable data boundaries according to the consistency point labels.

[0293] When the scheduled copy task starts, the master device sends a start message to the slave device, carrying a consistency point tag. The slave device receives the start message through a synchronization queue and returns a first acknowledgement message corresponding to the start message. After receiving the first acknowledgement message, the master device determines multiple target address change information in the log buffer based on the consistency point tag. For any of the first address change information, the master device checks whether there is any changed data reference information. If there is changed data reference information, the master device retrieves the first copied data corresponding to the first address change information from the data buffer; if there is no changed data reference information, the master device retrieves the first copied data corresponding to the first address change information from the log buffer. The master device summarizes the first copied data corresponding to the multiple target address change information to form the data to be copied, and performs compression processing on the data to be copied to obtain compressed data.

[0294] After receiving the data to be replicated from the master device through the synchronization queue, the slave device performs data verification based on the consistency point label carried in the startup message.

[0295] The slave device determines the sequence number and logical block address corresponding to multiple target address change information based on the consistency point label. Then, it combines the sequence number and logical block address to perform integrity and consistency checks on the data to be copied through the synchronization queue.

[0296] After the data verification is successful, the slave device writes the data to be copied to the storage medium through the production queue and sends a second confirmation message to the master device.

[0297] After receiving the second confirmation message, the master device clears the data to be copied and enters the next copy cycle divided by consistency point tags.

[0298] Figure 7 This is a schematic diagram of an asynchronous replication device provided in an embodiment of this application. Please refer to [link / reference]. Figure 7 It is applied to the master device, which is connected to both the host and slave devices. The asynchronous replication device 700 includes a recording module 701, an update module 702, a generation module 703, and a determination module 704.

[0299] The recording module 701 is used to determine the current address change information corresponding to the write request when processing the write request sent by the host;

[0300] Update module 702 is used to update the target buffer according to the current address change information. The target buffer includes the log buffer or the target buffer includes both the log buffer and the data buffer.

[0301] The generation module 703 is used to generate consistency point tags in the log buffer at preset intervals.

[0302] The determination module 704 is used to determine the data to be copied based on the target buffer and the consistency point label, and to transmit the data to be copied to the slave device.

[0303] Optionally, in the above-described apparatus, the determining module 704 is specifically used for:

[0304] In response to the start of the scheduled copy task, a start message is sent to the slave device. The start message includes a consistency point label.

[0305] Receive the first confirmation message corresponding to the start message sent by the slave device;

[0306] Based on the consistency point labels, identify multiple target address change information in the log buffer;

[0307] Based on multiple target address change information, determine the data to be copied.

[0308] Optionally, in the above-described apparatus, the determining module 704 is specifically used for:

[0309] For the first address change information, determine whether there is changed data reference information in the first address change information. If so, obtain the first copied data from the data buffer. If not, obtain the first copied data from the log buffer. The first address change information is one of multiple target address change information.

[0310] Based on the first copy data corresponding to each target address change information, determine the data to be copied.

[0311] Optionally, in the above-described apparatus, the update module 702 is specifically used for:

[0312] Determine if the log buffer contains a logical block address that is identical to the current address change information;

[0313] If so, copy the second address change information corresponding to the logical block address in the log buffer to the data buffer, update the changed data reference information corresponding to the second address change information in the log buffer, and write the current address change information into the log buffer;

[0314] If not, the current address change information is written to the log buffer.

[0315] Optionally, in the above-described apparatus, the determining module 704 is specifically used for:

[0316] The data to be copied is compressed to obtain compressed data.

[0317] Send compressed data to the slave device;

[0318] After receiving the second confirmation message from the slave device, clear the data to be copied.

[0319] Figure 8 This is a schematic diagram of another asynchronous replication apparatus provided in an embodiment of this application. Please refer to... Figure 8 It is applied to slave devices, which are connected to master devices. The asynchronous replication device 800 includes a transceiver module 801, a verification module 802, and a write module 803.

[0320] The transceiver module 801 is used to receive the start message sent by the master device through the synchronization queue and send the first confirmation message to the master device. The start message includes a consistency point label.

[0321] The transceiver module 801 is also used to receive data to be replicated sent by the master device through a synchronization queue. The data to be replicated is determined by the master device through a target buffer and a consistency point label. The target buffer includes a log buffer or a log buffer and a data buffer.

[0322] The verification module 802 is used to verify the data to be replicated through the synchronization queue based on the consistency point label;

[0323] The write module 803 is used to write the data to be copied to the storage medium through the production queue after the data verification is passed.

[0324] Optionally, in the above-described apparatus, the verification module 802 is specifically used for:

[0325] Based on the consistency point label, determine the sequence number and logical block address corresponding to multiple target address change information respectively;

[0326] Based on the sequence number and logical block address corresponding to multiple target address change information, the integrity and consistency checks of the data to be copied are performed through a synchronization queue.

[0327] The asynchronous replication device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0328] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Please refer to... Figure 9 The electronic device 900 may include: a memory 901, a processor 902, and a transceiver 903.

[0329] Memory 901 is used to store program instructions;

[0330] The processor 902 is used to execute the program instructions stored in the memory so that the electronic device 900 performs the above-described method.

[0331] Transceiver 903 may include a transmitter and / or a receiver. The transmitter may also be referred to as a transmitter, transmitter port, or transmitter interface, and the receiver may also be referred to as a receiver port, receiver interface, or similar descriptions. Exemplarily, memory 901, processor 902, and transceiver 903 are interconnected via bus 904.

[0332] This application also provides a computer program product that can be executed by a processor, and when the computer program product is executed, the above-described method can be implemented.

[0333] The asynchronous copying apparatus, electronic device, computer-readable storage medium, and computer program product of this application embodiment can execute the technical solutions shown in the above-described asynchronous copying method embodiments. Their implementation principles and beneficial effects are similar and will not be repeated here.

[0334] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0335] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0336] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0337] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0338] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0339] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0340] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0341] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0342] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An asynchronous replication method, characterized in that, Applied to a master device, wherein the master device is connected to both a host device and a slave device, the method includes: When processing the write request sent by the host, determine the current address change information corresponding to the write request; Update the target buffer based on the current address change information, wherein the target buffer includes a log buffer, or the target buffer includes the log buffer and a data buffer; Consistency point labels are generated in the log buffer at preset intervals. Based on the target buffer and the consistency point label, the data to be copied is determined and transmitted to the slave device.

2. The method according to claim 1, characterized in that, Based on the target buffer and the consistency point label, the data to be replicated is determined, including: In response to the initiation of the timed copy task, a start message is sent to the slave device, the start message including the consistency point tag; Receive the first confirmation message corresponding to the start message sent by the slave device; Based on the consistency point label, multiple target address change information is determined in the log buffer; Based on the multiple target address change information, the data to be copied is determined.

3. The method according to claim 2, characterized in that, Based on the multiple target address change information, the data to be copied is determined, including: For the first address change information, determine whether there is change data reference information in the first address change information. If yes, obtain the first copied data from the data buffer. If no, obtain the first copied data from the log buffer. The first address change information is one of the multiple target address change information. Based on the first copy data corresponding to each target address change information, determine the data to be copied.

4. The method according to any one of claims 1-3, characterized in that, The current address change information includes logical block addresses. Based on the current address change information, the target buffer is updated, including: Determine whether there is a logical block address in the log buffer that is the same as the current address change information; If so, copy the second address change information corresponding to the logical block address in the log buffer to the data buffer, update the changed data reference information corresponding to the second address change information in the log buffer, and write the current address change information into the log buffer; If not, the current address change information is written to the log buffer.

5. The method according to any one of claims 1-3, characterized in that, Transmitting the data to be copied to the slave device includes: The data to be copied is compressed to obtain compressed data; The compressed data is sent to the slave device; After receiving the second confirmation message sent by the slave device, the data to be copied is cleared.

6. An asynchronous replication method, characterized in that, Applied to a slave device, wherein the slave device is connected to a master device, the method includes: The system receives the startup message sent by the master device through a synchronization queue and sends a first confirmation message to the master device. The startup message includes a consistency point tag. The synchronization queue receives data to be replicated sent by the master device. The data to be replicated is determined by the master device through a target buffer and a consistency point label. The target buffer includes a log buffer, or the target buffer includes the log buffer and a data buffer. Based on the consistency point label, the data to be replicated is verified through the synchronization queue; After the data verification is passed, the data to be copied is written to the storage medium through the production queue.

7. The method according to claim 6, characterized in that, Based on the consistency point label, the data to be replicated is validated through the synchronization queue, including: Based on the consistency point label, determine the sequence number and logical block address corresponding to the multiple target address change information respectively; Based on the sequence number and logical block address corresponding to the multiple target address change information, the data to be copied is subjected to integrity and consistency checks through the synchronization queue.

8. An asynchronous replication device, characterized in that, Applied to a master device, wherein the master device is connected to both a host device and a slave device, the device includes: The recording module is used to determine the current address change information corresponding to the write request when processing the write request sent by the host; The update module is used to update the target buffer according to the current address change information, wherein the target buffer includes a log buffer, or the target buffer includes the log buffer and a data buffer; The generation module is used to generate consistency point tags in the log buffer at preset intervals; The determination module is used to determine the data to be copied based on the target buffer and the consistency point label, and to transmit the data to be copied to the slave device.

9. An asynchronous replication device, characterized in that, Applied to a slave device, wherein the slave device is connected to a master device, the device includes: The transceiver module is used to receive the start message sent by the master device through a synchronization queue, and send a first confirmation message to the master device. The start message includes a consistency point tag. The transceiver module is also used to receive data to be replicated sent by the master device through the synchronization queue. The data to be replicated is determined by the master device through a target buffer and a consistency point tag. The target buffer includes a log buffer, or the target buffer includes the log buffer and a data buffer. The verification module is used to verify the data to be replicated through the synchronization queue based on the consistency point label; The write module is used to write the data to be copied to the storage medium through the production queue after the data verification is passed.

10. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-5, or the method as described in claim 6 or 7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-5, or the method of claim 6 or 7.

Images