50 results about "Asynchronous replication" patented technology

An RDMA data transmission method, a network device and a network system is provided. An implementation of the method is applied to a network device comprising an xPU and an RNIC, the xPU comprises a first engine, and the RNIC comprises a second engine in communication with the first engine, a WQE Buffer, and an RDMA engine in communication with the second engine. The method comprises: assembling, by the first engine, a WE based on a hardware offload asynchronous copy instruction set, and transmitting the WE to the second engine; storing, by the second engine, the WE into the WQE Buffer and transmitting the WE to the RDMA engine; performing, by the RDMA engine, data processing based on the WE, the data processing comprising data transmitting, memory accessing and queue managing; and receiving, by the first engine, a feedback of the data processing transmitted via the second engine.

The invention provides a load migration method and system for balancing the energy efficiency of a server cluster, and relates to the technical field of distributed computing. According to the method, the power consumption data of the distributed cache nodes in the server cluster are acquired, and the high-power consumption nodes and the low-power consumption nodes are identified; meanwhile, access frequency data of the cache data fragments are obtained, and cross-node unified sorting is carried out to generate a global access popularity result; based on the popularity sorting and the power consumption data, determining a fragment with the highest access popularity in the high-power consumption nodes as a fragment to be migrated, and selecting a low-power consumption node with sufficient resources as a target node to form a migration decision; updating a cluster metadata mapping relation, establishing a data transmission channel, asynchronously copying data blocks in batches, and verifying the data blocks; and after completion, switching a client route, and pointing the request to a target node. Energy efficiency balance of the server cluster can be realized, the overall power consumption is effectively reduced, and the resource utilization rate is improved.

The invention provides a data disaster tolerance backup method and system of a security system, and relates to the technical field of data backup, and the method comprises the steps: carrying out content-based segmentation on a data object to obtain a fragment sequence; generating a fingerprint sequence and constructing a verifiable ordered index; encrypting the fragment sequence by using a session key and generating an authentication tag to form encrypted packaging; executing cross-domain asynchronous replication based on a multi-dimensional evaluation strategy; and carrying out consistency check on the submitted serial number and the ordered index, if the check is passed, carrying out persistent storage, otherwise, carrying out rollback and retransmission. According to the method, a data disaster recovery backup mechanism with high security, verifiability and flexibility is realized.

One embodiment provides a graphics processor comprising a base die including a plurality of chiplet sockets and a plurality of chiplets coupled with the plurality of chiplet sockets. A chiplet of the plurality of chiplets including a graphics core cluster including a plurality of graphics cores, a distributed shared local memory including shared local memory within each of the plurality of graphics cores, and a direct memory access engine within each of the plurality of graphics cores, the direct memory access engine configured to asynchronously copy data from a memory device to the distributed shared local memory.

Techniques for tracking incoming writes and snap creation / deletion in memory to improve asynchronous replication and support lower RPOs. In the techniques, a storage system uses its data cache to receive data specified in write requests issued by storage clients, while dedicating an amount of the cache memory to track and record offsets / lengths of writes directed to source volumes. At the end of each replication interval, the storage system obtains a list of the recorded offsets / lengths for each source volume, identifies and reads areas of the source volume that were written to during the replication interval using the list, and replicates data from the identified areas to a destination volume. Because the list of recorded offsets / lengths of incoming writes for the source volume is compiled and available from volatile cache memory, it can be generated and accessed very quickly using reduced processing / memory resources, allowing for lower RPOs in asynchronous replication processes.

Techniques can include: establishing an asynchronous replication configuration for a replication session of a source volume group VG1 and a target volume group VG2; and performing processing to transition the replication session to a low RPO (recovery point objective) replication mode comprising: setting the replication session to an alternative asynchronous replication (AAR) mode; transitioning the replication session to a hybrid mode; creating a first snapset of snapshots of source volumes of VG1; replicating each snapshot of the first snapset from a source to a target system in either the low RPO replication mode or the AAR mode based on an associated member level asynchronous replication mode corresponding to said each snapshot; and constructing a second snapset of snapshots of target volumes of VG2, wherein each snapshot in the second snapset denotes a same point in time copy as a corresponding snapshot in the first snapset.

A method for use in a storage system including a primary storage system and a secondary storage system, the method comprising: starting an asynchronous replication manager; detecting that an intrusion detector has detected an intrusion in the primary storage system; in response to the intrusion being detected, causing the asynchronous replication manager to stop transmission, to a secondary storage system, of replication data associated with any data buckets that are collected by the asynchronous replication manager while allowing the asynchronous replication manager to continue collecting new data buckets; generating an alert that indicates that the intrusion detector has detected the intrusion; receiving a response that is indicative of whether the intrusion is confirmed; when the response indicates that the intrusion is not confirmed, resuming the asynchronous replication manager, and when the intrusion is confirmed, causing the asynchronous replication manager to stop collecting data buckets.

Techniques for tracking incoming writes and snap creation / deletion in memory to improve asynchronous replication and support lower RPOs. In the techniques, a storage system uses its data cache to receive data specified in write requests issued by storage clients, while dedicating an amount of the cache memory to track and record offsets / lengths of writes directed to source volumes. At the end of each replication interval, the storage system obtains a list of the recorded offsets / lengths for each source volume, identifies and reads areas of the source volume that were written to during the replication interval using the list, and replicates data from the identified areas to a destination volume. Because the list of recorded offsets / lengths of incoming writes for the source volume is compiled and available from volatile cache memory, it can be generated and accessed very quickly using reduced processing / memory resources, allowing for lower RPOs in asynchronous replication processes.

The invention provides a data replication method of a distributed file system and the distributed file system, and belongs to the technical field of storage. In the method, a first storage cluster executes a first operation request and generates an operation log corresponding to the semantics of the first operation request based on the semantics of the first operation request for a file, and the operation log is sent to a second storage cluster; and the second storage cluster plays back the first operation request based on the operation log. In the method, the operation log is generated on the basis of the semantics of the operation request, and the operation request is played back on the basis of the operation log, so that data replication is realized on a file semantic layer, and compared with data replication realized on a block layer, the method does not depend on the replication capability of the block layer, and can realize synchronous replication and asynchronous replication capabilities by using a set of data replication scheme; therefore, synchronous replication and asynchronous replication can be realized without different block layer scheme designs, the complexity of system development and maintenance can be reduced, and the data replication performance is improved.

Techniques providing data path strategies for improving storage performance at DR sites. The techniques include receiving, in an asynchronous replication process, a large replication data transfer including data changes of a production volume since the last synchronization to a replica volume, partitioning the replication data into multiple small write requests, tagging each small write request as a write request to the replica volume, and performing early evicting, from cache memory, all cache pages used to cache host data specified in the small write requests; deep compression of contiguous host data specified in the small write requests; stream separation on the small write requests, each small write request being tagged as corresponding to a specific production site; and / or flushing host data having the same retention period to a specific region of physical storage space for the replica volume, each small write request being tagged with hint information indicating the retention period.

The invention discloses a switching method and device of a distributed disaster recovery system and electronic equipment, and relates to the technical field of distribution.The distributed disaster recovery system at least comprises a main cluster and a standby cluster, the main cluster and the standby cluster adopt an asynchronous replication mechanism, and the method comprises the steps that node information of L nodes included in the main cluster is collected through a monitoring module, the node information of the L nodes is transmitted to a control module to be analyzed, and the node state of each node is obtained; under the condition that the node state of the service node is an abnormal state, detecting the number of continuous abnormities of the IP address corresponding to the service node; and updating the service node as a main node in the standby cluster and generating a first signal under the condition that the number of continuous exceptions is greater than a preset number and the nodes in the same region corresponding to the service node are in an abnormal state. The technical problem that in the prior art, a disaster recovery system adopting a one-host and two-backup architecture is high in data loss risk during host and backup switching is solved.

The invention discloses a fund value data record distributed receiving and storing method and system, and belongs to the field of distributed big data, the method comprises the following steps: recording and analyzing the time consumption of each election cycle, the election success frequency, the failure frequency and network delay data, and adjusting the election timeout time in real time; collecting data of response time, log synchronization rate and stability of the nodes, scoring the nodes and allocating priorities to optimize the election process; a batch replication and asynchronous replication mechanism is adopted, so that communication delay between nodes is reduced; and during the election period, the voting strategy is adjusted according to the distribution priority, the node with the high priority preferentially obtains votes, and the priority of each node is calculated and updated in real time for the next election period. According to the method, the consistency and synchronism of data in a distributed database cluster can be effectively improved, the number of election failures is reduced, the election efficiency and stability of a system are optimized, and safe and efficient storage of fund numerical data is guaranteed.

The present application relates to the technical field of storage, and provides a data replication method for a distributed file system, and a distributed file system. In the method, on the basis of semantics of a first operation request for a file, a first storage cluster executes the first operation request and generates an operation log corresponding to the semantics of the first operation request; the first storage cluster sends the operation log to a second storage cluster; and the second storage cluster replays the first operation request on the basis of the operation log. In the described method, generating the operation log on the basis of the semantics of the operation request, and replaying the operation request on the basis of the operation log are to implement data replication at a file semantic layer; compared with implementing data replication at a block layer, the method does not rely on the replication capability of the block layer, and can use a single data replication scheme to implement both synchronous and asynchronous replication capabilities; thus, there is no need to design different block layer schemes to implement synchronous and asynchronous replication, thereby reducing the complexity of system development and maintenance, improving the data replication performance.

The invention provides a standby database data delay detection method and device, and relates to the technical field of databases, the method is implemented by a standby database application, and the method comprises the steps that heartbeat time in a standby database heartbeat table is inquired, and the heartbeat time in the standby database heartbeat table is obtained from a main database heartbeat table in an asynchronous replication mode; judging whether the delay between the heartbeat time and the current database time is lower than a preset threshold value or not; in response to the condition that the delay is lower than a preset threshold value, normally processing the application service; and blocking the processing of the application service in response to the fact that the delay is higher than a preset threshold value. According to the method, the illusion of no delay cannot be generated, and meanwhile, compared with the prior art, the condition of faults of the main and standby libraries is also included, so that the reliability is higher. Besides, the method is only related to synchronization of heartbeat time, so that service logic does not need to be sensed, and the method is a universal scheme and is low in implementation cost.

The application relates to the technical field of land and resource data processing, in particular to a land and resource planning data storage method, system, device and medium, which comprises the following steps: carrying out fragmentation processing on land and resource data to obtain data fragmentation information; carrying out real-time analysis on the data fragmentation information to obtain hot data and cold data, carrying out synchronous replication processing on the hot data, and carrying out asynchronous replication processing on the cold data; obtaining memory state information, dividing the memory state information to obtain hierarchical information, adjusting the hierarchical information according to the hot data, and storing the hot data into a plurality of hierarchical caches; obtaining data access traffic information in real time, dynamically adjusting the data fragmentation information according to the data access traffic information to obtain distribution update information for updating the hot data and the cold data, enabling the land and resource to be stably stored, improving the consistency of the data, and thus improving the reliability of the system and the user experience.

The application provides a data disaster backup method and system of a security system, relates to the technical field of data backup, and comprises the following steps: obtaining a segment sequence by performing content-based segmentation on a data object; generating a fingerprint sequence and constructing a verifiable ordered index; encrypting the segment sequence by using a session key and generating an authentication tag to form an encrypted package; performing cross-domain asynchronous replication based on a multi-dimensional evaluation strategy; performing consistency checking by submitting a serial number and the ordered index, and if the checking is passed, performing persistent storage, otherwise, rolling back and retransmitting. The method realizes a data disaster backup mechanism with high security, verifiability and flexibility.

To continuously maintain performance higher than a certain level regarding asynchronous copy between a plurality of sites by preparing processing performance required in the future in advance.SOLUTION: In a storage system including another storage device, in which processing performance of a virtual computer that executes asynchronous copy processing of asynchronously copying data with one storage device that executes input and output processing of data in response to a request from a host device may be changed, a processing performance adjustment unit changes in advance processing performance required at a second time point temporally later than a first time point by the virtual computer of the other storage device, before the second time point, when the asynchronous copy processing is executed at the second time point according to a state of the input and output processing at the first time point in the one storage device.SELECTED DRAWING: Figure 2

A method for use in a storage system including a primary storage system and a secondary storage system, the method comprising: starting an asynchronous replication manager; detecting that an intrusion detector has detected an intrusion in the primary storage system; in response to the intrusion being detected, causing the asynchronous replication manager to stop transmission, to a secondary storage system, of replication data associated with any data buckets that are collected by the asynchronous replication manager while allowing the asynchronous replication manager to continue collecting new data buckets; generating an alert that indicates that the intrusion detector has detected the intrusion; receiving a response that is indicative of whether the intrusion is confirmed; when the response indicates that the intrusion is not confirmed, resuming the asynchronous replication manager, and when the intrusion is confirmed, causing the asynchronous replication manager to stop collecting data buckets.

The invention discloses a distributed acquisition data consistency processing method, and belongs to the technical field of data acquisition and network management. The distributed acquisition data consistency processing method comprises the following steps: acquiring target data; calculating the priority of the target data to obtain high-priority data, medium-priority data and low-priority data; carrying out data processing on the high-priority data by adopting a practical Byzantine fault-tolerant algorithm to obtain first consistency data; performing data processing on the middle-priority data by adopting an asynchronous replication and periodic compensation strategy to obtain second consistency data; performing master-slave replication consistency and causal consistency processing on the low-priority data to obtain third consistency data; and summarizing the first consistency data, the second consistency data and the third consistency data to obtain consistency data.

The described technology is generally directed towards automatic storage of data / digital objects in a self-adaptive replication system. Data replication can be switched between synchronous (sync) replication and asynchronous (async) replication between a local storage system and a remote storage system. Latency between writing data to the local storage system and the remote storage system can be determined. In the event of the latency is less than a threshold value, sync replication can be utilized, and in the event of the latent is greater than the threshold value, async replication can be utilized. Accordingly, data replication can adapt based on infrastructure, volume of data to be write, period(s) of no-data writes, etc. Async replication can use a buffer to temporarily store data during a write operation at the remote storage system. Alternatively, a list can be maintained for data to write to the remote storage system for subsequent replication.

The present invention provides a distributed metadata remote asynchronous replication method, apparatus, and device for addressing the technical problem of low synchronization efficiency. Based on directory anchoring technology, the present invention attributes metadata modifications to the asynchronous remote replication feature directory to an anchored metadata service node. The metadata log for the asynchronous remote replication feature directory is read from the anchored metadata service node, and metadata modifications are extracted from the metadata log. After semantic conflict resolution, concurrent real-time synchronization is performed. This invention improves the synchronization efficiency of feature directory metadata.

Techniques can include: establishing an asynchronous replication configuration for a replication session of a source volume group VG1 and a target volume group VG2; and performing processing to transition the replication session to a low RPO (recovery point objective) replication mode comprising: setting the replication session to an alternative asynchronous replication (AAR) mode; transitioning the replication session to a hybrid mode; creating a first snapset of snapshots of source volumes of VG1; replicating each snapshot of the first snapset from a source to a target system in either the low RPO replication mode or the AAR mode based on an associated member level asynchronous replication mode corresponding to said each snapshot; and constructing a second snapset of snapshots of target volumes of VG2, wherein each snapshot in the second snapset denotes a same point in time copy as a corresponding snapshot in the first snapset.

The described technology is generally directed towards automatic storage of data / digital objects in a self-adaptive replication system. Data replication can be switched between synchronous (sync) replication and asynchronous (async) replication between a local storage system and a remote storage system. Latency between writing data to the local storage system and the remote storage system can be determined. In the event of the latency is less than a threshold value, sync replication can be utilized, and in the event of the latent is greater than the threshold value, async replication can be utilized. Accordingly, data replication can adapt based on infrastructure, volume of data to be write, period(s) of no-data writes, etc. Async replication can use a buffer to temporarily store data during a write operation at the remote storage system. Alternatively, a list can be maintained for data to write to the remote storage system for subsequent replication.

The embodiments of this specification provide a log backup method, apparatus, database system, and computing device. The log backup method is applied to the database system and includes: obtaining the logical log of the first data operation instruction in the target transaction generated by triggering, where the target transaction includes multiple data operation instructions, the first data operation instruction is any one of the multiple data operation instructions, and the logical log is generated by triggering the modification operation of the master database data when executing the first data operation instruction; replicating and sending the logical log to the standby database in real time, and returning to the step of obtaining the logical log of the first data operation instruction in the target transaction generated by triggering; when the standby database receives the logical logs of the data operation instructions of the target transaction, submitting the logical logs of the data operation instructions. By transmitting the logical log in real time, the risk of degradation to asynchronous replication due to timeout is reduced, and the stability, availability, and response speed of the database system are enhanced.

Energy aware data migration and data replication is disclosed. Data stored in a source storage may be migrated or replicated to a target storage based on a migration time or replication time predicted or estimated by an awareness engine. The awareness engine may include models configured to predict the migration time or the replication time so that the migration time or replication time accounts for a cost and / or source of energy. The migration operation or replication operation is performed at the estimated migration or replication time.

Techniques are provided for transitioning a first storage object and a second storage object from an asynchronous replication state to a synchronous replication state. Metadata operations are logged into a metadata log. Dirty data, modified by data operations executed upon the first storage object, are tracked using a dirty region log. Metadata operations are replicated from the metadata log to the second storage object during a metadata drain phase. During the metadata drain phase, incoming operations are logged into the metadata log when the metadata log has a non-zero size, otherwise, the metadata operations are replicated to the second storage object. After the metadata drain phase, a data drain phase is performed using the dirty region log.

The invention discloses a fault-tolerant distributed storage method and system supporting asynchronous and synchronous replication, and the method comprises the steps: deploying to-be-stored data to a main node, splitting the data into a plurality of sub-data, deploying the sub-data to a standby node by the main node, deploying a standby node of the sub-data and deploying redundant copies of other sub-data except the sub-data in all the sub-data at the same time; judging whether the data replication request adopts an asynchronous replication mode or a synchronous replication mode; when the judgment result is synchronous replication, the access flow of the data replication request is distributed to the main node, and when the judgment result is asynchronous replication, the access flow of the data replication request is distributed to the standby node; and when a difference value between the copied standby node data and the corresponding data of the main node exceeds a preset value, taking the corresponding data of the main node as the data responding to the data copying request. The method has the advantages that the balance between meeting user requirements and saving node resources can be realized, and the like.

The invention relates to a data synchronization method and device, computer equipment, a readable storage medium and a program product. When a data synchronization condition is met, a data level determined based on attributes of data and access information for the data is obtained, a corresponding target synchronization strategy is determined based on the data level, and the to-be-synchronized data is synchronized to a target data center according to the target synchronization strategy. Compared with a traditional method of synchronizing all the data according to an asynchronous copying mode, the method has the advantages that the data grades of the data are determined by combining the attributes of the data and the access information of the data, and different synchronization strategies are determined for different data grades, so that the data with different importance degrees are synchronized according to the graded synchronization strategies, and the synchronization efficiency is improved. And the synchronization consistency of important data is improved, so that the data synchronization efficiency is improved.

Techniques can include: establishing an asynchronous replication configuration of a volume V1 of a first system and a volume V2 of a second system; determining data changes between a successive pair of snapshots of V1; transferring the data changes from the first system to the second system; determining an average data transfer rate and an average data change rate; determining, based at least in part, on the average data transfer rate and the average data change rate, a predicted time denoting an amount of time expected to complete a final processing phase that transitions the asynchronous replication configuration from a first state to a second state; determining whether the predicted time exceeds a threshold; and responsive to determining that the predicted time does not exceed the threshold, performing the final processing phase that transitions the asynchronous replication configuration from the first state to the second state.

Systems and methods for asynchronously replicating mutable metadata fields in distributed environments comprise a lineage-based approach, wherein individual changes to metadata fields are stored as timestamped entries in lineage tables that enable efficient tracking and replication of updates. Advantageously, this approach to replicates individual entries independently, allows updates to be replied in any order while maintaining consistency across source and destination systems, and eliminates the need for strict sequencing, synchronization, or exactly-once delivery, which ensured reliability even in active-active configurations or high-latency networks. Metadata queries retrieve the most recent values based on timestamps to ensure consistency and eventual alignment of data between databases. Various embodiments support bidirectional replication, concurrent updates, and error handling, making them suitable for geographically distributed systems, hybrid cloud environments, and disaster recovery applications.