59 results about "Asynchronous replication" patented technology

A computer-implemented method comprises transitioning from a unidirectional asynchronous replication to initiating bi-directional synchronous replication between one or more storage objects of a first consistency group (CG1) of a primary storage site and one or more storage objects of a second consistency group (CG2) of a secondary storage site, converting the one or more storage objects of the CG2 from data protection read only access to read write access, and performing a reverse synchronization process between the one or more storage objects of the CG2 and the one or more storage objects of the CG1 including instantiating a reverse splitter on each volume of CG2, establishing reverse sync replication sessions for each storage object of the CG2, and allowing input output (IO) access to the one or more storage objects of the CG2.

The present application relates to the technical field of middleware, and in particular to a cache data semi-synchronization method, comprising comprises the following steps: writing and running a distributed cache proxy layer service using a preset programming language, receiving a read / write command request from an application client, parsing a Key consistent hashing value and sending same to a rear-end Redis sharding node. A master instance parses a message packet, marks a write operation and records same in an AOF file. Upon write to disk, the master instance sends an AOF file transfer command to a slave instance node, and the slave instance acknowledges reception. When receiving a reception response, the master instance updates an acknowledgment offset of the slave node and responds to the client. When network delay is high, data of a master node and a slave node is marked as "asynchronous", and it is automatically downgraded to asynchronous replication. According to the present application, by means of a self-developed Redis kernel, an asynchronous ack mechanism is used to optimize redis to achieve master-slave semi-synchronization, thereby enhancing data reliability of a single point of failure, and eliminating the influence introduced by a native Redis asynchronous replication mechanism such as data delay and data consistency.

A disaster recovery method, device and system can be applied to the technical field of storage. The method comprises the steps that a first data center synchronously copies backup data to a second data center through a first synchronous copying link; under the condition that a third data center is normal, the first data center asynchronously copies the backup data to the third data center through a first asynchronous copying link; under the condition that the third data center breaks down, the first data center asynchronously copies the backup data to a fourth data center through a second asynchronous copying link; wherein the first data center and the second data center are both production centers, the first data center and the second data center are located in a first area, the third data center and the fourth data center are both disaster recovery centers, and the third data center and the fourth data center are located in a second area. The disaster recovery performance of the disaster recovery system can be reduced in a stepped manner, so that user demands can be met to the greatest extent.

The embodiment of the invention provides a log backup method and device, a database system and computing device.The log backup method is applied to the database system and comprises the steps that a logic log, generated through triggering, of a first data operation instruction in a target transaction is obtained, and the target transaction comprises multiple data operation instructions; the first data operation instruction is any one of a plurality of data operation instructions, and the logic log is generated by executing the first data operation instruction to complete modification operation on main library data; copying the logic log in real time and sending the logic log to the standby library, and returning to execute the step of obtaining the logic log of the first data operation instruction generated by triggering in the target transaction; and under the condition that the standby library receives the logic logs of the data operation instructions of the target transaction, submitting the logic logs of the data operation instructions. Through real-time transmission of the logic log, the risk of degradation to asynchronous replication caused by timeout is reduced, and the stability, availability and response speed of a database system are enhanced.

Various examples are directed to systems and methods for operating a primary database management system and a secondary database management system. The secondary database management system may receive a takeover request indicating that the secondary database management system is to assume a role of the primary database system. The secondary database management system may determine that a last valid commit of a first host of the secondary database system is an oldest last valid commit. The secondary database management system may revert to a first state of the primary database management system corresponding to the last valid commit of the first host. The secondary database management system may be configured to assume the role of the primary database management system.

An RDMA data transmission method, a network device and a network system is provided. An implementation of the method is applied to a network device comprising an xPU and an RNIC, the xPU comprises a first engine, and the RNIC comprises a second engine in communication with the first engine, a WQE Buffer, and an RDMA engine in communication with the second engine. The method comprises: assembling, by the first engine, a WE based on a hardware offload asynchronous copy instruction set, and transmitting the WE to the second engine; storing, by the second engine, the WE into the WQE Buffer and transmitting the WE to the RDMA engine; performing, by the RDMA engine, data processing based on the WE, the data processing comprising data transmitting, memory accessing and queue managing; and receiving, by the first engine, a feedback of the data processing transmitted via the second engine.

The invention relates to the technical field of sparse matrix multiplication, and discloses a high-performance SpMM kernel implementation method based on a dense tensor core. The method comprises the following steps: storing a sparse matrix in a BlockNM sparse format, and pre-fetching metadata in the sparse matrix into a shared memory of a GPU (Graphics Processing Unit); asynchronously copying the sparse matrix and the dense matrix to a shared memory, setting a four-stage pipeline scheduling strategy, and controlling SpMM operations of different iteration processes; loading the sparse matrix and the dense matrix in the shared memory into a register, setting an offset for each thread accessing the shared memory, and enabling each thread to access different banks in the shared memory based on the offset; the matrix multiplication and addition operation is executed in the register to obtain a calculation result, the calculation result is temporarily stored in the shared memory from the register, then the calculation result in the shared memory is written back to the global memory, and the efficiency of sparse matrix multiplication is improved.

The invention provides a load migration method and system for balancing the energy efficiency of a server cluster, and relates to the technical field of distributed computing. According to the method, the power consumption data of the distributed cache nodes in the server cluster are acquired, and the high-power consumption nodes and the low-power consumption nodes are identified; meanwhile, access frequency data of the cache data fragments are obtained, and cross-node unified sorting is carried out to generate a global access popularity result; based on the popularity sorting and the power consumption data, determining a fragment with the highest access popularity in the high-power consumption nodes as a fragment to be migrated, and selecting a low-power consumption node with sufficient resources as a target node to form a migration decision; updating a cluster metadata mapping relation, establishing a data transmission channel, asynchronously copying data blocks in batches, and verifying the data blocks; and after completion, switching a client route, and pointing the request to a target node. Energy efficiency balance of the server cluster can be realized, the overall power consumption is effectively reduced, and the resource utilization rate is improved.

The invention provides a data disaster tolerance backup method and system of a security system, and relates to the technical field of data backup, and the method comprises the steps: carrying out content-based segmentation on a data object to obtain a fragment sequence; generating a fingerprint sequence and constructing a verifiable ordered index; encrypting the fragment sequence by using a session key and generating an authentication tag to form encrypted packaging; executing cross-domain asynchronous replication based on a multi-dimensional evaluation strategy; and carrying out consistency check on the submitted serial number and the ordered index, if the check is passed, carrying out persistent storage, otherwise, carrying out rollback and retransmission. According to the method, a data disaster recovery backup mechanism with high security, verifiability and flexibility is realized.

A disaster recovery method, apparatus and system, which can be applied in the technical field of storage. The method comprises: a first data center synchronously duplicating backup data to a second data center by means of a first synchronous duplication link; when a third data center is normal, the first data center asynchronously duplicating the backup data to the third data center by means of a first asynchronous duplication link; and when the third data center has a fault, the first data center asynchronously duplicating the backup data to a fourth data center by means of a second asynchronous duplication link, wherein the first data center and the second data center are both production centers and are located in a first area, and the third data center and the fourth data center are both disaster recovery centers and are located in a second area. The present application can achieve a step-by-step reduction in the disaster recovery performance of a disaster recovery performance, such that user demands are met to the greatest extent.

One embodiment provides a graphics processor comprising a base die including a plurality of chiplet sockets and a plurality of chiplets coupled with the plurality of chiplet sockets. A chiplet of the plurality of chiplets including a graphics core cluster including a plurality of graphics cores, a distributed shared local memory including shared local memory within each of the plurality of graphics cores, and a direct memory access engine within each of the plurality of graphics cores, the direct memory access engine configured to asynchronously copy data from a memory device to the distributed shared local memory.

Techniques for tracking incoming writes and snap creation / deletion in memory to improve asynchronous replication and support lower RPOs. In the techniques, a storage system uses its data cache to receive data specified in write requests issued by storage clients, while dedicating an amount of the cache memory to track and record offsets / lengths of writes directed to source volumes. At the end of each replication interval, the storage system obtains a list of the recorded offsets / lengths for each source volume, identifies and reads areas of the source volume that were written to during the replication interval using the list, and replicates data from the identified areas to a destination volume. Because the list of recorded offsets / lengths of incoming writes for the source volume is compiled and available from volatile cache memory, it can be generated and accessed very quickly using reduced processing / memory resources, allowing for lower RPOs in asynchronous replication processes.

Techniques can include: establishing an asynchronous replication configuration for a replication session of a source volume group VG1 and a target volume group VG2; and performing processing to transition the replication session to a low RPO (recovery point objective) replication mode comprising: setting the replication session to an alternative asynchronous replication (AAR) mode; transitioning the replication session to a hybrid mode; creating a first snapset of snapshots of source volumes of VG1; replicating each snapshot of the first snapset from a source to a target system in either the low RPO replication mode or the AAR mode based on an associated member level asynchronous replication mode corresponding to said each snapshot; and constructing a second snapset of snapshots of target volumes of VG2, wherein each snapshot in the second snapset denotes a same point in time copy as a corresponding snapshot in the first snapset.

A method for use in a storage system including a primary storage system and a secondary storage system, the method comprising: starting an asynchronous replication manager; detecting that an intrusion detector has detected an intrusion in the primary storage system; in response to the intrusion being detected, causing the asynchronous replication manager to stop transmission, to a secondary storage system, of replication data associated with any data buckets that are collected by the asynchronous replication manager while allowing the asynchronous replication manager to continue collecting new data buckets; generating an alert that indicates that the intrusion detector has detected the intrusion; receiving a response that is indicative of whether the intrusion is confirmed; when the response indicates that the intrusion is not confirmed, resuming the asynchronous replication manager, and when the intrusion is confirmed, causing the asynchronous replication manager to stop collecting data buckets.

Techniques for tracking incoming writes and snap creation / deletion in memory to improve asynchronous replication and support lower RPOs. In the techniques, a storage system uses its data cache to receive data specified in write requests issued by storage clients, while dedicating an amount of the cache memory to track and record offsets / lengths of writes directed to source volumes. At the end of each replication interval, the storage system obtains a list of the recorded offsets / lengths for each source volume, identifies and reads areas of the source volume that were written to during the replication interval using the list, and replicates data from the identified areas to a destination volume. Because the list of recorded offsets / lengths of incoming writes for the source volume is compiled and available from volatile cache memory, it can be generated and accessed very quickly using reduced processing / memory resources, allowing for lower RPOs in asynchronous replication processes.

Techniques can include: configuring stretched volumes for asynchronous replication; specifying replication settings denoting selected replication service levels of asynchronous replication optimizations for the stretched volumes; performing asynchronous replication for the stretched volumes based on the replication settings; monitoring a current amount denoting an amount of a resource that is free and available for use; and responsive to determining that the current amount is below a minimum, performing a corrective action to increase the current amount, wherein the corrective action includes: changing a replication setting for a stretched volume from a first replication service level to a second replication service level, wherein the second replication service level is expected to consume less of the resource than the first replication service level when performing asynchronous replication for the stretched volume.

The invention provides a data replication method of a distributed file system and the distributed file system, and belongs to the technical field of storage. In the method, a first storage cluster executes a first operation request and generates an operation log corresponding to the semantics of the first operation request based on the semantics of the first operation request for a file, and the operation log is sent to a second storage cluster; and the second storage cluster plays back the first operation request based on the operation log. In the method, the operation log is generated on the basis of the semantics of the operation request, and the operation request is played back on the basis of the operation log, so that data replication is realized on a file semantic layer, and compared with data replication realized on a block layer, the method does not depend on the replication capability of the block layer, and can realize synchronous replication and asynchronous replication capabilities by using a set of data replication scheme; therefore, synchronous replication and asynchronous replication can be realized without different block layer scheme designs, the complexity of system development and maintenance can be reduced, and the data replication performance is improved.

Techniques providing data path strategies for improving storage performance at DR sites. The techniques include receiving, in an asynchronous replication process, a large replication data transfer including data changes of a production volume since the last synchronization to a replica volume, partitioning the replication data into multiple small write requests, tagging each small write request as a write request to the replica volume, and performing early evicting, from cache memory, all cache pages used to cache host data specified in the small write requests; deep compression of contiguous host data specified in the small write requests; stream separation on the small write requests, each small write request being tagged as corresponding to a specific production site; and / or flushing host data having the same retention period to a specific region of physical storage space for the replica volume, each small write request being tagged with hint information indicating the retention period.

The invention discloses a switching method and device of a distributed disaster recovery system and electronic equipment, and relates to the technical field of distribution.The distributed disaster recovery system at least comprises a main cluster and a standby cluster, the main cluster and the standby cluster adopt an asynchronous replication mechanism, and the method comprises the steps that node information of L nodes included in the main cluster is collected through a monitoring module, the node information of the L nodes is transmitted to a control module to be analyzed, and the node state of each node is obtained; under the condition that the node state of the service node is an abnormal state, detecting the number of continuous abnormities of the IP address corresponding to the service node; and updating the service node as a main node in the standby cluster and generating a first signal under the condition that the number of continuous exceptions is greater than a preset number and the nodes in the same region corresponding to the service node are in an abnormal state. The technical problem that in the prior art, a disaster recovery system adopting a one-host and two-backup architecture is high in data loss risk during host and backup switching is solved.

The invention discloses a fund value data record distributed receiving and storing method and system, and belongs to the field of distributed big data, the method comprises the following steps: recording and analyzing the time consumption of each election cycle, the election success frequency, the failure frequency and network delay data, and adjusting the election timeout time in real time; collecting data of response time, log synchronization rate and stability of the nodes, scoring the nodes and allocating priorities to optimize the election process; a batch replication and asynchronous replication mechanism is adopted, so that communication delay between nodes is reduced; and during the election period, the voting strategy is adjusted according to the distribution priority, the node with the high priority preferentially obtains votes, and the priority of each node is calculated and updated in real time for the next election period. According to the method, the consistency and synchronism of data in a distributed database cluster can be effectively improved, the number of election failures is reduced, the election efficiency and stability of a system are optimized, and safe and efficient storage of fund numerical data is guaranteed.

The invention provides a distributed multi-copy high-performance high-availability scheme based on a data file, which comprises the following steps: acquiring a write request of a user, and executing the write request through a leader consensus layer in a distributed cluster; monitoring a data file generated in the storage engine; generating a loading item based on the data file, asynchronously sending the loading item to the follower to load the data file, and continuing to execute the next write-in request by the leader; and after the follower completes data loading through a direct loading technology, a loading result is updated to consensus data of the distributed cluster through an asynchronous callback mode. Compared with a current mainstream consensus protocol, the method adopts an asynchronous replication mechanism, returns a result that user write-in success does not depend on copy synchronization, does not need extra waiting, performs state synchronization by using a data file with a high compression ratio, effectively reduces the performance bottleneck of copy synchronization while ensuring the utilization efficiency of system resources, and improves the user experience. And the performance is linearly expanded along with the number of copies.

The present application relates to the technical field of storage, and provides a data replication method for a distributed file system, and a distributed file system. In the method, on the basis of semantics of a first operation request for a file, a first storage cluster executes the first operation request and generates an operation log corresponding to the semantics of the first operation request; the first storage cluster sends the operation log to a second storage cluster; and the second storage cluster replays the first operation request on the basis of the operation log. In the described method, generating the operation log on the basis of the semantics of the operation request, and replaying the operation request on the basis of the operation log are to implement data replication at a file semantic layer; compared with implementing data replication at a block layer, the method does not rely on the replication capability of the block layer, and can use a single data replication scheme to implement both synchronous and asynchronous replication capabilities; thus, there is no need to design different block layer schemes to implement synchronous and asynchronous replication, thereby reducing the complexity of system development and maintenance, improving the data replication performance.

The invention provides a standby database data delay detection method and device, and relates to the technical field of databases, the method is implemented by a standby database application, and the method comprises the steps that heartbeat time in a standby database heartbeat table is inquired, and the heartbeat time in the standby database heartbeat table is obtained from a main database heartbeat table in an asynchronous replication mode; judging whether the delay between the heartbeat time and the current database time is lower than a preset threshold value or not; in response to the condition that the delay is lower than a preset threshold value, normally processing the application service; and blocking the processing of the application service in response to the fact that the delay is higher than a preset threshold value. According to the method, the illusion of no delay cannot be generated, and meanwhile, compared with the prior art, the condition of faults of the main and standby libraries is also included, so that the reliability is higher. Besides, the method is only related to synchronization of heartbeat time, so that service logic does not need to be sensed, and the method is a universal scheme and is low in implementation cost.

The application relates to the technical field of land and resource data processing, in particular to a land and resource planning data storage method, system, device and medium, which comprises the following steps: carrying out fragmentation processing on land and resource data to obtain data fragmentation information; carrying out real-time analysis on the data fragmentation information to obtain hot data and cold data, carrying out synchronous replication processing on the hot data, and carrying out asynchronous replication processing on the cold data; obtaining memory state information, dividing the memory state information to obtain hierarchical information, adjusting the hierarchical information according to the hot data, and storing the hot data into a plurality of hierarchical caches; obtaining data access traffic information in real time, dynamically adjusting the data fragmentation information according to the data access traffic information to obtain distribution update information for updating the hot data and the cold data, enabling the land and resource to be stably stored, improving the consistency of the data, and thus improving the reliability of the system and the user experience.

The application provides a data disaster backup method and system of a security system, relates to the technical field of data backup, and comprises the following steps: obtaining a segment sequence by performing content-based segmentation on a data object; generating a fingerprint sequence and constructing a verifiable ordered index; encrypting the segment sequence by using a session key and generating an authentication tag to form an encrypted package; performing cross-domain asynchronous replication based on a multi-dimensional evaluation strategy; performing consistency checking by submitting a serial number and the ordered index, and if the checking is passed, performing persistent storage, otherwise, rolling back and retransmitting. The method realizes a data disaster backup mechanism with high security, verifiability and flexibility.

To continuously maintain performance higher than a certain level regarding asynchronous copy between a plurality of sites by preparing processing performance required in the future in advance.SOLUTION: In a storage system including another storage device, in which processing performance of a virtual computer that executes asynchronous copy processing of asynchronously copying data with one storage device that executes input and output processing of data in response to a request from a host device may be changed, a processing performance adjustment unit changes in advance processing performance required at a second time point temporally later than a first time point by the virtual computer of the other storage device, before the second time point, when the asynchronous copy processing is executed at the second time point according to a state of the input and output processing at the first time point in the one storage device.SELECTED DRAWING: Figure 2

A method for use in a storage system including a primary storage system and a secondary storage system, the method comprising: starting an asynchronous replication manager; detecting that an intrusion detector has detected an intrusion in the primary storage system; in response to the intrusion being detected, causing the asynchronous replication manager to stop transmission, to a secondary storage system, of replication data associated with any data buckets that are collected by the asynchronous replication manager while allowing the asynchronous replication manager to continue collecting new data buckets; generating an alert that indicates that the intrusion detector has detected the intrusion; receiving a response that is indicative of whether the intrusion is confirmed; when the response indicates that the intrusion is not confirmed, resuming the asynchronous replication manager, and when the intrusion is confirmed, causing the asynchronous replication manager to stop collecting data buckets.

The invention discloses a distributed acquisition data consistency processing method, and belongs to the technical field of data acquisition and network management. The distributed acquisition data consistency processing method comprises the following steps: acquiring target data; calculating the priority of the target data to obtain high-priority data, medium-priority data and low-priority data; carrying out data processing on the high-priority data by adopting a practical Byzantine fault-tolerant algorithm to obtain first consistency data; performing data processing on the middle-priority data by adopting an asynchronous replication and periodic compensation strategy to obtain second consistency data; performing master-slave replication consistency and causal consistency processing on the low-priority data to obtain third consistency data; and summarizing the first consistency data, the second consistency data and the third consistency data to obtain consistency data.

The described technology is generally directed towards automatic storage of data / digital objects in a self-adaptive replication system. Data replication can be switched between synchronous (sync) replication and asynchronous (async) replication between a local storage system and a remote storage system. Latency between writing data to the local storage system and the remote storage system can be determined. In the event of the latency is less than a threshold value, sync replication can be utilized, and in the event of the latent is greater than the threshold value, async replication can be utilized. Accordingly, data replication can adapt based on infrastructure, volume of data to be write, period(s) of no-data writes, etc. Async replication can use a buffer to temporarily store data during a write operation at the remote storage system. Alternatively, a list can be maintained for data to write to the remote storage system for subsequent replication.

The present invention provides a distributed metadata remote asynchronous replication method, apparatus, and device for addressing the technical problem of low synchronization efficiency. Based on directory anchoring technology, the present invention attributes metadata modifications to the asynchronous remote replication feature directory to an anchored metadata service node. The metadata log for the asynchronous remote replication feature directory is read from the anchored metadata service node, and metadata modifications are extracted from the metadata log. After semantic conflict resolution, concurrent real-time synchronization is performed. This invention improves the synchronization efficiency of feature directory metadata.