18 results about "Replica management" patented technology

The invention discloses an optical fiber data storage management system and method based on big data, and relates to the technical field of storage management. According to the optical fiber data storage management system and method based on big data, the spatial-temporal characteristics of the historical access records are extracted through the LSTM network, the data heat value is calculated in combination with the time decay function, and the thermal distribution diagram of the data blocks is generated. And based on the thermal distribution map, constructing a storage resource allocation model by using a deep reinforcement learning algorithm, and dynamically adjusting the optical path priority and copy distribution. A data prefetching strategy is optimized through the spatio-temporal joint index and the vector similarity index, and a high-frequency data cache index distribution diagram is generated. And comprehensively considering the data access delay, the index hit rate and the copy migration frequency, obtaining an optical fiber storage efficiency index, and adjusting a storage strategy according to the index. According to the method, the dynamic scheduling and utilization efficiency of data storage resources is effectively improved, the data access delay is reduced, the copy management is optimized, and the overall performance of a storage system is remarkably improved.

The invention is suitable for the technical field of computers, and provides a containerized multi-copy deployment management method and device, terminal equipment and a storage medium, and the method comprises the steps that after a container copy is started, state information of the container copy is periodically updated to a real-time state list, and the state information comprises an IP address and a node health level; reading state information of other container copies in the real-time state list; dynamically electing a master container copy according to a preset master-slave election rule based on the IP address and the node health level in the real-time state list; when it is detected that the current main container copy is abnormal, an automatic switching process is triggered, and the main container copy is elected again. According to the method and the device, the efficiency and the reliability of copy management in a containerized multi-copy deployment scene can be improved, so that the service continuity and the data consistency are guaranteed.

The invention provides a distributed multi-site operation cloud storage system, which comprises a data management module used for being responsible for storage, retrieval and copy management of data and ensuring the integrity and availability of the data; the storage pool management module is used for managing resources of a storage pool; the copy distribution module is used for automatically calculating a placement rule of a data copy and realizing distributed storage of data; the network scheduling module is used for transmitting and scheduling data in a cross-machine-room network; the fault detection module is used for monitoring the running state of the storage system, detecting faults and abnormal conditions, and giving an alarm and processing in time; and the arbitration module is used for being responsible for decision judgment and brain split prevention in a fault scene, and ensuring data consistency and system stability. The invention provides a distributed multi-site operation cloud storage system which solves the problem that once a machine room goes wrong, the whole cloud platform and a business application system borne by the cloud platform are both in an unavailable state.

The present invention discloses a fiber optic data storage management system and method based on big data, which relates to the technical field of storage management. The fiber optic data storage management system and method based on big data extracts the spatio-temporal features of historical access records through an LSTM network, calculates the data heat value in combination with a time decay function, and generates a heat distribution map of data blocks. Based on the heat distribution map, a storage resource allocation model is constructed using a deep reinforcement learning algorithm to dynamically adjust the optical path priority and replica distribution. The data prefetching strategy is optimized through spatio-temporal joint indexing and vector similarity indexing to generate a high-frequency data cache index distribution map. By comprehensively considering data access latency, index hit rate, and replica migration frequency, a fiber optic storage efficiency index is obtained, and the storage strategy is adjusted according to this index. This method effectively improves the dynamic scheduling and utilization efficiency of data storage resources, reduces data access latency, optimizes replica management, and significantly improves the overall performance of the storage system.

A method for use in a storage system including a primary storage system and a secondary storage system, the method comprising: starting an asynchronous replication manager; detecting that an intrusion detector has detected an intrusion in the primary storage system; in response to the intrusion being detected, causing the asynchronous replication manager to stop transmission, to a secondary storage system, of replication data associated with any data buckets that are collected by the asynchronous replication manager while allowing the asynchronous replication manager to continue collecting new data buckets; generating an alert that indicates that the intrusion detector has detected the intrusion; receiving a response that is indicative of whether the intrusion is confirmed; when the response indicates that the intrusion is not confirmed, resuming the asynchronous replication manager, and when the intrusion is confirmed, causing the asynchronous replication manager to stop collecting data buckets.

The present invention discloses a distributed all-flash storage data writing method, device and medium, which relate to the field of distributed storage technology and are used to optimize the writing performance of all-flash storage. In view of the problem that the current aggregated writing scheme still has room for further improvement in optimizing the writing performance, a distributed all-flash storage data writing method is provided. This method reuses EC data blocks as replica data blocks. Not only can it still provide the optimized writing purpose of aggregating small block IO into large block IO writing, but it also does not require the establishment of additional replica cache partitions, thereby avoiding additional replica management costs. In addition, when the business data is persistently stored in the form of EC, this method also reduces the repeated writing of correction and erasure data, thereby reducing data writing overhead, which is beneficial to improving the storage performance of the distributed all-flash storage system.

In some examples, a replication manager detects changed data caused by an input / output (I / O) operation, where the replication manager is to replicate data writes of I / O operations to a storage system. A classifier classifies the changed data to identify a sensitivity of the changed data. A system determines, based on the identified sensitivity of the changed data, an access control rule for a data object comprising the changed data. The system performs access control of the data object based on the determined access control rule.

A method for use in a storage system including a primary storage system and a secondary storage system, the method comprising: starting an asynchronous replication manager; detecting that an intrusion detector has detected an intrusion in the primary storage system; in response to the intrusion being detected, causing the asynchronous replication manager to stop transmission, to a secondary storage system, of replication data associated with any data buckets that are collected by the asynchronous replication manager while allowing the asynchronous replication manager to continue collecting new data buckets; generating an alert that indicates that the intrusion detector has detected the intrusion; receiving a response that is indicative of whether the intrusion is confirmed; when the response indicates that the intrusion is not confirmed, resuming the asynchronous replication manager, and when the intrusion is confirmed, causing the asynchronous replication manager to stop collecting data buckets.

An update performed to a first data set may be replicated to a second data set, where the first and second data sets may be organized according to different schemas. The replication of the update may comprise determining that a set-wide replication capacity for replicating across the second data set may suffice to perform the replication, as well as a store-specific replication capacity for replicating to an identified data store of the second data set. Based on a determination that the set-wide replication capacity and the store-specific replication capacity may suffice for replicating the update, the update may be caused to be replicated on the second data set.

To provide a management apparatus, a management method, and a storage system in which even in an environment where a connection speed and an amount of data written from a host are uncertain, remote copy can be stably operated. A management apparatus includes a processor and manages a plurality of storage apparatuses communicating with one another via a network and having a respective volume. In the management apparatus, a remote copy transferred from a replication source volume of a plurality of volumes to a replication destination volume via the network and duplicated is set for a combination of a plurality of replication source volumes and replication destination volumes. The management apparatus monitors a replication delay time, which a time required for duplicating a remote copy, and exercises control to limit an amount of transferred data of other remote copies based on the replication delay time.

The invention relates to access control based on classification of altered data. In some examples, a replication manager detects altered data caused by an input / output (I / O) operation, where the replication manager is to copy data writes of the I / O operation to a storage system. A classifier classifies the altered data to identify a sensitivity of the altered data. The system determines an access control rule for a data object including the altered data based on the identified sensitivity of the altered data. The system performs access control of the data object based on the determined access control rule.

A method performed by a replication manager in a system is described. The replication manager first modifies the schema definition of a subscriber database to remove index and constraint objects and c

In some examples, a replication manager detects changed data caused by an input / output (I / O) operation, where the replication manager is to replicate data written by I / O operations to a storage system. A classifier classifies the changed data to determine the sensitivity of the changed data. Based on the identified sensitivity of the changed data, a system determines an access control rule for a data object containing the changed data. The system performs access control of the data object based on the determined access control rule.

A repository replica management method. The method comprises: receiving a backup command, which backup command is used for backing up a primary repository of a user, wherein the primary repository comprises at least one piece of object data and reference data of the at least one piece of object data, and the reference data is used for pointing to the at least one piece of object data; and in response to the backup command, compressing the at least one piece of object data in the primary repository to generate a compressed file, and storing replicas of the compressed file in a replica storage medium corresponding to the primary repository. In the method, replicas are created only for object data that occupies a large storage space and has a low update frequency in a repository, and reference data that occupies a small storage space and is frequently updated remains shared, such that an object-level repository replica of the repository is implemented. A synchronization and consistency algorithm for the object-level repository replica is simple, and thus both the high algorithmic complexity associated with strong consistency algorithms used to resolve repository divergence due to concurrent writes and the risk of repository corruption are avoided.

This invention provides a method and system for integrating distributed edge data center computing power based on DPU, relating to the processor field. The method involves: transmitting parameter replica management data and consistency control function data to the DPU layer corresponding to the DPU card deployed within the edge data center, obtaining the target DPU layer; constructing a hardware-layered communication topology combining intra-data center aggregation and cross-data center controlled transmission within the target DPU layer, obtaining the target DPU, which performs bandwidth awareness, QoS scheduling, cross-data center parameter coordination, and traffic shaping control; transmitting gradient processing and parameter synchronization control logic data corresponding to the edge data center to the target DPU, and performing hardware-level offloading and cross-data center intelligent scheduling through the target DPU; and controlling GPU resource virtualization and network virtualization through the target DPU, so that multiple edge data centers logically form a unified training resource pool, resulting in the distributed edge data center computing power integration.

Example methods and systems relate to bulk APIs with block transport coding for cloud applications. A replication management service (RMS) is a cloud-based application, intended to replicate data from various data sources. The RMS provides a set of RESTful APIs to facilitate creation and execution of replication tasks. The RMS receives a plurality of tiles in the REST communication and generates a replicated stream in response. Since all replication tasks in the replication stream use the same constellation, applications using the RMS transmit constellation data only once regardless of the number of replication tasks in the replication stream. As a result, the total amount of data transmitted between the application and the RMS is reduced whenever multiple replication tasks are performed on the same constellation, thereby reducing network congestion and improving the performance of the cloud-based system.

Disclosed herein are system, method, and computer program product embodiments for extracting data to stream to a user interface using a client library and configuration file. An embodiment operates by receiving the configuration file from a replication management system (RMS). The embodiment accesses a RMS pod using the client library. The embodiment retrieves a list of tables in a repository using the client library. The embodiment chooses at least one table from the list of tables. The embodiment downloads data from the at least one table. The embodiment exports the data in a readable file format. The embodiment then streams the exported data to the UI.

Various embodiments for a replication assistant and log analyzer system are described herein. An embodiment operates by receiving a selection of a plurality of log files generated by an application. A query is received on the plurality of log files, the query comprising one or more terms. The plurality of log files are searched for the one or more terms corresponding to the query. A result of the searching is generated for display, the result comprising: a first indicator as to which of the plurality of log entries of which of the plurality of log files include the one or more terms, and a second indicator as to which of the different components of the application each of the log entries correspond. The result including is displayed, wherein the log entries are sorted by the timestamp of the log entries across the plurality of log files.