2089 results about "Data consistency" patented technology

The disclosed embodiments disclose techniques for managing a global namespace for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in a cloud storage system; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, a global namespace for the distributed filesystem is also split across these cloud controllers, with each cloud controller “owning” (e.g., managing write accesses for) a distinct portion of the global namespace and maintaining a set of namespace mappings that indicate which portion of the namespace is assigned to each cloud controller. During operation, an initial cloud controller receives a request from a client system to access a target file in the distributed system. This initial cloud controller uses the namespace mappings for the global namespace to determine a preferred cloud controller that will handle the request.

The disclosed embodiments disclose techniques for executing a cloud command for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller presents a distributed-filesystem-specific capability to a client system as a file in the distributed filesystem (e.g., using a file abstraction). Upon receiving a request from the client system to access and / or operate upon this file, the client controller executes an associated cloud command. More specifically, the cloud controller initiates a specially-defined operation that accesses additional functionality for the distributed filesystem that exceeds the scope of individual reads and writes to a typical data file.

A data management method wherein a real-time history of a database system is stored as a logical representation and the logical representation is then used for any point-in-time recovery of the database system. More specifically, a method for capturing transaction data, binary data changes, metadata, and events, and for tracking a real-time history of a database system according to the events. The method enables tracking and storing of consistent checkpoint images of the database system, and also enables tracking of transaction activities between checkpoints. The database system may be recovered to any consistent checkpoint or to any point between two checkpoints.

The disclosed embodiments disclose techniques that facilitate of avoiding client timeouts in a distributed filesystem. Multiple cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem in a local storage pool. During operation, a cloud controller receives from a client system a request for a data block in a target file that is stored in the distributed filesystem. Although the cloud controller is already caching the requested data block, the cloud controller delays transmission of the cached data block; this additional delay gives the cloud controller more time to access uncached data blocks for the target file from a cloud storage system, thereby ensuring that subsequent requests of such data blocks do not exceed a timeout interval on the client system.

The disclosed embodiments provide a system that distributes data for a distributed filesystem across multiple cloud storage systems. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. Whenever each cloud controller receives new data from a client, it outputs an incremental metadata snapshot for the new data that is propagated to the other cloud controllers and an incremental data snapshot containing the new data that is sent to a cloud storage system. During operation, data stored in the distributed filesystem can be distributed across two or more cloud storage systems to optimize performance and / or cost for the distributed filesystem.

The present invention provides a distributed clustering method to allow multiple active instances of consistency management processes that apply the same encoding scheme to be cooperative and function collectively. The techniques described herein facilitate an efficient method to apply an erasure encoding and decoding scheme across dispersed data stores that receive constant updates. The technique can be applied on many forms of distributed persistent data stores to provide failure resiliency and to maintain data consistency and correctness.

A system, method, and program for maintaining data consistency among updates to data storage areas are provided. Each update has an update time the update was made. There are multiple groups of data storage areas. For each group, updates to the data storage area in the group are stored in a journal for storing updates to the group, wherein there are multiple journals. An indication is made in a memory area for each group of a group update time comprising a most recent update time of the updates in the group. The update time for each update in the group is not greater than the group update time. A determination is made of a minimum group update time across all the groups. At least one update is applied to storage if the update time for the update does not exceed the minimum group update time. The data storage areas may be partitioned among a plurality of computer systems, and may be maintained in at least one primary and secondary storage. The data storage areas may also comprise volumes, with each group including at least one volume. The groups of updates may also be maintained in a journal.

The disclosed embodiments disclose techniques for restoring an archived file in a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, cloud controllers may archive infrequently-accessed files in an archival cloud storage system. During operation, a cloud controller receives a request from a client system to access an archived file, and restores this archived file from the archival cloud storage system.

Embodiments of the present invention are directed to methods for maintaining data consistency of data blocks during migration or reconfiguration of a current configuration within a distributed data-storage system to a new configuration. In one embodiment of the present invention, the current configuration is first determined to be reconfigured. The new configuration is then initialized, and data blocks are copied from the current configuration to the new configuration. Then, the configuration states maintained by component data-storage systems that store data blocks of the current and new configurations are synchronized. Finally, the current configuration is deallocated. In a second embodiment of the present invention, a current configuration is determined to be reconfigured, and, while carrying out continuing READ and WRITE operations directed to data blocks of the current configuration in a data-consistent manner, the new configuration is initialized, data blocks are copied from the current configuration to the new configuration, and the timestamp and data states for the data blocks of the current and new configurations are synchronized.

An efficient method to apply an erasure encoding and decoding scheme across dispersed data stores that receive constant updates. A data store is a persistent memory for storing a data block. Such data stores include, without limitation, a group of disks, a group of disk arrays, or the like. An encoding process applies a sequencing method to assign a sequence number to each data and checksum block as they are modified and updated onto their data stores. The method preferably uses the sequence number to identify data set consistency. The sequencing method allows for self-healing of each individual data store, and it maintains data consistency and correctness within a data block and among a group of data blocks. The inventive technique can be applied on many forms of distributed persistent data stores to provide failure resiliency and to maintain data consistency and correctness.

The present invention provides a distributed clustering method to allow multiple active instances of consistency management processes that apply the same encoding scheme to be cooperative and function collectively. The techniques described herein facilitate an efficient method to apply an erasure encoding and decoding scheme across dispersed data stores that receive constant updates. The technique can be applied on many forms of distributed persistent data stores to provide failure resiliency and to maintain data consistency and correctness.

A method and system is disclosed to maintain data consistency over an internet small computer system interface (iSCSI) network, for disaster recovery and remote data replication purposes. Data consistency and replication is maintained between primary and secondary sites geographically distant from each other. According to the method, a primary journal volume logs all changes (data writes) made to a primary volume, transmits the changes based on a preconfigured policy to a secondary journal volume, and thereafter merges the changes stored in the secondary journal volume with a secondary volume. Changes in the journal volumes are ordered in point-in-time (PiT) frames and transmitted using a vendor specific SCSI command utilizing the iSCSI protocol.

A system, method, and computer program product for a lease-based timeout scheme that addresses fencing's shortcomings. Unlike fencing, this scheme (or protocol) enables an isolated computer to realize it is disconnected from the distributed system and write its dirty data out to storage before its locks are stolen. In accordance with the invention, data consistency during a partition in a distributed system is ensured by establishing a lease based protocol between in the distributed system wherein a client can hold a lease with a server. The lease represents a contract between a client and a server in which the server promises to respect the client for a period of time. The server respects the contract even when it detects a partition between the client and itself.

In some embodiments, it is determined that a primary node has been elected from among a plurality of nodes in a replica set of nodes. The primary node accepts and processes client data access requests. The replica set includes the primary node and other secondary nodes. The primary node receives client data access requests. The primary node assigns a data sequence number to each client data access request that mutates state in the order the client data access requests are received. The data sequence numbers include a linearized processing order that is to be followed by each of the nodes in the replica set. The primary node sends the mutating client data access requests including any corresponding data sequence numbers to the secondary nodes. The primary node receives, from a threshold number of secondary nodes, an acknowledgement indicating reception of the client data access request. The primary node commits the data mutating access request.

A method for accurately synthesizing a full-aperture data map from a series of overlapped sub-aperture data maps. In addition to conventional alignment uncertainties, a generalized compensation framework corrects a variety of errors, including compensators that are independent in each sub-aperture. Another class of compensators (interlocked) include coefficients that are the same across all the sub-apertures. A constrained least-squares optimization routine maximizes data consistency in sub-aperture overlap regions. The stitching algorithm includes constraints representative of the accuracies of the hardware to ensure that the results are within meaningful bounds. The constraints also enable the computation of estimates of uncertainties in the final results. The method therefore automatically calibrates the system, provides a full-aperture surface map, and an estimate of residual uncertainties. Therefore, larger surfaces can be tested with greater departures from a best-fit sphere to greater accuracy than was possible in the prior art.

The present invention discloses devices Including a transparent client-connection manager for exchanging client data between an application server and the device: a request analyzer for analyzing query requests from at least one application server; a data-retrieval / command-execution module for executing query requests; a database connection manager for exchanging database data between at least one database server and the device: a cache-memory pool for storing data items from at least one database server: a cache-policy module for determining cache criteria for storing the data Items In the cache-memory pool; and a data-consistency invalidation module for determining invalidated data Items based on invalidation criteria for removing from the cache-memory pool. The cache-memory pool Is configured to utilize memory modules residing in data proxy devices and distributed cache management utility, enabling the memory capacity to be used as a cluster to balance workloads.

The system for end-to-end content publishing using XML with an object dependency graph is based on the following two design principles: First, separation of content and style: Information stored in the content management system is independent on how it is going to be presented. The presentation style is encapsulated elsewhere and can be used to customize the look and feel based on the end-user preferences as well as the delivery methods and devices. Second, reusability of information content: By encapsulating common information in fragments and subfragments and making these fragments insertable in other fragments, thereby avoid scattering and duplication of information. This enables a user to restrict the edit operations to a limited number of relevant fragments, to affect global changes. In addition, the present invention provides data consistency and data integrity in the content management.

The invention discloses a distributed type dynamic cache expanding method and system supporting load balancing, which belong to the technical field of software. The method comprises steps of: 1) monitoring respective resource utilization rate at regular intervals by each cache server; 2) calculating respective weighing load value Li according to the current monitored resource utilization rate, and sending the weighting load value Li to a cache clustering manager by each cache server; 3) calculating current average load value of a distributed cache system by the cache clustering manager according to the weighting load value Li, and executing expansion operation when the current average load value is higher than a threshold thremax; and executing shrink operation when the current average load value is lower than a set threshold thremin. The system comprises the cache servers, a cache client side and the cache clustering manager, wherein the cache servers are connected with the cache client side and the cache clustering manager through the network. The invention ensures the uniform distribution of the network flow among the cache nodes, optimizes the utilization rate of system resources, and solves the problems of ensuring data consistency and continuous availability of services.

The disclosed embodiments provide a system that archives data for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a request from a client for a data block of a file stored in the distributed filesystem. Upon determining that the requested data block is not currently cached in the cloud controller, the cloud controller sends a peer cache request for the requested data block to a peer cloud controller in the distributed filesystem.

A distributed, activity-based collaboration system can employ a data change request priority scheme for determining an order of execution of data change requests in effecting changes to local copies of data so as to optimize data consistency for collaborative activities. The data change request priority scheme can entail encoding sequence number information and dependency information in the data change requests, responsive to which data changes can be made, unmade and remade to the data.

A system and method for maintaining objects in storage elements includes maintaining information regarding which storage elements are storing particular objects and responding to a request to update an object by using maintained information to determine which of the storage elements store a copy of the object. Each storage element is instructed to invalidate the copy of the object, and an update of the object is performed after each storage element that includes the copy of the object indicates that the storage element has invalidated the copy of the object or the storage element is determined to be unresponsive.

Indirect access to local file systems is provided using storage tank protocols, allowing federation of a local file system into a distributed file system while preserving local access to the existing data in the local file system. The goal of the present system is to federate and migrate the data on a computer system with minimum disruption to applications operating on the computer system. Existing applications on a computer system continue to operate during data federation and migration and require little or no reconfiguration either when the data migration starts or when it ends. Data consistency is maintained: existing applications may modify data in the file system during migration or federation. During federation, other computer systems (or hosts) may modify the data in the file system if access control information allows them to do so. All changes in the file system are seen consistently on all hosts. Minimal downtime is required to install the present system and reconfigure the existing applications to communicate with the present system.

A method and apparatus to provide DBMS restart recovery that allows transactions to access data that does not have restart recovery work pending. Access to data requiring restart recovery work to be performed is restricted, and a transaction is denied access to this data. In another embodiment, the invention allows certain transactions to access restricted data that has restart recovery work pending. These transactions are transactions that do not require data consistency. Regardless of the embodiment, the invention allows full recovery to be completed concurrent to the processing of new transactions requiring access to the database. An amount of restart recovery processing may be postponed until after the DBMS has begun accepting new work requests.

In a system including a host, a primary storage subsystem coupled to the host, a cache coupled to the host and separate from the primary storage system, a secondary storage subsystem, and a data mover coupling the primary and secondary storage systems, data is temporarily cached for future storage in the primary storage subsystem so as to preserve timestamp information and maintain data consistency for asynchronously mirroring the data at a secondary subsystem.

The invention discloses a high-availability database cluster system based on message-oriented middleware, comprising a database service layer composed of a plurality of database servers and a middle layer arranged between an application layer and the database service layer; wherein, the middle layer comprises a plurality of application proxies, a plurality of database middleware and a message bus; the application proxy is used for connection redirection when fault occurs on service nodes so as to keep service uninterrupted and improve reliability; the database middleware is used for interception, analysis, processing and retransmission of the database request message; and the message bus runs through the database middleware and the database server, which is responsible for global sorting of reading and writing operation sent by the data middleware so as to ensure data consistency and sending of the sorting result to the database server. A feasible approach is provided for the invention in order to obtain a high-performance database with high availability. The invention has the advantages of high availability, good scalability and easy construction and management.

The invention discloses a distributed transaction processing method on a cloud computing platform. The method comprises the following steps that: at first, start base services in a data storage cluster of the cloud computing platform, and start transaction processing services on a node of a data management cluster; connect a client end to a data management service node by a RPC (Remote Procedure Call Protocol), request a new transaction, and a local transaction manager receives a request to instantiate a transaction object and assign a transaction identifier for sending to the client end; the client end issues a data operation request of the transaction, and the local transaction manager sends the request to a corresponding transaction object for processing according to the transaction identifier owned by the client end; the client end issues a transaction submitting command, and the local manager is responsible for submitting the transaction. The distributed transaction processing method is applied to the cloud computing platform, so that size of transaction logs and log operation times are reduced by caching and combining transaction logs; and moreover, a master lock and lease management solve a blocking problem; a strategy processing failure condition is recovered to maintain data consistency.

The disclosed embodiments disclose techniques that facilitate of avoiding client timeouts in a distributed filesystem. Multiple cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem in a local storage pool. During operation, a cloud controller receives from a client system a request for a data block in a target file that is stored in the distributed filesystem. Although the cloud controller is already caching the requested data block, the cloud controller delays transmission of the cached data block; this additional delay gives the cloud controller more time to access uncached data blocks for the target file from a cloud storage system, thereby ensuring that subsequent requests of such data blocks do not exceed a timeout interval on the client system.

Disclosed is a system for maintaining consistency of data across storage devices. A cut-off time value is provided to the system. The system then obtains information on data writes to a first storage device, including information on time stamp values associated with the data writes indicating an order of the data writes to the first storage device. At least one group of data writes having time stamp values earlier in time than the cut-off time value is then formed. The system then transfers the data writes in the groups to a second storage device for storage therein.