2086 results about "Data consistency" patented technology

The disclosed embodiments disclose techniques for managing a global namespace for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in a cloud storage system; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, a global namespace for the distributed filesystem is also split across these cloud controllers, with each cloud controller “owning” (e.g., managing write accesses for) a distinct portion of the global namespace and maintaining a set of namespace mappings that indicate which portion of the namespace is assigned to each cloud controller. During operation, an initial cloud controller receives a request from a client system to access a target file in the distributed system. This initial cloud controller uses the namespace mappings for the global namespace to determine a preferred cloud controller that will handle the request.

The disclosed embodiments disclose techniques for executing a cloud command for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller presents a distributed-filesystem-specific capability to a client system as a file in the distributed filesystem (e.g., using a file abstraction). Upon receiving a request from the client system to access and/or operate upon this file, the client controller executes an associated cloud command. More specifically, the cloud controller initiates a specially-defined operation that accesses additional functionality for the distributed filesystem that exceeds the scope of individual reads and writes to a typical data file.

A data management method wherein a real-time history of a database system is stored as a logical representation and the logical representation is then used for any point-in-time recovery of the database system. More specifically, a method for capturing transaction data, binary data changes, metadata, and events, and for tracking a real-time history of a database system according to the events. The method enables tracking and storing of consistent checkpoint images of the database system, and also enables tracking of transaction activities between checkpoints. The database system may be recovered to any consistent checkpoint or to any point between two checkpoints.

The disclosed embodiments provide a system that distributes data for a distributed filesystem across multiple cloud storage systems. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. Whenever each cloud controller receives new data from a client, it outputs an incremental metadata snapshot for the new data that is propagated to the other cloud controllers and an incremental data snapshot containing the new data that is sent to a cloud storage system. During operation, data stored in the distributed filesystem can be distributed across two or more cloud storage systems to optimize performance and/or cost for the distributed filesystem.

The present invention provides a distributed clustering method to allow multiple active instances of consistency management processes that apply the same encoding scheme to be cooperative and function collectively. The techniques described herein facilitate an efficient method to apply an erasure encoding and decoding scheme across dispersed data stores that receive constant updates. The technique can be applied on many forms of distributed persistent data stores to provide failure resiliency and to maintain data consistency and correctness.

A system, method, and program for maintaining data consistency among updates to data storage areas are provided. Each update has an update time the update was made. There are multiple groups of data storage areas. For each group, updates to the data storage area in the group are stored in a journal for storing updates to the group, wherein there are multiple journals. An indication is made in a memory area for each group of a group update time comprising a most recent update time of the updates in the group. The update time for each update in the group is not greater than the group update time. A determination is made of a minimum group update time across all the groups. At least one update is applied to storage if the update time for the update does not exceed the minimum group update time. The data storage areas may be partitioned among a plurality of computer systems, and may be maintained in at least one primary and secondary storage. The data storage areas may also comprise volumes, with each group including at least one volume. The groups of updates may also be maintained in a journal.

An efficient method to apply an erasure encoding and decoding scheme across dispersed data stores that receive constant updates. A data store is a persistent memory for storing a data block. Such data stores include, without limitation, a group of disks, a group of disk arrays, or the like. An encoding process applies a sequencing method to assign a sequence number to each data and checksum block as they are modified and updated onto their data stores. The method preferably uses the sequence number to identify data set consistency. The sequencing method allows for self-healing of each individual data store, and it maintains data consistency and correctness within a data block and among a group of data blocks. The inventive technique can be applied on many forms of distributed persistent data stores to provide failure resiliency and to maintain data consistency and correctness.

A method and system is disclosed to maintain data consistency over an internet small computer system interface (iSCSI) network, for disaster recovery and remote data replication purposes. Data consistency and replication is maintained between primary and secondary sites geographically distant from each other. According to the method, a primary journal volume logs all changes (data writes) made to a primary volume, transmits the changes based on a preconfigured policy to a secondary journal volume, and thereafter merges the changes stored in the secondary journal volume with a secondary volume. Changes in the journal volumes are ordered in point-in-time (PiT) frames and transmitted using a vendor specific SCSI command utilizing the iSCSI protocol.

A system, method, and computer program product for a lease-based timeout scheme that addresses fencing's shortcomings. Unlike fencing, this scheme (or protocol) enables an isolated computer to realize it is disconnected from the distributed system and write its dirty data out to storage before its locks are stolen. In accordance with the invention, data consistency during a partition in a distributed system is ensured by establishing a lease based protocol between in the distributed system wherein a client can hold a lease with a server. The lease represents a contract between a client and a server in which the server promises to respect the client for a period of time. The server respects the contract even when it detects a partition between the client and itself.

In some embodiments, it is determined that a primary node has been elected from among a plurality of nodes in a replica set of nodes. The primary node accepts and processes client data access requests. The replica set includes the primary node and other secondary nodes. The primary node receives client data access requests. The primary node assigns a data sequence number to each client data access request that mutates state in the order the client data access requests are received. The data sequence numbers include a linearized processing order that is to be followed by each of the nodes in the replica set. The primary node sends the mutating client data access requests including any corresponding data sequence numbers to the secondary nodes. The primary node receives, from a threshold number of secondary nodes, an acknowledgement indicating reception of the client data access request. The primary node commits the data mutating access request.

A method for accurately synthesizing a full-aperture data map from a series of overlapped sub-aperture data maps. In addition to conventional alignment uncertainties, a generalized compensation framework corrects a variety of errors, including compensators that are independent in each sub-aperture. Another class of compensators (interlocked) include coefficients that are the same across all the sub-apertures. A constrained least-squares optimization routine maximizes data consistency in sub-aperture overlap regions. The stitching algorithm includes constraints representative of the accuracies of the hardware to ensure that the results are within meaningful bounds. The constraints also enable the computation of estimates of uncertainties in the final results. The method therefore automatically calibrates the system, provides a full-aperture surface map, and an estimate of residual uncertainties. Therefore, larger surfaces can be tested with greater departures from a best-fit sphere to greater accuracy than was possible in the prior art.

The system for end-to-end content publishing using XML with an object dependency graph is based on the following two design principles: First, separation of content and style: Information stored in the content management system is independent on how it is going to be presented. The presentation style is encapsulated elsewhere and can be used to customize the look and feel based on the end-user preferences as well as the delivery methods and devices. Second, reusability of information content: By encapsulating common information in fragments and subfragments and making these fragments insertable in other fragments, thereby avoid scattering and duplication of information. This enables a user to restrict the edit operations to a limited number of relevant fragments, to affect global changes. In addition, the present invention provides data consistency and data integrity in the content management.

The invention discloses a distributed type dynamic cache expanding method and system supporting load balancing, which belong to the technical field of software. The method comprises steps of: 1) monitoring respective resource utilization rate at regular intervals by each cache server; 2) calculating respective weighing load value Li according to the current monitored resource utilization rate, and sending the weighting load value Li to a cache clustering manager by each cache server; 3) calculating current average load value of a distributed cache system by the cache clustering manager according to the weighting load value Li, and executing expansion operation when the current average load value is higher than a threshold thremax; and executing shrink operation when the current average load value is lower than a set threshold thremin. The system comprises the cache servers, a cache client side and the cache clustering manager, wherein the cache servers are connected with the cache client side and the cache clustering manager through the network. The invention ensures the uniform distribution of the network flow among the cache nodes, optimizes the utilization rate of system resources, and solves the problems of ensuring data consistency and continuous availability of services.

The disclosed embodiments provide a system that archives data for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a request from a client for a data block of a file stored in the distributed filesystem. Upon determining that the requested data block is not currently cached in the cloud controller, the cloud controller sends a peer cache request for the requested data block to a peer cloud controller in the distributed filesystem.

A system and method for maintaining objects in storage elements includes maintaining information regarding which storage elements are storing particular objects and responding to a request to update an object by using maintained information to determine which of the storage elements store a copy of the object. Each storage element is instructed to invalidate the copy of the object, and an update of the object is performed after each storage element that includes the copy of the object indicates that the storage element has invalidated the copy of the object or the storage element is determined to be unresponsive.

Indirect access to local file systems is provided using storage tank protocols, allowing federation of a local file system into a distributed file system while preserving local access to the existing data in the local file system. The goal of the present system is to federate and migrate the data on a computer system with minimum disruption to applications operating on the computer system. Existing applications on a computer system continue to operate during data federation and migration and require little or no reconfiguration either when the data migration starts or when it ends. Data consistency is maintained: existing applications may modify data in the file system during migration or federation. During federation, other computer systems (or hosts) may modify the data in the file system if access control information allows them to do so. All changes in the file system are seen consistently on all hosts. Minimal downtime is required to install the present system and reconfigure the existing applications to communicate with the present system.

A method and apparatus to provide DBMS restart recovery that allows transactions to access data that does not have restart recovery work pending. Access to data requiring restart recovery work to be performed is restricted, and a transaction is denied access to this data. In another embodiment, the invention allows certain transactions to access restricted data that has restart recovery work pending. These transactions are transactions that do not require data consistency. Regardless of the embodiment, the invention allows full recovery to be completed concurrent to the processing of new transactions requiring access to the database. An amount of restart recovery processing may be postponed until after the DBMS has begun accepting new work requests.

In a system including a host, a primary storage subsystem coupled to the host, a cache coupled to the host and separate from the primary storage system, a secondary storage subsystem, and a data mover coupling the primary and secondary storage systems, data is temporarily cached for future storage in the primary storage subsystem so as to preserve timestamp information and maintain data consistency for asynchronously mirroring the data at a secondary subsystem.

The invention discloses a high-availability database cluster system based on message-oriented middleware, comprising a database service layer composed of a plurality of database servers and a middle layer arranged between an application layer and the database service layer; wherein, the middle layer comprises a plurality of application proxies, a plurality of database middleware and a message bus; the application proxy is used for connection redirection when fault occurs on service nodes so as to keep service uninterrupted and improve reliability; the database middleware is used for interception, analysis, processing and retransmission of the database request message; and the message bus runs through the database middleware and the database server, which is responsible for global sorting of reading and writing operation sent by the data middleware so as to ensure data consistency and sending of the sorting result to the database server. A feasible approach is provided for the invention in order to obtain a high-performance database with high availability. The invention has the advantages of high availability, good scalability and easy construction and management.

The invention discloses a distributed transaction processing method on a cloud computing platform. The method comprises the following steps that: at first, start base services in a data storage cluster of the cloud computing platform, and start transaction processing services on a node of a data management cluster; connect a client end to a data management service node by a RPC (Remote Procedure Call Protocol), request a new transaction, and a local transaction manager receives a request to instantiate a transaction object and assign a transaction identifier for sending to the client end; the client end issues a data operation request of the transaction, and the local transaction manager sends the request to a corresponding transaction object for processing according to the transaction identifier owned by the client end; the client end issues a transaction submitting command, and the local manager is responsible for submitting the transaction. The distributed transaction processing method is applied to the cloud computing platform, so that size of transaction logs and log operation times are reduced by caching and combining transaction logs; and moreover, a master lock and lease management solve a blocking problem; a strategy processing failure condition is recovered to maintain data consistency.