248 results about "Distributed data store" patented technology

Embodiments of the present invention are directed to methods for maintaining data consistency of data blocks during migration or reconfiguration of a current configuration within a distributed data-storage system to a new configuration. In one embodiment of the present invention, the current configuration is first determined to be reconfigured. The new configuration is then initialized, and data blocks are copied from the current configuration to the new configuration. Then, the configuration states maintained by component data-storage systems that store data blocks of the current and new configurations are synchronized. Finally, the current configuration is deallocated. In a second embodiment of the present invention, a current configuration is determined to be reconfigured, and, while carrying out continuing READ and WRITE operations directed to data blocks of the current configuration in a data-consistent manner, the new configuration is initialized, data blocks are copied from the current configuration to the new configuration, and the timestamp and data states for the data blocks of the current and new configurations are synchronized.

The present invention generally relates to distributed data storage systems. In particular, the present invention is related to a method of data storing in a distributed data storage system that comprises a clustering of data blocks and the use of random linear combinations of data blocks that makes the distributed data store system efficient in terms of storage space needed and inter-device communication costs, both for the storage method, as for the associated repair method.

Various embodiments of the present invention provide methods, in distributed data-storage systems that associate one or more timestamps with each data block in each data-storage-component, for deciding whether or not a data block has been written. In certain embodiments of the present invention, a sparse database of timestamps associated with data blocks is maintained, each timestamp having a field that contains one of an indication of a time or sequence and a sentinel value indicating that the timestamp is garbage collected. When a timestamp is not found associated with a data block in a timestamp database, the data block is associated with a garbage-collected-timestamp state. In various embodiments of the present invention, data structures are maintained that store status information indicating whether or not any of a number of data blocks in each of a number of data-block-allocation units have been written. During replication, migration, or reconfiguration of a current segment of data blocks to a new segment of data blocks in these various embodiments, a data block is determined to be written or to be unwritten by determining, from the data structures, whether or not a data block-allocation unit containing the data block is written or unwritten.

A distributed data storage system and method comprising a highly integrated mass storage controller system permitting distributed management and control of data storage is disclosed. The present invention in some preferred embodiments permits mass storage media to be made available on a network in a fashion to permit global access, while automatically handling many high-level file access and data integrity/security/control functions normally associated with a host operating system. This integration and redistribution of functionality permits spatial diversification of data resources, automatic mirroring of data, fault isolation, data security, and a plethora of other control functions to be integrated into the mass storage device. This integration permits peer-to-peer communication between mass storage devices to both unburden the host data consumers but also isolate the data management functions from the data presentation functions normally associated with host systems. Exemplary embodiments of the present invention as applied to specific preferred system contexts include but are not limited to distributed data storage in a networked environment, brokered data access metering, database access/control, data backup, journaling, checkpointing, and automated software distribution.

The disclosed embodiments disclose techniques for synchronizing file updates between two cloud controllers of a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in the cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a request from a client to access a file in the distributed filesystem. The cloud controller sends a synchronization update request for the file to a second cloud controller and in response receives a synchronization update for the file from the second cloud controller.

A system and method for versioning of atomic units of data (such as XML fragments) in a distributed data store are disclosed. The distributed data store may maintain a linear version history for each fragment, such that each has no more than one successor and one predecessor, with no implicit branching. An application programming interface may provide an explicit branching request (indicating an intent to merge modified data in a child branch into its parent branch), along with read, create, create-merge, and/or integrate requests. A reconciliation policy may be specified on a per-application basis, and may include automatic reconciliation using a callback function or interactive reconciliation, with inputs returned to the application. Writes to a minority side of a network partition may be refused unless an explicit branch is requested. Transactions may not be supported in client applications or may be supported through explicit branching. Transactions may be used internally.

Version indicators within an existing range can be associated with a data partition in a distributed data store. A partition reconfiguration can be associated with one of multiple partitions in the data store, and a new version indicator that is outside the existing range can be assigned to the reconfigured partition. Additionally, a broadcast message can be sent to multiple nodes, which can include storage nodes and/or client nodes that are configured to communicate with storage nodes to access data in a distributed data store. The broadcast message can include updated location information for data in the data store. In addition, a response message can be sent to a requesting node of the multiple nodes in response to receiving from that node a message that requests updated location information for the data. The response message can include the requested updated location information.

Provided is a process including: receiving a request to access a distributed data store, wherein the distributed data store stores a plurality of units of content that are each distributed among multiple computing entities hosting different subsets of data of the distributed data store; and causing, with one or more processors of a computing device configured to participate in combining the information from the plurality of computing entities to access units of content, logging of the request in an entry in a tamper-evident log.

A file protection scheme for fixed content in a distributed data archive uses computations that leverage permutation operators of a cyclic code. In an illustrative embodiment, an N+K coding technique is described for use to protect data that is being distributed in a redundant array of independent nodes (RAIN). The data itself may be of any type, and it may also include system metadata. According to the invention, the data to be distributed is encoded by a dispersal operation that uses a group of permutation ring operators. In a preferred embodiment, the dispersal operation is carried out using a matrix of the form [I_{N<sub2>—</sub2>}C] where I_N is an n×n identity sub-matrix and C is a k×n sub-matrix of code blocks. The identity sub-matrix is used to preserve the data blocks intact. The sub-matrix C preferably comprises a set of permutation ring operators that are used to generate the code blocks. The operators are preferably superpositions that are selected from a group ring of a permutation group with base ring Z₂.

Load balancing includes receiving, from a client, a connection request to establish a connection with a server; determining load balancing state information based at least in part on the connection request; synchronizing the determined load balancing state information across a plurality of service engines, including to invoke an atomic read-miss-create (RMC) function on a distributed data store service; and distributing the connection to a selected server among a plurality of servers according to a result of the RMC function.

Embodiments include a database controller for a database of information encoded as a set of data items, the database controller comprising: a segment control module configured to divide the set of data items into a series of segments, each segment comprising each of the data items falling within defined lower limit and upper limit values of an ordering metric used to place the data items in order; and an adaptive segment locator module configured to manage the distribution of the series of segments among a plurality of storage units, including selecting which segments from among the series of segments to co-locate on the same storage unit based on historical data representing read operations performed on the database.

Described herein are methods and systems for locating and aggregating data from disparate sources. Data related to an audience of media consumers is accessed from a plurality of sources. At least a portion of the data from the plurality of sources is aggregated into a unified data store, data from the unified data store is extracted at least partly based on significance information.

The embodiment of the invention provides a distributed file system, and a data storage processing method and a data storage processing device thereof. The method comprises the following steps of: receiving an access request sent by a client of the distributed file system; determining a storage unit to be accessed by the client of the distributed file system and the status of a virtual resource pool corresponding to the storage unit according to the access request, wherein same data is stored in the storage unit and the virtual resource pool corresponding to the storage unit and the virtual resource pool is generated by clustering according to the client of the distributed file system having a same activity cycle; and returning an access response message including a client identifier to the client of the distributed file system initiating the access request, when determining that the virtual resource pool corresponding to the storage unit is in an on-line status, wherein the client of the distributed file system identified by the client identifier is one of the virtual resource pools. The invention further provides the corresponding device and the distributed file system. By the technical scheme, the system stability of the distributed file system can be improved.

A distributed data system may include nodes each including a process that may access distributed data. A lock mechanism coupled to the distributed data store may grant locks for portions of the distributed data. Only threads of a process holding a lock for a portion of the distributed data may access the portion. A process may obtain a lock for a distributed data portion. In one embodiment, the process may release the lock if no more threads require access to the portion so may initiate a "dummy" thread requester indicating a thread requiring access to the portion. If other threads requiring access are initiated, it may not be necessary to obtain the lock again. This may help reduce the use of resources associated with using locks. If the process receives a request to release the lock, the process may cancel the dummy thread requester and release the lock.

A distributed data store may maintain versioned hierarchical data structures. Different versions of a hierarchical data structure may be maintained consistent with a transaction log for the hierarchical data structure. When access requests directed to the hierarchical data structure are received, a version of the hierarchical data structure may be identified for processing an access request. For access requests with snapshot isolation, the identified version alone may be sufficient to consistently process the access request. For access requests with higher isolation requirements, such as serializable isolation, transactions based on the access request may be submitted to the transaction log so that access requests resulting in committed transactions may be allowed, whereas access requests resulting in conflicting transactions may be denied.

Data storage reliability is maintained in a write-back distributed data storage system including multiple nodes. Information is stored as a stripe including a collection of a data strips and associated parity strips, the stripe distributed across data and parity nodes. Each data node maintains the data strip holding a first copy of data, and each parity node maintains a parity strip holding a parity for the collection of data strips. A driver node initiates a full-stripe-write parity update protocol for maintaining parity coherency in conjunction with other nodes, to keep the relevant parity strips coherent. Parity is determined directly by computing parity strips for all data strips of a stripe. Any node may function as a driver node.

A distributed data store labels operations with globally unique identifiers that contain approximate timestamps. The labels are used to address causes of inconsistency in the distributed data store while ensuring progress. A first mode is provided that stores the latest label for each entry is useful if re-inserts and deletes are rare. Another mode is provided that stores a history of labels for each entry can be used if there are many re-inserts and deletes. A further mode is provided that stores a history of labels for queries can report updates to query answers as inserts and deletes settle across the distributed data store.

A data storage and retrieval method and system. The method includes defining, by a computing system, a meta-object, a schema based structured definition for the meta-object, and a taxonomy with configuration data. The computing system creates content associated with the taxonomy and associates topics of the content with a logical storage room representations. Reference coordinates associated with the logical storage room representations are associated with the taxonomy. The computing system generates updated configuration data comprising the reference coordinates and a uniform resource identifier associated with the content is generated. The computing system generates an account associated with the meta-object for a user. Metering charges for usage of the account and a report associated with the account and usage are generated.

The invention discloses an Internet financial cloud transaction platform based on a block chain and an implementation method, wherein the block chain system is at least used for carrying out distributed data storage, point-to-point transmission, a consensus mechanism or encryption calculation so as to facilitate the non-tampering and permanent storage of the transaction data, and comprises a blockchain bottom layer platform used for carrying out data transaction; a block chain network used for providing an information environment for transaction and information transmission; a bottom layer structure which forms a block chain system framework structure; a block chain used for information transaction, storage and transmission; an online loan client used for providing online loan and borrowing information data; a borrower client used for providing the loan or borrowing for the online loan client; and a cloud server used for providing a service platform with comprehensive business capability for various Internet users. According to the invention, the public, transparent and tamper-free functions of the transaction are realized, and the safety factor during the transaction process is improved.

A text annotation structured storage system stores text annotations with associated type information in a structured data store. The present system persists or stores annotations in a structured data store in an indexable and queryable format. Exemplary structured data stores comprise XML databases and relational databases. The system exploits type information in a type system to develop corresponding schemas in a structured data model. The system comprises techniques for mapping annotations to an XML data model and a relational data model. The system captures various features of the type system, such as complex types and inheritance, in the schema for the persistent store. In particular, the repository provides support for path navigation over the hierarchical type system starting at any type.