218results about How to "Ensure data consistency" patented technology

The disclosed embodiments provide a system that archives data for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a request from a client for a data block of a file stored in the distributed filesystem. Upon determining that the requested data block is not currently cached in the cloud controller, the cloud controller sends a peer cache request for the requested data block to a peer cloud controller in the distributed filesystem.

The present invention is related to databases and the exchange of scientific information. Specifically the invention discloses a laboratory data management system including a knowledge base that stores characteristics of laboratory reagents, experiment subjects, tissues and cells. The invention also includes an experiment input module that accepts input of parameters of an experiment and data from a knowledge base module. Further, the invention includes a protocol creator connected to the knowledge base module and the experiment input module that creates a laboratory protocol for an experiment based on data supplied from the knowledge base module and the experiment input module. The invention also includes methods to use the described system.

A block chain-based distributed logistic tracking method is disclosed and comprises the following steps: an order is placed by a user via a conventional information system, the order is distributed to an order node, and order information is generated; upper node private key decryption verification is adopted for a warehouse node, package information is generated and stored in a block chain after successful verification, and packaging operation is conducted by a sorter; the package is picked up via a package pickup and sending node, the upper node private key decryption verification is adopted, package pickup information is generated and stored in the block chain after successful verification, an optimal transport route is calculated, the upper node private key decryption verification is adopted for a transport node, transport information is generated and stored in the block chain after successful verification, the upper node private key decryption verification is adopted for a package delivery node, package delivery information is generated and stored in the block chain after successful verification, an express delivery worker sends the package to the user who places the order, the upper node private key decryption verification is adopted by the user, and signing information is generated and stored in the block chain after successful verification. Via the block chain-based distributed logistic tracking method, encryption performance and safety of data flows can be greatly improved, the optimal route is provided, and whole course data can be monitored and traced.

The invention discloses a database access platform and access method, which solves problems of high requirements on technician, impalpable code and safety of traditional database operation mode. The platform mainly comprises an application access interface for supplying setting of database configuration information and feeding back database operation result; a data source acquisition unit for connecting corresponding database according to the database names in the configuration information; a database single table operation unit for calling corresponding table structure information according to the access parameters in the configuration information and executing corresponding SQL operation in combination with generation of SQL sentence; a database complex operation unit for calling corresponding SQL template according to the access parameter in the configuration information and executing corresponding SQL operation in combination with generation of SQL sentence. Because of shielding direct SQL operation, the platform facilitates developers unnecessary to concern detailed SQL sentence spelling and database operation and greatly reduces the technical requirements on developers.

Disclosed is a system for replicating data. The system may comprise a plurality of nodes preferably organised in groups with one of the nodes acting as a coordinator node. The nodes are configured to receive write requests from an external server and to apply these write requests to a data storage source of the data storage system. The write requests typically belong to a batch of independent write actions identified by a batch sequence number. Each node stores the write request in non-volatile memory with the coordinator node monitoring which batches are secured in their entirety in non-volatile memory. The coordinator node authorises all other nodes to sequentially replicate the write requests in their non-volatile memory to the data storage source for all writes up to the highest batch sequence number for which all writes have been secured in non-volatile memory.

The invention provides a cloud storage data synchronizing framework based on analyzing the shortcoming of a single point failure of central server nodes Namenode, which exists in a hadoop distributed file system (HDFS) module in the current Hadoop framework. The cloud storage data synchronizing framework adopts a double-center-server architecture, and double center servers serve on line simultaneously, so that data consistency is realized; and a three-machine Paxos algorithm aiming at the double center servers and based on a distribution consistency Paxos algorithm is designed, so that a data synchronizing framework Quorum is formed, and read-write operation on the architecture is standardized. By the data synchronizing framework Quorum, the problem of service interruption under the condition of the signal point failure of the nodes Namenode is well solved, so that a system can provide correct data read-write access outward under the condition that a certain server fails, and final consistency of data can be ensured.

The invention discloses a multi-copy storage method and a multi-copy storage system for a distributed database system, and belongs to the field of computer software. The multi-copy storage system comprises a main Master node, a standby Master node, a plurality of storage nodes and a connection network. The multi-copy storage method includes steps of 1, storing data in different data pieces; 2, only allowing a user to operate the data in a primary version, and enabling the multi-copy storage system to synchronize change of the data to various copies in real time and guarantee the atomicity; 3, enabling the multi-copy storage system to detect survival conditions of the nodes via synchronous heartbeat detection, and enabling a main Master host to instantly select an available copy to replace the primary version when a certain storage node of the data in the primary version dies; 4, enabling the multi-copy storage system to carry out detection and repair operation for the storage copies regularly, and migrating the data pieces with excessively low quantities of copies to other available storage nodes by means of local storage migration. A main storage model and a standby storage model are adopted for each data piece, and the copies are of completely identical logical structures and are distributed on different physical storage nodes. The multi-copy storage method and the multi-copy storage system have the advantages that the integrity and the availability of the data are protected against the death of optional nodes of the system.

The invention discloses a data management method and a system of an HBase database. The management method comprises the steps of writing response data into a request, generating HBase records to be stored in the HBase database; using fields with the content sizes larger preset field threshold values and included in the HBase records to be stored as large fields and extracting contents of the large fields to generate large field data blocks; writing the large field data blocks into container files stored in an HDFS file system; using positions of the large field data blocks in the HDFS file system as references and using the references to take the place of the contents of the large fields in the HBase records to be stored; storing the HBase records to be stored in the HBase. By adopting the data management method and the system of the HBase database, the advantages of an HBase and an HDFS can be well played, the large-field processing defect of the HBase is overcome, the shortcoming of overlarge resource consumption of the HDFS during small file storage is overcome, the business demand is met well, and good effect is obtained in the actual application.

The invention relates to a cache processing method and system of a distributed database. The system comprises a receiving module, an order matching module, an analyzing module, a judging module, a cache analyzing module, a table name judging module, a routing module and a feedback module. The receiving module is in one-way connection with the analyzing module and the order matching module. The order matching module is in one-way connection with a client side and the analyzing module. The analyzing module is in one-way connection with the judging module. The judging module is in one-way connection with the cache analyzing module and the routing module. The cache analyzing module is connected with the feedback module through the table name judging module. The routing module sends a buffering result to the feedback module. The feedback module returns the buffering result to the client side. According to the cache processing method and system, the system concurrent processing capacity and the system resource utilization ratio are improved, a load on a low-end server is utilized for sharing the storage space of the database to process data, and performance and safety are improved.

A method and system for integrated modality worklists management for hybrid scanners is disclosed. The present invention retrieves schedules of multiple modalities in a hybrid scanner in response to a single action by the operator of the hybrid scanner, thereby improving operator efficiency. The retrieved schedules are collated and stored in a common database. The automated maintenance of a common database improves data integrity.

The invention discloses a method for data migration of a non-relational database HBase. According to the method, by reading configuration file information of users, data within a time period designated by the users are migrated from an old HBase cluster to a new HBase cluster. The method has the advantages that by designing RowKey, group information and time information of the data are embodied on the RowKey, and whether the data corresponding to the RowKey are within a migration time range designated by the users can be directly identified by scanning the RowKey during data migration; groups serve as data migration units, every group serves as a check point, the range of the non-synchronous data between the old HBase cluster and the new HBase cluster can be rigidly controlled within one group when abnormal conditions occur, and time required for restoring data synchronization is reduced; log information is recorded in the data migration process to be used for treating the abnormal conditions in the data migration process, synchronization of the data between the old cluster and the new cluster is guaranteed, and data migration between the HBase clusters can be finished accurately and efficiently.

The embodiment of the invention provides a primary/backup switching method and device of a database. The method comprises the following steps: monitoring the operation of a first database which serves as a current primary database, and sending a backup-to-primary request to a second database which serves as a current backup database when a the operation exception of the first database is detected; obtaining feedback data reported when he second database is successfully switched to a current primary database since the second database detects that data in the first database and the second database is consistent, and obtaining data switching information which is finally synchronized from the first database by the second database according to the feedback data; and sending a primary-to-backup request which contains the data switching information to the first database to cause the first database to judge that the data synchronized to the second database is not subjected to data loss according to the data switching information, and switching to the current backup database.

The invention discloses a multi-business user data managing system based on a mixed database and a method for the same. The multi-business user data managing system comprises a system inlet/outlet module for importing and exporting user data, a distributed processing module having a data classifying module, a data screening recording module, a data integrating module and a data checking module, and a data storage cluster comprising a relational database, a Hadoop cluster and a MongoDB cluster. Data is classified and stored via a method of combing a traditional relational database and a non-relational database; the mixed database storage model has advantages of strong versatility and data consistency of the relational database during structural data processing and possesses characteristics of great reading-writing capacity and high framework module extendibility of the non-relational database during the process of the great number of non-structural data.

The invention discloses a database migration method and device and a storage medium which are used for reducing the impact on the normal access operation of a database during a database migration process. The database migration method includes a step of acquiring first location information of database operation log data, a step of buffering log change data for a source database operation, a step of querying a database table to be migrated from the source database for the database table to be migrated and resetting the database table to be migrated in the target database, and a step of resetting the log change data in the target database according to cached log change data until second location information corresponding to the reset log change data is not smaller than the first location information.

The present invention relates to a method for ensuring data consistency upon mapping of a hierarchical data format to a relational database management system.

According to the invention, a method for ensuring data consistency upon storage of a data item in a database, the data item having an identifier and a locator for linking the data item to a destination, whereby the identifier and the locator are stored in the database in addition to the data item, comprises the steps of:

• • checking (11) whether the current combination of identifier and locator has already been stored in the database, and using (12) the stored combination of the data item if this is the case,
  • in case the current combination of identifier and locator has not yet been stored in the database, checking (13, 15) whether either the identifier or the locator has already been stored in the database, replacing (14, 16) the current combination of identifier and locator with a univocal combination if this is the case, and storing (17) the univocal combination together with the data item, and
  • storing (17) the current combination of identifier and locator together with the data item else.

The invention relates to a high-availability virtual machine pooling management system based on roles. According to the virtual machine pooling management method, a master node, a slave node and a backup node are included. The master node is the only logic entrance of a virtual machine resource pool and responsible for managing all nodes in the resource pool. The slave node is mainly responsible for managing a life cycle of a virtual machine of the slave node. The backup node is used for backing up state information of the master node, ensuring consistency with the state information of the master node, and taking over the work of the master node when the master node fails. During a virtual machine pooling management process, all the nodes can switch over three roles along with different conditions, wherein the three roles are the master node, the slave node and the backup node. A serial method is used for ensuring that only one master node and one backup node exist in the system and a double-direction asynchronous communication mechanism is used for ensuring consistency of data of the master node and the backup node. The high-availability virtual machine pooling management system based on the roles improves reliability of a virtualized system and flexibility of management.

The invention provides a method and a device for executing multiple tasks, electronic equipment and a computer readable medium, being capable of reducing the workload of repeated development of a developer, and ensuring data consistency during interaction between multiple systems, without affecting the performance of the whole system at the same time. The method for executing the multiple tasks provided by the invention comprises the steps of receiving task data sent by each service system, wherein each piece of task data comprises a respective task type that is corresponding to a service logic for executing the task, and the service logic is stored in a server; according to a preset task distribution rule, distributing each task to the preset quantity of task queues; and generating a taskexecuter for each task of the task queues according to a corresponding task type and executing the corresponding task by the task executer according to a preset task execution rule, wherein the taskexecuter is used for executing the service logic corresponding to the server.

The invention provides a method for improving data transmission efficiency on the basis of TCP (Transmission Control Protocol) long connection. Data communication is carried out between a client and a server by adopting TCP long connection; the receiving and processing of the client and the server are subjected to parallel processing by adopting an asynchronous mode; a communication data packet adopts a numbering mechanism to ensure the data reliability and the processing consistency; the communication reliability is ensured by a heartbeat detection and failure reconnection mechanism; the parallel transmission efficiency is improved by using a connection pool; and the client and the server process a plurality of queues and a plurality of threads, the parallelism of data processing is improved and the processing efficiency is improved.

A bladed storage servicing system comprising a RAID layout facilitates the removal of one or more functional storage devices in a tray of the bladed storage subsystem in order to service a failed storage device. Upon removal of the tray from the bladed storage subsystem, the system designates the functional storage devices on the tray as temporarily removed. The system logs writes to the removed functional storage devices so that the functional storage devices can be returned to consistency when replaced in the bladed storage subsystem. The system manages reads to the removed functional storage devices through reconstruction from active storage devices in the bladed storage subsystem. Upon reinsertion of the tray in the bladed storage subsystem, the system verifies the identities of the functional storage devices and resynchronizes the functional storage devices to ensure data consistency.