Distributed key value database system with strong consistency and read-write method thereof

Abstract

The invention discloses a distributed key value database system, which comprises a client end and a data server cluster, wherein the client end comprises a Hash module, a positioning module and a forwarding module; the data server cluster comprises a plurality of data intervals; each data interval comprises a host node and a plurality of slave nodes; the host node comprises a first read module, a write module, a first recovery module, a first log module, a first decision module and a first storage engine module; the Hash module is used for receiving a write request sent from a client, and positions the write request to the data intervals according to a key of the write request; each write request is provided with a number distributed by a system automatically; the positioning module is used for positioning the write request to the host nodes of the data intervals; and the forwarding module is used for sending the write request to the write module of the main node. According to the distributed key value database system, strong consistency between data copies is provided under the condition of smaller expenditure.

Description

technical field [0001] The invention belongs to the technical field of databases and data storage, and more specifically relates to a strongly consistent distributed key-value database system and a reading and writing method thereof. Background technique [0002] With the advent of the digital age, data information is showing an explosive growth trend. Especially with the advent of the web2.0 era in recent years, the emergence of large-scale Internet applications and the rise of cloud storage, a large amount of unstructured data has accumulated, and these applications have gradually expanded their storage requirements to exceed the traditional enterprise database. Manageable data capacity. [0003] For this type of scenario, a successful solution is to use a distributed key-value (key-value) Nosql database combined with technologies such as compression and partitioning to improve data compression efficiency and storage scale. Typical distributed key-value Nosql databases i...

AI Technical Summary

Problems solved by technology

One solution is to adopt the master-standby copy protocol, but the master-standby protocol is not a good solution, it is difficult to guarantee the consistency between copies and the availability of the system when two nodes fail; in the system there are 3 When the number of replicas is greater than or equal to 1, the paxos protocol is proven to be the only protocol that can guarantee strong consistency of replicas. For a system with 2F+1 replicas, the paxos protocol can tolerate failure of F nodes, making the system highly available

[0005] The above typical solutions have the following defects: poor consistency, or lead to high overhead

Amazon's Dynamo can only provide final consistency between replicas. When data inconsistencies between different replicas occur, complex conflict avoidance and resolution mechanisms need to be used; Google's Bigtable can provide strong consistency of replicas and support single-operation transactions, but Bigtable relies on the GFS file system, and Bigtable has a lot of overhead when processing transaction operations; Yahoo's PNUTS provides weak consistency guarantees, and it focuses on replica consistency across data centers

Images