Distributed key-value query method and query engine system

Abstract

The invention provides a distributed key-value query engine system and a query method. The system comprises a metadata server, at least one proxy node and at least one storage node, wherein the metadata server is used for maintaining the information of a global routing table pointing at a storage node; the proxy node comprises a query forwarding module and a data forwarding module, wherein the query forwarding module is used for forwarding a client request to a corresponding storage node in response to a request including a target KEY sent from a client based on the information of the global routing table, and the data forwarding module is used for transmitting a response packet including a target Value, returned from the corresponding storage node, to the client; and the storage node is used for acquiring the needed data from a local storage based on the received client request and transmitting the response packet including the target Value to the proxy node. According to the distributed key-value query engine system and the query method, the key-value data can be stored and queried in a distributed manner, massive key-value data can be accommodated and the adaptability to highly concurrent access is better.

Description

technical field [0001] The present application relates to the technical field of distributed data storage and query, in particular to a distributed key-value query method and a query engine system. Background technique [0002] For the support of large-scale Internet applications and cloud computing, relational databases are generally used to store related data. [0003] Traditional relational databases usually store a certain type of information of an application (such as website user information) in a single database table of the database system, and provide a communication interface for reading and writing operations to the application. In terms of implementation, typical relational databases such as MySQL have a certain upper limit for single-table query and write operations. If the upper limit is exceeded, SQL query and write operations will become very slow, and the disk input / output (IO) module will also A bottleneck occurs. In today's massive data on the Internet, ...

AI Technical Summary

Problems solved by technology

There are two problems here. One is that for each application, the application layer needs to split the data, and the splitting strategy of each application will be different, which is inefficient; the other problem is that the cost of data re-splitting after adding machines is very high High, requires manual downtime for maintenance operations, does not support automatic elastic expansion

[0006] Secondly, in traditional relational databases, the concurrency capability of a single machine has a certain upper limit, and high-concurrency access will exhaust machine resources.

One of the reasons why high concurrent access cannot be supported is that relational databases need to perform SQL parsing operations during retrieval, which consumes a lot of CPU resources.

Images