Key-value data distributed caching system and method thereof

Abstract

The invention discloses a key-value data distributed caching system and a method thereof. The system comprises a key-value access proxy cluster and a key-value data storage cluster, wherein the key-value access proxy cluster is composed of a plurality of data access proxy nodes, and the key-value data storage cluster is composed of a plurality of data fragmentation storage replication clusters; the data access proxy nodes are used for sending a node business storage request, routing a data read and write request to a target data storage node, and storing or reading business data; and in the data fragmentation storage replication clusters, a master-slave data replication mechanism is adopted to determine that a single point of failure does not exist in the data, read data are read from a plurality of slave nodes, and load balance of the read data is determined. By adopting the technical scheme provided by the invention, key-value data automatic fragmentation storage is realized through a consistency hash algorithm, and horizontal expansion of data storage capacity of the system is guaranteed.

Description

technical field [0001] The invention relates to the field of data storage, in particular to a system and a method for distributing and storing massive key-value data in memory of multiple nodes. Background technique [0002] Because most Internet services need to obtain data from the database to provide access to user data, and database services are generally limited by hard disk io bottlenecks, and cannot provide high concurrent access to large data services. [0003] In the existing data distributed cache system, because the data sharding technology uses a fixed hash algorithm, when the system expands and new storage nodes are added, a large amount of stored data will become invalid, resulting in huge data migration costs. In addition, the sharded data does not use the replication cluster technology, and the system has a single point of failure. Once the sharding node goes down, it will directly lead to data loss. Without connection pool and long connection technology, da...

AI Technical Summary

Problems solved by technology

[0003] In the existing data distributed cache system, because the data sharding technology uses a fixed hash algorithm, when the system expands and new storage nodes are added, a large amount of stored data will become invalid, resulting in huge data migration costs

In addition, the sharded data does not use the replication cluster technology, and the system has a single point of failure. Once the sharded node goes down, it will directly lead to data loss

Without connection pool and long connection technology, data loses a lot of performance during network transmission, and there is no data group configuration management function, so it is impossible to store multiple business data on the same cluster

In addition, key-value data and value data only support the character format, which cannot satisfy the finer control of the business on the key-associated data.

Finally, since no plug-in technology is used in the existing data storage, the flexible integration of the system cannot be realized

Images