Distributed data center based on kafka and Hash shared memory

Abstract

The invention discloses a distributed data center based on kafka and Hash shared memory, comprising: a central database which is used for establishing a data update table and writing added / modified data into the data update table; a newly-added data reading server side which reads the newly-added data of the data updating table and pushes the newly-added data to the kafka; a root data center server which reads the total data and then consumes the newly added data in the kafka when the shared memory has no data, directly consumes the newly added data in the kafka when the shared memory has data, and writes the read data into the shared memory; and a plurality of pages of node data center servers which synchronize full data to own nodes and consume newly added data in the kafka when the shared memory has no data, and directly consume the newly added data in the kafka when the shared memory has data, wherein the data are stored in the shared memory. According to the invention, distributed query is realized, the query and check speed is high, and the quick requirement of local services on query and check is met.

Description

technical field [0001] The invention relates to the technical field of data centers, in particular to a distributed data center based on kafka and hash shared memory. Background technique [0002] For data centers, redis (redis is a key-value storage system) or database is generally used for storage at present, and there is only one central database or a database with read-write separation. As the amount of data increases, the data in the central database becomes larger and larger, and the frequency of writing and accessing the data center also increases. The performance of the center also limits the continuous growth of the business. Since the data center uses redis or database to store data, the business has a certain speed requirement on the data access speed. When it is cross-regional or even transnational, the data access efficiency cannot meet the business requirements. The access efficiency of the data center cannot meet the business processing capacity, it is diffic...

AI Technical Summary

Problems solved by technology

As the amount of data increases, the data in the central database becomes larger and larger, and the frequency of writing and accessing the data center also increases. The performance of the center also limits the continuous growth of the business. Since the data center uses redis or database to store data, the business has a certain speed requirement on the data access speed. When it is cross-regional or even transnational, the data access efficiency cannot meet the business requirements. In other words, the access efficiency of the data center cannot meet the business processing capabilities, it is difficult to achieve fast and real-time data access and inspection, and the data access speed of cross-regional access to the central database is slow

Images