A high-availability time-series database cluster system based on message queue

Abstract

This application relates to a high-availability time-series database cluster system based on message queues, which belongs to the field of database technology, and includes message queue clusters, connector modules and time-series database clusters; the connector module includes multiple database connectors, and each database connection The servers are all connected to the message queue cluster; the time series database cluster includes multiple time series database nodes connected one by one with multiple database connectors; the message queue cluster is used to receive and cache the time series data of data producers; multiple database connectors are used to Read time-series data from the message queue cluster, and write time-series data to multiple time-series database nodes in parallel based on multiple threads; the database connector that sends write failure information is used to write failure information corresponding to the first preset time interval Time series database nodes continue to write time series data. This application can meet the high-frequency concurrent writing and high-level horizontal cluster expansion capabilities of time series databases.

Description

technical field [0001] The present application relates to the technical field of databases, in particular to a high-availability time-series database cluster system implemented based on message queues. Background technique [0002] Time series data is mainly collected and generated by various types of real-time monitoring, inspection and analysis equipment in the power industry and chemical industry. Because time series data has the characteristics of fast production efficiency, heavy dependence on collection time, and large amount of information at multiple measurement points, a database system that can efficiently store and quickly process massive time series data is needed, and time series databases have emerged as the times require. [0003] The Chinese patent document with the patent publication number CN111752758A discloses a high-availability system of influxDB with a dual-master architecture. The system uses the access module to send the user's write request to two i...

AI Technical Summary

Problems solved by technology

[0004] However, the system alternately sends write requests to two influxdb nodes, which is not suitable for high-frequency concurrent write scenarios. The data writing module may become a system bottleneck. At the same time, if both host nodes fail, the newly collected data will be permanently lost, and the architecture is not very capable of horizontal horizontal cluster expansion

Images