Automatic disaster-tolerant system based on Greenplum database

Abstract

The invention discloses an automatic disaster-tolerant system based on a Greenplum database. The system comprises a task initiation layer, a task synchronization layer and a result analysis layer, wherein the task initiation layer queries a production data cluster, monitors data variation in the Greenplum database and judges whether data synchronization is needed or not; the task synchronization layer defines a metadata monitoring task, records metadata changes and then synchronizes changed metadata in the production data cluster into a disaster-tolerant data cluster through a pipeline; the result analysis layer calculates changes of the disaster-tolerant data cluster according to the synchronized data, compares the synchronized data of the production data cluster with the synchronized data of the disaster-tolerant data cluster, records daily synchronization and outputs an analysis result. The system has certain redundancy, high performance and high availability, can guarantee efficient running of the database, effectively avoids business performance problems caused by faults and is simpler and more reasonable in structure and wide in application range.

Description

technical field [0001] The invention relates to a database disaster recovery system, in particular to an automatic disaster recovery system based on Greenplum database. Background technique [0002] With the continuous development of domestic telecommunication enterprises, the competition in the telecommunication industry tends to be fierce. On the one hand, there is more and more room for customers to choose telecommunication services and telecommunication companies, and competition among telecommunication companies for customers is becoming more and more fierce. After continuous "price wars" by operators, a serious phenomenon of "increasing revenue without increasing revenue" has emerged in the telecom market, and a large number of low-loyalty customers switch networks or change services. Although telecommunications companies have adopted activities with a certain discount period to reduce the churn rate of customers, after the discount period ended, many customers left t...

AI Technical Summary

Problems solved by technology

(2) Data needs real-time feedback

[0009] 1) Unpredictability The host hardware used by the MPP architecture is an X86 machine. Compared with a minicomputer, the stability of an X86 machine is far from that of a minicomputer. Faults cannot be predicted, and usually there is no way to repair them in the first place, which may cause The absence of large batches of data

When the situation is serious, the problem is unpredictable, and the mirror of the default Greenplum architecture cannot take over the work, and the entire business system is paralyzed at this time;

[0010] 2) The default Greenplum architecture has low high availability, host hardware is not predictable, and database high availability is low. Therefore, in addition to production data clusters, building disaster recovery data clusters is also a top priority;

[0011] 3) The performance is reduced because Greenplum's own architecture is that an X86 machine deploys Primary instances and Mirror instances (belonging to different groups) at the same time

The MPP architecture has a "short-board effect" in performance. When a machine in the cluster has low computing power, the entire cluster will feedback results based on the machine with the lowest computing power. The time-consuming is twice the normal speed, and the performance is reduced by 50. %

Based on this reasoning, if two hosts are damaged at the same time, the calculation running time will be greatly extended, and the performance may be reduced by about 75%;

[0012] 4), unable to perform incremental backup (table level) Greenplum database is widely used in the OLAP field, with a huge amount of data, if the Oracle incremental backup method (for the table level) is used, it will greatly increase the difficulty and pressure of backup, so the MPP architecture database is for This point has not been considered, but in the actual business production process, important business data needs to be backed up on a regular basis, and the default Greenplum architecture can only perform full or incremental backups at the database level, but cannot perform full or incremental backups at the table level ;

[0013] 5) If two or more hosts are damaged in the system, the entire cluster will be unavailable. The Greenplum architecture is the default for no disaster recovery solution. When the hosts belonging to the same group are in active / standby mode (mutual data backup), if the two hosts are damaged at the same time, It will cause the entire cluster data to be lost, the business system will be in a paralyzed state, and there will be no disaster recovery plan;

In addition, in order to shorten the length of time that the business is affected, maintenance personnel are required to deal with it quickly, so there is also a dependence on the technical level of maintenance personnel.

If the technical level of maintenance personnel cannot meet these two aspects at the same time, it will affect the normal performance of the business system

It will greatly reduce the perception of front-end customers, and will also greatly delay the recovery speed of faults

Images