Distributed system data consistency processing method, device and system

Abstract

The invention relates to the field of data processing, and in particular to a distributed system data consistency processing device. The distributed system data consistency processing device comprises a configuration module, a database message generation module and a data consistency comparison task module, wherein the configuration module is used for pre-configuring a monitoring object and a monitoring rule corresponding to the monitoring object; the database message generation module is used for monitoring and analyzing a transaction log file corresponding to the preset monitoring object, generating a business data change message when determining that the preset monitoring object changes, and sending the business data change message to the data consistency comparison task module; the business data change message contains business data to be compared; and the data consistency comparison task module is used for receiving the business data change message sent by the database message generation module, and comparing the consistency of the business data to be compared contained in the business data change message according to the preset monitoring rule to obtain a comparison result.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a distributed system data consistency processing method, device and system. Background technique [0002] A database is a warehouse that organizes, stores and manages data according to its data structure. Databases are widely used in data storage and processing. At present, the distributed database transaction processing of large-scale Internet applications mostly adopts the BASE model. This model is different from the ACID model of relational databases in that it adopts an asynchronous update method, allowing the data state to be out of sync for a period of time, as long as the data is finally consistent. That's it. That is to say, the BASE model obtains high availability and high reliability by sacrificing real-time high consistency. Therefore, how to monitor and process inconsistent data has become a problem to be solved. [0003] In the prior art, there is...

AI Technical Summary

Problems solved by technology

[0004] However, because the methods provided by the existing technology need to trigger the data consistency monitoring according to the verification request sent by the application system, the old application system needs to be modified to realize the data consistency monitoring, which is not suitable for all applications. System and database, higher cost

In addition, since it is necessary to access and query the business database in real time to obtain specific data during data consistency monitoring, when the amount of data involved is large, it will put a greater pressure on the business database, and real-time reading of the business database will also Cause system performance loss, and then produce defects such as slow system response and inability to obtain fast and accurate monitoring results

Images