Method for distributing data and data synchronization system

Abstract

The invention discloses a method for distributing data and a data synchronization system. The method in an embodiment of the invention includes acquiring synchronous data in a source database subsystem by the aid of a data synchronization file generating subsystem and generating SQL (structured query language) data files according to the synchronous data; acquiring the SQL data files from the data synchronization file generating subsystem by the aid of an internal network and transmitting the SQL data files to an external network; receiving the SQL data files by the aid of the external network and verifying the received SQL data files so as to determine whether the received SQL data files are consistent with the SQL data files transmitted by the internal network; synchronizing the synchronous data in the SQL data files to a target database subsystem by the aid of the external network if the received SQL data files are consistent with the SQL data files transmitted by the internal network; or generating first prompting information by the aid of the external network if the received SQL data files are not consistent with the SQL data files transmitted by the internal network. The first prompting information is used for indicating that the SQL data files received by the external network are not consistent with the SQL data files transmitted by the internal network. The method and the data synchronization system have the advantages that the data synchronization and integration efficiency of the data synchronization system can be improved, operation and maintenance pressures can be relieved, and the consistency of the synchronous data can be guaranteed.

Description

Technical field [0001] The invention relates to the field of computer technology, in particular to a data distribution method and a data synchronization system. Background technique [0002] With the advent of the cloud era, big data has also attracted more and more attention. The analyst team of "Zhuyuntai" believes that Big data is usually used to describe a large amount of unstructured data and semi-structured data created by a company. These data will cost when downloaded to a relational database for analysis. Too much time and money. Big data requires special technology to effectively process a large amount of data within a tolerable elapsed time. Technologies applicable to big data include: Massive Parallel Processor (MPP) databases, data mining grids, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems. [0003] Some people compare data to a coal mine with energy. Coal is classified into coking coal, anth...

AI Technical Summary

Problems solved by technology

[0006] 1. The data synchronization and integration between heterogeneous databases is not perfect. For example, the two databases that need data sharing are Oracle database and MySQL database. Because the concurrency, consistency and transaction characteristics of these two databases are different , then the data types are different, and the current big data analysis and data mining technologies are not combined, so that data synchronization also requires data integration, and data integration is not processed uniformly, making data synchronization and integration inefficient;

[0007] 2. If there are a large number of business systems, that is, when there are many databases, the amount of business data that needs to be synchronized and integrated is huge, which will put a lot of pressure on operation and maintenance

[0008] 3. In the case of massive data that needs to be synchronized, the consistency and integrity of the data cannot be guaranteed, and if there is a problem, it cannot respond in time

[0009] Combining the above three points, it can be seen that the existing data synchronization system has low data processing efficiency and heavy operation and maintenance pressure, and cannot guarantee data consistency

Images