Heterogeneous data synchronization method applying big data and storage medium

Abstract

The invention relates to the technical field of heterogeneous data processing, in particular to a heterogeneous data synchronization method applying big data and a storage medium, the storage medium and the synchronization method are included, and the method comprises the following steps that 1, data source user information is configured in a user-defined mode; step 2, packaging a plug-in data source; 3, transmitting a parameter table name; step 4, filtering data; step 5, customizing the number of partitions; step 6, defining an insertion mode; 7, starting the script; 8, a main method is obtained through reflection; step 9, executing data encapsulation; step 10, starting a corresponding App (Application); step 11, after the feedback operation is finished, the system applies for resources; step 12, after the resources are obtained, creating an excitor object process; step 13, transmitting the distributed allocation task to a random machine; step 14, calling and checking a Log log; and step 15, ending the operation. According to the invention, the synchronization performance and the enterprise production service efficiency are improved.

Description

technical field [0001] The invention relates to the technical field of heterogeneous data processing, in particular to a heterogeneous data synchronization method and storage medium applying big data. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not constitute prior art. [0003] The term big data is being mentioned more and more. With the development of big data technology, various components and new technologies are constantly being added, so that massive and multi-source data can be used to exert its value. As we all know, business The sources of data are always diversified. Based on this, higher requirements are put forward for data synchronization methods and data storage methods, in which data storage is recorded in a certain format on the internal or external storage medium of the computer; in the current technology, data Storage should be named, and this naming should refl...

AI Technical Summary

Problems solved by technology

[0005] The inventor found through research that: the core technical difficulties of cross-architecture data synchronization are due to the fact that the current public data synchronization methods all have long synchronization cycles, and the data is prone to data loss or data duplication during the synchronization process. The complexity of the system will also cause the system to stand by, and then the database will crash.

[0006] The purpose of this disclosure is to provide a heterogeneous data synchronization method and storage medium using big data, and solve the technical problem of long synchronization period of heterogeneous data under the same architecture in the prior art through the processing logic of step 1-step 15; At the same time, it also solves the technical problems of easy data loss and duplication when heterogeneous data is synchronized.

Images