Data exchange control method and device

Abstract

The embodiment of the invention provides a data exchange control method and device. According to the embodiment of the invention, after the first information is received, the received data volume corresponding to the current data exchange operation of the working node is obtained from a destination database, and the data submission of the current data exchange operation is controlled according tothe received data volume and the submitted data volume corresponding to the current data exchange operation, therefore, the restart frequency of a data submission task is reduced, the exchange time ofthe whole data exchange is reduced, and the exchange efficiency of a data exchange system is improved.

Description

technical field

[0001] The invention relates to the technical field of data processing, in particular to a data exchange control method and device. Background technique

[0002] The ETL (Extract-Transform-Load, Extract-Transform-Load) system, also known as the data exchange system, is used to extract, transform, and load data from the source to the destination. In order to improve the ability of ETL exchange and support more data source access, ETL tools will be deployed in the form of clusters, and the control node will manage multiple work nodes in a unified manner, and assign data exchange jobs to one or more work nodes for data processing. exchange.

[0003] According to the data exchange job issued by the control node and its parameter settings, the working node extracts data from the source database, and submits the data to the destination database after its own processing.

[0004] When the following two scenarios exist (1. The network environment where the data exc...

Examples

Embodiment Construction

[0026] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the embodiments of the invention as recited in the appended claims.

[0027] Terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments of the present invention, and are not intended to limit the embodiments of the present invention. As used in the embodiments of the present invention and the appended claims, the singular forms "a", "said" and "the" are also intended to include the plural f...

technical field

[0001] The invention relates to the technical field of data processing, in particular to a data exchange control method and device. Background technique

[0002] The ETL (Extract-Transform-Load, Extract-Transform-Load) system, also known as the data exchange system, is used to extract, transform, and load data from the source to the destination. In order to improve the ability of ETL exchange and support more data source access, ETL tools will be deployed in the form of clusters, and the control node will manage multiple work nodes in a unified manner, and assign data exchange jobs to one or more work nodes for data processing. exchange.

[0003] According to the data exchange job issued by the control node and its parameter settings, the working node extracts data from the source database, and submits the data to the destination database after its own processing.

[0004] When the following two scenarios exist (1. The network environment where the data exc...

Examples

Embodiment Construction

[0026] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the embodiments of the invention as recited in the appended claims.

[0027] Terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments of the present invention, and are not intended to limit the embodiments of the present invention. As used in the embodiments of the present invention and the appended claims, the singular forms "a", "said" and "the" are also intended to include the plural f...

Images