Data table association method and device, storage medium and electronic equipment

Abstract

The invention discloses a data table association method and device, a storage medium and electronic equipment, according to a key value of each to-be-associated data table, an association rate of each data table corresponding to each other data table is determined, and according to attribute information of each data table and the corresponding association rate between every two data tables, the association rate of each data table corresponding to each other data table is determined. And determining an association weight between every two data tables, determining an association sequence of the to-be-associated data tables based on the association weights between the data tables, and performing association according to the association sequence. Wherein the association weight represents the data size of the virtual intermediate table after the two data tables are associated, and the association weight between the two data tables is negatively correlated with the association sequence. According to the method, the association sequence of the data tables is determined on the basis of the data volume of the virtual intermediate table after every two data tables are associated in the to-be-associated data tables, so that the frequency of loading the frequency of the data tables with large data volume in the association process is reduced, a large amount of computing resources are saved, and the multi-table association duration is shortened.

Description

technical field [0001] The present application relates to the technical field of data processing, and in particular, to a data table association method, apparatus, storage medium and electronic device. Background technique [0002] With the continuous development of big data technology, when analyzing massive data information, it is necessary to correlate data tables of different data sources. [0003] When performing data table association in the prior art, a serial association method is usually used, and each data table is sequentially associated. E.g figure 1 As shown, when performing the association of data tables A, B, C, and D, usually first associate table A with table B to obtain intermediate result table A-B, and then associate intermediate result table A-B with table C to obtain intermediate result Table A-B-C, and finally, associate the intermediate result table A-B-C with table D to obtain the final association result A-B-C-D. [0004] However, since the above...

AI Technical Summary

Problems solved by technology

[0004] However, since the above-mentioned method of serial association does not limit the sequence of association of each data table, tables with a large amount of data may be frequently loaded and calculated during the association process, especially for the association of hundreds of millions of large-scale data tables, it takes a lot of time massive computing resources

Images