Efficient parallel data alignment method, system and device

Abstract

The embodiment of the invention provides an efficient parallel data alignment method, system and device, relates to the technical field of data processing, is applied to a first server, and comprises the following steps: distributing stored data into different preset first arrays respectively according to data identifiers of the stored data; for each first array, subjecting data contained in the first array and data contained in a target second array in a second server to data alignment processing, obtaining a data alignment sub-result, and presetting second arrays corresponding to the first arrays one to one in the second server, wherein the target second array is a second array corresponding to the first array; and combining the data alignment sub-results to obtain a data alignment result between the first server and the second server. By applying the scheme provided by the embodiment of the invention, data transmission resources consumed in a single data alignment processing process can be reduced.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to an efficient parallel data alignment method, system and device. Background technique [0002] An object often has multiple types of feature data. For example, in the case that the object is a user, the feature data of the user includes feature data of age, gender, occupation, educational background, and the like. In addition, the types of characteristic data of objects stored in servers of different organizations are different. For example, the communication company's server stores feature data such as the user's call data and short message data, and the shopping website's server stores feature data of the user's purchased commodity data and purchase time data. [0003] In some application scenarios, it is necessary to combine the feature data of the objects stored in the servers of multiple institutions for data processing, but since the servers of different in...

AI Technical Summary

Problems solved by technology

[0004] In the existing technology, when the servers of multiple institutions jointly perform data alignment processing, the servers can be divided into a sending end and a receiving end. The sending end needs to send all the data stored by itself to the receiving end before the receiving end can start data alignment. Processing, a large amount of data needs to be transmitted between the sender and the receiver at one time, which consumes a large amount of data transmission resources at one time

Images