Data alignment method, system, electronic device, and storage medium

Abstract

The application relates to the technical field of data processing, and discloses a data alignment method, which comprises the following steps: a data providing end generates a key pair containing a public key and a private key, generates a Bloom filter binary file through the private key after hash processing of to-be-aligned data, and sends the public key and the Bloom filter binary file to a data alignment end; the data alignment end encrypts first encrypted data obtained through the public key after hash processing of a data alignment sample, and sends the first encrypted data to the data providing end; the data providing end sends second encrypted data obtained through alignment encryption of the first encrypted data through the private key to the data alignment end; the data alignment end decrypts the second encrypted data to obtain third encrypted data, collides the third encrypted data with the Bloom filter binary file to obtain aligned data, and returns the aligned data to the data providing end. The data alignment method improves the speed and efficiency of data alignment under the premise of meeting data privacy, safety and supervision.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data alignment method, system, electronic device, and computer-readable storage medium. Background Technology

[0002] Given the importance of data privacy, awareness of data protection is gradually increasing both domestically and internationally. Federated learning, in essence, is a distributed machine learning technology or framework based on data privacy protection. Its goal is to achieve collaborative modeling without data loss, thereby improving the performance of AI models and empowering business operations, while ensuring data privacy, security, and legal compliance. The data used for model training in federated learning systems is often sensitive user data. With increasing emphasis on data privacy protection and increasingly stringent data usage regulations, how to connect data silos through federated learning technology and achieve secure, efficient, and rapid data alignment and fusion among various institutions has become an urgent technical challenge.

[0003] Conventional data fusion methods compare data one by one. When the dataset is very large, the problems with conventional data alignment and fusion methods become apparent; the alignment speed increases exponentially with the amount of data to be aligned. However, Bloom filters align data with less memory, offer fast insertion and query speeds, and have a time complexity of O(k), independent of the number of elements in the dataset, significantly improving alignment speed. This invention, while satisfying data privacy, security, and regulatory requirements, allows data collaborating parties to quickly, accurately, and securely use their respective data for alignment and fusion. This helps enterprises protect their data while obtaining more accurate user data to support continuous business optimization. Summary of the Invention

[0004] In view of the above, it is necessary to provide a data alignment method to improve the speed and efficiency of data alignment while ensuring data security.

[0005] To achieve the above objectives, the present invention provides a data alignment method applied at the data providing end, comprising:

[0006] Generate a key pair containing a public key and a private key, perform hash processing on the data to be aligned, generate a Bloom filter binary file from the processed data using the private key, and send the public key and the Bloom filter binary file to the data alignment end;

[0007] The first encrypted data is obtained by hashing the data alignment sample and then encrypting it with the public key.

[0008] The first encrypted data is aligned and encrypted using the private key to obtain the second encrypted data, and the second encrypted data is sent to the data alignment end;

[0009] The alignment data returned by the data alignment end is received. The alignment data is obtained by colliding the third encrypted data obtained by the data alignment end decrypting the second encrypted data with the Bloom filter binary file.

[0010] Optionally, the data to be aligned contains multiple sample identifiers, and the step of hashing the data to be aligned and generating a Bloom filter binary file from the processed data using the private key includes:

[0011] The hash value is calculated based on the value of the sample identifier of the data to be aligned;

[0012] Generate a Bloom filter of the corresponding size based on the number of rows of the data to be aligned;

[0013] The hash value of the data to be aligned is encrypted using the private key and inserted into the Bloom filter to obtain a Bloom filter binary file.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a data alignment method applied to a data alignment end, comprising:

[0015] The receiver receives a public key and a Bloom filter binary file sent by the data provider. The Bloom filter binary file is generated by the data provider after hashing the data to be aligned and then encrypting it with a private key that is in the same key pair as the public key.

[0016] The data alignment sample is hashed, and the processed data is encrypted using the public key to obtain the first encrypted data. The first encrypted data is then sent to the data provider.

[0017] The receiving end encrypts the first encrypted data using the private key to obtain the second encrypted data.

[0018] The second encrypted data is decrypted to obtain the third encrypted data. The third encrypted data is then collided with the Bloom filter binary file to obtain aligned data, and the aligned data is sent to the data provider.

[0019] Optionally, hashing the data alignment sample and encrypting the processed data using the public key to obtain the first encrypted data includes:

[0020] Filter out data columns in the data alignment sample that share a common identifier with the data to be aligned from the data provider;

[0021] Based on the calculation formula for calculating the hash value corresponding to the data to be aligned by the data provider, the hash value corresponding to the data alignment sample of the data alignment end is calculated for the selected data column;

[0022] Generate a first number of random numbers corresponding to the number of data rows in the data alignment sample;

[0023] A second random number is generated based on the first random number;

[0024] A third random number is calculated from the first random number based on the public key;

[0025] The first encrypted data is obtained by calculating the hash value of the data alignment sample based on the third random number.

[0026] Optionally, colliding the third encrypted data with the Bloom filter binary file to obtain aligned data includes:

[0027] Obtain the data line containing the third encrypted data from the Bloom filter binary file, and store the data line as aligned data.

[0028] Optionally, when the number of data providers is not less than two, colliding the third encrypted data with the Bloom filter binary file to obtain aligned data, and sending the aligned data to the data providers includes:

[0029] Obtain the data lines containing the third encrypted data from each Bloom filter binary file and store them as corresponding data tables;

[0030] The intersection of all data tables based on their index columns is used to obtain the alignment data for each table.

[0031] Each alignment data is sent to the corresponding data provider.

[0032] Furthermore, to achieve the above objectives, the present invention also provides a data alignment system, the system comprising:

[0033] Data Provider: This component generates a key pair containing a public and private key, hashes the data to be aligned, generates a Bloom filter binary file from the processed data using the private key, and sends the public key and the Bloom filter binary file to the data alignment provider. It also receives first encrypted data obtained by hashing the data alignment sample and encrypting it using the public key; aligns and encrypts the first encrypted data using the private key to obtain second encrypted data, and sends the second encrypted data to the data alignment provider; and receives aligned data returned by the data alignment provider, which is obtained by collision between the third encrypted data (obtained by decrypting the second encrypted data by the data alignment provider) and the Bloom filter binary file.

[0034] Data Alignment End: This end receives a public key and a Bloom filter binary file from a data provider. The Bloom filter binary file is generated by the data provider hashing the data to be aligned and then encrypting it with a private key in the same key pair as the public key. It then hashes the data alignment sample, encrypts the processed data with the public key to obtain first encrypted data, and sends the first encrypted data to the data provider. Next, it receives second encrypted data obtained by aligning and encrypting the first encrypted data with the private key. Finally, it decrypts the second encrypted data to obtain third encrypted data, collides the third encrypted data with the Bloom filter binary file to obtain aligned data, and sends the aligned data to the data provider.

[0035] In addition, to achieve the above objectives, the present invention also provides an electronic device applied to a data provider, the electronic device comprising: a memory and a processor, the processor being electrically connected to the memory, the memory storing a data provider alignment program that can run on the processor, the data provider alignment program being executed by the processor to implement the data alignment method applied to the data provider as described above.

[0036] In addition, to achieve the above objectives, the present invention also provides an electronic device for use at a data alignment end. The electronic device includes a memory and a processor, the processor being electrically connected to the memory. The memory stores a data alignment end alignment program that can run on the processor. When the data alignment end alignment program is executed by the processor, it implements the data alignment method applied to the data alignment end as described above.

[0037] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a data alignment program, which can be executed by one or more processors to implement the data alignment method described above. Attached Figure Description

[0038] Figure 1 This is a flowchart of the first embodiment of the data alignment method of the present invention;

[0039] Figure 2 This is a flowchart of a second embodiment of the data alignment method of the present invention;

[0040] Figure 3 This is a schematic diagram of the first embodiment of the electronic device of the present invention;

[0041] Figure 4 This is a schematic diagram of a second embodiment of the electronic device of the present invention;

[0042] Figure 5 This is a schematic diagram of the first embodiment of the data alignment system of the present invention;

[0043] Figure 6 This is a schematic diagram of a second embodiment of the data alignment system of the present invention.

[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0046] It should be noted that the descriptions involving "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0047] like Figure 1 The diagram shown is a flowchart of the first embodiment of the data alignment method of the present invention. The data alignment method is applied to the data providing end and includes steps S11-S14.

[0048] S11. Generate a key pair containing a public key and a private key, perform hash processing on the data to be aligned, generate a Bloom filter binary file from the processed data using the private key, and send the public key and the Bloom filter binary file to the data alignment end.

[0049] Specifically, the data to be aligned contains multiple sample identifiers, and the hash processing of the data to be aligned includes: calculating a hash value based on the value of the sample identifier of the data to be aligned.

[0050] In one embodiment, the data to be aligned at the data provider includes a data column consisting of the values ​​corresponding to the sample identifiers tel and cnid, and an index data column index, for example:

[0051] Index Tel Cnid 1 13922222222 4******2 2 13933333333 4******3 3 13944444444 4******4 … … … 100000 15911111111 5******3

[0052] The formula for calculating the hash value is as follows:

[0053] Hash = md5(tel) + sha256(cnid)

[0054] The calculation results for the above example are as follows:

[0055] Index Hash 1 md5(13922222222)+sha256(4******2) 2 md5(13933333333)+sha256(4******3) 3 md5(13944444444)+sha256(4******4) … … 100000 md5(15911111111)+sha256(5******3)

[0056] Specifically, generating a Bloom filter binary file from the processed data using the private key includes: generating a Bloom filter of a corresponding size based on the number of rows in the data to be aligned; encrypting the hash value of the data to be aligned using the private key and inserting it into the Bloom filter to obtain the Bloom filter binary file. The hash encryption calculation formula is (hash). d mod N, where d and N are the RSA private keys generated by the data provider.

[0057] S12. Receive the first encrypted data obtained by the data alignment end after hashing the data alignment sample and encrypting it with the public key.

[0058] Specifically, the data alignment end performs hash processing on the data alignment samples, including: the data alignment end filters out data columns in the data alignment samples that have a common identifier with the data to be aligned at the data provider; based on the calculation formula of the data provider for calculating the hash value corresponding to the data to be aligned, the data alignment end calculates the hash value of the data alignment sample corresponding to the data provider for the filtered data columns.

[0059] For example: The data to be aligned at the data provider end contains sample identifiers tel and cnid, and its hash value is calculated using the formula: Hash = md5(tel) + sha256(cnid). The data alignment sample at the data alignment end is as follows:

[0060] Index Tel Cnid Name 1 13911111111 4******1 Zhao** 2 13922222222 4******2 money** 3 13933333333 4******3 open**

[0061] Filter out the data columns corresponding to the sample identifiers tel and cnid in the data alignment samples, and calculate hash = md5(tel) + sha256(cnid) as follows:

[0062] Index Hash 1 md5(13911111111)+sha256(4******1) 2 md5(13922222222)+sha256(4******2) 3 md5(13933333333)+sha256(4******3)

[0063] Specifically, the data alignment end encrypts the processed data using the public key to obtain the first encrypted data, including: the data alignment end generates a corresponding number of first random numbers r1, r2...r based on the number of data rows in the data alignment sample. i ; Generate a second random number rr1, rr2...rr based on the first random number. i The third random number r1', r2'...r is calculated from the first random number using the public key. i The data alignment end calculates the first encrypted data α based on the hash value of the data alignment sample using the third random number.

[0064] Specifically, the formula for generating the second random number based on the first random number is rr. i =r i -1 mod N;

[0065] The formula for calculating the third random number from the first random number using the public key is r. i '=r i e mod N, where e and N are the RSA public keys generated by the data provider;

[0066] The formula for calculating the first encrypted data based on the hash value of the data-aligned sample using the third random number is α = hash × r. i 'mod N.

[0067] S13. Align and encrypt the first encrypted data using the private key to obtain the second encrypted data, and send the second encrypted data to the data alignment end.

[0068] Specifically, the formula for calculating the second encrypted data β is β = α. d mod N.

[0069] In one embodiment, when the number of data providers participating in data alignment and fusion is not less than two, each data provider calculates the corresponding hash value using MD5 and / or SHA algorithms based on the value of the sample identifier of the data to be aligned. For example: the data to be aligned from the first data provider contains sample identifiers A1 and A2, and its hash value is calculated using the formula: Hash = md5(A1) + sha256(A2); the data to be aligned from the second data provider contains sample identifiers A1 and A3, and its hash value is calculated using the formula: Hash = md5(A1) + md5(A3); the data to be aligned from the third data provider contains sample identifiers A1, A2, and A3, and its hash value is calculated using the formula: Hash = md5(A1) + sha256(A2) + md5(A3).

[0070] S14. Receive the alignment data returned by the data alignment end. The alignment data is obtained by colliding the third encrypted data obtained by the data alignment end decrypting the second encrypted data with the Bloom filter binary file.

[0071] Specifically, the formula for decrypting the second encrypted data β at the data alignment end to obtain the third encrypted data γ is γ=(β×rr) i )mod N.

[0072] Specifically, the data alignment end collides the third encrypted data with the Bloom filter binary file to obtain aligned data, including: obtaining the data line containing the third encrypted data γ in the Bloom filter binary file, and storing the data line as aligned data γ. s .

[0073] Specifically, when the number of data providers participating in data alignment and fusion is not less than two, the data alignment end collides the third encrypted data with the Bloom filter binary file to obtain aligned data, and sends the aligned data to the data provider, including:

[0074] Retrieve the data rows containing the third encrypted data γ from each Bloom filter binary file and store them as data table γ respectively. s 1,γ s 2, ...,γ s n;

[0075] All data tables γ s 1,γ s 2, ...,γ s n Obtain the alignment data for each data table by taking the intersection of the index columns. s 1', γ s 2',...,γ s n';

[0076] Align data γ s 1', γ s 2',...,γ s n' is sent to the corresponding data provider respectively.

[0077] like Figure 2 The diagram shown is a flowchart of a second embodiment of the data alignment method of the present invention. This data alignment method is applied to the data alignment end and includes steps S21-S24.

[0078] S21. Receive the public key and Bloom filter binary file sent by the data provider. The Bloom filter binary file is generated by the data provider after hashing the data to be aligned and encrypting it with the private key in the same key pair as the public key.

[0079] Specifically, the data to be aligned contains multiple sample identifiers, and the data provider performs hash processing on the data to be aligned by: calculating a hash value based on the value of the sample identifier of the data to be aligned.

[0080] In one embodiment, the data to be aligned at the data provider includes a data column consisting of the values ​​corresponding to the sample identifiers tel and cnid, and an index data column index, for example:

[0081] Index Tel Cnid 1 13922222222 4******2 2 13933333333 4******3 3 13944444444 4******4 … … … 100000 15911111111 5******3

[0082] The formula for calculating the hash value is as follows:

[0083] Hash = md5(tel) + sha256(cnid)

[0084] The calculation results for the above example are as follows:

[0085] Index Hash 1 md5(13922222222)+sha256(4******2) 2 md5(13933333333)+sha256(4******3) 3 md5(13944444444)+sha256(4******4) … … 100000 md5(15911111111)+sha256(5******3)

[0086] Specifically, the data provider generates a Bloom filter binary file from the processed data using the private key, including: generating a Bloom filter of a corresponding size based on the number of rows in the data to be aligned; encrypting the hash value of the data to be aligned using the private key; and inserting the encrypted hash value into the Bloom filter to obtain the Bloom filter binary file. The hash encryption calculation formula is (hash). d mod N, where d and N are the RSA private keys generated by the data provider.

[0087] S22. The data alignment sample is hashed, and the processed data is encrypted using the public key to obtain the first encrypted data. The first encrypted data is then sent to the data provider.

[0088] Specifically, hashing the data alignment samples includes: the data alignment end selecting data columns in the data alignment samples that have a common identifier with the data to be aligned at the data provider end; and calculating the hash value of the data alignment sample corresponding to the data provider end based on the calculation formula for calculating the hash value of the data to be aligned at the data provider end.

[0089] For example: The data to be aligned at the data provider end contains sample identifiers tel and cnid, and its hash value is calculated using the formula: Hash = md5(tel) + sha256(cnid). The data alignment sample at the data alignment end is as follows:

[0090] Index Tel Cnid Name 1 13911111111 4******1 Zhao** 2 13922222222 4******2 money** 3 13933333333 4******3 open**

[0091] Filter out the data columns corresponding to the sample identifiers tel and cnid in the data alignment samples, and calculate hash =

[0092] The md5(tel) + sha256(cnid) results are as follows:

[0093] Index Hash 1 md5(13911111111)+sha256(4******1) 2 md5(13922222222)+sha256(4******2) 3 md5(13933333333)+sha256(4******3)

[0094] Specifically, the data alignment end encrypts the processed data using the public key to obtain the first encrypted data, including: the data alignment end generates a corresponding number of first random numbers r1, r2...r based on the number of data rows in the data alignment sample. i ; Generate a second random number rr1, rr2...rr based on the first random number. i The third random number r1', r2'...r is calculated from the first random number using the public key. i The data alignment end calculates the first encrypted data α based on the hash value of the data alignment sample using the third random number.

[0095] Specifically, the formula for generating the second random number based on the first random number is rr. i =r i -1 mod N;

[0096] The formula for calculating the third random number from the first random number using the public key is r. i '=r i e mod N, where e and N are the RSA public keys generated by the data provider;

[0097] The formula for calculating the first encrypted data based on the hash value of the data-aligned sample using the third random number is α = hash × r. i 'mod N.

[0098] S23. Receive the second encrypted data obtained by aligning and encrypting the first encrypted data using the private key from the data provider.

[0099] Specifically, the formula for calculating the second encrypted data β is β = α. d mod N.

[0100] In one embodiment, when the number of data providers participating in data alignment and fusion is not less than two, each data provider calculates the corresponding hash value using MD5 and / or SHA algorithms based on the value of the sample identifier of the data to be aligned. For example: the data to be aligned from the first data provider contains sample identifiers A1 and A2, and its hash value is calculated using the formula: Hash = md5(A1) + sha256(A2); the data to be aligned from the second data provider contains sample identifiers A1 and A3, and its hash value is calculated using the formula: Hash = md5(A1) + md5(A3); the data to be aligned from the third data provider contains sample identifiers A1, A2, and A3, and its hash value is calculated using the formula: Hash = md5(A1) + sha256(A2) + md5(A3).

[0101] S24. Decrypt the second encrypted data to obtain the third encrypted data, collide the third encrypted data with the Bloom filter binary file to obtain aligned data, and send the aligned data to the data provider.

[0102] Specifically, the formula for decrypting the second encrypted data β at the data alignment end to obtain the third encrypted data γ is γ=(β×rr) i )mod N.

[0103] Specifically, the data alignment end collides the third encrypted data with the Bloom filter binary file to obtain aligned data, including: obtaining the data line containing the third encrypted data γ in the Bloom filter binary file, and storing the data line as aligned data γ. s .

[0104] Specifically, when the number of data providers participating in data alignment and fusion is not less than two, the data alignment end collides the third encrypted data with the Bloom filter binary file to obtain aligned data, and sends the aligned data to the data provider, including:

[0105] Retrieve the data rows containing the third encrypted data γ from each Bloom filter binary file and store them as data table γ respectively. s 1,γ s 2, ...,γ s n;

[0106] All data tables γ s 1,γ s 2, ...,γ s n Obtain the alignment data for each data table by taking the intersection of the index columns. s 1', γ s 2',...,γ s n';

[0107] Align data γ s 1', γ s 2',...,γ s n' is sent to the corresponding data provider respectively.

[0108] As can be seen from the above embodiments, the data alignment method proposed in this invention, based on the combined use of MD5 / SHA / RSA encryption technology and Bloom filters, greatly improves the speed and efficiency of data alignment while satisfying data privacy, security and supervision. Data collaborating parties can quickly, accurately and securely use their respective data for alignment and fusion.

[0109] like Figure 3 The diagram shown is a schematic representation of the first embodiment of the electronic device of the present invention. The electronic device 1 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. The electronic device 1 can be a computer, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing, which is a type of distributed computing consisting of a super virtual computer composed of a group of loosely coupled computers.

[0110] In this embodiment, the electronic device 1 includes, but is not limited to, a memory 11, a processor 12, and a network interface 13 that can be interconnected via a system bus. The memory 11 stores a data provider end alignment program 10, which can be executed by the processor 12. Figure 3 Only the electronic device 1 with components 11-13 and data providing end alignment procedure 10 is shown. Those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0111] The memory 11 includes RAM and at least one type of readable storage medium. The RAM provides a cache for the operation of the electronic device 1; the readable storage medium can be volatile or non-volatile. Specifically, the readable storage medium can be a storage medium such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the readable storage medium can be an internal storage unit of the electronic device 1, such as the hard disk of the electronic device 1; in other embodiments, the storage medium can also be an external storage device of the electronic device 1, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. In this embodiment, the readable storage medium of the memory 11 mainly includes a program storage area and a data storage area. The program storage area is typically used to store the operating system and various application software installed on the electronic device 1, such as the code of the data provider alignment program 10 in one embodiment of the present invention. The data storage area can store data created based on the use of blockchain nodes, such as various types of data that have been output or will be output.

[0112] In some embodiments, processor 12 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 12 is typically used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication with other devices. In this embodiment, processor 12 is used to run program code stored in memory 11 or process data, such as running data provider end-alignment program 10.

[0113] The network interface 13 may include a wireless network interface or a wired network interface, which is used to establish a communication connection between the electronic device 1 and a client (not shown in the figure).

[0114] Optionally, the electronic device 1 may further include a user interface, which may include a display, an input unit such as a keyboard, and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an organic light-emitting diode (OLED) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0115] In one embodiment of the present invention, when the data providing end alignment program 10 is executed by the processor 12, it performs the following steps S11-S14.

[0116] S11. Generate a key pair containing a public key and a private key, perform hash processing on the data to be aligned, generate a Bloom filter binary file from the processed data using the private key, and send the public key and the Bloom filter binary file to the data alignment end.

[0117] S12. Receive the first encrypted data obtained by the data alignment end after hashing the data alignment sample and encrypting it with the public key.

[0118] S13. Align and encrypt the first encrypted data using the private key to obtain the second encrypted data, and send the second encrypted data to the data alignment end.

[0119] S14. Receive the alignment data returned by the data alignment end. The alignment data is obtained by colliding the third encrypted data obtained by the data alignment end decrypting the second encrypted data with the Bloom filter binary file.

[0120] The specific operation steps implemented in steps S11-S14 above are largely the same as those in the first embodiment of the data alignment method of the present invention, and will not be repeated here.

[0121] In other embodiments, the data provider alignment program 10 may be divided into one or more modules, which are stored in the memory 11 and executed by one or more processors (processor 12 in this embodiment) to complete the present invention. The module referred to in the present invention refers to a series of computer program instruction segments capable of performing specific functions, used to describe the execution process of the data provider alignment program 10 in the electronic device 1.

[0122] like Figure 4The diagram shown is a schematic representation of a second embodiment of the electronic device of the present invention. The electronic device 2 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. The electronic device 2 can be a computer, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing, which is a type of distributed computing consisting of a super virtual computer composed of a group of loosely coupled computers.

[0123] In this embodiment, the electronic device 2 includes, but is not limited to, a memory 21, a processor 22, and a network interface 23 that can be interconnected via a system bus. The memory 21 stores a data alignment end alignment program 20, which can be executed by the processor 22. Figure 4 Only electronic device 2 with components 21-23 and data alignment end alignment procedure 20 is shown. Those skilled in the art will understand that... Figure 4 The structure shown does not constitute a limitation on electronic device 2, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0124] The memory 21 includes RAM and at least one type of readable storage medium. The RAM provides a cache for the operation of the electronic device 2; the readable storage medium can be volatile or non-volatile. Specifically, the readable storage medium can be a storage medium such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the readable storage medium can be an internal storage unit of the electronic device 2, such as the hard disk of the electronic device 2; in other embodiments, the storage medium can also be an external storage device of the electronic device 2, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 2. In this embodiment, the readable storage medium of the memory 21 mainly includes a program storage area and a data storage area. The program storage area is typically used to store the operating system and various application software installed on the electronic device 2, such as the code of the data alignment end alignment program 20 in one embodiment of the present invention. The data storage area can store data created according to the use of blockchain nodes, such as various types of data that have been output or will be output.

[0125] In some embodiments, processor 22 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 22 is typically used to control the overall operation of the electronic device 2, such as performing control and processing related to data interaction or communication with other devices. In this embodiment, the processor 22 is used to run program code stored in the memory 21 or process data, such as running a data alignment end-alignment program 20.

[0126] The network interface 23 may include a wireless network interface or a wired network interface, which is used to establish a communication connection between the electronic device 2 and the client (not shown in the figure).

[0127] Optionally, the electronic device 2 may further include a user interface, which may include a display, an input unit such as a keyboard, and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an organic light-emitting diode (OLED) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 2 and to display a visual user interface.

[0128] In one embodiment of the present invention, when the data alignment end alignment program 20 is executed by the processor 22, it performs the following steps S21-S24.

[0129] S21. Receive the public key and Bloom filter binary file sent by the data provider. The Bloom filter binary file is generated by the data provider after hashing the data to be aligned and encrypting it with the private key in the same key pair as the public key.

[0130] S22. The data alignment sample is hashed, and the processed data is encrypted using the public key to obtain the first encrypted data. The first encrypted data is then sent to the data provider.

[0131] S23. Receive the second encrypted data obtained by aligning and encrypting the first encrypted data using the private key from the data provider.

[0132] S24. Decrypt the second encrypted data to obtain the third encrypted data, collide the third encrypted data with the Bloom filter binary file to obtain aligned data, and send the aligned data to the data provider.

[0133] The specific operation steps implemented in steps S21-S24 above are largely the same as those in the second embodiment of the data alignment method of the present invention, and will not be repeated here.

[0134] In other embodiments, the data alignment end alignment program 20 may be divided into one or more modules, which are stored in the memory 21 and executed by one or more processors (processor 22 in this embodiment) to complete the present invention. The module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions and are used to describe the execution process of the data alignment end alignment program 20 in the electronic device 2.

[0135] As can be seen from the above embodiments, the electronic devices 1 and 2 proposed in this invention, based on the combined use of MD5 / SHA / RSA encryption technology and Bloom filters, greatly improve the speed and efficiency of data alignment while satisfying data privacy, security and supervision. Data collaborating parties can quickly, accurately and securely use their respective data for alignment and fusion.

[0136] like Figure 5 The diagram shown is a schematic diagram of the first embodiment of the data alignment system of the present invention.

[0137] In a first embodiment of the present invention, the data alignment system 10 includes a data providing end 110 and a data alignment end 120, exemplarily:

[0138] The data providing end 110 is used to generate a key pair containing a public key and a private key, hash the data to be aligned, generate a Bloom filter binary file from the processed data using the private key, and send the public key and the Bloom filter binary file to the data alignment end 120; receive the first encrypted data obtained by the data alignment end 120 after hashing the data alignment sample and encrypting it using the public key; align and encrypt the first encrypted data using the private key to obtain the second encrypted data, and send the second encrypted data to the data alignment end 120; receive the alignment data returned by the data alignment end 120, wherein the alignment data is obtained by collision between the third encrypted data obtained by the data alignment end 120 after decrypting the second encrypted data and the Bloom filter binary file;

[0139] The data alignment end 120 is used to receive a public key and a Bloom filter binary file sent by the data provider 110. The Bloom filter binary file is generated by the data provider 110 by hashing the data to be aligned and then encrypting it with a private key in the same key pair as the public key. The data alignment sample is hashed, and the processed data is encrypted with the public key to obtain first encrypted data. The first encrypted data is sent to the data provider 110. The data provider 110 receives second encrypted data obtained by aligning and encrypting the first encrypted data with the private key. The second encrypted data is decrypted to obtain third encrypted data. The data line containing the third encrypted data in the Bloom filter binary file is obtained and stored as alignment data. The alignment data is sent to the data provider 110.

[0140] The specific operation steps implemented when the data providing end 110 and the data alignment end 120 are executed are largely the same as those in the above embodiments, and will not be repeated here.

[0141] like Figure 6 The diagram shown is a schematic representation of a second embodiment of the data alignment system of the present invention.

[0142] In a second embodiment of the present invention, the data alignment system 20 includes a first data providing end 210, a second data providing end 220, a third data providing end 230, and a data alignment end 240, as exemplarily:

[0143] The first data provider 210 is used to generate a key pair containing a public key and a private key, hash the data to be aligned, generate a Bloom filter binary file from the processed data using the private key, and send the public key and the Bloom filter binary file to the data alignment provider 240; receive the first encrypted data obtained by hashing the data alignment sample and encrypting it using the public key from the data alignment provider 240; align and encrypt the first encrypted data using the private key to obtain the second encrypted data, and send the second encrypted data to the data alignment provider 240; receive the alignment data returned by the data alignment provider 240, wherein the alignment data is obtained by collision between the third encrypted data obtained by decrypting the second encrypted data from the data alignment provider 240 and the Bloom filter binary file;

[0144] The functions or operation steps implemented by the second data provider 220 and the third data provider 230 when they are executed are the same as those of the data provider 210 described above, and will not be repeated here.

[0145] The data alignment end 240 is used to receive a public key 1 and a Bloom filter binary file 1 sent by the first data provider 210. The Bloom filter binary file 1 is generated by the first data provider 210 by hashing the data to be aligned 1 and encrypting it with a private key 1 in the same key pair as the public key 1. The data alignment sample is hashed, and the processed data is encrypted with the public key 1 to obtain first encrypted data 1, which is then sent to the first data provider 210. The data alignment end 240 receives a second encrypted data 1 obtained by aligning and encrypting the first encrypted data 1 with the private key from the first data provider 210. The second encrypted data is decrypted to obtain a third encrypted data 1, and the data alignment end 240 is then obtained. The Bloom filter binary file 1 contains a data line 1 of third encrypted data 1, which is stored as a first data table. The same operation steps as those for the first data provider 210 are performed on the second data provider 220 and the third data provider 230 to obtain the corresponding second data table and third data table. The first data table, the second data table, and the third data table are intersected according to the index data column to obtain the first aligned data corresponding to the first data table, the second aligned data corresponding to the second data table, and the third aligned data corresponding to the third data table. The first aligned data is sent to the first data provider 210, the second aligned data is sent to the second data provider 220, and the third aligned data is sent to the third data provider 230.

[0146] The specific operation steps implemented when the first data provider 210, the second data provider 220, the third data provider 230 and the data alignment provider 240 are executed are largely the same as those in the above embodiments, and will not be repeated here.

[0147] Furthermore, this invention also proposes a computer-readable storage medium, which can be volatile or non-volatile. Specifically, the computer-readable storage medium can be any one or any combination of several of the following: hard disk, multimedia card, SD card, flash memory card, SMC, read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, etc. The computer-readable storage medium stores a data alignment program, which includes a data provider end alignment program 10 and a data alignment end alignment program 20.

[0148] When the data providing end-alignment procedure 10 is executed by the processor, it performs the following operations:

[0149] A11. Generate a key pair containing a public key and a private key, perform hash processing on the data to be aligned, generate a Bloom filter binary file from the processed data using the private key, and send the public key and the Bloom filter binary file to the data alignment end.

[0150] A12. Receive the first encrypted data obtained by the data alignment end after hashing the data alignment sample and encrypting it with the public key;

[0151] A13. Align and encrypt the first encrypted data using the private key to obtain the second encrypted data, and send the second encrypted data to the data alignment end;

[0152] A14. Receive the alignment data returned by the data alignment end. The alignment data is obtained by colliding the third encrypted data obtained by the data alignment end decrypting the second encrypted data with the Bloom filter binary file.

[0153] When the data alignment end alignment procedure 20 is executed by the processor, it performs the following operations:

[0154] A21. Receive the public key and Bloom filter binary file sent by the data provider. The Bloom filter binary file is generated by the data provider after hashing the data to be aligned and encrypting it with the private key in the same key pair as the public key.

[0155] A22. The data alignment sample is hashed, and the processed data is encrypted using the public key to obtain the first encrypted data. The first encrypted data is then sent to the data provider.

[0156] A23. Receive the second encrypted data obtained by the data provider aligning and encrypting the first encrypted data using the private key;

[0157] A24. Decrypt the second encrypted data to obtain the third encrypted data, collide the third encrypted data with the Bloom filter binary file to obtain aligned data, and send the aligned data to the data provider.

[0158] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0161] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A data alignment method, applied at the data provider end, characterized in that, include: Generate a key pair containing a public key and a private key, calculate a hash value based on the value of the sample identifier of the data to be aligned, generate a Bloom filter of the corresponding size based on the number of rows of the data to be aligned, encrypt the hash value of the data to be aligned based on the private key and insert it into the Bloom filter to obtain a Bloom filter binary file, and send the public key and the Bloom filter binary file to the data alignment end. The data alignment end receives the first encrypted data obtained by hashing the data alignment sample and encrypting it with the public key. The hashing of the data alignment sample by the data alignment end includes: filtering out data columns in the data alignment sample that have a common identifier with the data to be aligned by the data provider; calculating the hash value corresponding to the data to be aligned based on the calculation formula of the data provider; and calculating the hash value corresponding to the data alignment sample for the filtered data columns. The first encrypted data is aligned and encrypted using the private key to obtain the second encrypted data, and the second encrypted data is sent to the data alignment end; The alignment data returned by the data alignment end is received. The alignment data is obtained by colliding the third encrypted data obtained by the data alignment end decrypting the second encrypted data with the Bloom filter binary file.

2. A data alignment method, applied to a data alignment end, characterized in that, include: The system receives a public key and a Bloom filter binary file sent by a data provider. The Bloom filter binary file is obtained by the data provider calculating the hash value based on the value of the sample identifier of the data to be aligned, generating a Bloom filter of the corresponding size based on the number of rows of the data to be aligned, encrypting the hash value of the data to be aligned using the private key in the same key pair as the public key, and inserting it into the Bloom filter. The data columns in the data alignment sample that have a common identifier with the data to be aligned from the data provider are selected. Based on the calculation formula of the data provider to calculate the hash value corresponding to the data to be aligned, the hash value corresponding to the data alignment sample is calculated for the selected data columns. The hash value corresponding to the data alignment sample is encrypted with the public key to obtain the first encrypted data. The first encrypted data is sent to the data provider. The receiving end encrypts the first encrypted data using the private key to obtain the second encrypted data. The second encrypted data is decrypted to obtain the third encrypted data. The data line containing the third encrypted data is obtained from the Bloom filter binary file. The data line is stored as alignment data and sent to the data provider.

3. The data alignment method as described in claim 2, characterized in that, The first encrypted data is obtained by encrypting the hash value corresponding to the data alignment sample using the public key, including: Generate a first number of random numbers corresponding to the number of data rows in the data alignment sample; A second random number is generated based on the first random number; A third random number is calculated from the first random number based on the public key; The first encrypted data is obtained by calculating the hash value corresponding to the data alignment sample based on the third random number.

4. The data alignment method as described in claim 2, characterized in that, When the number of data providers is not less than two, colliding the third encrypted data with the Bloom filter binary file to obtain aligned data, and sending the aligned data to the data providers includes: Obtain the data lines containing the third encrypted data from each Bloom filter binary file and store them as corresponding data tables; The intersection of all data tables based on their index columns is used to obtain the alignment data for each table. Each alignment data is sent to the corresponding data provider.

5. A data alignment system, characterized in that, The system includes: Data Provider: Generates a key pair containing a public and private key; calculates a hash value based on the sample identifier of the data to be aligned; generates a Bloom filter of corresponding size based on the number of rows in the data to be aligned; encrypts the hash value of the data to be aligned using the private key and inserts it into the Bloom filter to obtain a Bloom filter binary file; sends the public key and the Bloom filter binary file to the data alignment provider; receives first encrypted data obtained by hashing the data alignment sample and encrypting it using the public key; aligns and encrypts the first encrypted data using the private key to obtain second encrypted data, and sends the second encrypted data to the data alignment provider; receives aligned data returned by the data alignment provider, which is obtained by collision between the third encrypted data obtained by decrypting the second encrypted data by the data alignment provider and the Bloom filter binary file. Data Alignment End: This end receives a public key and a Bloom filter binary file from the data provider. The Bloom filter binary file is generated by the data provider using a hash value calculated from the sample identifier of the data to be aligned. A Bloom filter of the corresponding size is generated based on the number of rows in the data to be aligned. The hash value of the data to be aligned is encrypted using a private key in the same key pair as the public key and inserted into the Bloom filter. The end then filters out data columns in the data alignment samples that share a common identifier with the data to be aligned from the data provider. Based on the calculation formula used by the data provider to calculate the hash value corresponding to the data to be aligned, the end calculates the hash value corresponding to the data alignment sample for the filtered data columns. The hash value corresponding to the data alignment sample is then encrypted using the public key to obtain first encrypted data, which is sent to the data provider. The end also receives second encrypted data obtained by the data provider aligning and encrypting the first encrypted data using the private key. The second encrypted data is decrypted to obtain third encrypted data. The data rows containing the third encrypted data are obtained from the Bloom filter binary file, stored as alignment data, and sent to the data provider.

6. An electronic device used in a data providing end, characterized in that, The electronic device includes: a memory and a processor, the processor being electrically connected to the memory, the memory storing a data provider alignment program that can run on the processor, the data provider alignment program implementing the data alignment method as described in claim 1 when executed by the processor.

7. An electronic device used in a data alignment terminal, characterized in that, The electronic device includes: a memory and a processor, the processor being electrically connected to the memory, the memory storing a data alignment end alignment program that can run on the processor, the data alignment end alignment program implementing the data alignment method as described in any one of claims 2-4 when executed by the processor.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a data alignment program, which can be executed by one or more processors to implement the data alignment method as described in any one of claims 1-4.

Images