An information mining method and system based on transverse federated learning

Abstract

The application discloses a kind of information mining method and system based on transverse federal learning.The method comprises the following steps: a participant node server obtains at least two groups of sample data of local participants;The participant node server calculates the local total number of the sample data, and the local occurrence times of each feature in sample data;The participant node server filters out local frequent feature set from each feature according to the local total number and the local occurrence times;The participant node server sends the occurrence information of the local frequent feature set to at least two federal node servers in the information mining system after random splitting;Federal node server according to the received data, the global frequent feature set of different participant sample data is counted, the first independent right.The association between features is deeply mined based on association learning while ensuring the privacy and security of participant data.

Description

Technical Field

[0001] This invention relates to the field of federated learning, and in particular to an information mining method and system based on horizontal federated learning. Background Technology

[0002] Federated learning is an emerging foundational technology for artificial intelligence. It was first proposed by Google in 2016 and was originally intended to solve the problem of Android mobile phone users updating models locally. Its design goal is to carry out efficient machine learning among multiple participants or multiple computing nodes while ensuring information security during big data exchange, protecting terminal data and personal data privacy, and ensuring legality and compliance.

[0003] Horizontal federated learning is a type of federated learning. It is suitable for situations where the data features of the participating parties overlap significantly, while the sample IDs overlap only slightly. For example, customer data from two banks in different regions [1,2]. The term "horizontal" comes from the "horizontal partitioning" of the data. Federated learning combines multiple rows of samples with the same characteristics from multiple participating parties; that is, the training data for each participating party is horizontally partitioned. This is called horizontal federated learning. Horizontal federation increases the total number of training samples.

[0004] However, in the era of big data, how to deeply mine the correlation information between features based on association learning while ensuring the data privacy and security of the participants has always been a hot research issue in the industry. Summary of the Invention

[0005] This invention provides an information mining method and system based on horizontal federated learning, which enables in-depth mining of the correlation information between features based on association learning while ensuring the data privacy and security of the participants.

[0006] In a first aspect, embodiments of the present invention provide an information mining method based on horizontal federated learning, applied to an information mining system based on horizontal federated learning, wherein the system includes at least two participating node servers and at least two federated node servers, including:

[0007] The participating node server obtains at least two sets of sample data from the local participating party;

[0008] The participating node server calculates the total number of local occurrences of the sample data and the number of times each feature appears locally in the sample data;

[0009] The participating node server filters out a set of frequently occurring local features from the features based on the total number of local features and the frequency of occurrence of the local features.

[0010] The participating node server randomly splits the occurrence information of the local frequent feature set and sends it to at least two federated node servers in the information mining system;

[0011] The federated node server calculates the global frequent feature set of sample data from different participants based on the received data, in order to obtain the correlation between the features in the global frequent feature set.

[0012] Secondly, embodiments of the present invention also provide an information mining system based on horizontal federated learning, wherein the system includes at least two participating node servers and at least two federated node servers, and the system applies any of the information mining methods described in the embodiments of the present invention.

[0013] This invention randomly splits the occurrence information of local frequent feature sets and sends it to a federated node server. The federated node server then uses the received data to statistically analyze the global frequent feature sets of sample data from different participants, thereby enabling the mining of correlation information between features based on multi-participant sample data. Because of the large amount of sample data, the mined correlation information is more general and reliable.

[0014] Furthermore, because only randomly distributed information is sent without sending sample data to the federated node server, the data privacy and security of the participants are guaranteed.

[0015] By filtering local frequent feature sets and sending only the relevant information of these local frequent feature sets to the federated node server, the exposure probability of participant sample data is further reduced, while also improving data transmission and statistical efficiency.

[0016] By merging the FP-Trees of each participant based on the N-PDZ protocol to construct a global FP-Tree, a global frequent feature set of tree structure is obtained, which solves the following problem: When using the FP-Growth algorithm for association rule analysis, it is necessary to build an FP-Tree based on the item header. However, in federated learning, the item header is distributed on multiple participant node servers. If an FP-Tree based on global data items is built, it is necessary to traverse the data of all participants multiple times and perform multiple rounds of interaction among all participants.

[0017] Because when merging the FP-Trees of each participant based on the N-PDZ protocol, only one frequent 1-itemset interaction and one local FP-Tree interaction with the participant are required, thus achieving the beneficial effect of requiring only a small amount of joint computation to statistically analyze frequent feature set information with low time and space complexity.

[0018] Because the N-PDZ protocol is a multi-party computation (MPC) protocol, it is easily extended to multiple parties, where MPC stands for Multi-Party Secure Computation. Furthermore, the construction of a global FP-Tree can be achieved by merging the FP-Tree paths of each party using the N-PDZ protocol, avoiding repeated traversal of the participants' original data items and a large number of interactions, resulting in high efficiency and suitability for large-scale datasets. Attached Figure Description

[0019] Figure 1 A flowchart illustrating an information mining method based on lateral federated learning, provided as an embodiment of the present invention;

[0020] Figure 2 This is a flowchart of another information mining method based on horizontal federated learning provided in the embodiments of the present invention;

[0021] Figure 3 This is a schematic diagram of a federated node server receiving data from one party, provided in an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of a federated node server receiving data from two parties, provided in an embodiment of the present invention.

[0023] Figure 5 This is a flowchart of another information mining method based on horizontal federated learning provided in the embodiments of the present invention;

[0024] Figure 6 This is a structure tree of participant data provided in an embodiment of the present invention;

[0025] Figure 7 This is another data structure tree of participating parties provided in the embodiments of the present invention;

[0026] Figure 8 This is a tree structure for merging the data from both sides, provided in an embodiment of the present invention;

[0027] Figure 9 This is a schematic diagram of the structure of an information mining system based on horizontal federated learning provided in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0029] Figure 1This is a flowchart illustrating an information mining method based on horizontal federated learning, provided as an embodiment of the present invention. This embodiment is applicable to situations where feature association rules are mined for the same features from different participants. The method can be executed by an information mining system based on horizontal federated learning, which includes at least two participant node servers and at least two federated node servers. The participant node servers are used to process local sample data from each participant, while the federated node servers are used to perform global processing of multi-party data uploaded by each participant.

[0030] See Figure 1 The specific information mining method based on horizontal federated learning includes the following steps:

[0031] S110, The participating node server obtains at least two sets of sample data from the local participating party.

[0032] The sample data has already undergone feature alignment.

[0033] S120. The participating node server calculates the total number of local occurrences of the sample data and the number of times each feature appears locally in the sample data.

[0034] The total number of local occurrences describes the total number of sample data in a participant's local data. The number of local occurrences refers to the number of times a feature appears in a participant's local sample data.

[0035] S130. The participating node server selects a set of frequent local features from the features based on the total number of local features and the number of times the local features appear.

[0036] Among them, the local frequent feature set refers to local frequent features or combinations of local frequent features that occur more frequently than a set threshold.

[0037] S140. The participating node server randomly splits the occurrence information of the local frequent feature set and sends it to at least two federated node servers in the information mining system.

[0038] The occurrence information can be any form of information reflecting the frequent occurrence of local feature sets. Specifically, it can be the number of times it occurs.

[0039] S150: The federated node server, based on the received data, statistically analyzes the global frequent feature set of sample data from different participants to obtain the correlation between features in the global frequent feature set.

[0040] This invention, in its embodiments, randomly splits the occurrence information of the local frequent feature set and sends it to the federated node server. The federated node server then uses the received data to statistically analyze the global frequent feature set of sample data from different participants, thereby enabling the mining of inter-feature correlation information based on multi-participant sample data. Due to the large volume of sample data, the mined correlation information is more general and reliable.

[0041] Furthermore, because only randomly distributed information is sent without sending sample data to the federated node server, the data privacy and security of the participants are guaranteed.

[0042] By filtering local frequent feature sets and sending only the relevant information of these local frequent feature sets to the federated node server, the exposure probability of participant sample data is further reduced, while also improving data transmission and statistical efficiency.

[0043] Furthermore, after the federated node server calculates the global frequent feature set of sample data from different participants based on the received data, and obtains the correlation between features in the global frequent feature set, it also includes:

[0044] Participating node servers or federation node servers analyze user needs based on a global set of frequent features;

[0045] Based on the analysis results, participating node servers or federated node servers make personalized recommendations to users.

[0046] For example, if milk and diapers appear frequently together in the global frequent feature set, then diapers can be recommended to users who buy milk, or milk can be recommended to users who buy diapers.

[0047] Figure 2 This is a flowchart of another information mining method based on horizontal federated learning provided in an embodiment of the present invention. The information mining method can be described as follows:

[0048] S210, The participating node server obtains at least two sets of sample data from the local participating party.

[0049] S220, The participating node server calculates the total number of local occurrences of the sample data and the number of times each feature appears locally in the sample data.

[0050] S230. The participating node server calculates the ratio of the occurrence frequency of each feature or different feature combinations to the total number of sample data, and uses the ratio as the support of that feature or feature combination.

[0051] S240. The participating node server filters out features or feature combinations that are less than the preset minimum support threshold to obtain the local frequent feature set.

[0052] S250. The participating node server randomly splits the local occurrence count of the local frequent feature set and the total number of local samples into N numbers, where N is greater than or equal to 2, and sends the split data to N different federated node servers in the horizontal federated learning system.

[0053] S260: The federated node server calculates the total number of sample data from different participants and the global occurrence count of local frequent feature sets based on the received data.

[0054] Specifically, the federated node server, based on the received data, counts the total global number of sample data from different participants and the global occurrence count of local frequent feature sets, including:

[0055] The federated node server sums up the N random numbers received from the local total to obtain the global total number of sample data from different participants;

[0056] The federated node server sums up the N random numbers representing the number of occurrences of the received local frequent features to obtain the global occurrence count of the local frequent feature set.

[0057] S270. The federated node server determines the global frequent feature set from the local frequent feature set based on the global total number and the global occurrence count, so as to obtain the correlation between the features in the global frequent feature set.

[0058] Specifically, the federated node server determines the global frequent feature set from the local frequent feature set based on the global total number and the global occurrence count, including:

[0059] The federated node server calculates the support of the local frequent feature set in the total number of occurrences in the global total number of occurrences based on the total number of occurrences in the global total number of occurrences.

[0060] The federated node server determines the global frequent feature set from the local frequent feature set based on the calculated support.

[0061] For example, taking a sample as a product purchase order, a feature as product type, and based on the Apriori algorithm, where frequent feature sets correspond to frequent itemsets, the information mining method provided in this embodiment of the invention can be described as follows:

[0062] (1) Participant P i (i = 1, 2, 3... p, where p is the number of participants) Local statistics k-frequent itemsets (k = 1, 2, ..., n, where n is the number of product types), the local total number of sample data.

[0063] The sample data of the first participant is as follows:

[0064]

[0065]

[0066] From the above sample data, we can statistically determine that: the total number of local samples for the first participant is AI = 5, the number of times product 1 appears is A(1) = 4, and the support of product 1 is A(1) / AI, which is 4 / 5. Based on this, we can calculate that the support of product 2 is 4 / 5, the support of product 3 is 1, the support of product 4 is 2 / 5, and the support of product 5 is 3 / 5.

[0067] Assuming the minimum support is 0.5, after filtering out item 4 (with support less than 0.5), the locally frequent 1-itemsets are obtained as follows:

[0068] Product Item Set Support 1 4 / 5 2 4 / 5 3 1 5 3 / 5

[0069] The sample data for the second participant is as follows:

[0070] Order Number Purchase goods 1 1、2、3、4 2 2、3、4、5 3 1、2、3、6 4 1、2、4、5、6

[0071] From the above sample data, we can statistically determine that: the total number of local samples from the second participant is BI = 4, the number of times product 1 appears is B(1) = 3, and the support of product 1 is equal to B(1) / BI, which is 3 / 4. Based on this, we can calculate that the support of product 2 is 1, the support of product 3 is 3 / 4, the support of product 4 is 3 / 4, the support of product 5 is 1 / 2, and the support of product 6 is 1 / 2.

[0072] Assuming the minimum support is 0.5, filtering out items with support less than 0.5 yields the following locally frequent 1-itemsets:

[0073] Product Item Set Support 1 3 / 4 2 1 3 3 / 4 4 3 / 4 5 1 / 2 6 1 / 2

[0074] (2) The federated node server calculates the global k-frequent itemsets, the total number of product types, and the global total number of sample data through the N-PDZ protocol.

[0075] First participant: Divide AI and A(1) into N (assuming N = 3) parts each, see [reference] Figure 3 Taking AI as an example, AI is randomly split into three numbers: x1 = 1, x2 = 1, and x3 = 3. Then, x1, x2, and x3 are sent to three computing nodes: Node1, Node2, and Node3. AI = x1 + x2 + x3.

[0076] Second participant: Divide BI and B(1) into N (assuming N=3) parts each. Taking BI as an example, randomly divide BI into 3 numbers y1=0.5, y2=1.5, y3=2, and send y1, y2, y3 to the three computing nodes Node1, Node2, and Node3 of the federated node server. Wherein, BI=y1+y2+y3.

[0077] After AI and BI complete the data splitting, the servers at each federated node receive the data as follows: Figure 4 .

[0078] Summation on each federated node server:

[0079] Z1 = x1 + y1 = 1.5

[0080] Z² = x² + y² = 2.5

[0081] Z3 = x3 + y3 = 5

[0082] Summing Z1, Z2, and Z3 gives the total number of sample data N = AI + BI = Z1 + Z2 + Z3 = 9.

[0083] Based on the above calculation logic, the global occurrence count of frequent 1-itemsets can be calculated.

[0084] Calculate the support of frequent 1-itemsets based on the global occurrence count of frequent 1-itemsets and the total number of sample data; then filter the global frequent 1-itemsets based on the support.

[0085] (3) Participants filter out item combinations with support less than the minimum support based on the support of item k=1. If the itemset is empty, the result of the corresponding itemset k-1 is the final result. Otherwise, k=k+1.

[0086] (4) Repeat step (3) until all frequent itemsets are obtained.

[0087] By obtaining frequent itemsets, we can uncover the implicit relationships between different products.

[0088] The embodiments of the present invention are implemented based on the Apriori algorithm, which makes the information mining method of the present invention simple in logic and easy to implement.

[0089] Figure 5 This is a flowchart of another information mining method based on horizontal federated learning provided in an embodiment of the present invention. The information mining method can be described as follows:

[0090] S310, The participating node server obtains at least two sets of sample data from the local participating party.

[0091] S320. The participating node server calculates the total number of local occurrences of the sample data and the number of times each feature appears locally in the sample data.

[0092] S330. The participating node server filters out local frequent 1-itemsets from the features based on the total number of local items and the number of times the local items appear.

[0093] S340. The federated node server calculates the support of locally frequent 1-itemsets in the global sample data based on the received data, filters out locally frequent 1-itemsets with support less than the preset minimum support threshold, and obtains globally frequent 1-itemsets.

[0094] The S350 and federated node servers are based on the FP-Growth algorithm. They place globally frequent 1-itemsets into an item header table based on their support and sort them according to their support.

[0095] S360: The participating node server filters out infrequent 1-itemsets in the sample data based on the item header table, and sorts the frequent 1-itemsets in the sample data in descending order based on support.

[0096] S370, Participating node servers construct a local FP-Tree based on the sorted sample data.

[0097] S380 and the federated node server merge the local FP-Tree of each participant based on the N-PDZ protocol to construct a global FP-Tree, thus obtaining a global frequent feature set of tree structure.

[0098] For example, taking a sample as a product purchase order, a feature as product type, and implemented based on the FP-Growth algorithm, with frequent feature sets corresponding to frequent itemsets, the information mining method provided in this embodiment of the invention can be described as follows:

[0099] The sample data of the first participant is as follows:

[0100] Order Number Purchase goods 1 A, B 2 B, C, D 3 A, C, D, E 4 A, D, E 5 A, B, C

[0101] Setting the minimum support threshold to 0.5, the first participant calculates locally frequent 1-itemsets based on local sample data as follows:

[0102] Product Item Set Support A 4 / 5 B 3 / 5 C 3 / 5 D 3 / 5

[0103] Product E was filtered out because its support score was less than 0.5.

[0104] The sample data of the second participant is as follows:

[0105] Order Number Purchase goods 1 A, B, C, D 2 B, C 3 A, B, C 4 A, B, D 5 B, C, E

[0106] With a minimum support threshold set to 0.5, the second participant calculates locally frequent 1-itemsets based on local sample data as follows:

[0107] Product Item Set Support A 3 / 5 B 1 C 4 / 5

[0108] Product E and Product D were filtered out because their support was less than 0.5.

[0109] The first and second participants, based on the N-PDZ protocol, randomly send the local frequent 1-itemsets and the total number of sample data to the federated node server after splitting them.

[0110] The federated node server calculates the support of locally frequent 1-itemsets in the global sample data based on the received data, and the results are as follows:

[0111] Product Item Set Support A 7 / 10 B 8 / 10 C 7 / 10 D 3 / 10

[0112] The minimum support threshold is set to 0.5. The federated node server filters out items D with support less than the preset minimum support threshold, obtaining a global frequent feature set. Based on the FP-Growth algorithm, the global frequent feature set is placed into the item header table according to the support of frequent 1-itemsets, and sorted by support. The results are as follows:

[0113] commodity Number of occurrences B 8 A 7 C 7

[0114] The first participating section filters out infrequent 1-itemsets from the sample data based on the item header table, and sorts the frequent 1-itemsets in the sample data in descending order according to their support. The results are as follows:

[0115]

[0116]

[0117] See Figure 6 The first participant constructs a local FP-Tree based on the sorted sample data.

[0118] The second participating section filters out infrequent 1-itemsets from the sample data based on the item header table, and sorts the frequent 1-itemsets in the sample data in descending order according to their support. The results are as follows:

[0119] Order Number Purchase goods 1 B, A, C 2 B, C 3 B, A, C 4 B, A 5 B, C

[0120] See Figure 7 The first participant constructs a local FP-Tree based on the sorted sample data.

[0121] The federated node server merges the local FP-Trees of each participant based on the N-PDZ protocol to construct a global FP-Tree, obtaining a globally frequent feature set in a tree structure. See [link / reference]. Figure 8 .

[0122] By obtaining frequent itemsets, we can uncover the implicit relationships between products B, A, and C.

[0123] The embodiments of the present invention are implemented based on the FP-Growth algorithm, which enables the information mining method of the present invention to utilize a tree structure, directly obtain the frequent set without generating candidate frequent sets, greatly reduce the number of times sample data is scanned, and thus improve the computational efficiency of the information mining method.

[0124] Figure 9 This is a schematic diagram of the structure of an information mining system based on horizontal federated learning provided in an embodiment of the present invention. See also Figure 9 The information mining system based on horizontal federated learning provided in this embodiment of the invention includes at least M participating node servers and at least N federated node servers. The system applies any of the information mining methods described in this embodiment of the invention. Wherein, M is greater than or equal to 2, N is greater than or equal to 2, and M and N can be the same or different, depending on actual needs; this embodiment does not impose any limitation on them.

[0125] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. An information mining method based on horizontal federated learning, applied to an information mining system based on horizontal federated learning, wherein the system includes at least two participating node servers and at least two federated node servers, characterized in that, include: The participating node server obtains at least two sets of sample data from the local participating party; The participating node server calculates the total number of local occurrences of the sample data and the number of times each feature appears locally in the sample data; The participating node server filters out a set of frequently occurring local features from the features based on the total number of local features and the frequency of occurrence of the local features. The participating node server randomly splits the occurrence information of the local frequent feature set and sends it to at least two federated node servers in the information mining system. Only the occurrence information is randomly split and sent, without sending sample data, so as to ensure the data privacy and security of the participants. Based on the received data, the federated node server statistically analyzes the global frequent feature set of sample data from different participants to obtain the correlation between features in the global frequent feature set. The participating node server randomly splits the information of the local frequent feature set and sends it to at least two federated node servers in the information mining system, including: The participating node server randomly splits the local occurrence count of the local frequent feature set and the total number of local samples into N numbers, where N is greater than or equal to 2, and sends the split data to N different federated node servers in the horizontal federated learning system. In this way, only the relevant information of the local frequent feature set is sent to the federated node server, which further reduces the exposure probability of the participating sample data and also improves the data transmission and statistical efficiency. The federated node server, based on the received data, statistically analyzes the global frequent feature set of sample data from different participants to obtain the correlation relationships between features in the global frequent feature set, including: The federated node server sums up the N random numbers received from the local total to obtain the global total number of sample data from different participants; The federated node server sums up the N random numbers representing the number of occurrences of the local frequent features received, to obtain the global occurrence count of the local frequent feature set; The federated node server calculates the support of the local frequent feature set in the total number of occurrences in the global total number of occurrences based on the total number of occurrences in the global total number of occurrences. The federated node server determines the global frequent feature set from the local frequent feature set based on the calculated support. The federated node server, based on the received data, statistically analyzes the global frequent feature set of sample data from different participants. After obtaining the correlation relationships between features in the global frequent feature set, it also includes: Participating node servers or federation node servers analyze user needs based on a global set of frequent features; Based on the analysis results, participating node servers or federated node servers make personalized recommendations to users.

2. The method according to claim 1, characterized in that, The participating node server, based on the total number of local features and the frequency of local occurrences, filters out a set of frequently occurring local features from the features, including: The participating node server calculates the ratio of the occurrence frequency of each feature or different feature combinations to the total number of sample data, and uses the ratio as the support of that feature or feature combination; The participating node servers filter out features or feature combinations that are less than a preset minimum support threshold to obtain the local frequent feature set.

3. The method according to claim 1, characterized in that, The method further includes, after the federated node server calculates the global frequent feature set of sample data from different participants based on the received data, and obtains the correlation between features in the global frequent feature set, the method also includes: If the global frequent feature set is a frequent 1-itemset, the federated node server, based on the FP-Growth algorithm, puts the global frequent feature set into the item header table according to the support of the frequent 1-itemset, and sorts them according to the support. The participating node servers filter out infrequent 1-itemsets in the sample data based on the item header table, and sort the frequent 1-itemsets in the sample data in descending order based on support. Participating node servers construct local FP-Trees based on the sorted sample data; The federated node server merges the local FP-Trees of each participant based on the N-PDZ protocol to construct a global FP-Tree, thus obtaining a global frequent feature set of tree structure.

4. The method according to claim 3, characterized in that, The federated node server, based on the received data, statistically analyzes the global frequent feature set of sample data from different participants to obtain the correlation relationships between features in the global frequent feature set, including: The federated node server calculates the support of locally frequent 1-itemsets in the global sample data based on the received data. Locally frequent 1-itemsets with support less than a preset minimum support threshold are filtered out to obtain the global frequent feature set.

5. An information mining system based on horizontal federated learning, wherein the system includes at least two participant node servers and at least two federated node servers, characterized in that, The system employs any one of the information mining methods described in claims 1 to 4.

Images