A federated downsampling method, system, device and storage medium

Abstract

The present invention discloses a federated downsampling method, system, device and storage medium, including: determining multiple candidate nearest neighbors of each piece of data in a data set to be sampled; encrypting the ID of each piece of data in the data set to be sampled and the IDs of multiple candidate nearest neighbors to obtain a plurality of encrypted IDs; constructing a graph topology structure using the obtained encrypted IDs; and downsampling the graph topology structure. The method, system, device and storage medium can effectively avoid the problem of data privacy leakage that occurs when performing federated downsampling.

Description

Technical Field

[0001] The present invention belongs to the field of data processing, and relates to a federated downsampling method, and in particular to a federated downsampling method, system, device and storage medium. Background Art

[0002] With the development of computer science, machine learning methods have been widely used in actual production and life. In these practical applications, data often contains private information from individual citizens. Therefore, in order to protect the privacy of user data, it is necessary to apply the method of vertical federated learning to build machine learning models. However, in actual production, data often faces serious imbalance problems. Whether it is the traditional sampling-based solution or the popular data augmentation and cleaning method based on deep learning, when solving the data imbalance problem, the privacy information in the data will be leaked. Therefore, these methods cannot be applied in the scenario of vertical federated learning.

[0003] Existing non-random downsampling methods all require the use of distance information between data. However, the distance between data samples itself is also one of the privacy information of the data. Considering the need for data privacy protection, this information cannot be sent between different data owners, so existing methods cannot be applied.

[0004] In summary, existing technologies face the problem of data privacy leakage caused by calculating the distance between data samples when performing federated downsampling, thereby affecting the security of the data. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a federal downsampling method, system, device and storage medium, which can effectively avoid the problem of data privacy leakage when performing federal downsampling.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] In one aspect, the present invention provides a federated downsampling method, comprising:

[0008] Determine multiple candidate nearest neighbors for each piece of data in the data set to be sampled;

[0009] The ID of each piece of data in the sampled data set and the IDs of its multiple candidate nearest neighbors are encrypted to obtain several encrypted IDs;

[0010] Use the obtained encrypted IDs to construct a graph topology structure;

[0011] Downsample the graph topology.

[0012] The federated downsampling method of the present invention is further improved in that:

[0013] The specific process of determining multiple candidate nearest neighbors for each piece of data in the to-be-sampled data set is as follows:

[0014] Use the data owned by each participant in federated learning to construct a data set to be sampled, use the data xi in the data set to be sampled as the central data node, and randomly select several candidate data from the data set to be sampled;

[0015] Calculate the distance between data xi and each candidate data;

[0016] The candidate data with the closest distance is taken as the jth candidate nearest neighbor of data xi, and multiple candidate nearest neighbors of data xi are obtained.

[0017] The specific process of constructing the graph topology using each encrypted ID is as follows:

[0018] Determine whether the data represented by each encrypted ID appears in the data owned by each participant in federated learning;

[0019] When the data represented by any encrypted ID appears in the data owned by all participants in federated learning, the encrypted ID is added to the graph topology.

[0020] Also includes:

[0021] When any candidate nearest neighbor nij of data xi appears in the data owned by all participants in federated learning, the node corresponding to the data xi is connected to the node corresponding to the candidate nearest neighbor nij that appears in the data owned by all participants in federated learning in the graph topology.

[0022] The specific process of downsampling the graph topology is as follows:

[0023] In the graph topology, for each encrypted ID, determine whether the data represented by each encrypted ID is a minority class sample;

[0024] When there are minority class samples in the data represented by the encrypted ID, the nodes whose corresponding data are majority class samples in the neighbors of the encrypted ID of the minority class samples are deleted, and the data corresponding to the remaining encrypted IDs in the graph topology are used as the downsampling result.

[0025] In a second aspect of the present invention, the present invention provides a federated downsampling system, comprising:

[0026] A determination module, used to determine multiple candidate nearest neighbors for each piece of data in the data set to be sampled;

[0027] An encryption module is used to encrypt the ID of each piece of data in the sampled data set and the IDs of multiple candidate nearest neighbors to obtain a number of encrypted IDs;

[0028] A construction module, used for constructing a graph topology structure using each encrypted ID obtained;

[0029] Sampling module, used to downsample the graph topology.

[0030] The modules are determined to include:

[0031] The random selection module is used to construct a data set to be sampled using the data owned by each participant in the federated learning, taking the data xi in the data set to be sampled as the central data node, and randomly selecting several candidate data from the data set to be sampled;

[0032] A calculation module is used to calculate the distance between the data xi and each candidate data;

[0033] The setting module is used to take the candidate data with the closest distance as the jth candidate nearest neighbor of the data xi, and obtain multiple candidate nearest neighbors of the data xi.

[0034] In a third aspect of the present invention, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the federal downsampling method when executing the computer program.

[0035] In a fourth aspect of the present invention, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the federal downsampling method are implemented.

[0036] The present invention has the following beneficial effects:

[0037] The federated downsampling method, system, device and storage medium described in the present invention, in specific operation, based on the characteristic that not all distance information is required during federated downsampling, constructs a graph topology structure based on the candidate nearest neighbors of the data. Specifically, the ID of each data in the sampled data set and the IDs of multiple candidate nearest neighbors are encrypted to obtain several encrypted IDs, and then a graph topology structure is constructed based on the obtained encrypted IDs. Finally, downsampling is performed using the constructed graph topology structure to avoid calculating the distance between data samples during the downsampling process, thereby avoiding the problem of data privacy leakage, improving data security, being simple to operate, and being highly practical. It can be widely used in binary classification problems with unbalanced samples, expanding the application scenarios of federated learning, and being able to empower more industries that need to apply federated learning.

[0038] Furthermore, each participant in the federated learning determines K candidate nearest neighbors for each data in the data set to be sampled, and obtains K candidate nearest neighbors for each data in the data set to be sampled, that is, the graph topology structure is constructed in combination with each participant in the federated learning, thereby avoiding the problem of poor downsampling effect caused by relying on a single participant to construct the graph topology structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0040] Figure 1 It is the principle diagram of the present invention;

[0041] Figure 2 is a flow chart of the present invention;

[0042] Figure 3 It is a structural schematic diagram of the present invention.

[0043] Among them, 1 is the determination module, 2 is the encryption module, 3 is the construction module, and 4 is the sampling module. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0045] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0047] As we all know, Federated machine learning (Federated Learning), also known as federated learning, joint learning, and alliance learning, is a machine learning framework that can effectively help multiple institutions use data and perform machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations.

[0048] For example, suppose there are two different companies A and B (participants in federated learning), and they have different data. For example, company A has user feature data, and company B has product feature data and labeled data. According to the GDPR guidelines mentioned above, companies A and B cannot roughly merge the data of both parties, because the original providers of the data, that is, their respective users, may not agree to do so. Assuming that both parties establish a task model, each task can be classification or prediction, and these tasks have been recognized by their respective users when obtaining data, then the question is how to establish high-quality models at each end of company A and B. Due to incomplete data (for example, company A lacks label data, and company B lacks user feature data), or insufficient data, the model at each end may not be established or the effect is not ideal. Federated learning is to solve this problem: it hopes that the data of each company will not leave the local area, and then the federated system can establish a virtual shared model through parameter exchange under the encryption mechanism, that is, without violating data privacy regulations. This virtual model is like the optimal model established by everyone aggregating data together. However, when establishing the virtual model, the data of each company itself is not moved, and privacy is not leaked or data compliance is affected. In this way, the constructed models only serve local goals in their respective regions.

[0049] Downsampling, also known as down sampling, is a multi-rate digital signal processing technique or the process of reducing the signal sampling rate, usually used to reduce the data transmission rate or data size. Complementary to interpolation, interpolation is used to increase the sampling frequency, and filters are used in the process of downsampling to reduce the distortion caused by aliasing. Because downsampling will cause aliasing, the part of the system that has the downsampling function is called a downconverter.

[0050] Embodiment 1

[0051] See also Figure 1 and Figure 2In this embodiment, a federated downsampling method based on nearest neighbor estimation is provided to overcome the problem of data privacy information leakage caused by calculating the distance between data samples during federated downsampling in the prior art, so as to ensure data privacy during federated downsampling and improve data security. It should be noted that the present invention can also effectively solve the problem of sample imbalance in practical applications of federated learning, as well as the problem of poor sampling processing effect caused by using only one party's data in the prior art. To this end, a method for estimating the nearest neighbor without leaking distance information and performing downsampling based on this is provided.

[0052] The federated downsampling method of the present invention comprises:

[0053] 1) Determine multiple candidate nearest neighbors for each piece of data in the data set to be sampled;

[0054] Specifically, each participant in the federated learning determines K candidate nearest neighbors for each piece of data in the data set to be sampled, and obtains K candidate nearest neighbors for each piece of data in the data set to be sampled. It should be noted that in the present invention, when obtaining the K candidate nearest neighbors for each piece of data, repeated random sampling is adopted. A participant in the federated learning determines a candidate nearest neighbor for a piece of data once to improve the randomness of the data and ensure the security of the data. For example, the participants in the federated learning include enterprise A and enterprise B, then enterprise A and enterprise B both need to determine K candidate nearest neighbors for each piece of data.

[0055] In addition, it should be noted that the specific process for any participant in federated learning to determine the jth candidate nearest neighbor for the data xi in the to-be-sampled data set is:

[0056] The federated learning method is used to take the data xi in the data set to be sampled as the central data node, randomly select several candidate data from the data set to be sampled, calculate the distance between data xi and each candidate data, and take the candidate data with the closest distance as the jth candidate nearest neighbor of data xi.

[0057] For example, in federated learning, participant a randomly selects several candidate data from the federated learning sampled data set for data xi, calculates the distance between the data xi and the candidate data, and takes the candidate data with the closest distance as the candidate nearest neighbor of the data xi. The above process is repeated K-1 times to obtain K candidate nearest neighbors ni0, ni1, … niK of the data xi.

[0058] 2) Encrypt the ID of each piece of data in the sampled data set and the IDs of multiple candidate nearest neighbors to obtain several encrypted IDs;

[0059] It should be noted that commonly used encryption methods include the DES algorithm (Data Encryption Standard), the 3DES algorithm (Triple DES), the RC2 and RC4 algorithms, the IDEA algorithm (International Data Encryption Algorithm) and the RSA algorithm.

[0060] In actual operation, the encrypted ID needs to be sent to the third party of federated learning. At the same time, the party with the data label in federated learning sends the encrypted ID and its corresponding label to the third party of federated learning. This method can ensure the security of data processing.

[0061] 3) Using the obtained encrypted IDs to construct a graph topology structure;

[0062] Specifically, the specific process of constructing the graph topology structure using each encrypted ID is as follows:

[0063] Determine whether the data represented by each encrypted ID appears in the data owned by each participant in the federated learning. When the data represented by any encrypted ID appears in the data owned by all participants in the federated learning, the encrypted ID is added to the graph topology structure. For example, when the candidate nearest neighbor nij among multiple candidate nearest neighbors ni0, ni1, ... niK of the data xi appears in the data owned by all participants in the federated learning, the encrypted ID of the candidate nearest neighbor nij is added to the graph topology structure.

[0064] In addition, for the graph topology structure, when any candidate nearest neighbor nij of data xi appears in the data owned by all participants in federated learning, then in the graph topology structure, the node corresponding to the data xi is connected to the node corresponding to the candidate nearest neighbor nij that appears in the data owned by all participants in federated learning. The final form of the graph topology structure can be constructed through the nodes and the edges formed by the connections.

[0065] In actual operation, a trusted third party of federated learning can use the obtained encrypted IDs to build a graph topology structure to improve the security of data processing while reducing the computational complexity of the system.

[0066] It should be noted that the present invention constructs a graph topology structure based on the candidate nearest neighbors of each data in federated learning, solves the problem of data privacy leakage caused by calculating the distance between samples, and completes the construction of the sample neighbor graph topology structure under the condition of protecting data privacy from leakage.

[0067] At the same time, it should be noted that the present invention combines the graph topology information of different federated learning participants. Compared with using only one party's data for processing, it can better model the distance topological relationship between different samples and effectively improve the effect of downsampling processing.

[0068] 4) Downsample the graph topology.

[0069] Specifically, the specific process of downsampling the graph topology is as follows:

[0070] In the graph topology, for each encrypted ID, determine whether the data represented by each encrypted ID is a minority class sample. When the data represented by any encrypted ID is a minority class sample, delete the nodes whose corresponding data are majority class samples in the neighbors of the encrypted ID, and use the data corresponding to the remaining encrypted IDs in the graph topology as the downsampling result.

[0071] For example, when the candidate nearest neighbor nik is a minority class sample, and the data corresponding to the neighbor nodes of the encrypted ID of the candidate nearest neighbor nik include candidate nearest neighbor nig, candidate nearest neighbor nij, candidate nearest neighbor nil, candidate nearest neighbor nir and candidate nearest neighbor nib, when the candidate nearest neighbor nil and the candidate nearest neighbor nib are majority class samples, the candidate nearest neighbors nig, candidate nearest neighbors nij and candidate nearest neighbors nir are minority class samples, the nodes corresponding to the candidate nearest neighbors nil and the candidate nearest neighbor nib in the graph topology structure are deleted, and the nodes corresponding to the candidate nearest neighbors nig, candidate nearest neighbors nij and candidate nearest neighbors nir in the graph topology structure are retained. After traversing each node in the graph topology structure, the encrypted IDs corresponding to the remaining nodes in the graph topology structure are sent to each participant in the federated learning.

[0072] It should be noted that the problem of sample imbalance often occurs in the actual application of federated learning. The present invention provides an effective solution for applying federated learning to the binary classification problem of sample imbalance, expands the application scenarios of federated learning, and empowers more industries that need to apply federated learning.

[0073] Embodiment 2

[0074] refer to Figure 3 The present invention provides a federated downsampling system, comprising:

[0075] Determination module 1, used to determine multiple candidate nearest neighbors of each data in the data set to be sampled;

[0076] Encryption module 2, used for encrypting the ID of each data in the sampled data set and the IDs of multiple candidate nearest neighbors to obtain a number of encrypted IDs;

[0077] A construction module 3 is used to construct a graph topology structure using each encrypted ID obtained;

[0078] The sampling module 4 is used to downsample the graph topology structure.

[0079] Determine module 1 includes:

[0080] The random selection module is used to use the federated learning method to take the data xi in the to-be-sampled data set as the central data node and randomly select several candidate data from the to-be-sampled data set;

[0081] A calculation module is used to calculate the distance between the data xi and each candidate data;

[0082] The setting module is used to take the candidate data with the closest distance as the jth candidate nearest neighbor of the data xi, and obtain multiple candidate nearest neighbors of the data xi.

[0083] Embodiment 3

[0084] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the federated downsampling method are implemented. The memory may include a memory, such as a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk memory, etc. The processor, the network interface, and the memory are interconnected through an internal bus. The internal bus may be an industrial standard architecture bus, a peripheral component interconnection standard bus, an extended industrial standard structure bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory is used to store programs. Specifically, the program may include a program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0085] Embodiment 4

[0086] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the federated downsampling method are implemented. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0087] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0088] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0089] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0090] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A federated downsampling method, characterized in that: include: Determine multiple candidate nearest neighbors for each piece of data in the data set to be sampled; The ID of each piece of data in the sampled data set and the IDs of its multiple candidate nearest neighbors are encrypted to obtain several encrypted IDs; Use the obtained encrypted IDs to construct a graph topology structure; Downsample the graph topology; The specific process of determining multiple candidate nearest neighbors for each piece of data in the to-be-sampled data set is as follows: Use the data owned by each participant in federated learning to construct a data set to be sampled, use the data xi in the data set to be sampled as the central data node, and randomly select several candidate data from the data set to be sampled; Calculate the distance between data xi and each candidate data; The candidate data with the closest distance is taken as the jth candidate nearest neighbor of data xi, and multiple candidate nearest neighbors of data xi are obtained; The specific process of constructing the graph topology using each encrypted ID is as follows: Determine whether the data represented by each encrypted ID appears in the data owned by each participant in federated learning; When the data represented by any encrypted ID appears in the data owned by all participants in federated learning, the encrypted ID is added to the graph topology.

2. The federated downsampling method according to claim 1, characterized in that: Also includes: When the candidate nearest neighbor nij of data xi appears in the data owned by all participants in federated learning, the node corresponding to the data xi is connected to the node corresponding to the candidate nearest neighbor nij that appears in the data owned by all participants in federated learning in the graph topology.

3. The federated downsampling method according to claim 1, characterized in that: The specific process of downsampling the graph topology is as follows: In the graph topology, for each encrypted ID, determine whether the data represented by each encrypted ID is a minority class sample; When there are minority class samples in the data represented by the encrypted ID, the nodes whose corresponding data are majority class samples in the neighbors of the encrypted ID of the minority class samples are deleted, and the data corresponding to the remaining encrypted IDs in the graph topology are used as the downsampling result.

4. A federated downsampling system, characterized in that: include: A determination module (1) is used to determine multiple candidate nearest neighbors of each piece of data in the data set to be sampled; An encryption module (2) is used to encrypt the ID of each piece of data in the sampled data set and the IDs of multiple candidate nearest neighbors to obtain a plurality of encrypted IDs; A construction module (3) is used to construct a graph topology structure using the obtained encrypted IDs; A sampling module (4), used for downsampling the graph topology; The determination module (1) comprises: The random selection module is used to construct a data set to be sampled using the data owned by each participant in the federated learning, taking the data xi in the data set to be sampled as the central data node, and randomly selecting several candidate data from the data set to be sampled; A calculation module is used to calculate the distance between the data xi and each candidate data; A setting module is used to take the candidate data with the closest distance as the jth candidate nearest neighbor of the data xi, and obtain multiple candidate nearest neighbors of the data xi; The specific process of constructing the graph topology using each encrypted ID is as follows: Determine whether the data represented by each encrypted ID appears in the data owned by each participant in federated learning; When the data represented by any encrypted ID appears in the data owned by all participants in federated learning, the encrypted ID is added to the graph topology.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the federated downsampling method according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the federated downsampling method according to any one of claims 1 to 3 are implemented.

Images