Federated learning system, feature selection method and device in federated learning system

Abstract

The present application provides a federated learning system, a feature selection method and device in the federated learning system, the system includes at least a first participant and a second participant, the first participant holds a first feature of a sample and a sample label, and the second participant holds a second feature of the sample; the first participant is configured to: calculate the first distance between each sample in the joint sample set and the central sample based on the first feature, and obtain the second distance between each sample in the joint sample set and the central sample calculated by the second participant based on the second feature; update the first distance based on the second distance to obtain the comprehensive distance between each sample in the joint sample set and the central sample; screen out the first similar sample and the first heterogeneous sample based on the sample label and the comprehensive distance; update the weight of the first feature, and screen the first feature. The present application improves the accuracy of the first participant in feature selection of the first feature it holds.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically to a federated learning system, and a feature selection method and device in the federated learning system. Background Art

[0002] In the field of artificial intelligence, in order to effectively solve the problem of data islands, federated learning technology can be used, and multiple participants can work together to train machine learning models. During the training process of machine learning models, feature selection can be used to reduce the features contained in the samples to improve the training efficiency of machine learning models.

[0003] In the prior art, for feature selection in a federated learning system, the participants first determine the feature selection requirements based on offline communication, and each participant selects features for its own samples. This offline communication method for feature selection is limited by the limitations of its communication channels, which leads to insufficient data utilization in the federated learning system, resulting in a low match between feature selection in the federation process and actual feature requirements, and the accuracy of feature selection is difficult to meet the requirements. Summary of the invention

[0004] One purpose of the present application is to propose a federated learning system, a feature selection method and a device in the federated learning system. Under the premise of ensuring that the second feature held by the second participant is not disclosed to the first participant, the data source used by the first participant for feature selection is expanded from the first feature to a comprehensive feature including the second feature, thereby improving the first participant's utilization rate of the data in the federated learning system, thereby improving the first participant's accuracy in feature selection of the first feature it holds.

[0005] According to one aspect of an embodiment of the present application, a federated learning system is disclosed, the system comprising at least a first participant and a second participant, the first participant holds a first feature of a sample and a sample label of the sample, the second participant holds a second feature of the sample, the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R;

[0006] The first participant is configured as:

[0007] Calculate a first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain a second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature;

[0008] The first distance is updated based on the second distance to obtain a comprehensive distance between each sample in the joint sample set and the central sample R;

[0009] Based on the sample labels and the comprehensive distances, k samples with the closest comprehensive distances are selected from samples with the same label as the central sample R as first similar samples, and k samples with the closest comprehensive distances are respectively selected from samples with different labels from the central sample R as first heterogeneous samples, where k is a natural number greater than 0;

[0010] Based on the feature distance between the first similar sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, the weight of each first feature is updated, and the first features are screened according to the updated weights of each first feature.

[0011] In an exemplary embodiment of the present application, the first participant is configured as:

[0012] A random number seed is agreed upon in advance with the second participant, so that the first participant and the second participant perform sampling according to the random number seed each time and respectively select the joint sample set.

[0013] In an exemplary embodiment of the present application, the first participant is configured as:

[0014] A sample is randomly selected from the joint sample set as a central sample R, and an identifier of the central sample R is sent to the second participant, so that the second participant selects the central sample R according to the identifier.

[0015] In an exemplary embodiment of the present application, the first participant is configured as:

[0016] A vector distance between the first distance and the second distance is calculated, the vector distance is used as the updated first distance, and the updated first distance is used as the comprehensive distance between the corresponding sample and the central sample R.

[0017] In an exemplary embodiment of the present application, the first participant is configured as:

[0018] Screening out the first feature whose updated weight is greater than a preset weight threshold;

[0019] Alternatively, the first features are sorted in descending order of updated weights, and a preset number of first features with top rankings are obtained by screening.

[0020] According to one aspect of an embodiment of the present application, a federated learning system is disclosed, wherein the system includes at least a first participant and a second participant, wherein the first participant holds a first feature of a sample and a sample label of the sample, and the second participant holds a second feature of the sample, and the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively use the same sample in the joint sample set as a central sample R;

[0021] The second participant is configured as follows:

[0022] The first matrix generated by the first participant and the second matrix generated by the second participant are processed by means of privacy computing to obtain feature distances between the second similar samples and the central sample R on each second feature, and feature distances between the second heterogeneous samples and the central sample R on each second feature, wherein the first matrix is ​​used to describe the sample labels of each sample in the joint sample set, the second matrix is ​​used to describe the second feature of each sample in the joint sample set, the second similar samples are k samples with the same label as the central sample R and closest to the central sample R determined based on the second feature, and the second heterogeneous samples are k samples in each class of samples with different labels from the central sample R determined based on the second feature and closest to the central sample R;

[0023] Based on the feature distance between the second similar sample and the central sample R on each second feature, and the feature distance between the second heterogeneous sample and the central sample R on each second feature, the weight of each second feature is updated, and the second features are screened according to the updated weights of each second feature.

[0024] In an exemplary embodiment of the present application, the second participant is configured as:

[0025] If the second feature is a continuous value, the second feature is normalized, and the second matrix is ​​generated based on the normalized second feature.

[0026] In an exemplary embodiment of the present application, the system further includes a trusted third party, and the trusted third party is configured as:

[0027] receiving the first matrix from the first party and receiving the second matrix from the second party;

[0028] Based on the first matrix and the second matrix, calculate the feature distance between the second homogeneous sample and the central sample on each second feature, and the feature distance between the second heterogeneous sample and the central sample on each second feature;

[0029] The feature distances between the second homogeneous samples and the central sample on each second feature, and the feature distances between the second heterogeneous samples and the central sample on each second feature are sent to the second participant.

[0030] According to one aspect of an embodiment of the present application, a feature selection method in a federated learning system is disclosed. The system includes at least a first participant and a second participant. The first participant holds a first feature of a sample and a sample label of the sample, and the second participant holds a second feature of the sample. The first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R. The method is applied to the first participant, and the method includes:

[0031] Calculate a first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain a second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature;

[0032] The first distance is updated based on the second distance to obtain a comprehensive distance between each sample in the joint sample set and the central sample R;

[0033] Based on the sample labels and the comprehensive distances, k samples with the closest comprehensive distances are selected from samples with the same label as the central sample R as first similar samples, and k samples with the closest comprehensive distances are respectively selected from samples with different labels from the central sample R as first heterogeneous samples, where k is a natural number greater than 0;

[0034] Based on the feature distance between the first similar sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, the weight of each first feature is updated, and the first features are screened according to the updated weights of each first feature.

[0035] According to one aspect of an embodiment of the present application, a feature selection device in a federated learning system is disclosed, wherein the system includes at least a first participant and a second participant, wherein the first participant holds a first feature of a sample and a sample label of the sample, and the second participant holds a second feature of the sample, wherein the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R, wherein the device is provided at the first participant, and comprises:

[0036] a distance calculation module, configured to calculate a first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain a second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature;

[0037] a distance updating module, configured to update the first distance based on the second distance, to obtain a comprehensive distance between each sample in the joint sample set and the central sample R;

[0038] a sample screening module configured to screen out k samples with the closest comprehensive distance from samples with the same label as the central sample R as first similar samples, and to screen out k samples with the closest comprehensive distance from samples with different labels from the central sample R as first heterogeneous samples, based on the sample labels and the comprehensive distance, wherein k is a natural number greater than 0;

[0039] The first feature screening module is configured to update the weights of each first feature based on the feature distance between the first similar sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, and screen the first features according to the updated weights of each first feature.

[0040] According to one aspect of an embodiment of the present application, a feature selection method in a federated learning system is disclosed. The system includes at least a first participant and a second participant. The first participant holds a first feature of a sample and a sample label of the sample, and the second participant holds a second feature of the sample. The first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R. The method is applied to the second participant, and the method includes:

[0041] The first matrix generated by the first participant and the second matrix generated by the second participant are processed by means of privacy computing to obtain feature distances between the second similar samples and the central sample R on each second feature, and feature distances between the second heterogeneous samples and the central sample R on each second feature, wherein the first matrix is ​​used to describe the sample labels of each sample in the joint sample set, the second matrix is ​​used to describe the second feature of each sample in the joint sample set, the second similar samples are k samples with the same label as the central sample R and closest to the central sample R determined based on the second feature, and the second heterogeneous samples are k samples in each class of samples with different labels from the central sample R determined based on the second feature and closest to the central sample R;

[0042] Based on the feature distance between the second similar sample and the central sample R on each second feature, and the feature distance between the second heterogeneous sample and the central sample R on each second feature, the weight of each second feature is updated, and the second features are screened according to the updated weights of each second feature.

[0043] According to one aspect of an embodiment of the present application, a feature selection device in a federated learning system is disclosed, wherein the system includes at least a first participant and a second participant, wherein the first participant holds a first feature of a sample and a sample label of the sample, and the second participant holds a second feature of the sample, wherein the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R, wherein the device is disposed at the second participant, and the device includes:

[0044] a privacy computing module configured to process the first matrix generated by the first participant and the second matrix generated by the second participant by means of privacy computing to obtain the feature distances between the second similar samples and the central sample R on each second feature, and the feature distances between the second heterogeneous samples and the central sample R on each second feature, wherein the first matrix is ​​used to describe the sample labels of each sample in the joint sample set, the second matrix is ​​used to describe the second feature of each sample in the joint sample set, the second similar samples are k samples with the same label as the central sample R and closest to the central sample R determined based on the second feature, and the second heterogeneous samples are k samples in each class of samples with different labels from the central sample R determined based on the second feature and closest to the central sample R;

[0045] The second feature screening module is configured to update the weights of each second feature based on the feature distance between the second similar sample and the central sample R on each second feature, and the feature distance between the second heterogeneous sample and the central sample R on each second feature, and screen the second features according to the updated weights of each second feature.

[0046] According to one aspect of an embodiment of the present application, an electronic device is disclosed, comprising: a memory storing computer-readable instructions; and a processor reading the computer-readable instructions stored in the memory to execute any one of the methods described in the above claims.

[0047] According to one aspect of an embodiment of the present application, a computer program medium is disclosed, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the method described in any one of the above claims.

[0048] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various optional implementations.

[0049] In an embodiment of the present application, the first participant holding the sample label obtains the second distance calculated with the second feature as the data source from the second participant, and then updates the first distance calculated with the first feature as the data source based on the second distance, thereby obtaining the comprehensive distance calculated with the comprehensive feature as the data source. The comprehensive feature refers to a set consisting of the first feature and the second feature. In this process, the data source used by the first participant for feature selection is expanded from the first feature to the comprehensive feature, so the accuracy of the first participant's feature selection for the first feature it holds is improved. At the same time, the second feature of the second participant is not disclosed to the first participant, so the data security of the second participant is guaranteed. It can be seen from this that, under the premise of ensuring that the second feature held by the second participant is not disclosed to the first participant, the data source used by the first participant for feature selection is expanded from the first feature to the comprehensive feature containing the second feature, thereby improving the utilization rate of the first participant for data in the federated learning system, thereby improving the accuracy of the first participant's feature selection for the first feature it holds.

[0050] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0051] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other objects, features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0053] Figure 1 A flow chart of a feature selection method in a federated learning system according to an embodiment of the present application is shown.

[0054] Figure 2 A flow chart of a feature selection method in a federated learning system according to yet another embodiment of the present application is shown.

[0055] Figure 3 A schematic diagram of feature selection of each participant in a federated learning system according to an embodiment of the present application is shown.

[0056] Figure 4 A block diagram of a feature selection device in a federated learning system according to an embodiment of the present application is shown.

[0057] Figure 5 A block diagram of a feature selection device in a federated learning system according to yet another embodiment of the present application is shown.

[0058] Figure 6 A hardware diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present application will be more comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The accompanying drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0060] In addition, the described features, structures or characteristics may be combined in one or more example embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present application. However, those skilled in the art will appreciate that the technical solution of the present application may be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring the present application and making the various aspects of the present application obscure.

[0061] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0062] Figure 1 A flow chart of a feature selection method in a federated learning system in one embodiment of the present application is shown.

[0063] A federated learning system usually includes at least two participants. In order to distinguish the participants in this application, the federated system of this application includes a first participant and a second participant. The first participant holds the first feature of the sample and the sample label of the sample, and the second participant holds the second feature of the sample. In a complete feature selection process, there is generally only one first participant, and there can be multiple second participants.

[0064] In order to perform feature selection, n samples are selected as the joint sample set of the first participant and the second participant, and a sample R is selected from the joint sample set as the central sample. n is a natural number greater than 1. For example: the first participant and the second participant both select 10 samples with IDs 1 to 10 as the joint sample set, and both select the sample with ID 5 as the central sample R.

[0065] Figure 1 The method is applied to a first party, and the method includes:

[0066] Step S110: Calculate the first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain the second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature;

[0067] Step S120: updating the first distance based on the second distance to obtain the comprehensive distance between each sample in the joint sample set and the central sample R;

[0068] Step S130: Based on the sample labels and the comprehensive distances, k samples with the closest comprehensive distances are selected from the samples with the same sample label as the central sample R as the first similar samples, and k samples with the closest comprehensive distances are respectively selected from the samples with different sample labels from the central sample R as the first heterogeneous samples, where k is a natural number greater than 0;

[0069] Step S140: based on the feature distance between the first homogeneous sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, the weight of each first feature is updated, and the first features are screened according to the updated weights of each first feature.

[0070] In detail, in the embodiment of the present application, the joint sample set includes multiple feature dimension combinations, the first feature of the sample refers to the feature of the sample in the first feature dimension combination, the second feature of the sample refers to the feature of the sample in the second feature dimension combination, and the first feature dimension combination is usually different from the second feature dimension combination. For example: the first participant holds a sample in feature dimension x a1 、x a2 …x af The first participant has af first features X_A of the sample, denoted as X_A = (x a1 ,x a2 ,...,x af ); The second participant holds samples in feature dimension x b1 、x b2 …x bfThe second participant has bf second features X_B of the sample, denoted as X_B = (x b1 ,x b2 ,...,x bf ).

[0071] After selecting the central sample R, the first participant calculates the first distance between each sample in the joint sample set and the central sample R based on the first feature it holds, and the second participant calculates the second distance between each sample in the joint sample set and the central sample R based on the second feature it holds. The first distance describes the sample distance calculated using the first feature as the data source, and the second distance describes the sample distance calculated using the second feature as the data source.

[0072] The first participant obtains the second distance from the second participant, and then updates the first distance based on the second distance (for example, the vector distance between the first distance and the second distance is used as the updated first distance), and the updated first distance is used as the comprehensive distance between each sample in the joint sample set and the central sample R. Since the comprehensive distance is obtained by updating the first distance based on the second distance, and the second distance describes the sample distance calculated based on the second feature as the data source, the comprehensive distance is used to describe the sample distance calculated based on the comprehensive feature as the data source (the comprehensive feature here refers to the set of the first feature and the second feature).

[0073] After obtaining the comprehensive distances corresponding to each sample in the joint sample set, the first participant selects k samples with the same sample label as the central sample R and the closest comprehensive distance as the first similar samples based on the sample labels it holds, and selects k samples with the closest comprehensive distance from each type of samples with different sample labels from the central sample R as the first heterogeneous samples. Specifically, the first participant determines the samples with the same sample label as the central sample R as similar samples based on the sample labels it holds, and then sorts these similar samples in order of comprehensive distance from near to far, and selects the first k as the first similar samples; similarly, determines the samples with different sample labels from the central sample R as heterogeneous samples, and then sorts each heterogeneous sample separately in order of comprehensive distance from near to far, and selects the first k from each heterogeneous sample as the first heterogeneous samples. k is a natural number greater than 0. If there are F types of heterogeneous samples, the number of first heterogeneous samples screened is F*k.

[0074] After screening out the first homogeneous samples, the first participant determines the characteristic distance between the first homogeneous samples and the central sample R on each first feature, that is, the characteristic distance between the first homogeneous samples as a whole and the central sample R on each first feature. Similarly, after screening out the first heterogeneous samples, the first participant determines the characteristic distance between the first heterogeneous samples and the central sample R on each first feature.

[0075] Then, the first participant updates the weight of each first feature based on the feature distance between the first homogeneous sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature. The weight of the first feature is used to describe the strength of the category differentiation function played by the corresponding first feature when the corresponding first feature is used to distinguish the sample category. For a specific first feature, the larger its weight, the greater the similarity between the homogeneous sample and the central sample R on the first feature compared to the similarity between the heterogeneous sample and the central sample R on the first feature, indicating that the category differentiation function played by the first feature is stronger; conversely, for a specific first feature, the smaller its weight, the weaker the category differentiation function played by the first feature.

[0076] Then, the first participant divides all the first features into two parts according to the updated weights: the weight of any first feature in the first part is greater than or equal to the weight of any first feature in the second part. Then, the first features in the second part are eliminated, and the first features in the first part are retained to achieve feature selection. The information described by the sample after feature selection is approximately as effective as the information described by the sample before feature selection; at the same time, compared with the number of first features used in the sample description information before feature selection, the number of first features used in the sample description information after feature selection is less, so after feature selection of the sample, the efficiency of the first participant in training the machine learning model using the data it holds can be improved.

[0077] In an embodiment of the present application, the first participant holding the sample label obtains the second distance calculated with the second feature as the data source from the second participant, and then updates the first distance calculated with the first feature as the data source based on the second distance, thereby obtaining the comprehensive distance calculated with the comprehensive feature as the data source. The comprehensive feature refers to a set consisting of the first feature and the second feature. In this process, the data source used by the first participant for feature selection is expanded from the first feature to the comprehensive feature, so the accuracy of the first participant's feature selection for the first feature it holds is improved. At the same time, the second feature of the second participant is not disclosed to the first participant, so the data security of the second participant is guaranteed. It can be seen from this that, under the premise of ensuring that the second feature held by the second participant is not disclosed to the first participant, the data source used by the first participant for feature selection is expanded from the first feature to the comprehensive feature containing the second feature, thereby improving the utilization rate of the first participant for data in the federated learning system, thereby improving the accuracy of the first participant's feature selection for the first feature it holds.

[0078] In order to facilitate the demonstration of the specific implementation process of the embodiment of the present application, the first participant is A; the second participant is B; the first distance calculated based on the first feature is d1; the second distance calculated based on the second feature is d2; the comprehensive distance is d; the jth sample closest to R among the same type of samples is H j , j = 1, ..., k; the jth sample closest to R among all types of heterogeneous samples C is M j (C), j=1,...,k.

[0079] In one embodiment, the first participant and the second participant agree in advance on a random number seed, so that the first participant and the second participant perform sampling according to the random number seed each time and respectively select a joint sample set composed of the same samples.

[0080] In this embodiment, A and B pre-agreed on the same random number seed. A samples the sample set according to the random number seed to obtain a joint sample set containing n samples. B also samples the sample set according to the random number seed to obtain a joint sample set containing n samples. The n samples selected according to the same random number seed have the same sample ID column, so that both parties select a joint sample set composed of the same samples.

[0081] In one embodiment, the first participant and the second participant may also perform sampling by pre-agreed sampling point distribution (for example, starting from the same sample ID, sampling n times at the same sampling interval), thereby respectively selecting a joint sample set composed of the same samples.

[0082] In one embodiment, the first participant randomly selects a sample from the joint sample set as the central sample R, and sends the identifier of the central sample R to the second participant, so that the second participant selects the central sample R according to the identifier.

[0083] In one embodiment, the first participant calculates the vector distance between the first distance and the second distance, uses the vector distance as the updated first distance, and uses the updated first distance as the comprehensive distance between the corresponding sample and the center sample R.

[0084] In this embodiment, for each sample in the joint sample set, A determines the first distance d1 between the sample and R, and determines the second distance d2 between the sample and R, and then calculates the vector distance between d1 and d2. The vector distance is used as the updated first distance, and the updated first distance is used as the comprehensive distance d between the sample and R.

[0085] In one embodiment, the first participant updates the weight of the first feature by the following formula:

[0086]

[0087]

[0088] In this embodiment, f is the first feature, w f is the weight of the first feature, diff(f,R,H j ) is the feature distance between the first similar sample and R on the first feature f, diff(f,R,M j (C)) is the feature distance between the first heterogeneous sample in category C and R on the first feature f, p(C) is the proportion of category C in the total samples, and p(class(R)) is the proportion of the same type of samples in R in the total samples.

[0089] In one embodiment, the first participant screens and obtains a first feature whose updated weight is greater than a preset weight threshold.

[0090] In this embodiment, the preset weight threshold is σ. After the weight of the first feature stops updating, A removes the first features whose weight is less than or equal to σ, and retains the first features whose weight is greater than σ.

[0091] In one embodiment, the first participant sorts the first features in descending order of updated weights, and screens out a preset number of first features that are ranked high.

[0092] In this embodiment, the preset number is z. After the weight of the first feature stops updating, A sorts the first features in descending order of the updated weight, and then retains the first z first features and eliminates other first features.

[0093] In one embodiment, the weights of the first features are updated multiple times, and after the number of updates reaches a preset threshold, the first features are screened according to the latest weights.

[0094] Figure 2 A flow chart of a feature selection method in a federated learning system in another embodiment of the present application is shown. The method is applied to a second participant, and the method includes:

[0095] Step S210: Process the first matrix generated by the first participant and the second matrix generated by the second participant by means of privacy computing to obtain the characteristic distance between the second similar samples and the central sample R on each second feature, and the characteristic distance between the second heterogeneous samples and the central sample R on each second feature.

[0096] Among them, the first matrix is ​​used to describe the sample labels of each sample in the joint sample set, the second matrix is ​​used to describe the second feature of each sample in the joint sample set, the second similar samples are the k samples with the same label as the central sample R and closest to the central sample R determined based on the second feature, and the second heterogeneous samples are the k samples closest to the central sample R in each category of samples with different labels from the central sample R determined based on the second feature.

[0097] Step S220: based on the feature distance between the second similar samples and the central sample R on each second feature, and the feature distance between the second heterogeneous samples and the central sample R on each second feature, the weight of each second feature is updated, and the second features are screened according to the updated weights of each second feature.

[0098] In this embodiment, A generates a first matrix mat1 for describing the sample labels of each sample in the joint sample set according to the sample labels of the samples it holds. B generates a second matrix mat2 for describing the second features of each sample in the joint sample set according to the second features of the samples it holds.

[0099] Then the second participant processes the first matrix mat1 and the second matrix mat2 by means of privacy computing (for example, by means of multi-party secure computing or by means of a trusted execution environment) to obtain the diff(f,R,H j ) and diff(f,R,M j (C)). During this process, the sample labels of the samples held by A are not disclosed to Party B.

[0100] Then B updates the weight of the second feature by the following formula:

[0101]

[0102]

[0103] In this embodiment, w f is the weight of the second feature, diff(f,R,H j ) is the feature distance between the second similar sample and R on the second feature f, diff(f,R,M j (C)) is the feature distance between the second heterogeneous sample in category C and R on the second feature f, p(C) is the proportion of category C in the total samples, and p(class(R)) is the proportion of the same type of samples in R in the total samples.

[0104] It should be noted that, since the second participant does not hold the sample label, the second participant cannot distinguish the category of the sample on its own, and cannot determine the feature distance between the second sample of the same type and the central sample R on each second feature on its own, and cannot determine the feature distance between the second heterogeneous sample and the central sample R on each second feature on its own. Therefore, the second participant cannot perform feature selection on the second feature based on the above two types of feature distances on its own. Through this method of the present embodiment, the first participant does not need to disclose the sample label it holds to the second participant, and the second participant can also indirectly determine the feature distance between the second sample of the same type and the central sample R on each second feature based on the sample label held by the first participant, and determine the feature distance between the second heterogeneous sample and the central sample R on each second feature, thereby performing feature selection on the second feature it holds.

[0105] It can be seen that this embodiment enables the second participant, who originally could not independently perform feature selection on the second feature he holds because he does not hold the sample label, to perform feature selection on the second feature he holds.

[0106] In one embodiment, in a first matrix of size n*1, the value of R is 1, and the value of H j and M j The value of (C) is -1, and the others are 0.

[0107] In one embodiment, the second participant screens and obtains a second feature whose updated weight is greater than a preset weight threshold.

[0108] In this embodiment, the preset weight threshold is σ. After the weight of the second feature stops updating, B removes the second features whose weight is less than or equal to σ, and retains the second features whose weight is greater than σ.

[0109] In one embodiment, the second participant sorts the second features in descending order of updated weights, and screens out a preset number of second features with top rankings.

[0110] In this embodiment, the preset number is z. After the weight of the second feature stops updating, B sorts the second features in descending order of the updated weight, and then retains the first z second features and eliminates the other second features.

[0111] In one embodiment, if the second feature is a continuous value, the second participant normalizes the second feature and generates a second matrix based on the normalized second feature.

[0112] In this embodiment, if the second feature is a continuous value, B normalizes the second feature and then generates the second matrix mat2; if the second feature is a discrete value, B directly generates the second matrix mat2 according to the second feature.

[0113] In one embodiment, the system further includes a trusted third party as an intermediary. The trusted third party receives the first matrix from the first participant and receives the second matrix from the second participant. Based on the first matrix and the second matrix, the characteristic distance between the second similar sample and the central sample R on each second feature, and the characteristic distance between the second heterogeneous sample and the central sample R on each second feature are calculated. The characteristic distance between the second similar sample and the central sample R on each second feature, and the characteristic distance between the second heterogeneous sample and the central sample R on each second feature are sent to the second participant.

[0114] In this embodiment, through the secure multi-party computation (MPC), the trusted third party receives the first matrix mat1 from A and the second matrix mat2 from B, and obtains the diff (f, R, H) of B through matrix multiplication. j ) and diff(f,R,M j (C)), and then the diff(f,R,H j ) and diff(f,R,M j (C)) is provided to B so that B can screen the second feature it holds.

[0115] In one embodiment, a trusted execution environment (TEE) may be set up in the system to implement privacy computing for the processing of the first matrix and the second matrix through the trusted execution environment.

[0116] In one embodiment, the weights of the second features are updated multiple times, and after the number of updates reaches a preset threshold, the second features are screened according to the latest weights.

[0117] Various distances in the embodiments of the present application can be measured using Euclidean distance, cosine similarity, or Manhattan distance, which are not listed here one by one.

[0118] Figure 3 A schematic diagram of feature selection of each participant in a federated learning system according to an embodiment of the present application is shown.

[0119] In this embodiment, after each participant agrees on a random number seed in advance, they each take samples according to the random number seed, thereby selecting n samples as the joint sample set. A randomly selects a sample from the joint sample set as the central sample R, and sends the identifier of R to B. Thus, A calculates the first distance d1 between each sample in the joint sample set and R, and B calculates the second distance d2 between each sample in the joint sample set and R.

[0120] Then each participant screens the features they hold according to the algorithm process shown below.

[0121] 1. Let each feature weight w f =0;

[0122] 2.for i=1to m do;

[0123] 3. A randomly selects a central sample R and calculates the first distance d1; B calculates the second distance d2; A calculates the comprehensive distance d, and then finds the nearest k samples from the samples with the same label as R as the first same-class samples H, and finds the nearest k samples from each class of samples with different labels from R as the first different-class samples M.

[0124] 4.for f=1to F do

[0125] 5. A updates the weights of its features according to the determined H and M according to the following formula; B updates the diff(f,R,H) of B calculated by MPC j ) and diff(f,R,M j (C)) also updates the weights of the features it holds according to the following formula:

[0126]

[0127] Then each participant selects the first z features with the largest weights or features with weights greater than a preset weight threshold for model training.

[0128] Figure 4 A feature selection device in a federated learning system of an embodiment of the present application is shown. The system includes at least a first participant and a second participant. The first participant holds a first feature of a sample and a sample label of the sample, and the second participant holds a second feature of the sample. The first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R. The device is provided at the first participant, and the device includes:

[0129] The distance calculation module 310 is configured to calculate a first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain a second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature;

[0130] The distance updating module 320 is configured to update the first distance based on the second distance to obtain a comprehensive distance between each sample in the joint sample set and the central sample R;

[0131] The sample screening module 330 is configured to screen out k samples with the closest comprehensive distance from the samples with the same label as the central sample R as first similar samples, and screen out k samples with the closest comprehensive distance from each type of samples with different labels from the central sample R as first heterogeneous samples, based on the sample labels and the comprehensive distance, wherein k is a natural number greater than 0;

[0132] The first feature screening module 240 is configured to update the weights of each first feature based on the feature distance between the first similar sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, and screen the first features according to the updated weights of each first feature.

[0133] Figure 5 A feature selection device in a federated learning system of another embodiment of the present application is shown, wherein the system includes at least a first participant and a second participant, wherein the first participant holds a first feature of a sample and a sample label of the sample, and the second participant holds a second feature of the sample, wherein the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R, wherein the device is provided at the second participant, and comprises:

[0134] The privacy computing module 410 is configured to process the first matrix generated by the first participant and the second matrix generated by the second participant by means of privacy computing to obtain the feature distances between the second similar samples and the central sample R on each second feature, and the feature distances between the second heterogeneous samples and the central sample R on each second feature, wherein the first matrix is ​​used to describe the sample labels of each sample in the joint sample set, the second matrix is ​​used to describe the second feature of each sample in the joint sample set, the second similar samples are k samples with the same label as the central sample R and closest to the central sample R determined based on the second feature, and the second heterogeneous samples are k samples in each class of samples with different labels from the central sample R determined based on the second feature and closest to the central sample R;

[0135] The second feature screening module 420 is configured to update the weights of each second feature based on the feature distance between the second similar sample and the central sample R on each second feature, and the feature distance between the second heterogeneous sample and the central sample R on each second feature, and screen the second features according to the updated weights of each second feature.

[0136] Reference below Figure 6 An electronic device 50 according to an embodiment of the present application is described. Figure 6 The electronic device 50 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0137] like Figure 6 As shown, the electronic device 50 is in the form of a general computing device. The components of the electronic device 50 may include but are not limited to: at least one processing unit 510, at least one storage unit 520, and a bus 530 connecting different system components (including the storage unit 520 and the processing unit 510).

[0138] The storage unit stores program codes, which can be executed by the processing unit 510, so that the processing unit 510 performs the steps according to various exemplary embodiments of the present invention described in the description of the exemplary method described above in this specification. For example, the processing unit 510 can perform the following steps: Figure 1 or Figure 2 The steps shown in .

[0139] The storage unit 520 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 5201 and / or a cache storage unit 5202 , and may further include a read-only storage unit (ROM) 5203 .

[0140] The storage unit 520 may also include a program / utility 5204 having a set (at least one) of program modules 5205, such program modules 5205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0141] Bus 530 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0142] The electronic device 50 may also communicate with one or more external devices 600 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 50, and / or may communicate with any device that enables the electronic device 50 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 550. The input / output (I / O) interface 550 is connected to a display unit 540. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 560. As shown, the network adapter 560 communicates with other modules of the electronic device 50 via a bus 530. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0144] In an exemplary embodiment of the present application, a computer-readable storage medium is further provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the method described in the above method embodiment.

[0145] According to one embodiment of the present application, a program product for implementing the method in the above method embodiment is also provided, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, the readable storage medium can be any tangible medium containing or storing the program, and the program can be used by or in combination with the instruction execution system, device or device.

[0146] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0147] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0148] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0149] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as JAVA, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0150] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0151] In addition, although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in a specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0152] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0153] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

Claims

1. A federated learning system, characterized in that: The system includes at least a first participant and a second participant, the first participant holds a first feature of a sample and a sample label of the sample, the second participant holds a second feature of the sample, the first participant and the second participant respectively select a joint sample set consisting of the same samples, and respectively select a same sample from the joint sample set as a central sample R; The first participant is configured as follows: Calculate a first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain a second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature; The first distance is updated based on the second distance to obtain a comprehensive distance between each sample in the joint sample set and the central sample R; Based on the sample labels and the comprehensive distances, k samples with the closest comprehensive distances are selected from samples with the same label as the central sample R as first similar samples, and k samples with the closest comprehensive distances are respectively selected from samples with different labels from the central sample R as first heterogeneous samples, where k is a natural number greater than 0; Based on the feature distance between the first similar sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, the weight of each first feature is updated, and the first features are screened according to the updated weight.

2. The system according to claim 1, characterized in that The first participant is configured as: A random number seed is agreed upon in advance with the second participant, so that the first participant and the second participant perform sampling according to the random number seed each time and respectively select the joint sample set.

3. The system according to claim 1, characterized in that The first participant is configured as follows: A sample is randomly selected from the joint sample set as a central sample R, and an identifier of the central sample R is sent to the second participant, so that the second participant selects the central sample R according to the identifier.

4. The system according to claim 1, characterized in that The first participant is configured as: A vector distance between the first distance and the second distance is calculated, the vector distance is used as the updated first distance, and the updated first distance is used as the comprehensive distance between the corresponding sample and the central sample R.

5. The system according to any one of claims 1 to 4, characterized in that: The first participant is configured as follows: Screening out the first feature whose updated weight is greater than a preset weight threshold; Alternatively, the first features are sorted in descending order of updated weights, and a preset number of first features with top rankings are obtained by screening.

6. A federated learning system, characterized in that: The system includes at least a first participant and a second participant, the first participant holds a first feature of a sample and a sample label of the sample, the second participant holds a second feature of the sample, the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively use the same sample in the joint sample set as a central sample R; The second participant is configured as follows: The first matrix generated by the first participant and the second matrix generated by the second participant are processed by means of privacy computing to obtain feature distances between the second similar samples and the central sample R on each second feature, and feature distances between the second heterogeneous samples and the central sample R on each second feature, wherein the first matrix is ​​used to describe the sample labels of each sample in the joint sample set, the second matrix is ​​used to describe the second feature of each sample in the joint sample set, the second similar samples are k samples with the same label as the central sample R and closest to the central sample R determined based on the second feature, and the second heterogeneous samples are k samples in each class of samples with different labels from the central sample R determined based on the second feature and closest to the central sample R; Based on the feature distance between the second similar sample and the central sample R on each second feature, and the feature distance between the second heterogeneous sample and the central sample R on each second feature, the weight of each second feature is updated, and the second features are screened according to the updated weight.

7. The system according to claim 6, characterized in that The second participant is configured as follows: If the second feature is a continuous value, the second feature is normalized, and the second matrix is ​​generated based on the normalized second feature.

8. The system according to claim 6 or 7, characterized in that: The system further includes a trusted third party, wherein the trusted third party is configured as: receiving the first matrix from the first party and receiving the second matrix from the second party; Based on the first matrix and the second matrix, calculate the feature distance between the second homogeneous sample and the central sample on each second feature, and the feature distance between the second heterogeneous sample and the central sample on each second feature; The feature distances between the second homogeneous samples and the central sample on each second feature, and the feature distances between the second heterogeneous samples and the central sample on each second feature are sent to the second participant.

9. A feature selection method in a federated learning system, characterized in that: The system includes at least a first participant and a second participant, the first participant holds a first feature of a sample and a sample label of the sample, the second participant holds a second feature of the sample, the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select a same sample from the joint sample set as a central sample R, the method is applied to the first participant, and the method includes: Calculate a first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain a second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature; The first distance is updated based on the second distance to obtain a comprehensive distance between each sample in the joint sample set and the central sample R; Based on the sample labels and the comprehensive distances, k samples with the closest comprehensive distances are selected from samples with the same label as the central sample R as first similar samples, and k samples with the closest comprehensive distances are respectively selected from samples with different labels from the central sample R as first heterogeneous samples, where k is a natural number greater than 0; Based on the feature distance between the first similar sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, the weight of each first feature is updated, and the first features are screened according to the updated weight.

10. A feature selection device in a federated learning system, characterized in that: The system includes at least a first participant and a second participant, the first participant holds a first feature of a sample and a sample label of the sample, the second participant holds a second feature of the sample, the first participant and the second participant respectively select a joint sample set composed of the same samples, and respectively select the same sample from the joint sample set as a central sample R, the device is provided at the first participant, and the device includes: a distance calculation module, configured to calculate a first distance between each sample in the joint sample set and the central sample R based on the first feature, and obtain a second distance between each sample in the joint sample set and the central sample R calculated by the second participant based on the second feature; a distance updating module, configured to update the first distance based on the second distance, to obtain a comprehensive distance between each sample in the joint sample set and the central sample R; a sample screening module configured to screen out k samples with the closest comprehensive distance from samples with the same label as the central sample R as first similar samples, and to screen out k samples with the closest comprehensive distance from samples with different labels from the central sample R as first heterogeneous samples, based on the sample labels and the comprehensive distance, wherein k is a natural number greater than 0; The first feature screening module is configured to update the weight of each first feature based on the feature distance between the first similar sample and the central sample R on each first feature, and the feature distance between the first heterogeneous sample and the central sample R on each first feature, and screen the first features according to the updated weight.

Images