A method, apparatus, and electronic device for determining feature contribution in federated learning.

Abstract

The embodiment of the present invention provides a method, device and electronic device for determining the contribution of features in federated learning. The method includes: obtaining each first feature value included in a first training sample locally used to train a federated learning model and the weight corresponding to each first feature, calculating the Shapley value corresponding to the first feature based on each first feature value and its corresponding weight as the contribution corresponding to the first feature, and then obtaining the Shapley value corresponding to each second feature included in a second training sample corresponding to a second device as the contribution corresponding to the second feature, and determining the importance of each feature according to the contribution corresponding to the first feature and the contribution corresponding to the second feature. Since the Shapley value can ensure the fairness of distribution, applying the Shapley value to the federated learning model can more accurately determine the contribution of each feature included in the training sample to the model, thereby making the output result of the model more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a method, device and electronic device for determining feature contribution in federated learning. Background Art

[0002] In the generalized linear federated learning model, the model initiator needs to evaluate the contribution of each feature in the training sample to the model after the model training is completed. According to the contribution of each feature to the model, the importance of each input information during the use of the model can be judged. In other words, the contribution of each feature to the model can guide the judgment of the importance of input information during the use of the model, thereby enhancing the interpretability of the model.

[0003] For example, the risk control model trained by the bank in cooperation with the operator institution is initiated by the bank. After the risk control model training is completed, the bank can evaluate the user's corresponding online time, call package amount, call duration in the past 6 months, number of calls in the past month and other features. The evaluated importance levels are: online time: high; call package amount: low; call duration in the past 6 months: high; number of calls in the past month: medium, indicating that the online time has a greater impact on the output results of the model, while the call package amount has a smaller impact on the output results of the model. Then, during the use of the model, the changes in the model effect can be monitored according to the importance of each piece of information. For example, if the online time of the bank user group and the number of calls in the past month change significantly, the model effect is very likely to deteriorate, and the iterative model needs to be updated again.

[0004] It can be seen that it is very important to evaluate the contribution of each feature to the model. Therefore, there is an urgent need for a method that can accurately determine the contribution of each feature in the federated learning model. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method, device and electronic device for determining the contribution of features in federated learning, so as to accurately determine the contribution of each feature in the federated learning model and achieve the purpose of making the output result of the model more accurate. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for determining feature contribution in federated learning, which is applied to a first device in a federated learning model training system, the system also includes a second device, and the method includes:

[0007] After the training of the federated learning model is completed, obtaining each first feature included in the first training sample used to train the federated learning model locally and the weight corresponding to each first feature;

[0008] Based on each first feature and its corresponding weight, calculating the Shapley value corresponding to the first feature as the contribution degree corresponding to the first feature;

[0009] Obtaining a Shapley value corresponding to each second feature included in a second training sample corresponding to the second device as a contribution degree corresponding to the second feature;

[0010] The importance of each feature is determined according to the contribution corresponding to the first feature and the contribution corresponding to the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model.

[0011] Optionally, the step of obtaining the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device includes:

[0012] Obtaining each second feature and the weight corresponding to each second feature included in the second training sample corresponding to the second device for training the federated learning model, and calculating the Shapley value corresponding to the second feature based on each second feature and its corresponding weight; or,

[0013] Receive the Shapley values ​​corresponding to each second feature included in the second training sample for training the federated learning model sent by the second device, wherein the Shapley values ​​corresponding to the second feature are calculated by the second device based on each second feature and its corresponding weight.

[0014] Optionally, the step of determining the importance of each feature according to the contribution of the first feature and the contribution of the second feature includes:

[0015] Sorting the contribution degree corresponding to the first feature and the contribution degree corresponding to the second feature to obtain a sorting result;

[0016] The importance of each feature is determined based on the ranking result.

[0017] Optionally, the step of calculating the Shapley value corresponding to the first feature based on each first feature and its corresponding weight includes:

[0018] Calculating an average value of each of the first features;

[0019] Based on the average value, according to the formula Decentralize each of the first features to obtain a processed first feature, where Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, is the average value corresponding to the i-th first feature;

[0020] Based on the processed first feature and its corresponding weight, a Shapley value corresponding to the first feature is calculated.

[0021] Optionally, the step of calculating the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight includes:

[0022] Based on the processed first feature and its corresponding weight, the Shapley value corresponding to the first feature is calculated according to the following formula:

[0023]

[0024] in, is the Shapley value of the i-th first feature, β i is the weight corresponding to the i-th first feature.

[0025] In a second aspect, an embodiment of the present invention provides a device for determining feature contribution in federated learning, which is applied to a first device in a federated learning model training system, the system also includes a second device, and the device includes:

[0026] An acquisition module, used to acquire, after the training of the federated learning model is completed, each first feature included in the first training sample used to train the federated learning model locally and the weight corresponding to each first feature;

[0027] A first calculation module, configured to calculate, based on each first feature and its corresponding weight, a Shapley value corresponding to the first feature as a contribution degree corresponding to the first feature;

[0028] A second calculation module, used to obtain a Shapley value corresponding to each second feature included in a second training sample corresponding to the second device as a contribution degree corresponding to the second feature;

[0029] A determination module is used to determine the importance corresponding to each feature according to the contribution corresponding to the first feature and the contribution corresponding to the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model.

[0030] Optionally, the second calculation module includes:

[0031] a first calculation unit, configured to obtain each second feature included in a second training sample corresponding to the second device for training the federated learning model and a weight corresponding to each second feature, and calculate a Shapley value corresponding to the second feature based on each second feature and its corresponding weight; or

[0032] A second computing unit is used to receive the Shapley values ​​corresponding to each second feature included in the second training sample sent by the second device for training the federated learning model, wherein the Shapley value corresponding to the second feature is calculated by the second device based on each second feature and its corresponding weight.

[0033] Optionally, the first calculation module includes:

[0034] an average value calculation unit, used to calculate the average value of each first feature;

[0035] A central calculation unit is used for calculating the average value according to the formula Decentralize each of the first features to obtain a processed first feature, where Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, is the average value corresponding to the i-th first feature;

[0036] A Shapley value calculation unit is used to calculate the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight.

[0037] Optionally, the Shapley value calculation unit includes:

[0038] The Shapley value calculation subunit is used to calculate the Shapley value corresponding to the first feature according to the following formula based on the processed first feature and its corresponding weight:

[0039]

[0040] in, is the Shapley value of the i-th first feature, β i is the weight corresponding to the i-th first feature.

[0041] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0042] Memory, used to store computer programs;

[0043] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0044] In the scheme provided by the embodiment of the present invention, after the training of the federated learning model is completed, the first device in the federated learning model training system obtains the first eigenvalues ​​and the weights corresponding to each first feature included in the first training sample used to train the federated learning model locally, and calculates the Shapley value corresponding to the first feature based on each first eigenvalue and its corresponding weight as the contribution corresponding to the first feature, and then obtains the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device as the contribution corresponding to the second feature, and then determines the importance corresponding to each feature according to the contribution corresponding to the first feature and the contribution corresponding to the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model. Through the above scheme, since the Shapley value can accurately identify the contribution of the participant to the alliance, that is, the Shapley value can ensure the fairness of the distribution, so by applying the Shapley value to the federated learning model, the contribution of each feature included in the training sample to the model can be more accurately determined, and then the importance corresponding to each feature is determined by the contribution, and the importance is used to guide whether the model needs to be updated and iterated, so as to make the output result of the model more accurate. Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0046] Figure 1 A flowchart of a method for determining feature contribution in federated learning provided by an embodiment of the present invention;

[0047] Figure 2 Based on Figure 1 A specific flow chart of step S104 in the illustrated embodiment;

[0048] Figure 3 Based on Figure 1 A specific flow chart of step S102 in the illustrated embodiment;

[0049] Figure 4 A schematic diagram of the structure of a device for determining feature contribution in federated learning provided by an embodiment of the present invention;

[0050] Figure 5 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on the present invention belong to the scope of protection of the present invention.

[0052] In order to accurately determine the contribution of each feature in the federated learning model and achieve the purpose of making the output effect of the model more accurate, the embodiments of the present invention provide a method, device, electronic device, computer-readable storage medium and computer program product for determining the contribution of features in federated learning. The following first introduces a method for determining the contribution of features in federated learning provided by an embodiment of the present invention.

[0053] The method for determining the contribution of features in federated learning provided by the embodiment of the present invention can be applied to a first device in a federated learning model training system, wherein the first device can be: a server, a processing chip, a computer, etc., which is not specifically limited here. The federated learning model training system also includes a second device, and the first device can be combined with the second device to perform federated learning model training.

[0054] like Figure 1 As shown, a method for determining feature contribution in federated learning is applied to a first device in a federated learning model training system, the system also includes a second device, and the method includes:

[0055] S101, after the training of the federated learning model is completed, obtaining each first feature included in a first training sample locally used for training the federated learning model and a weight corresponding to each first feature;

[0056] S102, based on each first feature and its corresponding weight, calculating a Shapley value corresponding to the first feature as a contribution degree corresponding to the first feature;

[0057] S103, obtaining a Shapley value corresponding to each second feature included in a second training sample corresponding to the second device as a contribution degree corresponding to the second feature;

[0058] S104, determining the importance of each feature according to the contribution of the first feature and the contribution of the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model.

[0059] It can be seen that in the scheme provided by the embodiment of the present invention, after the training of the federated learning model is completed, the first device in the federated learning model training system obtains the first eigenvalues ​​and the weights corresponding to each first feature included in the first training sample used to train the federated learning model locally, and calculates the Shapley value corresponding to the first feature based on each first eigenvalue and its corresponding weight, as the contribution corresponding to the first feature, and then obtains the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device, as the contribution corresponding to the second feature, and then determines the importance corresponding to each feature according to the contribution corresponding to the first feature and the contribution corresponding to the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model. Through the above scheme, since the Shapley value can accurately identify the contribution of the participant to the alliance, that is, the Shapley value can ensure the fairness of the distribution, so by applying the Shapley value to the federated learning model, the contribution of each feature included in the training sample to the model can be more accurately determined, and then the importance corresponding to each feature is determined by the contribution, and the importance is used to guide whether the model needs to be updated and iterated, so as to make the output result of the model more accurate.

[0060] Federated learning is a technology that can effectively help multiple institutions use data and conduct machine learning modeling while meeting the requirements of user privacy protection and data security. Generally, the federated learning model training system includes initiators, participants, and regulatory devices. During the training of the federated learning model, the initiator and the participants do not directly interact with each other, but send sample data to the regulatory device through encrypted transmission for model training. After the model training is completed, a federated learning model that can be used by both the model initiator and the model participants is obtained.

[0061] In the federated training system of the model, the party with the label of the stored data usually takes the initiative to initiate the model training task as the initiator, and the participating party supplements the data dimension as the data provider. For example, a bank is usually the party with the label and can take the initiative to initiate the model training task. The data source institution can act as a participant and provide data with more dimensions to participate in the model training.

[0062] Among them, the initiator trains the federated learning model through the first device and the participant's second device. After the federated learning model training is completed, in order to determine the contribution of the features of each dimension in the sample to the model training, the first device can obtain the first features included in the first training sample and the weight corresponding to each first feature.

[0063] The first training sample is a sample local to the first device for training the federated learning model, that is, multiple training samples provided by the initiator of the model training. The first feature is a feature included in the first training sample, which may be multiple. The weight corresponding to each first feature can be determined according to the model parameters of the federated learning model. Since the federated learning model has been trained, the model parameters have been determined, and the first device can obtain the model parameters corresponding to each first feature as the corresponding weight.

[0064] For example, as the initiator of model training, the bank jointly conducts federated learning model training with shopping malls. After the training of the federated learning model is completed, the first device corresponding to the bank can obtain the first features included in the first training sample for training the federated learning model and the weights corresponding to each first feature. The first feature may include product type, business direction, location, event, condition level and related person. Among them, product type is the type of financial products or services provided to customers; business direction is the direction in which the bank or participant conducts business; location is the address of the participant's location; event is the specific matter of the interaction between the participant and the bank; condition level is the level that describes the prerequisites, qualification standards and requirements required for the normal operation of the bank's business; related persons are all parties involved in the bank's business development process.

[0065] The number of first training samples may be 100, and the weight corresponding to each first feature is in percentage, which may be specifically shown in the following Table 1:

[0066] The first training sample Product Type Business Direction Place event Condition level Contact Person The first training sample 1 Product A Business Direction Location 11 deposit Low personal First training sample 2 Product B Business Direction Location 12 Withdrawals High mechanism … … … … … … … The first training sample is 100 Product N Business Direction Location Query middle personal Weight 68 83 20 10 42 5

[0067] After obtaining the above-mentioned first features and the weight corresponding to each first feature, the first device can calculate the Shapley value corresponding to the first feature based on each first feature and its corresponding weight as the contribution corresponding to the first feature, that is, execute step S102.

[0068] The Shapley value can construct a compromise utility allocation scheme that comprehensively considers the requirements of the conflicting parties, thereby ensuring the fairness of the allocation. The Shapley value can be calculated according to the following formula:

[0069]

[0070] in, is the Shapley value corresponding to the i-th feature, i∈[1,p], S is the set of input features, y is the input feature, p is the number of features, and val(S) refers to the predicted value of the model composed of the features in S.

[0071]

[0072] Among them, val yi (S) is the marginal contribution of feature yi in S to the benefit, is the model prediction value, y is the input model feature, is the expectation of the model's predictions.

[0073] As an implementation method, since the first training sample in the federated learning model contains multiple training samples, each training sample contains multiple features, and each feature corresponds to a Shapley value, in order to calculate the Shapley value corresponding to each first feature, the first device can first calculate the Shapley value corresponding to a feature in a sample, and then calculate the Shapley value corresponding to each first feature according to a preset processing method.

[0074] Since in the linear regression model, the predicted value of the model can be obtained by the formula To calculate, and then calculate the contribution of each feature in each sample to the model, where x1, x2, ...x p Represents the features of the same dimension in different samples, β0, β1, ...β p Represents the weight corresponding to the feature. Therefore, the calculation of the Shapley value corresponding to a feature can be done using the following formula:

[0075]

[0076] in, is the Shapley value corresponding to the jth feature in sample x, β j is the weight corresponding to the jth feature in sample x, E(β j X j ) is the average effect estimate of the jth feature in sample x, that is, the expectation of the product of the feature of the jth dimension and its weight in multiple samples, E(X j ) is the expectation of the j-th feature.

[0077] After obtaining the Shapley value corresponding to each feature in each sample, the first device can calculate the Shapley value corresponding to the first feature according to a preset processing method as the contribution corresponding to the first feature. The preset processing method includes at least the following processing methods:

[0078] In the first method, the first device can form a matrix by listing the Shapley values ​​corresponding to each feature in each sample according to the Shapley values ​​of the features corresponding to samples with different behaviors, and then list the Shapley values ​​of the features corresponding to the same sample. After obtaining the matrix, the first device can superimpose the Shapley values ​​of the features corresponding to each column of the matrix to obtain the Shapley values ​​corresponding to each first feature as the contribution degree corresponding to the first feature.

[0079] For example, for 100 user information samples, each sample includes 6 features. After calculating the Shapley value corresponding to each feature in each sample, a 100*6 matrix A can be constructed. At this time, an element in the matrix A represents the Shapley value corresponding to a feature in a user information sample, that is, the contribution of the feature to the sample. It represents the contribution of the j-th feature in the ith sample to the model. The first device can superimpose the j-th column of the matrix A to obtain the Shapley value of the j-th feature of all samples as the contribution corresponding to the j-th feature.

[0080] In the second method, the first device can form a matrix of Shapley values ​​corresponding to each feature in each sample, according to the Shapley values ​​of the features corresponding to the same sample, and list them as Shapley values ​​of the features corresponding to different samples. After obtaining the matrix, the first device can superimpose the Shapley values ​​of the features corresponding to each row of the matrix to obtain the Shapley values ​​corresponding to each first feature as the contribution degree corresponding to the first feature.

[0081] For example, for 100 user information samples, each sample includes 6 features. After calculating the Shapley value corresponding to each feature in each sample, a 6*100 matrix B can be constructed. At this time, an element in the matrix B represents the Shapley value corresponding to a feature in a user information sample, that is, the contribution of the feature to the sample. It represents the contribution of the j-th feature in the ith sample to the model. The first device can superimpose the j-th row of the matrix B to obtain the Shapley value of the j-th feature of all samples as the contribution corresponding to the j-th feature.

[0082] In the above step S103, the first device may obtain the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device as the contribution corresponding to each second feature. The specific calculation method of the contribution corresponding to each second feature is the same as the calculation method of the contribution corresponding to each first feature, which will not be repeated here.

[0083] Among them, the execution order of the above-mentioned step S103 and the above-mentioned steps S101-S102 is not limited. The above-mentioned step S103 can be executed first and then the above-mentioned steps S101-S102; the above-mentioned steps S101-S102 can be executed first and then the above-mentioned step S103; the above-mentioned step S103 and the above-mentioned steps S101-S102 can also be executed at the same time, which is all reasonable.

[0084] After obtaining the contribution corresponding to the first feature and the contribution corresponding to the second feature, the first device can determine the importance corresponding to each feature according to the contribution corresponding to the first feature and the contribution corresponding to the second feature. Since the contribution corresponding to the first feature and the contribution corresponding to the second feature are obtained by calculating the Shapley values ​​corresponding to the first feature and the second feature, and the Shapley value can ensure the fairness of distribution, when the input information changes greatly, it can be determined whether the model needs to be updated and iterated according to its importance, that is, the importance can guide whether the model needs to be updated and iterated, thereby making the output result of the model more accurate.

[0085] As an implementation manner of an embodiment of the present invention, the step of obtaining the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device can be implemented in at least the following two ways:

[0086] The first method: The first device can obtain the second features and the weight corresponding to each second feature included in the second training sample corresponding to the second device for training the federated learning model, and calculate the Shapley value corresponding to the second feature based on each second feature and its corresponding weight.

[0087] The first device can send a second feature contribution request to the second device. After receiving the second feature contribution request, the second device can obtain the second features and the weight corresponding to each second feature included in the second training sample used locally for training the federated learning model, and send them to the first device.

[0088] For example, the second device corresponding to the shopping mall organization can obtain the second features included in the second sample used to train the federated learning model, namely, the product identification, product type and membership type, and the weights corresponding to each second feature, wherein the number of second training samples is 100, and the weight corresponding to each second feature is in percentage. The specific corresponding relationship is shown in Table 2 below:

[0089]

[0090]

[0091] In this way, the first device can obtain each second feature included in the second training sample sent by the second device and the weight corresponding to each second feature, and then the first device can calculate the Shapley value corresponding to the second feature based on each second feature and its corresponding weight. The specific method for the first device to calculate the Shapley value corresponding to the second feature is the same as the method for calculating the Shapley value corresponding to the first feature, which will not be repeated here.

[0092] The second method: the first device may receive the Shapley values ​​corresponding to each second feature included in the second training sample for training the federated learning model sent by the second device.

[0093] The Shapley value corresponding to the second feature is calculated by the second device based on each second feature and its corresponding weight. The second device can calculate the Shapley value corresponding to the second feature based on each second feature included in the second training sample used locally for training the federated learning model and the weight corresponding to each second feature, and then send the Shapley value corresponding to the second feature to the first device.

[0094] In this way, the first device can receive the Shapley values ​​corresponding to each second feature included in the second training sample sent by the second device for training the federated learning model. The specific method for the second device to calculate the Shapley value corresponding to the second feature is the same as the method for the first device to calculate the Shapley value corresponding to the first feature, which will not be repeated here.

[0095] It can be seen that in this embodiment, the first device can obtain the second features and the weight corresponding to each second feature included in the second training sample for training the federated learning model corresponding to the second device, and calculate the Shapley value corresponding to the second feature based on each second feature and its corresponding weight, or, it can receive the Shapley values ​​corresponding to the second features included in the second training sample for training the federated learning model sent by the second device, and then determine the importance of each feature. Regardless of which method is used, the Shapley values ​​corresponding to the second features included in the second training sample can be accurately determined, thereby ensuring the accuracy of the importance of each feature determined subsequently.

[0096] As an implementation method of the embodiment of the present invention, Figure 2 As shown, the step of determining the importance of each feature according to the contribution of the first feature and the contribution of the second feature may include:

[0097] S201, sorting the contribution degree corresponding to the first feature and the contribution degree corresponding to the second feature to obtain a sorting result;

[0098] S202: Determine the importance of each feature based on the ranking result.

[0099] In one embodiment, after the first device calculates the Shapley value corresponding to each first feature, that is, the contribution corresponding to the first feature, and obtains the Shapley value corresponding to each second feature, that is, the contribution corresponding to the second feature, the first device can sort the contribution corresponding to each first feature and the contribution corresponding to each second feature, wherein the sorting method can be sorting from small to large or from large to small to obtain a sorting result, and determine the importance corresponding to each feature according to the sorting result.

[0100] The first device may sort the contribution corresponding to each first feature and the contribution corresponding to each second feature in descending order to obtain a sorting result. In the sorting result, the higher the position, the higher the importance of the feature.

[0101] If the contribution degree corresponding to the first feature and the contribution degree corresponding to the second feature include both positive values ​​and negative values, in order to facilitate sorting, the first device may sort according to the absolute values ​​of the contributions to obtain a sorting result.

[0102] For example, the first device may construct a histogram with the horizontal axis being the Shapley values ​​corresponding to each first feature and the Shapley values ​​corresponding to each second feature, and the vertical axis being the importance level corresponding to each feature, so that the user can view the importance level corresponding to each feature.

[0103] It can be seen that in this embodiment, the first device can sort the contribution corresponding to the first feature and the contribution corresponding to the second feature to obtain a sorting result, and then determine the importance corresponding to each feature based on the sorting result. The importance of each feature can be accurately determined, and the importance can guide whether the model needs to be updated and iterated, thereby further improving the accuracy of the output results of the model.

[0104] As an implementation method of the embodiment of the present invention, Figure 3 As shown, the step of calculating the Shapley value corresponding to the first feature based on each first feature and its corresponding weight may include:

[0105] S301, calculating the average value of each first feature;

[0106] After obtaining each first feature and its corresponding weight, the first device may calculate the average value of each first feature. For example, for the product type feature of the first sample in Table 1 above, the electronic device may convert the product type feature into a first feature represented by a numerical value, and then calculate the average value of 100 first features.

[0107] S302, based on the average value, according to the formula Decentralize each of the first features to obtain a processed first feature, where Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, is the average value corresponding to the i-th first feature;

[0108] In one implementation, as described in the above embodiment, the first device may construct a matrix of the Shapley values ​​corresponding to each feature in each sample in a preset manner. After obtaining the matrix, the elements of each row or column may be superimposed according to different matrix construction methods. The value obtained after superposition is the individual difference contribution of all features to the i-th sample, and the calculation formula may be:

[0109]

[0110] Where p is the number of features, is the individual difference contribution of the jth feature, E(X j ) is the expectation of the jth feature in the matrix X composed of multiple samples, is the model prediction value, x j is the jth feature in sample x, β j is the weight corresponding to the jth feature, β0 is the bias term, which is a constant term, Predict expectations for the model.

[0111] Since the same contribution of each first feature to the sample is ineffective in determining the contribution of each first feature to the model training process, the individual difference contribution is the key to determining the contribution of each first feature to the model training process. Therefore, the electronic device can be calculated according to the formula Decentralize each of the first features, that is, remove the influence of the same contribution, and then obtain the processed first feature, where Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, is the average value of the i-th first feature.

[0112] S303: Calculate the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight.

[0113] After obtaining the processed first feature, the first device may calculate the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight.

[0114] It can be seen that in this embodiment, the first device can calculate the average value of each first feature, based on the average value, according to the formula Decentralize each first feature to obtain the processed first feature, where Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, The average value corresponding to the i-th first feature is calculated based on the processed first feature and its corresponding weight. In this way, the Shapley value corresponding to the first feature can be obtained more quickly and accurately.

[0115] As an implementation manner of an embodiment of the present invention, the step of calculating the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight may include:

[0116] Based on the processed first feature and its corresponding weight, the Shapley value corresponding to the first feature is calculated according to the following formula:

[0117]

[0118] in, is the Shapley value of the i-th first feature, β i is the weight corresponding to the i-th first feature.

[0119] The electronic device determines the Shapley value Z of the i-th first feature i Then, we can calculate Z i The weight β corresponding to the i-th first feature i multiply the product The Shapley value corresponding to the i-th first feature is determined. In this way, the Shapley value corresponding to each feature can be calculated.

[0120] It can be seen that in this embodiment, the first device can calculate the Shapley value corresponding to each first feature by using the above calculation method, and can accurately calculate the Shapley value corresponding to each first feature.

[0121] Corresponding to the above method for determining feature contribution in federated learning, an embodiment of the present invention further provides a device for determining feature contribution in federated learning. The following is an introduction to a device for determining feature contribution in federated learning provided by an embodiment of the present invention.

[0122] like Figure 4 As shown, a device for determining feature contribution in federated learning is applied to a first device in a federated learning model training system, the system also includes a second device, and the device may include:

[0123] An acquisition module 410 is used to acquire, after the training of the federated learning model is completed, each first feature included in the first training sample used to train the federated learning model locally and a weight corresponding to each first feature;

[0124] A first calculation module 420, configured to calculate, based on each first feature and its corresponding weight, a Shapley value corresponding to the first feature as a contribution degree corresponding to the first feature;

[0125] A second calculation module 430 is used to obtain the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device as the contribution degree corresponding to each second feature;

[0126] The determination module 440 is used to determine the importance of each feature according to the contribution of the first feature and the contribution of the second feature.

[0127] The importance is used to determine the composition of the input information of the federated learning model.

[0128] It can be seen that in the scheme provided by the embodiment of the present invention, after the training of the federated learning model is completed, the first device in the federated learning model training system obtains the first eigenvalues ​​and the weights corresponding to each first feature included in the first training sample used to train the federated learning model locally, and calculates the Shapley value corresponding to the first feature based on each first eigenvalue and its corresponding weight, as the contribution corresponding to the first feature, and then obtains the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device, as the contribution corresponding to the second feature, and then determines the importance corresponding to each feature according to the contribution corresponding to the first feature and the contribution corresponding to the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model. Through the above scheme, since the Shapley value can accurately identify the contribution of the participant to the alliance, that is, the Shapley value can ensure the fairness of the distribution, so by applying the Shapley value to the federated learning model, the contribution of each feature included in the training sample to the model can be more accurately determined, and then the importance corresponding to each feature is determined by the contribution, and the importance is used to guide whether the model needs to be updated and iterated, so as to make the output result of the model more accurate.

[0129] As an implementation of the embodiment of the present invention, the second calculation module 430 may include:

[0130] a first calculation unit, configured to obtain each second feature included in a second training sample corresponding to the second device for training the federated learning model and a weight corresponding to each second feature, and calculate a Shapley value corresponding to the second feature based on each second feature and its corresponding weight; or

[0131] The second computing unit is used to receive the Shapley values ​​corresponding to each second feature included in the second training sample for training the federated learning model sent by the second device.

[0132] The Shapley value corresponding to the second feature is calculated by the second device based on each second feature and its corresponding weight.

[0133] As an implementation of the embodiment of the present invention, the first calculation module 420 may include:

[0134] an average value calculation unit, used to calculate the average value of each first feature;

[0135] A central calculation unit is used for calculating the average value according to the formula Decentralize each of the first features to obtain a processed first feature.

[0136] Among them, Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, is the average value corresponding to the i-th first feature.

[0137] A Shapley value calculation unit is used to calculate the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight.

[0138] As an implementation manner of an embodiment of the present invention, the Shapley value calculation unit may include:

[0139] The Shapley value calculation subunit is used to calculate the Shapley value corresponding to the first feature according to the following formula based on the processed first feature and its corresponding weight:

[0140]

[0141] in, is the Shapley value of the i-th first feature, β i is the weight corresponding to the i-th first feature.

[0142] An embodiment of the present invention further provides an electronic device, which may be the first device mentioned above. Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.

[0143] Memory 503, used for storing computer programs;

[0144] The processor 501 is used to implement the method steps for determining the feature contribution in federated learning described in any of the above embodiments when executing the program stored in the memory 503.

[0145] It can be seen that in the scheme provided by the embodiment of the present invention, after the training of the federated learning model is completed, the first device in the federated learning model training system obtains the first eigenvalues ​​and the weights corresponding to each first feature included in the first training sample used to train the federated learning model locally, and calculates the Shapley value corresponding to the first feature based on each first eigenvalue and its corresponding weight, as the contribution corresponding to the first feature, and then obtains the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device, as the contribution corresponding to the second feature, and then determines the importance corresponding to each feature according to the contribution corresponding to the first feature and the contribution corresponding to the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model. Through the above scheme, since the Shapley value can accurately identify the contribution of the participant to the alliance, that is, the Shapley value can ensure the fairness of the distribution, so by applying the Shapley value to the federated learning model, the contribution of each feature included in the training sample to the model can be more accurately determined, and then the importance corresponding to each feature is determined by the contribution, and the importance is used to guide whether the model needs to be updated and iterated, so as to make the output result of the model more accurate.

[0146] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0147] The communication interface is used for communication between the above electronic device and other devices.

[0148] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0149] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0150] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, the steps of the method for determining feature contribution in federated learning described in any of the above embodiments are implemented.

[0151] In another embodiment provided by the present invention, a computer program product containing instructions is also provided. When the computer is run on the computer, the computer executes the method for determining feature contribution in federated learning described in any of the above embodiments.

[0152] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0153] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0154] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for devices, electronic devices, computer-readable storage media, and computer program products, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for determining feature contribution in federated learning, characterized in that: A first device is applied to a federated learning model training system, the system further comprising a second device, the federated learning model is a risk control model, and the method comprises: After the training of the federated learning model is completed, obtaining first features and weights corresponding to each first feature included in the first training sample used to train the federated learning model locally; wherein the first features include: product type, business direction, location, event, condition level, and related person; Based on each first feature and its corresponding weight, calculating the Shapley value corresponding to the first feature as the contribution degree corresponding to the first feature; Obtaining the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device as the contribution corresponding to the second feature; wherein the second feature includes: product identification, product type and member type; Determine the importance of each feature according to the contribution of the first feature and the contribution of the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model; The step of obtaining the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device includes: Receiving the Shapley value corresponding to each second feature included in the second training sample for training the federated learning model sent by the second device, wherein the Shapley value corresponding to the second feature is calculated by the second device based on each second feature and its corresponding weight; The step of calculating the Shapley value corresponding to the first feature based on each first feature and its corresponding weight comprises: Calculating an average value of each of the first features; Based on the average value, according to the formula Decentralize each of the first features to obtain a processed first feature, where Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, is the average value corresponding to the i-th first feature; Based on the processed first feature and its corresponding weight, a Shapley value corresponding to the first feature is calculated.

2. The method according to claim 1, characterized in that The step of determining the importance of each feature according to the contribution of the first feature and the contribution of the second feature includes: Sorting the contribution degree corresponding to the first feature and the contribution degree corresponding to the second feature to obtain a sorting result; The importance of each feature is determined based on the ranking result.

3. The method according to claim 1, characterized in that The step of calculating the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight comprises: Based on the processed first feature and its corresponding weight, the Shapley value corresponding to the first feature is calculated according to the following formula: in, is the Shapley value of the i-th first feature, β i is the weight corresponding to the i-th first feature.

4. A device for determining feature contribution in federated learning, characterized in that: A first device applied to a federated learning model training system, the system further comprising a second device, the federated learning model being a risk control model, and the device comprising: An acquisition module, configured to acquire, after the training of the federated learning model is completed, each first feature included in the first training sample used to train the federated learning model locally and the weight corresponding to each first feature; wherein the first feature includes: product type, business direction, location, event, condition level and related person; A first calculation module, configured to calculate, based on each first feature and its corresponding weight, a Shapley value corresponding to the first feature as a contribution degree corresponding to the first feature; A second calculation module, used to obtain the Shapley value corresponding to each second feature included in the second training sample corresponding to the second device as the contribution corresponding to the second feature; wherein the second feature includes: product identification, product type and member type; a determination module, configured to determine the importance of each feature according to the contribution of the first feature and the contribution of the second feature, wherein the importance is used to determine the composition of the input information of the federated learning model; The second calculation module includes: A second calculation unit is used to receive the Shapley value corresponding to each second feature included in the second training sample for training the federated learning model sent by the second device, wherein the Shapley value corresponding to the second feature is calculated by the second device based on each second feature and its corresponding weight; The first calculation module includes: an average value calculation unit, used to calculate the average value of each first feature; A central calculation unit is used for calculating the average value according to the formula Decentralize each of the first features to obtain a processed first feature, where Z i is the first feature of the i-th feature after processing, X i is the i-th first feature, is the average value corresponding to the i-th first feature; A Shapley value calculation unit is used to calculate the Shapley value corresponding to the first feature based on the processed first feature and its corresponding weight.

5. The device according to claim 4, characterized in that The Shapley value calculation unit comprises: The Shapley value calculation subunit is used to calculate the Shapley value corresponding to the first feature according to the following formula based on the processed first feature and its corresponding weight: in, is the Shapley value of the i-th first feature, β i is the weight corresponding to the i-th first feature.

6. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-3 when executing a program stored in a memory.

Images