Method and apparatus for determining contribution degrees of participants in federated learning

Abstract

The present disclosure provides a method and apparatus for determining the contribution degree of participants in federated learning. The method includes: generating a plurality of participant groups based on the architecture of federated learning, and calculating the weights of the participant groups; determining the utility change value of the federated learning model before and after the aggregation period and establishing a lookup table, and judging whether to calculate the contribution value of the participants according to the utility change value; randomly generating a full permutation combination by using the participant groups, and calculating the estimated value of the marginal contribution value when a participant joins a sub-combination, and judging whether to calculate the utility value of the new participant group by using an interpolation function according to the estimated value and the weight; updating the lookup table according to the utility value of the new participant group, and calculating the marginal contribution value of the participants based on the updated lookup table until the contribution value of the converged participants is calculated, and determining the contribution degree of the participants in federated learning according to the contribution value. The present disclosure can improve the calculation accuracy and calculation efficiency of the contribution value in federated learning.

Description

Technical Field

[0001] The present disclosure relates to the technical field of federated learning, and in particular, to a method and apparatus for determining the contribution degree of participants in federated learning. Background Art

[0002] With the development of artificial intelligence and distributed machine learning technologies, the federated learning method of performing machine learning by combining different participants has become a mainstream trend for training artificial intelligence models. As a new type of distributed machine learning framework, federated learning meets the needs of multiple clients to perform model training under the requirement of data security.

[0003] In the prior art, when measuring the contribution degree of participants in federated learning, the federated learning system usually determines the contribution degree of each participant to the federated learning model based on the local data volume of each participant. However, due to problems such as uneven data quality and different forms of local data among the local data of each participant, the existing calculation methods for federated learning contribution degree have problems such as low calculation accuracy, inaccurate calculation results, large calculation data volume, and low calculation efficiency.

[0004] In view of the above problems in the prior art, there is a need to provide a method for determining the contribution degree of participants in federated learning that can improve the calculation accuracy of the contribution value, reduce the calculation data volume, make the calculation result of the contribution value more accurate, and have higher calculation efficiency. Summary of the Invention

[0005] In view of this, embodiments of the present disclosure provide a method and apparatus for determining the contribution degree of participants in federated learning to solve the problems of the existing calculation method for federated learning contribution degree, such as low calculation accuracy, inaccurate calculation results, large calculation data volume, and low calculation efficiency.

[0006] In the first aspect of the embodiments of the present disclosure, a method for determining the contribution degree of participants in federated learning is provided, including: generating a plurality of participant groups based on the architecture of federated learning, determining a set of participant groups composed of the plurality of participant groups, and calculating the weights of the participant groups, where each participant group includes at least two participants; determining the aggregation period in federated learning, obtaining the utility change value corresponding to the federated learning model before and after the aggregation period and establishing a lookup table, and judging whether to calculate the contribution values of each participant during the aggregation period according to the utility change value; when the judgment result is yes, randomly generating a full permutation combination using the participant groups in the set of participant groups, generating a plurality of sub-combinations according to the order of the participants in the participant groups in the full permutation combination, calculating the estimated value of the marginal contribution value when a participant joins a sub-combination, and judging whether to use an interpolation function to calculate the utility value of the new participant group formed after the participant joins the sub-combination according to the estimated value of the marginal contribution value and the weight of the participant group; when the judgment result is yes, calculating the utility value of the new participant group using the interpolation function, when the judgment result is no, calculating the utility value of the new participant group using a preset model deduction method, and updating the lookup table according to the calculated utility value of the new participant group; based on the updated lookup table, calculating the marginal contribution value of the participant, and judging whether the marginal contribution value of the participant converges, when the judgment result is yes, taking the converged marginal contribution value as the contribution value of the participant, when the judgment result is no, generating a new full permutation combination until the contribution values of all converged participants are calculated, and determining the contribution degree of the participant in federated learning according to the contribution values.

[0007] In a second aspect of the embodiments of the present disclosure, there is provided an apparatus for determining the contribution degree of participants in federated learning, including: a generation module configured to generate a plurality of participant groups based on the architecture of federated learning, determine a participant group set composed of the plurality of participant groups, and calculate the weights of the participant groups, where each participant group includes at least two participants; a establishment module configured to determine the aggregation period in federated learning, obtain the utility change value corresponding to the federated learning model before and after the aggregation period and establish a lookup table, and determine whether to calculate the contribution values of each participant during the aggregation period according to the utility change value; a judgment module configured to, when the judgment result is yes, randomly generate a full permutation combination using the participant groups in the participant group set, generate a plurality of sub-combinations according to the order of the participants in the participant groups in the full permutation combination, calculate the estimated value of the marginal contribution value when a participant joins the sub-combination, and determine whether to calculate the utility value of the new participant group formed after the participant joins the sub-combination using an interpolation function according to the estimated value of the marginal contribution value and the weight of the participant group; an update module configured to, when the judgment result is yes, calculate the utility value of the new participant group using the interpolation function, and when the judgment result is no, calculate the utility value of the new participant group using a preset model deduction method, and update the lookup table according to the calculated utility value of the new participant group; a calculation module configured to calculate the marginal contribution value of the participant based on the updated lookup table, and determine whether the marginal contribution value of the participant converges. When the judgment result is yes, use the converged marginal contribution value as the contribution value of the participant. When the judgment result is no, generate a new full permutation combination until the contribution values of all converged participants are calculated, and determine the contribution degree of the participant in federated learning according to the contribution values.

[0008] The above at least one technical solution adopted in the embodiments of the present disclosure can achieve the following beneficial effects:

[0009] Through a federated learning-based architecture, multiple participant groups are generated, and a set of participant groups composed of multiple participant groups is determined. The weights of the participant groups are calculated, where each participant group contains at least two participants; the aggregation period in federated learning is determined, the utility change values corresponding to the federated learning model before and after the aggregation period are obtained and a lookup table is established, and it is judged whether to calculate the contribution values of each participant within the aggregation period according to the utility change values; when the judgment result is yes, a full permutation combination is randomly generated using the participant groups in the set of participant groups, and multiple sub-combinations are generated according to the order of the participants in the participant groups in the full permutation combination. The estimated value of the marginal contribution value when a participant joins a sub-combination is calculated, and according to the estimated value of the marginal contribution value and the weights of the participant groups, it is judged whether to calculate the utility value of the new participant group formed after the participant joins the sub-combination using an interpolation function; when the judgment result is yes, the utility value of the new participant group is calculated using the interpolation function, and when the judgment result is no, the utility value of the new participant group is calculated using a preset model deduction method, and the lookup table is updated according to the calculated utility value of the new participant group; based on the updated lookup table, the marginal contribution value of the participant is calculated, and it is judged whether the marginal contribution value of the participant converges. When the judgment result is yes, the converged marginal contribution value is used as the contribution value of the participant, and when the judgment result is no, a new full permutation combination is generated until the contribution values of all converged participants are calculated, and the contribution degree of the participant in federated learning is determined according to the contribution values. The present disclosure can improve the calculation accuracy of the contribution values of participants in federated learning, reduce the amount of calculation data, make the calculation result of the contribution values more accurate, and have higher calculation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0011] Figure 1 is a schematic diagram of a federated learning architecture provided by an embodiment of the present disclosure;

[0012] Figure 2 is a schematic flowchart of a method for determining the contribution degree of participants in federated learning provided by an embodiment of the present disclosure;

[0013] Figure 3 is a schematic flowchart of a program for calculating the contribution value of a participant provided by an embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram of the structure of a device for determining the contribution degree of participants in federated learning provided by an embodiment of the present disclosure;

[0015] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. Specific embodiments

[0016] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present disclosure. However, those skilled in the art should clearly understand that the present disclosure can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present disclosure.

[0017] Federated learning refers to comprehensively utilizing various AI (Artificial Intelligence) technologies to jointly explore data value by multiple parties in cooperation on the premise of ensuring data security and user privacy, giving birth to new intelligent business forms and models based on joint modeling. Federated learning has at least the following characteristics:

[0018] (1) A weakly centralized joint training mode in which participating nodes control their own data, ensuring data privacy and security during the co-creation of intelligence.

[0019] (2) In different application scenarios, establish various model aggregation and optimization strategies by using screening and / or combining AI algorithms and privacy-preserving computing to obtain high-level and high-quality models.

[0020] (3) On the premise of ensuring data security and user privacy, based on various model aggregation and optimization strategies, obtain methods to improve the efficiency of the federated learning engine, where the efficiency methods can be to improve the overall efficiency of the federated learning engine by solving problems including computational architecture parallelism, information interaction under large-scale cross-domain networks, intelligent perception, and exception handling mechanisms.

[0021] (4) Obtain the requirements of multiple parties of users in each scenario, and through a mutual trust mechanism, determine a reasonable evaluation of the true contribution degrees of each federated participant and conduct distribution incentives.

[0022] Based on the above methods, an AI technology ecosystem based on federated learning can be established to fully utilize the value of industry data and promote the implementation of scenarios in vertical fields.

[0023] At present, with the increase in the number of participants in federated learning and the amount of computing data, how to accurately and quickly evaluate the contribution of each participant to the training of the federated learning model has become an urgent problem to be solved. In the prior art, the federated learning system determines the contribution of each participant to the federated learning model according to the local data volume of each participant. However, due to problems such as uneven data quality, different formats or forms of local data, and a large overlap of data features among the local data of each participant, the calculation efficiency of the contribution to the federated learning model in federated learning is low, and the accuracy of the federated learning contribution of each participant is also low. This will lead to insufficient evaluation of the fairness and impartiality of the contribution of each participant when the benefits of each participant are distributed using the federated learning contribution in the later stage.

[0024] In view of the above problems in the prior art, it is necessary to provide a method for calculating the contribution of each participant in federated learning based on the Shapley value calculation rule, combined with the marginal contribution value generated when each participant joins the participant group. Based on the embodiments of the present disclosure, it is possible to improve the accurate calculation of the contribution value of each participant to the training of the federated learning model in federated learning, reduce the calculation amount, make the calculation result of the contribution value more accurate, and improve the calculation efficiency.

[0025] Next, a method and device for determining the contribution of a participant in federated learning according to an embodiment of the present disclosure will be described in detail with reference to the accompanying drawings.

[0026] Figure 1 It is a schematic diagram of the architecture of a federated learning provided by an embodiment of the present disclosure. As Figure 1 shown, the architecture of federated learning may include a server (central node) 101 and participants 102, 103, and 104.

[0027] In the federated learning process, the basic model can be established by the server 101, and the server 101 sends the model to the participants 102, 103, and 104 that have established communication connections with it. The basic model can also be established by any participant and uploaded to the server 101, and the server 101 sends the model to other participants that have established communication connections with it. The participants 102, 103, and 104 construct a model based on the downloaded basic structure and model parameters, use local data for model training, obtain updated model parameters, and encrypt and upload the updated model parameters to the server 101. The server 101 aggregates the model parameters sent by the participants 102, 103, and 104 to obtain global model parameters, and sends the global model parameters back to the participants 102, 103, and 104. The participants 102, 103, and 104 iterate their respective models according to the received global model parameters until the model finally converges, thereby realizing the training of the model. In the federated learning process, the data uploaded by the participants 102, 103, and 104 are model parameters, and the local data will not be uploaded to the server 101, and all participants can share the final model parameters. Therefore, common modeling can be achieved on the basis of ensuring data privacy. It should be noted that the number of participants is not limited to the three mentioned above, but can be set as needed, and the embodiments of the present disclosure do not limit this.

[0028] Figure 2 It is a schematic flowchart of a method for determining the contribution degree of participants in federated learning provided by an embodiment of the present disclosure. Figure 2 The method for determining the contribution degree of participants in federated learning can be executed by the server of federated learning. As Figure 2 shown, the method for determining the contribution degree of participants in federated learning can specifically include:

[0029] S201, based on the architecture of federated learning, generate multiple participant groups, determine a set of participant groups composed of the multiple participant groups, and calculate the weights of the participant groups, where each participant group includes at least two participants;

[0030] S202, determine the aggregation period in federated learning, obtain the utility change value corresponding to the federated learning model before and after the aggregation period and establish a lookup table, and determine whether to calculate the contribution values of each participant within the aggregation period according to the utility change value;

[0031] S203. When the judgment result is yes, randomly generate a full permutation combination using the participant groups in the participant group set, generate multiple sub - combinations according to the order of the participants in the participant groups in the full permutation combination, calculate the estimated value of the marginal contribution value when a participant joins a sub - combination, and judge whether to use an interpolation function to calculate the utility value of the new participant group formed after the participant joins the sub - combination according to the estimated value of the marginal contribution value and the weight of the participant group;

[0032] S204. When the judgment result is yes, calculate the utility value of the new participant group using the interpolation function; when the judgment result is no, calculate the utility value of the new participant group using a preset model deduction method, and update the lookup table according to the calculated utility value of the new participant group;

[0033] S205. Based on the updated lookup table, calculate the marginal contribution value of the participant, and judge whether the marginal contribution value of the participant converges. When the judgment result is yes, use the converged marginal contribution value as the contribution value of the participant; when the judgment result is no, generate a new full permutation combination until the contribution values of all converged participants are calculated, and determine the contribution degree of the participant in the federated learning according to the contribution value.

[0034] Specifically, each participant corresponds to a node in the federated learning framework, and each node corresponds to a participant device. The participant device can be a PC, a tablet computer, a smart phone, a smart wearable device, etc. There is a client terminal of the federated learning participant on each participant device, but the participant device is not limited to the above - mentioned devices or clients. There is also a node (i.e., the server) in the federated learning framework that provides services for the client. The server can be a server for performing aggregation operations. The server can coordinate multiple clients to perform federated learning to obtain a federated learning model. The server can be an independent physical server, or a server cluster or a cloud computing server composed of multiple physical servers.

[0035] Furthermore, a participant group refers to the mutual permutation combination among all individuals of the participants in the federated learning, which is a participant group composed of individual participants. For example, in a federated learning framework, there are 3 participants, namely A, B, and C. Then the following participant groups can be formed among them: A, B, C, AB, BC, AC.

[0036] Furthermore, in federated learning, an aggregation period refers to one round of training for the federated learning model. Each participating client uses local data to train its local model. When the local model training converges, the trained local model parameters are obtained and sent to the server. All participants upload their local model parameters in each aggregation round, and the server performs weighted averaging to obtain a federated model. Therefore, each participant makes its own contribution in each round. Here, the round represents a complete training of the federated learning model by the server.

[0037] According to the technical solution provided by the embodiments of the present disclosure, multiple participant groups are generated based on a federated learning architecture, and a set of participant groups composed of multiple participant groups is determined. The weights of the participant groups are calculated, where each participant group includes at least two participants; the aggregation period in federated learning is determined, the utility change values corresponding to the federated learning model before and after the aggregation period are obtained and a lookup table is established, and it is determined whether to calculate the contribution values of each participant during the aggregation period according to the utility change values; when the judgment result is yes, a full permutation combination is randomly generated using the participant groups in the set of participant groups, and multiple sub-combinations are generated according to the order of the participants in the participant groups in the full permutation combination. The estimated value of the marginal contribution value when a participant joins the sub-combination is calculated, and it is determined whether to calculate the utility value of the new participant group formed after the participant joins the sub-combination using an interpolation function according to the estimated value of the marginal contribution value and the weights of the participant groups; when the judgment result is yes, the utility value of the new participant group is calculated using the interpolation function, and when the judgment result is no, the utility value of the new participant group is calculated using a preset model deduction method, and the lookup table is updated according to the calculated utility value of the new participant group; based on the updated lookup table, the marginal contribution value of the participant is calculated, and it is determined whether the marginal contribution value of the participant converges. When the judgment result is yes, the converged marginal contribution value is used as the contribution value of the participant, and when the judgment result is no, a new full permutation combination is generated until the contribution values of all converged participants are calculated, and the contribution degree of the participant in federated learning is determined according to the contribution values. The present disclosure can improve the calculation accuracy of the contribution value in federated learning, reduce the amount of calculation data, make the calculation result of the contribution value more accurate, and have higher calculation efficiency.

[0038] The following combines a specific program flow diagram to elaborate in detail on the loop process of calculating the contribution values of each participant in the present disclosure in federated learning. Figure 3 It is the program flow diagram for calculating the contribution value of a participant provided by the embodiments of the present disclosure. As Figure 3 shown, the specific content in the program for calculating the contribution value of a participant may include the following:

[0039] In some embodiments, multiple sub - combinations are generated according to the order of participants in the participant group in the full permutation and combination, and an estimated value of the marginal contribution value when a participant joins a sub - combination is calculated, including: dividing the full permutation and combination into multiple sub - combinations according to the order of participants, determining the next participant corresponding to the last participant in the sub - combination in the full permutation and combination, and calculating an estimated value of the marginal contribution value when the next participant joins the sub - combination.

[0040] Specifically, when it is determined that the contribution value of each participant in this round needs to be calculated, first randomly generate a full permutation and combination P from the set of participant groups Ps in this round of aggregation period, and let k = 0. For example, assume that a set of participant groups Ps contains elements 1, 2, 3, 4, 5, and these 5 elements correspond to 5 participants respectively. The following full permutation and combinations can be generated based on these 5 participants: (1, 2, 3, 4, 5), (2, 3, 4, 5, 1), (3, 4, 5, 1, 2), (4, 5, 1, 2, 3), (5, 4, 3, 2, 1).... After randomly generating a full permutation and combination, then divide the participants in the full permutation and combination into multiple sub - combinations.

[0041] Furthermore, when calculating the marginal contribution value of a participant according to the full permutation and combination, the following method can be adopted:

[0042] Take out the first j participants from the full permutation and combination P in sequence to form a sub - combination S, and calculate an estimated value of the marginal contribution generated when the (j + 1) - th participant joins the sub - combination S. For example, if the full permutation and combination is (5, 4, 3, 2, 1), then the first 1 participant (i.e., element 5) can be taken out first to form a sub - combination (5), and calculate an estimated value of the marginal contribution generated when the 2 - nd participant (i.e., element 4) joins the sub - combination (5). In practical applications, the following formula can be used to estimate the marginal contribution, that is, Δ j+1_est = v N - v S = v_lut[N] - v_lut[S]. Since v S has been calculated in the previous combination S', it only needs to be obtained from the lookup table v_lut here, and there is no need to calculate V(M S (t) ).

[0043] Furthermore, use the above - mentioned calculation method to perform a loop on all formed sub - combinations S in the current full permutation and combination, and the marginal contribution value generated when the (j + 1) - th participant joins the sub - combination S, until each case of the sub - combinations in the current full permutation and combination is calculated once, and finally an estimated value of the marginal contribution value when a participant joins a sub - combination is obtained.

[0044] In some embodiments, determining whether to use an interpolation function to calculate the utility value of the new participant group formed after a participant joins a sub - combination according to the estimated value of the marginal contribution value and the weight of the participant group includes: calculating the product of the estimated value of the marginal contribution value and the weight of the participant group, and comparing the product with a preset second truncation threshold; when the product is less than or equal to the second truncation threshold, it is determined to use the interpolation function to calculate the utility value of the new participant group; otherwise, the utility value of the new participant group is calculated using a preset model deduction method.

[0045] Specifically, according to the relationship between the product of the estimated value of the marginal contribution value and the weight of the participant group and the second truncation threshold, it is determined whether to calculate the utility value of the new participant group; the following combines specific embodiments to detail the calculation and judgment process of the above - mentioned product, which may specifically include the following content:

[0046] Calculate the product |Δ j+1_est *w |S| | of the marginal contribution value generated when the (j + 1)-th participant joins the sub - combination S and the weight corresponding to the sub - combination S. If this product satisfies |Δ j+1_est *w |S| |≤η*|vN - v0|, it is determined to use the interpolation function to calculate the utility value of the new participant group; otherwise, it is determined to calculate the utility value of the new participant group using a preset model deduction method.

[0047] That is to say, if the marginal contribution value generated when the (j + 1)-th participant joins the sub - combination S satisfies the above formula, then at this time, the utility value of the new participant group corresponding to the (j + 1)-th participant joining the sub - combination S does not need to be deduced anymore, and the interpolation function is directly used to calculate the utility value of the new participant group. If it does not satisfy the above formula, the new participant group needs to be model - deduced and the utility value is calculated.

[0048] According to the technical solution provided by the embodiments of the present disclosure, in order to pre - determine whether to perform model deduction on the new participant group, by magnifying the utility value, the marginal contribution value generated when the (j + 1)-th participant joins the sub - combination S is estimated, and the estimated value is multiplied by the weight of the participant group. The product is compared with the second truncation threshold, so as to determine whether to use the interpolation function to calculate the utility value of the new participant group or use the model deduction method to calculate the utility value of the new participant group; since the complexity of model deduction is very high and the calculation amount is large, by adding the above - mentioned judgment means, for the new participant groups that do not need to perform model deduction, the weighted sum of the utility values of the sub - combinations calculated in the previous iteration process can be directly used, thereby improving the calculation speed of the contribution value.

[0049] In some embodiments, an interpolation function is used to calculate the utility value of a new participant group, including: based on the utility value of the sub-combination calculated during the historical iteration process, and the corresponding utility value when the participant group is the full set participant group, the utility value of the new participant group is calculated using a preset interpolation function, and the lookup table is updated according to the calculation result.

[0050] Specifically, the principle of calculating the utility value of the interpolation function is to use the utility value of the already calculated sub-combination to estimate the utility value of the new participant group. The calculation formula of the interpolation function is V(S∪{j+1})=interpolate(v S ,v N , S, N), and obtain the utility estimate of the new participant group according to the calculation formula, that is, obtain the utility estimate corresponding to the new participant group S∪{j+1}, and update the lookup table v_lut[S∪{j+1}]=V(S∪{j+1}). Among them, the implementation method of the interpolation function interpolate(·) is:

[0051]

[0052] In the calculation formula of the above interpolation function, v S , v N , the sub-combination S and the full combination N are taken as the input of the function, calculated and output.

[0053] In some embodiments, the utility value of the new participant group is calculated using a preset model deduction method, and the lookup table is updated based on the calculated utility value of the new participant group, including: aggregating the model parameters of the new participant group, and performing model deduction on the model of the new participant group, aggregating the weight of each participant in the new participant group to obtain the weight of the new participant group, performing model deduction on the model of the new participant group on a standard validation set, calculating the true utility value of the new participant group, and updating the lookup table using the true utility value.

[0054] Specifically, the model is derived using the formula V(S∪{j+1})=V(M S∪{j+1} (t) )=V(Agg(S∪{j+1})), and the real utility value of the new participant group is obtained according to the calculation formula and the lookup table v_lut[S∪{j+1}]=V(S∪{j+1}) is updated.

[0055] In some embodiments, based on the updated lookup table, the marginal contribution value of the participating party is calculated, and it is determined whether the marginal contribution value of the participating party converges. When the determination result is yes, the converged marginal contribution value is used as the contribution value of the participating party, including: calculating the marginal contribution value of the participating party according to the utility value after the participating party joins the sub-combination in the updated lookup table by using a preset Shapley value calculation formula; and determining whether the marginal contribution value of the participating party converges. When the determination result is convergence, the converged marginal contribution value is used as the contribution value of the participating party; wherein, the contribution value is used to represent the contribution degree of the participating party to the federated learning model trained in the aggregation period in federated learning.

[0056] Specifically, according to the updated v_lut lookup table, the marginal contribution value of the participating party i is calculated If it indicates that the marginal contribution value of the participating party i in the current full permutation combination has converged. At this time, directly use as the contribution value of each participating party i in the t-th aggregation period Otherwise, let k = k + 1, and regenerate a new full permutation combination P, and repeat the steps of the above embodiments to calculate the contribution values of each participating party in the full permutation combination P.

[0057] Here, for the setting of the threshold θ, θ can represent the judgment condition for whether the Monte Carlo method converges. In practical applications, θ can be set to 1e-3 to 1e-5.

[0058] Based on the calculation process of the contribution values of each participating party in the full permutation combination P in the above embodiments, all full permutation combinations are cycled through to obtain the contribution values of each participating party i in all T aggregation periods, and the contribution values of the participating party i to the federated model are accumulated

[0059] Further, the Shapley Value is a method for fairly distributing benefits based on the average of the marginal contributions generated by an individual i joining a combination S, and its computational complexity is O(2 n ), where n is the total number of individuals. Its calculation formula is:

[0060]

[0061] The Shapley value (i.e., the Shapley value) considers all possible orders in which an individual i joins a sub-combination. Here, N represents the full combination, S represents a sub-combination in a certain permutation case, V(·) represents the utility function, the |·| symbol represents the number of elements in the set, and [V(S ∪ {i}) – V(S)] represents the marginal utility after i joins the sub-combination S. The weight w |S| = |S|!(|N| - |S| - 1)! / |N|! represents the probability of the occurrence of this combination.

[0062] Further, repeat the above steps to obtain the contribution value of each participant i in all T aggregation cycles, and accumulate to obtain the contribution value of participant i to the joint model; that is, calculate all the aggregation cycles in the above manner to obtain the contribution value corresponding to each participant in each aggregation cycle, and then accumulate to obtain the total contribution value.

[0063] Further, set the first truncation threshold λ in the following manner: Let the marginal gain of the final joint model utility function relative to the initial model be Δ U =|V(M (T) ) - V(M (0) )|, where T is the total number of communication rounds, and λ can be set to λ=Δ U *0.01. The second truncation threshold η can be used to represent the error level of the contribution value, and it can be set to η = 1e - 3 to 1e - 5.

[0064] In some embodiments, based on the architecture of federated learning, generate multiple participant groups, determine the set of participant groups composed of multiple participant groups, and calculate the weights of the participant groups, including: determining all participants in the federated learning, enumerating the participants in ascending order of the number of participants to obtain multiple participant groups, taking the set composed of the multiple participant groups as the set of participant groups, and calculating the corresponding weight of each participant group based on the number of participants in each participant group; wherein, the weight is used to represent the probability of the participant group appearing in the set of participant groups.

[0065] Specifically, the following combines a specific embodiment to detail the participants in the federated learning and the process of constructing the set of participant groups, which can specifically include the following content:

[0066] Suppose there is a federated learning of N participants 1, 2,... i... n - 1, n, and the training has a total of T cycles of aggregation. Record each aggregation cycle t during the training, the local model M i (t) uploaded by each participant i, and the jointly aggregated joint model M (t) , initialize the model M (0) , have an evaluation function or utility function V(·) for the model performance (such as accuracy, loss, etc.), a federated learning model aggregation method Agg(·), thresholds λ, η; where λ represents the first truncation threshold and η represents the second truncation threshold.

[0067] Further, first, according to all the participating parties in the federated learning, enumerate all possible groups of participating parties Ps = [(1,), (2,), (3), …, (1, 2), (1, 3), (2, 3), … P, … N] in ascending order of the number of participating parties; for each sub - combination S with 0, 1, 2, … n - 1 participating parties, calculate the weight w |S| = |S|!(|N| - |S| - 1)! / |N|!.

[0068] It should be noted that each group of participating parties corresponds to one of the above - mentioned sub - combinations S. When calculating the weight of the sub - combination S, it is based on the number of participating parties in each sub - combination. In a group of participating parties, one participating party corresponds to an element in a set, that is, the weight corresponding to the group of participating parties is calculated according to the number of elements in the group of participating parties. The weight corresponding to each sub - combination can be regarded as the probability of the sub - combination appearing in the overall group of participating parties.

[0069] In some embodiments, to determine the aggregation period in the federated learning, obtain the utility change value corresponding to the federated learning model before and after the aggregation period and establish a lookup table, including: for each aggregation period in the federated learning, determine the initial utility value and the final utility value of the federated learning model corresponding to the aggregation period, calculate the difference between the final utility value and the initial utility value, use the difference as the utility change value, and establish a lookup table corresponding to the aggregation period that includes all groups of participating parties; perform an initialization operation on the lookup table so that the initial utility values of other groups of participating parties except the empty - set group of participating parties and the full - set group of participating parties in the lookup table are 0; where the lookup table is used to store the utility values corresponding to all groups of participating parties.

[0070] Specifically, calculate the utility values corresponding to the federated learning model before and after the start and end of each aggregation period, and establish a lookup table, that is, for each aggregation period, the final utility value and the initial utility value of this aggregation period can be calculated first. The following combines a specific embodiment to detail the calculation process of the initial utility value and the final utility value of the aggregation period, which may specifically include the following content:

[0071] For each aggregation period t, calculate v N = V(M (t) ), v0 = V(M (t-1) ), and establish a lookup table v_lut = {(): v0, (1,): 0, (2,): 0, (3,): 0…, (1, 2): 0, (1, 3): 0, (2, 3): 0, … N: vN}, where v NIt represents the final utility value of the joint model after the end of the current aggregation cycle, and v0 represents the utility value of the joint model after the end of the previous aggregation cycle corresponding to the current aggregation cycle. Of course, v0 can also be understood as the initial utility value of the current aggregation cycle before the start of the current aggregation cycle. The difference in different expressions does not constitute a limitation on the essential meaning of v0, and the above two expressions are equivalent.

[0072] Furthermore, when the lookup table is initialized, except for the participant groups corresponding to the empty set () and the full set N, the utility values ​​of other participant groups in the participant group set Ps are set to 0. By establishing a v_lut lookup table and using the lookup table to cache the utility values ​​of the participant groups, thereby recording the calculated utility values, the calculation amount for subsequent contribution value calculations can be reduced to avoid repeated calculations.

[0073] In some embodiments, whether to calculate the contribution value of each participant in the aggregation cycle is determined based on the utility change value, including: comparing the utility change value of the joint learning model corresponding to before and after the aggregation cycle with a preset first truncation threshold; when the utility change value is less than the first truncation threshold, and the utility change values ​​corresponding to multiple consecutive aggregation cycles are all less than the first truncation threshold, it is determined that the contribution value of each participant in the aggregation cycle is 0; otherwise, the contribution value of each participant in the aggregation cycle is recalculated.

[0074] Specifically, by calculating the final utility value of the current aggregation cycle and the initial utility value of the current aggregation cycle, if the difference between the final utility value and the initial utility value corresponding to the current aggregation cycle is less than the first cutoff threshold, and if the difference between the final utility value and the initial utility value is less than the first cutoff threshold for two consecutive aggregation cycles, the calculation is terminated, and the contribution value of each participant in the current aggregation cycle is regarded as 0, that is, each participant has not made a contribution in the current aggregation cycle. The following is a detailed description of the process of using a calculator to determine whether to calculate the contribution value of each participant in the current aggregation cycle in conjunction with a specific embodiment, which may specifically include the following contents:

[0075] If |v N -v0|≤λ, then the counter is incremented by 1. If the counter exceeds 1 (the counter value is greater than or equal to 2), then the contribution value of each participant i in this aggregation period t is Then return to the previous step, otherwise continue to the next step; in other words, by subtracting the final utility value corresponding to the aggregation model generated before and after the current aggregation cycle from the initial utility value, and comparing the difference with the first truncation threshold, when the difference is less than the first truncation threshold, the value of the counter is increased by 1. If the counter is greater than or equal to 2 (that is, the utility change value of the aggregation model generated by two consecutive aggregation cycles is less than the threshold), the contribution value of each participant in this aggregation cycle is judged to be 0.

[0076] The purpose of the embodiments of the present disclosure is to evaluate the change in the utility value of the joint model in this round before officially calculating the contribution value of each participant. According to the evaluation result, it can be judged whether the utility value of the joint model after this round of aggregated training has been improved, that is, whether the performance of the joint model itself has been improved. If the improvement in model performance is very small, it can be considered that the contribution value of each participant in this round is 0; if it is found in this round that the performance of the joint learning model has been greatly improved, then continue to execute the following calculation, that is, specifically calculate the contribution value of each participant in this round. The present disclosure can pre-judge whether it is necessary to further calculate the contribution value of the participant or directly calculate the contribution value of the participant in this round as 0, thereby avoiding an ineffective calculation process and improving the calculation efficiency.

[0077] The following is an embodiment of the device of the present disclosure, which can be used to execute the method embodiment of the present disclosure. For details not disclosed in the embodiment of the device of the present disclosure, please refer to the method embodiment of the present disclosure.

[0078] Figure 4 It is a schematic structural diagram of a device for determining the contribution degree of participants in joint learning provided by the embodiments of the present disclosure. As Figure 4 shown, the device for determining the contribution degree of participants in joint learning includes:

[0079] A generation module 401, configured to generate a plurality of participant groups based on the architecture of joint learning, determine a set of participant groups composed of the plurality of participant groups, and calculate the weights of the participant groups, where each participant group includes at least two participants;

[0080] A establishment module 402, configured to determine the aggregation period in joint learning, obtain the utility change value corresponding to the joint learning model before and after the aggregation period and establish a lookup table, and judge whether to calculate the contribution value of each participant during the aggregation period according to the utility change value;

[0081] A judgment module 403, configured to, when the judgment result is yes, randomly generate a full permutation combination using the participant groups in the set of participant groups, generate a plurality of sub-combinations according to the order of the participants in the participant groups in the full permutation combination, calculate the estimated value of the marginal contribution value when the participant joins the sub-combination, and judge whether to use an interpolation function to calculate the utility value of the new participant group formed after the participant joins the sub-combination according to the estimated value of the marginal contribution value and the weight of the participant group;

[0082] An update module 404, configured to, when the judgment result is yes, calculate the utility value of the new participant group using an interpolation function, and when the judgment result is no, calculate the utility value of the new participant group using a preset model deduction method, and update the lookup table according to the calculated utility value of the new participant group;

[0083] The calculation module 405 is configured to calculate the marginal contribution value of the participant based on the updated lookup table, and determine whether the marginal contribution value of the participant has converged. When the judgment result is yes, the converged marginal contribution value is used as the contribution value of the participant. When the judgment result is no, a new full permutation combination is generated until the contribution values ​​of all converged participants are calculated, and the contribution degree of the participant in the joint learning is determined according to the contribution value.

[0084] In some embodiments, Figure 4 The judgment module 403 divides the full permutation combination into multiple sub-combinations according to the order of the participants, determines the next participant corresponding to the last participant of the sub-combination in the full permutation combination, and calculates the estimated value of the marginal contribution value of the next participant when joining the sub-combination.

[0085] In some embodiments, Figure 4 The judgment module 403 calculates the product of the estimated value of the marginal contribution value and the weight of the participant group, and compares the product with a preset second truncation threshold; when the product is less than or equal to the second truncation threshold, it is determined to use an interpolation function to calculate the utility value of the new participant group; otherwise, the utility value of the new participant group is calculated using a preset model deduction method.

[0086] In some embodiments, Figure 4 The updating module 404 calculates the utility value of the new participant group using a preset interpolation function based on the utility value of the sub-combination calculated in the historical iteration process and the corresponding utility value when the participant group is the full set participant group, and updates the lookup table according to the calculation result.

[0087] In some embodiments, Figure 4 The update module 404 aggregates the model parameters of the new participant group, performs model deduction on the model of the new participant group, aggregates the weight of each participant in the new participant group to obtain the weight of the new participant group, performs model deduction on the model of the new participant group on the standard validation set, calculates the real utility value of the new participant group, and uses the real utility value to update the lookup table.

[0088] In some embodiments, Figure 4 The generation module 401 determines all the participants in the joint learning, enumerates the participants in order from the least to the most to obtain multiple participant groups, takes the set formed by the multiple participant groups as the participant group set, and calculates the weight corresponding to each participant group based on the number of participants in each participant group; wherein the weight is used to represent the probability of the participant group appearing in the participant group set.

[0089] In some embodiments, Figure 4The first judgment module 402 of determines, for each aggregation period in the federated learning, the initial utility value and the final utility value of the federated learning model corresponding to the aggregation period, calculates the difference between the final utility value and the initial utility value, takes the difference as the utility change value, and establishes a lookup table corresponding to the aggregation period that includes all participating party groups; performs an initialization operation on the lookup table so that the initial utility values of other participating party groups except the empty set participating party group and the full set participating party group in the lookup table are 0; wherein, the lookup table is used to store the utility values corresponding to all participating party groups.

[0090] In some embodiments, Figure 4 The establishment module 402 of compares the utility change value of the federated learning model corresponding before and after the aggregation period with a preset first truncation threshold. When the utility change value is less than the first truncation threshold and the utility change values corresponding to multiple consecutive aggregation periods are all less than the first truncation threshold, it is determined that the contribution values of each participating party within the aggregation period are 0; otherwise, the contribution values of each participating party within the aggregation period are recalculated.

[0091] In some embodiments, Figure 4 The calculation module 405 of calculates the marginal contribution value of the participating party by using a preset Shapley value calculation formula according to the utility value after the participating party joins the sub - combination in the updated lookup table; and according to whether the marginal contribution value of the participating party converges, when the judgment result is convergence, takes the converged marginal contribution value as the contribution value of the participating party; wherein, the contribution value is used to represent the contribution degree of the participating party to the federated learning model trained in the aggregation period in the federated learning.

[0092] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present disclosure.

[0093] Figure 5 is a schematic structural diagram of the electronic device 5 provided by the embodiments of the present disclosure. As Figure 5 shown, the electronic device 5 of this embodiment includes: a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the steps in each of the above - mentioned method embodiments. Or, when the processor 501 executes the computer program 503, it implements the functions of each module / unit in each of the above - mentioned device embodiments.

[0094] Exemplarily, the computer program 503 can be divided into one or more modules / units, which are stored in the memory 502 and executed by the processor 501 to complete the present disclosure. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 503 in the electronic device 5.

[0095] The electronic device 5 can be a desktop computer, a notebook, a palm computer, a cloud server, and other electronic devices. The electronic device 5 can include, but is not limited to, the processor 501 and the memory 502. Those skilled in the art can understand that Figure 5 merely examples of the electronic device 5, which do not constitute a limitation on the electronic device 5, may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may further include input / output devices, network access devices, a bus, etc.

[0096] The processor 501 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0097] The memory 502 can be an internal storage unit of the electronic device 5. For example, the hard disk or memory of the electronic device 5. The memory 502 can also be an external storage device of the electronic device 5. For example, a plug-in hard disk equipped on the electronic device 5, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 502 can also include both the internal storage unit and the external storage device of the electronic device 5. The memory 502 is used to store computer programs and other programs and data required by the electronic device. The memory 502 can also be used to temporarily store data that has been output or will be output.

[0098] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0099] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this disclosure.

[0101] In the embodiments provided by this disclosure, it should be understood that the disclosed device / computer device and method can be implemented in other ways. For example, the device / computer device embodiments described above are only illustrative. For example, the division of modules or units is only a logical functional division. In actual implementation, there may be other division methods. Multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] In addition, in each of the various embodiments of the present disclosure, the functional units may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, all or part of the processes in the above-mentioned embodiment methods of the present disclosure can also be completed by instructing relevant hardware through a computer program. The computer program may be stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program may include computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0105] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. A method for determining the contribution degree of participants in federated learning, characterized in that Including: Based on a federated learning architecture, generate multiple participant groups, determine a set of participant groups composed of the multiple participant groups, and calculate the weights of the participant groups, where each participant group includes at least two participants; Determine the aggregation period in the federated learning, obtain the utility change value corresponding to the federated learning model before and after the aggregation period and establish a lookup table, and judge whether to calculate the contribution values of each participant within the aggregation period according to the utility change value; When the judgment result is yes, randomly generate a full permutation combination using the participant groups in the set of participant groups, generate multiple sub-combinations according to the order of the participants in the participant groups in the full permutation combination, calculate the estimated value of the marginal contribution value when a participant joins the sub-combination, and judge whether to use an interpolation function to calculate the utility value of the new participant group formed after the participant joins the sub-combination according to the estimated value of the marginal contribution value and the weight of the participant group; When the judgment result is yes, calculate the utility value of the new participant group using the interpolation function, and when the judgment result is no, calculate the utility value of the new participant group using a preset model deduction method, and update the lookup table according to the calculated utility value of the new participant group; Based on the updated lookup table, calculate the marginal contribution value of the participant, and judge whether the marginal contribution value of the participant converges. When the judgment result is yes, use the converged marginal contribution value as the contribution value of the participant. When the judgment result is no, generate a new full permutation combination until the contribution values of all converged participants are calculated, and determine the contribution degree of the participant in the federated learning according to the contribution values; Among them, during the federated learning process, the basic model is established by the server. The server sends the basic model to the participants that have established a communication connection with it, or the basic model is established by any participant and uploaded to the server. The server sends the basic model to other participants that have established a communication connection with it. The participants build a model according to the downloaded basic structure and model parameters, train the model using local data to obtain updated model parameters, and encrypt and upload the updated model parameters to the server.

2. The method according to claim 1, wherein The generating multiple sub-combinations according to the order of the participants in the participant groups in the full permutation combination and calculating the estimated value of the marginal contribution value when a participant joins the sub-combination includes: Divide the full permutation combination into multiple sub-combinations according to the order of the participants, and determine the next participant corresponding to the last participant in the sub-combination in the full permutation combination, and calculate the estimated value of the marginal contribution value when the next participant joins the sub-combination.

3. The method according to claim 2, wherein The judging whether to use an interpolation function to calculate the utility value of the new participant group formed after the participant joins the sub-combination according to the estimated value of the marginal contribution value and the weight of the participant group includes: Calculate the product of the estimated value of the marginal contribution value and the weight of the participant group, and compare the product with a preset second truncation threshold; When all of the products are less than or equal to the second truncation threshold, it is determined to calculate the utility value of the new participant group using an interpolation function; otherwise, the utility value of the new participant group is calculated using a preset model deduction method.

4. The method according to claim 3, wherein The calculating the utility value of the new participant group using an interpolation function includes: Based on the utility values of the sub - combinations calculated in the historical iteration process and the utility value corresponding to when the participant group is the complete - set participant group, the utility value of the new participant group is calculated using a preset interpolation function, and the lookup table is updated according to the calculation result.

5. The method according to claim 1, characterized in that, The calculating the utility value of the new participant group using a preset model deduction method and updating the lookup table according to the calculated utility value of the new participant group includes: Aggregating the model parameters of the new participant group, performing model deduction on the model of the new participant group, aggregating the weights of each participant in the new participant group to obtain the weight of the new participant group, performing model deduction on the model of the new participant group on the standard validation set, calculating the true utility value of the new participant group, and updating the lookup table using the true utility value.

6. The method according to claim 1, characterized in that The generating multiple participant groups based on the federated learning architecture, determining a participant - group set composed of multiple participant groups, and calculating the weights of the participant groups includes: Determining all participants in the federated learning, enumerating the participants in ascending order of the number of participants to obtain multiple participant groups, taking the set composed of the multiple participant groups as the participant - group set, and calculating the weight corresponding to each participant group based on the number of participants in each participant group; wherein, the weight is used to represent the probability of the participant group appearing in the participant - group set.

7. The method according to claim 1, characterized in that, The determining the aggregation period in the federated learning, obtaining the utility change value corresponding to the federated learning model before and after the aggregation period and establishing a lookup table includes: For each aggregation period in the federated learning, determining the initial utility value and the final utility value of the federated learning model corresponding to the aggregation period, calculating the difference between the final utility value and the initial utility value, taking the difference as the utility change value, and establishing a lookup table corresponding to the aggregation period that includes all the participant groups. Performing an initialization operation on the lookup table so that the initial utility values of other participant groups except the empty - set participant group and the complete - set participant group in the lookup table are 0; wherein, the lookup table is used to store the utility values corresponding to all the participant groups.

8. The method according to claim 7, wherein The determining whether to calculate the contribution values of each participant during the aggregation period according to the utility change value includes: Compare the utility change value of the federated learning model corresponding to before and after the aggregation period with a preset first truncation threshold. When the utility change value is less than the first truncation threshold and the utility change values corresponding to consecutive multiple rounds of aggregation periods are all less than the first truncation threshold, determine that the contribution values of all parties in the aggregation period are 0; otherwise, recalculate the contribution values of all parties in the aggregation period.

9. The method according to claim 1, wherein Based on the updated lookup table, calculate the marginal contribution value of the party and determine whether the marginal contribution value of the party converges. When the judgment result is yes, use the converged marginal contribution value as the contribution value of the party, including: According to the utility value of the party after joining the sub - combination in the updated lookup table, use a preset Shapley value calculation formula to calculate the marginal contribution value of the party; and according to whether the marginal contribution value of the party converges, when the judgment result is convergence, use the converged marginal contribution value as the contribution value of the party; Wherein, the contribution value is used to represent the contribution degree of the party to the federated learning model trained in the aggregation period in the federated learning.

10. An apparatus for determining the contribution degree of participants in federated learning, characterized in that, Including: A generation module, configured to generate multiple groups of parties based on the architecture of federated learning, determine a set of groups of parties composed of multiple groups of parties, and calculate the weights of the groups of parties, where each group of parties includes at least two parties; A building module, configured to determine the aggregation period in the federated learning, obtain the utility change value corresponding to the federated learning model before and after the aggregation period and build a lookup table, and judge whether to calculate the contribution values of all parties in the aggregation period according to the utility change value; A judgment module, configured to when the judgment result is yes, randomly generate a full permutation combination using the groups of parties in the set of groups of parties, generate multiple sub - combinations according to the order of the parties in the groups of parties in the full permutation combination, calculate the estimated value of the marginal contribution value when the party joins the sub - combination, and judge whether to use an interpolation function to calculate the utility value of the new group of parties formed after the party joins the sub - combination according to the estimated value of the marginal contribution value and the weight of the group of parties; An update module, configured to when the judgment result is yes, calculate the utility value of the new group of parties using an interpolation function, when the judgment result is no, calculate the utility value of the new group of parties using a preset model deduction method, and update the lookup table according to the calculated utility value of the new group of parties; A calculation module, configured to based on the updated lookup table, calculate the marginal contribution value of the party and determine whether the marginal contribution value of the party converges. When the judgment result is yes, use the converged marginal contribution value as the contribution value of the party. When the judgment result is no, generate a new full permutation combination until the converged contribution values of all parties are calculated, and determine the contribution degree of the party in the federated learning according to the contribution value; Among them, during the federated learning process, the basic model is established by the server, and the server sends the basic model to the participating parties that have established a communication connection with it. Alternatively, the basic model is established by any participating party and uploaded to the server, and the server sends the basic model to other participating parties that have established a communication connection with it. The participating parties construct a model based on the downloaded basic structure and model parameters, use local data to train the model, obtain updated model parameters, and encrypt and upload the updated model parameters to the server.

Images