Unbiased federated learning incentive method and device based on stackelberg game

Abstract

Embodiments of the present application disclose a non-biased federated learning incentive method and device based on a Stackelberg game, which is used to guarantee the non-bias of a model in the case of random and indefinite number of target participants in each round of training, and to realize faster and better convergence of the model. The method comprises: determining a user utility function and a service utility function based on an expected participation level of each alternative participant; determining a target optimization problem corresponding to the service utility function and the user utility function based on a Stackelberg game model; solving the target optimization problem to obtain a target participation level of each alternative participant and a corresponding target incentive function; performing federated learning based on the target participants in each round of training until a target model is obtained through model convergence; and randomly sampling a plurality of alternative participants according to the target participation level of the target participants in each round of training to determine the target model, which is obtained by aggregating actual participation levels of the target participants in each round of training.

Description

Technical Field

[0001] This application relates to the field of federated learning, and more particularly to an unbiased federated learning incentive method and related equipment based on Stackelberg games. Background Technology

[0002] Federated learning, as an emerging distributed machine learning paradigm, enables numerous user endpoints to collaboratively train machine learning models under the coordination of a central server, while maintaining the privacy of the training data. In the era of big data, where companies and institutions need to use massive amounts of user data to develop valuable machine learning models for profit, and where countries are increasingly emphasizing data privacy protection, the status of federated learning is rising significantly. To encourage active participation from potential participants in federated learning, sufficient compensation, or incentives, needs to be provided to each participant.

[0003] However, existing incentive mechanisms for federated learning are designed with a fixed number of target participants in each training round. In practical applications, candidate participants are not guaranteed to participate in each training round, and the training data of the target participants in each round is unlikely to represent the data characteristics of all candidate participants, leading to serious model bias. In other words, the model only performs well on specific datasets. Therefore, there is an urgent need for a federated learning incentive method that takes into account the randomness of target participants in each round while ensuring the overall performance of the model. Summary of the Invention

[0004] This application provides an unbiased federated learning incentive method and device based on Stackelberg game, which can achieve faster and better model convergence when the target participants in each round of training are random.

[0005] The first aspect of this application provides an unbiased federated learning incentive method based on Stackelberg game theory, applied to an aggregation server, including:

[0006] The user utility function of each candidate participant and the service utility function of the aggregation server are determined based on the expected participation level of each candidate participant.

[0007] The objective optimization problem corresponding to the service utility function and the user utility function is determined based on the Stackelberg game model;

[0008] Solve the target optimization problem to obtain the target participation level of each candidate participant and the target incentive function corresponding to the target participation level. Under the target participation level, the user utility function and the service utility function reach Stackelberg equilibrium.

[0009] Federated learning is performed based on the target participants in each round of training until the model converges to obtain the target model; the target participants in each round of training are determined by random sampling of multiple candidate participants according to the target participation level; the target model is obtained by aggregating the actual participation levels of the target participants in each round of training.

[0010] The learning incentives for each target participant are calculated based on the corresponding target incentive function.

[0011] In one specific implementation, determining the user utility function of each candidate participant and the service utility function of the aggregation server based on the expected participation level of each candidate participant includes:

[0012] The service utility function of the aggregation server is calculated based on the expected participation level of each candidate participant and the preset loss function.

[0013] Send a preset incentive function to each of the candidate participants;

[0014] The system receives a user utility function sent by each candidate participant. The user utility function is constructed by the candidate participant based on the expected participation level, a preset incentive function, and a preset cost function.

[0015] In one specific implementation, the user utility function is constructed by the candidate participant based on the expected participation level, a preset incentive function, and a preset cost function, including:

[0016] The user utility function for each candidate participant is constructed according to the following formula;

[0017] U c (q n ,P n ) = P n q n -C n

[0018] Among them, C n Let q be the preset cost function for the nth candidate participant. n Let P be the expected participation level of the nth candidate participant. n U is the preset activation function for the nth candidate participant. c (q n ,P n ) is the nth alternative participant in q n and P n The user utility function below.

[0019] In one specific implementation, calculating the service utility function of the aggregation server based on the expected participation level of each candidate participant and a preset loss function includes:

[0020] Construct the service utility function of the aggregation server according to the following formula;

[0021]

[0022] Where P is the set of preset incentive functions for each target participant, q is the set of expected participation levels for each target participant, F is the preset loss function, and w r Let r be the aggregation model parameters. U represents the expected value. s Let be the service utility function of the aggregation server under P and q.

[0023] In one specific implementation, determining the objective optimization problem corresponding to the service utility function and the user utility function based on the Stackelberg game model includes:

[0024] Based on the Stackelberg game model, the service utility function, and the user utility function of each backup server, the following objective optimization problem can be obtained through mathematical derivation.

[0025]

[0026] Among them, a n q represents the proportion of the training data of the nth candidate participant to the sum of the training data of all other candidate participants. n G represents the expected participation level of the nth alternative participant. n Let F represent the local gradient of the nth candidate participant, where α and β are system parameters. * This represents the minimum value of the global objective loss function.

[0027] In one specific implementation, solving the objective optimization problem to obtain the target participation level of each candidate participant and the target incentive function corresponding to the target participation level includes:

[0028] Under the constraints of expected costs and the preset maximum participation level for each candidate participant, the objective optimization problem is solved to calculate the target participation level for each candidate participant and the target incentive function corresponding to the target participation level.

[0029] In one specific implementation, federated learning is performed based on the target participants in each round of training until the model converges to obtain the target model, including:

[0030] Send the aggregated model parameters of round r to each target participant participating in the (r+1)th round of training;

[0031] The parameters of the aggregated model trained in the (r+1)th round are calculated using the following formula;

[0032]

[0033] Among them, w r+1 w represents the aggregation model parameters in round r+1. r Let S(q) represent the aggregation model parameters in the r-th round. r Let a represent the set of target participants in round r. n q represents the proportion of the training data of the nth candidate participant to the total training data. n ′ represents the actual participation level of the nth candidate participant. Indicates that the nth alternative participant is based on w r The parameters of the model to be aggregated in the (r+1)th round obtained from local training;

[0034] If the w r+1 If the preset convergence condition is met, then based on w... r+1 Determine the target model.

[0035] A second aspect of this application provides a computer device, including:

[0036] The determining unit is used to determine the user utility function of each candidate participant and the service utility function of the aggregation server based on the expected participation level of each candidate participant;

[0037] The determining unit is further configured to determine the objective optimization problem corresponding to the service utility function and the user utility function based on the Stackelberg game model;

[0038] The solution unit is used to solve the target optimization problem, obtain the target participation level of each candidate participant and the target incentive function corresponding to the target participation level, and under the target participation level, the user utility function and the service utility function reach the Stackelberg equilibrium.

[0039] The training unit is used to perform federated learning based on the target participants in each round of training until the model converges to obtain the target model; the target participants in each round of training are determined by random sampling of multiple candidate participants according to the target participation level; and the target model is obtained by aggregating the actual participation levels of the target participants in each round of training.

[0040] The incentive unit is used to calculate the learning incentive for each of the target participants according to the corresponding target incentive function.

[0041] In one specific implementation, the determining unit is specifically used to calculate the service utility function of the aggregation server based on the expected participation level of each candidate participant and a preset loss function.

[0042] Send a preset incentive function to each of the candidate participants;

[0043] The system receives a user utility function sent by each candidate participant. The user utility function is constructed by the candidate participant based on the expected participation level, a preset incentive function, and a preset cost function.

[0044] In one specific implementation, the user utility function is constructed by the candidate participant based on the expected participation level, a preset incentive function, and a preset cost function, including:

[0045] The user utility function for each candidate participant is constructed according to the following formula;

[0046] U c (q n ,P n ) = P n q n -C n

[0047] Among them, C n Let q be the preset cost function for the nth candidate participant. n Let P be the expected participation level of the nth candidate participant. n U is the preset activation function for the nth candidate participant. c (q n ,P n ) is the nth alternative participant in q n and P n The user utility function below.

[0048] In one specific implementation, the determining unit is specifically used to construct the service utility function of the aggregation server according to the following formula;

[0049]

[0050] Where P is the set of preset incentive functions for each target participant, q is the set of expected participation levels for each target participant, F is the preset loss function, and w r Let r be the aggregation model parameters. U represents the expected value. s Let be the service utility function of the aggregation server under P and q.

[0051] In one specific implementation, the determining unit is specifically used to perform mathematical derivation based on the Stackelberg game model, the service utility function, and the user utility function of each backup server to obtain the following objective optimization problem;

[0052]

[0053] a n q represents the proportion of the training data of the nth candidate participant to the sum of the training data of all other candidate participants. n G represents the expected participation level of the nth alternative participant. n Let F represent the local gradient of the nth candidate participant, where α and β are system parameters. * This represents the minimum value of the global objective loss function.

[0054] In one specific implementation, the solving unit is specifically used to solve the objective optimization problem under the constraints of the expected cost and the preset maximum participation level of each candidate participant, and to calculate the target participation level of each candidate participant and the target incentive function corresponding to the target participation level.

[0055] In one specific implementation, the training unit is specifically used to send the aggregated model parameters of the rth round to each target participant participating in the (r+1)th round of training;

[0056] The parameters of the aggregated model trained in the (r+1)th round are calculated using the following formula;

[0057]

[0058] Among them, w r+1 w represents the aggregation model parameters in round r+1. r Let S(q) represent the aggregation model parameters in the r-th round. r Let a represent the set of target participants in round r. n q represents the proportion of the training data of the nth candidate participant to the total training data. n ′ represents the actual participation level of the nth candidate participant. Indicates that the nth alternative participant is based on w r The parameters of the model to be aggregated in the (r+1)th round obtained from local training;

[0059] If the w r+1 If the preset convergence condition is met, then based on w... r+1 Determine the target model.

[0060] A third aspect of this application provides a computer device, including:

[0061] Central processing unit, memory, and input / output interfaces;

[0062] The memory is either a short-term storage memory or a persistent storage memory;

[0063] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method described in the first aspect.

[0064] A fourth aspect of this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in the first aspect.

[0065] A fifth aspect of this application provides a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect.

[0066] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: They fully consider the randomness of the target participants in each round of training, and randomly sample multiple candidate participants by pre-calculating the target participation level of each candidate participant to determine the target participants for each round of training until the model parameters converge. The target participation level of each candidate participant is determined when the user utility function of each candidate participant and the service utility function of the aggregation server reach a balance. The aforementioned steps simulating the randomness of the target participants ensure that an unequal number of users randomly participate in training in each round, conforming to system heterogeneity and simulating situations where users may be unable to participate in training in a timely manner due to network or resource issues. Compared with traditional methods of uniform sampling and determining target participants based on user data volume weights, this approach achieves faster and better model convergence. Attached Figure Description

[0067] Figure 1 This is a flowchart illustrating an unbiased federated learning incentive method disclosed in an embodiment of this application.

[0068] Figure 2 This is a schematic diagram of the structure of a computer device disclosed in an embodiment of this application;

[0069] Figure 3 This is another schematic diagram of the computer device disclosed in the embodiments of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0071] In federated learning, each data provider computes and updates its local model using its local dataset, and the model buyer periodically aggregates these local models to update the global model. Because data providers and model buyers are often distinct entities in federated learning, data providers may not be as interested as model buyers in obtaining high-performance models (i.e., higher accuracy and lower cost). For example, a company might use a federated learning framework to train a business model, specifically by having users use local data and local models on their devices, even though the users themselves may not be interested in the models. Therefore, without sufficient compensation or incentives, users may be unwilling to participate in federated learning for local training due to the training costs required for local training, such as computational and communication resource consumption. This makes the design of incentive mechanisms crucial in federated learning systems.

[0072] This application provides an unbiased federated learning incentive method and device based on Stackelberg game, which can achieve faster and better model convergence when the target participants in each round of training are random.

[0073] Please see Figure 1 This application provides an unbiased federated learning incentive method based on Stackelberg games, comprising the following steps:

[0074] 101. Determine the user utility function of each candidate participant and the service utility function of the aggregation server based on the expected participation level of each candidate participant.

[0075] Considering that the incentive design needs to simultaneously take into account the interests of both the participants and the aggregation server, and maximize the interests of both sides, it is necessary to construct a Stackelberg game equilibrium of their utility functions. Furthermore, considering the unbiasedness of the model, the model convergence speed, and the need to simulate users who may be unable to participate in training in a timely manner due to network or resource issues, the user utility function of each candidate participant and the service utility function of the aggregation server can be determined based on the expected participation level of each candidate participant.

[0076] 102. Determine the objective optimization problem corresponding to the service utility function and the user utility function based on the Stackelberg game model.

[0077] After determining the user utility function of each candidate participant and the service utility function of the aggregation server based on the expected participation level of each candidate participant in step 101, the objective optimization problem that needs to be met to construct the Stackelberg game equilibrium can be determined according to the user utility function of each candidate participant and the service utility function of the aggregation server.

[0078] Specifically, this involves obtaining the objective optimization problem that each candidate participant's user utility function and the aggregation server's service utility function must satisfy to reach the Stackelberg game equilibrium, and then solving this objective optimization problem.

[0079] 103. Solve the objective optimization problem to obtain the target participation level of each candidate participant and the target incentive function corresponding to the target participation level. Under the target participation level, the user utility function and the service utility function reach the Stackelberg equilibrium.

[0080] Because the objective optimization problem is the mathematical expression that each candidate participant's user utility function and the aggregation server's service utility function must satisfy to reach the Stackelberg game equilibrium, solving the objective optimization problem can determine the target participation level of each candidate participant when the user utility function and the aggregation server's service utility function reach the Stackelberg game equilibrium.

[0081] In some specific implementations, this can be achieved by: solving the objective optimization problem under the constraints of expected cost and the preset maximum participation level of each candidate participant, and calculating the target participation level of each candidate participant and the target incentive function corresponding to the target participation level.

[0082] Specifically, before purchasing a model, the model buyer (i.e., the aggregation server) has an expected cost and a pricing strategy. The pricing strategy is a set of preset incentive functions for each candidate participant, which can be used to calculate the incentives that each candidate participant can obtain at different participation levels. Meanwhile, the model buyer's goal is to reduce training error within a limited cost; the data provider (i.e., the candidate participant) aims to obtain the highest possible profit (i.e., the profit remaining after subtracting the training cost from the obtained incentives). Therefore, the conditions for the Stackelberg game model to reach equilibrium include: maximizing the user utility function of each candidate participant and the service utility function of the aggregation server while minimizing the training error.

[0083] After constructing the objective optimization problem, it is necessary to determine the objective problem corresponding to the minimum risk function and the maximum utility function based on the expected cost and the Stackelberg game model. Finally, the objective problem is solved mathematically under constraints to calculate the target participation level of each candidate participant.

[0084] It should be noted that the target participation level for each candidate participant is calculated based on the expected cost, a preset incentive function, a loss function, and the cost function of each candidate participant. The goal is to minimize the loss value (calculated based on the loss function) corresponding to the final target model parameters obtained through convergence, while maximizing the profit of each candidate participant, without exceeding the expected cost. In some specific implementations, the target participation level is also subject to the following constraints: each candidate participant is configured with a maximum participation level as needed, and the target participation level for each candidate participant cannot exceed the corresponding maximum participation level.

[0085] 104. Federated learning is performed based on the target participants in each round of training until the model converges to obtain the target model; the target participants in each round of training are determined by random sampling of multiple candidate participants according to the target participation level, and the target model is obtained by aggregating the actual participation levels of the target participants in each round of training.

[0086] After determining the target participants for each round of training, federated learning can be performed according to the predetermined target participants for each round of training until the model converges and the target model is obtained.

[0087] In this approach, the target participation level of each candidate participant, pre-calculated, serves as one of the sampling constraints. Random sampling is performed among multiple candidate participants to determine the target participant for each training round. Candidate participants are those who may participate in any round of this federated learning, while the target participant for each training round is the participant participating in that specific round. Participants can be user terminals and / or data servers—any device storing training data—and this embodiment is not limited to any particular device. The specific sampling method can be Bernoulli sampling or random number-based sampling. For example, (in the random number-based sampling method) after obtaining the target participation level of each candidate participant, because the target participation level is always less than or equal to 1, a random number can be randomly generated for each candidate participant (the size of any random number is always less than or equal to 1). The target participation level of the candidate participant and the size of the corresponding random number are compared. If the corresponding random number is greater than or equal to the corresponding target participation level, the corresponding candidate participant is determined as the target participant; if the corresponding random number is less than the corresponding target participation level, the operation of determining the corresponding candidate participant as the target participant is not performed.

[0088] In addition, the aggregation server needs to aggregate the model parameters to be aggregated locally trained by each target participant in round r+1 according to the target participation level of each target participant, so as to obtain the aggregated model parameters in round r+1.

[0089] 105. Calculate the learning incentives for each target participant based on the corresponding target incentive function.

[0090] In practical applications, incentive allocation is required after federated learning to encourage participation from all parties involved. Specifically, the learning incentive for each target participant can be calculated based on the target incentive function determined in step 103 and the actual participation level of each target participant in each round of training. It should be noted that participants in each round of training typically participate in federated learning according to the target participation level calculated in step 103. Therefore, unless otherwise specified, the actual participation level referred to in the foregoing and subsequent embodiments of this application generally refers to the target participation level of the corresponding target participant.

[0091] In this embodiment, the randomness of the target participants in each training round is fully considered. Multiple candidate participants are randomly sampled based on the pre-calculated target participation level of each candidate participant to determine the target participants for each training round, until the model parameters converge. The target participation level of each candidate participant is determined when the user utility function of each candidate participant and the service utility function of the aggregation server reach a balance. The aforementioned steps simulating the randomness of the target participants ensure that a unequal number of users randomly participate in training each round, conforming to system heterogeneity and simulating situations where users may be unable to participate in training in a timely manner due to network or resource issues. Compared with traditional methods of uniform sampling and determining target participants based on user data volume weights, this approach achieves faster and better model convergence.

[0092] In some specific implementations, step 101 can be implemented in the following ways: calculate the service utility function of the aggregation server based on the expected participation level of each candidate participant and the preset loss function; send the preset incentive function to each candidate participant; receive the user utility function sent by each candidate participant, wherein the user utility function is constructed by the candidate participant based on the expected participation level, the preset incentive function and the preset cost function.

[0093] Specifically, the user utility function for each candidate participant is typically net profit, which is the net profit obtained by subtracting costs from the incentives received at a certain participation level (i.e., the expected participation level). The service utility function works similarly; however, the goal of the service utility function is to obtain a better target model. Therefore, the minimum of the expected model loss is the minimum of the service utility function.

[0094] Furthermore, the user utility function for each candidate participant is constructed according to the following formula;

[0095] U c (q n ,P n ) = P n q n -C n

[0096] Among them, C n Let q be the preset cost function for the nth candidate participant. n Let P be the expected participation level of the nth candidate participant. n U is the preset incentive function for the nth candidate participant. c (q n ,P n ) is the nth alternative participant in q n and P n The user utility function below.

[0097] Similarly, the service utility function of the aggregation server can also be constructed according to the following formula;

[0098]

[0099] Where P is the set of preset incentive functions for each target participant, q is the set of expected participation levels for each target participant, F is the preset loss function, and w r Let r be the aggregation model parameters. U represents the expected value. s This is the service utility function of the aggregation server under P and q.

[0100] Furthermore, the user utility function of each of the aforementioned candidate participants and the service utility function of the aggregation server can be used to determine the objective optimization problem based on the Stackelberg game model and the following steps.

[0101] The main process of the game is that the buyer in the model first determines the cost B and the pricing strategy P = {p1,...p}. N The data provider determines its level of participation (or probability of participation) q based on its own interests. n The two sides then play the game again based on the new parameters until a Stackelberg equilibrium is reached (SE: P and q no longer change). The following are the decisions made by both sides:

[0102] First, the decision-making process is led by the model purchaser. Because the model purchaser's goal is to reduce training error within a limited cost, they need to formulate a strategy targeting models with q = {q1,...q}. NThe pricing strategy P of the model provider (i.e., the candidate participant who has the capability and potential to participate in training) determines the participation probability. Therefore, the incentive for the model buyer to purchase user n is P. n q n Assume w r (q) represents the aggregated model parameters after the r-th round of training. Given that the pricing strategy is P and the participation level is q, the objective of the model buyer is... Constraints are Where F is the loss function of the currently trained model, U s This is the service utility function for the model purchaser.

[0103] Next, data providers act as followers in making decisions. This is because the data provider's goal is to maximize its own U. n User utility functions are therefore constructed for the data provider.

[0104] First, a pre-defined cost function needs to be constructed for data providers. This function calculates the resource consumption incurred by data providers during their participation in federated learning, including but not limited to computational and / or communication costs, as well as the lost opportunities to participate in other activities for monetary rewards. Intuitively, the higher the level of participation, the higher the cost; therefore, the pre-defined cost function can be... i>1, 0≤q n ≤q n,max Among them, q n,max The maximum level of participation for the data provider can be configured by the data provider. In this embodiment, i is set to 2, which is a standard assumption in economic models where the decision variable is a capacity-constrained variable.

[0105] Secondly, considering the incentives received by the data provider and local costs, the user utility function can be expressed as U c (q n ,P n ) = P n q n -C n , where 0≤q n ≤q n,max The data provider's game-theoretic objective is to minimize the user's utility function.

[0106] Finally, a Stackelberg game model is performed. Specifically, the decisions of the data provider and the model buyer can be reduced to a second-order Stackelberg game: in the first stage, the model buyer, as the leader, formulates the pricing strategy, and in the second stage, the data provider, as the follower, adjusts its participation level. The game continues until the SE (Stake) is reached.

[0107] Specifically, through mathematical derivation, SE can be reduced to a targeted problem, namely...

[0108]

[0109]

[0110] Among them, a n q represents the proportion of the training data of the nth candidate participant to the sum of the training data of all other candidate participants. n G represents the expected participation level of the nth alternative participant. n Let F represent the local gradient of the nth candidate participant, where α and β are system parameters. * The objective pricing strategy, which minimizes the global objective loss function, can be expressed as follows: in The target participation level is [value]. It should be noted that, in the process of solving the objective optimization problem, F [value]... * During the calculation process, it will be eliminated, and no explicit numerical value is needed to help determine the target participation level. The above expression means that in the constraint ( 0≤q n ≤q n,max In ), find The minimum value.

[0111] It should be noted that, based on the L-smoothness and μ-strong convexity theory, we can obtain the following expression:

[0112]

[0113] in, Furthermore, L is the L-smooth L-parameter, μ is the μ-strongly convex μ-parameter, and E is the number of local training epochs for the participant. It is the upper bound of the variance of the stochastic gradient. It is the upper bound of the expected square norm of the stochastic gradient, ω0 is the initialized machine learning model parameters, ω n These are the final machine learning model parameters.

[0114] This application also provides an unbiased model aggregation method, in which the data provider can independently decide its participation level without being determined by the model purchaser, and can obtain the same unbiased convergence effect as full participation under random sampling.

[0115] In some specific implementations, step 104 can be implemented in the following way: send the aggregated model parameters of round r to each target participant participating in round r+1 of training; calculate the aggregated model parameters of round r+1 of training according to the following formula; Among them, w r+1 w represents the aggregation model parameters in round r+1. rLet S(q) represent the aggregation model parameters in the r-th round. r Let a represent the set of target participants in round r. n q represents the proportion of the training data of the nth candidate participant to the total training data. n ′ represents the actual participation level of the nth candidate participant. Indicates that the nth alternative participant is based on w r The parameters of the model to be aggregated in the (r+1)th round obtained from local training; if w r+1 If the preset convergence condition is met, then based on w r+1 Determine the target model.

[0116] In this training process, the parameters of the model to be trained in each round are provided by the initial model of the aggregation server if it is the first round of training; otherwise, they are aggregated from the parameters of the model to be aggregated obtained from the local training of each target participant in the previous round. The specific aggregation method can be found in step 103 and related steps, and is not limited in this embodiment. In practical applications, the model parameters of the initial model in the first round of training can be randomly generated by the aggregation server, or the aggregation server can perform a training operation based on the amount of local training data and / or gradient information of each candidate participant, and then use the model obtained from this training as the initial model for the first round of training. No specific limitations are imposed here.

[0117] In addition, each target participant uses the aggregated model parameters obtained after training in round r as the initial model parameters for round r+1. They use local data to iteratively train the initial model for this round, which is built based on the initial model parameters for this round, until the model converges and finally obtains the converged model for this round. The model parameters in the converged model for this round are the model parameters to be aggregated in this round.

[0118] It should be noted that, in order to ensure the unbiasedness of the aggregation model parameters as much as possible, the proportion of training data of each target participant, that is, the proportion of the training data of each target participant to the sum of the training data of each candidate participant, can be used as the basis for aggregation.

[0119] In some specific implementations, this step can be achieved by calculating the aggregate model parameters for each r-th training round according to the following formula;

[0120]

[0121] Among them, w r+1 w represents the aggregation model parameters in round r+1. r Let S(q) represent the aggregation model parameters in the r-th round. r Let a represent the set of target participants in round r. nq represents the proportion of the training data of the nth candidate participant to the total training data. n ′ represents the actual participation level of the nth candidate participant. Indicates that the nth alternative participant is based on w r The parameters of the model to be aggregated in the (r+1)th round obtained from local training.

[0122] It is understood that the model parameters (including but not limited to the parameters of the model to be trained, the parameters of the model to be aggregated, and the parameters of the aggregated model) referred to in the foregoing and following embodiments of this application may refer to the gradient parameters of the currently trained model and / or the weight, bias, and other values ​​of the currently trained model. This embodiment does not make any specific limitation.

[0123] In addition, the preset convergence model can be that the loss value of the (r+1)th training round (based on the preset loss function, the aggregate model parameters of the (r+1)th round, and the aggregate model parameters of the rth round) is not greater than the preset loss threshold, or it can be that the difference between the loss value of the (r+1)th training round and the loss value of the rth round is less than the preset stabilization threshold. This embodiment does not make specific limitations.

[0124] It should be noted that, based on the above aggregation method, the following can be concluded: This aggregation method is similar to importance sampling. That is, the updated models from each target participant (i.e., the parameters of the models to be aggregated obtained through local training by each target participant) are re-weighted and aggregated (e.g., ...). This ensures that the final aggregated model parameters are as unbiased as possible compared to those of each candidate participant in each round of training. In traditional sampling methods... The participation level in the embodiments of this application is independent, therefore Where N is the number of alternative participants.

[0125] Existing federated learning incentive mechanisms primarily determine a fixed subset of participants in each round based on the amount of user data and computational or communication resources. This approach fails to guarantee convergence for a given objective; alternatively, a full-participation design violates the system heterogeneity inherent in federated learning. The embodiments described in this application comprehensively consider the model purchaser's cost, the amount of local training data provided by the data provider, computational costs, and / or communication costs. A Stackelberg game is used to calculate the target participation level under the target pricing strategy. Data providers choose whether to participate in each round of training based on their participation level, ensuring that a unequal number of target participants randomly participate in each round. This invention conforms to system heterogeneity, simulating situations where users may be unable to participate in training in a timely manner due to network resource limitations. Compared to traditional uniform sampling and sampling based on user data volume weights, it achieves faster and better convergence. Furthermore, the aforementioned incentive design effectively measures the contribution of user data characteristics (such as data volume and data distribution) to model performance and obtains a new convergence constraint based on an imbalanced and non-independent identically distributed dataset at the user end. The present invention was tested on a computer device using a real dataset. The experimental results show that the present invention not only allows model buyers to obtain higher performance models, but also brings higher profits to data providers.

[0126] In addition, it should be added that, firstly, different upper bounds of convergence can be used in the aforementioned target problem, such as other convex or non-convex upper bounds of convergence; secondly, other machine learning constraints, such as non-convex constraints, can be adopted, and this application does not impose specific restrictions.

[0127] Please see Figure 2 This application provides a computer device, including:

[0128] The determining unit 201 is used to determine the user utility function of each candidate participant and the service utility function of the aggregation server based on the expected participation level of each candidate participant;

[0129] The determination unit 201 is also used to determine the objective optimization problem corresponding to the service utility function and the user utility function based on the Stackelberg game model;

[0130] Solver 202 is used to solve the objective optimization problem, obtain the objective participation level of each candidate participant and the objective incentive function corresponding to the objective participation level, and achieve the Stackelberg equilibrium under the objective participation level.

[0131] Training unit 203 is used to perform federated learning based on the target participants in each round of training until the model converges to obtain the target model; the target participants in each round of training are determined by random sampling of multiple candidate participants according to the target participation level, and the target model is obtained by aggregating the actual participation levels of the target participants in each round of training.

[0132] The incentive unit 204 is used to calculate the learning incentive for each target participant according to the corresponding target incentive function.

[0133] In one specific implementation, the determining unit 201 is specifically used to calculate the service utility function of the aggregation server based on the expected participation level of each candidate participant and a preset loss function;

[0134] Send a preset incentive function to each candidate participant;

[0135] Receive the user utility function sent by each candidate participant. The user utility function is constructed by the candidate participant based on the expected participation level, the preset incentive function and the preset cost function.

[0136] In one specific implementation, the user utility function is constructed by the candidate participant based on the expected participation level, a preset incentive function, and a preset cost function, including:

[0137] The user utility function for each candidate participant is constructed according to the following formula;

[0138] U c (q n ,P n ) = P n q n -C n

[0139] Among them, C n Let q be the preset cost function for the nth candidate participant. n Let P be the expected participation level of the nth candidate participant. n U is the preset incentive function for the nth candidate participant. c (q n ,P n ) is the nth alternative participant in q n and P n The user utility function below.

[0140] In one specific implementation, the determining unit 201 is specifically used to construct the service utility function of the aggregation server according to the following formula;

[0141]

[0142] Where P is the set of preset incentive functions for each target participant, q is the set of expected participation levels for each target participant, F is the preset loss function, and w r Let r be the aggregation model parameters. U represents the expected value. s This is the service utility function of the aggregation server under P and q.

[0143] In one specific implementation, the unit 201 is specifically used to perform mathematical derivation based on the Stackelberg game model, the service utility function, and the user utility function of each backup server, to obtain the following objective optimization problem;

[0144]

[0145] Among them, a n q represents the proportion of the training data of the nth candidate participant to the sum of the training data of all other candidate participants. n G represents the expected participation level of the nth alternative participant. n Let F represent the local gradient of the nth candidate participant, where α and β are system parameters. * This represents the minimum value of the global objective loss function.

[0146] In one specific implementation, the solution unit 202 is specifically used to solve the objective optimization problem under the constraints of the expected cost and the preset maximum participation level of each candidate participant, and to calculate the target participation level of each candidate participant and the target incentive function corresponding to the target participation level.

[0147] In one specific implementation, the training unit 203 is specifically used to send the aggregated model parameters of the rth round to each target participant participating in the (r+1)th round of training;

[0148] The parameters of the aggregated model trained in the (r+1)th round are calculated using the following formula;

[0149]

[0150] Among them, w r+1 w represents the aggregation model parameters in round r+1. r Let S(q) represent the aggregation model parameters in the r-th round. r Let a represent the set of target participants in round r. n q represents the proportion of the training data of the nth candidate participant to the total training data. n ′ represents the actual participation level of the nth candidate participant. Indicates that the nth alternative participant is based on w r The parameters of the model to be aggregated in the (r+1)th round obtained from local training;

[0151] If w r+1 If the preset convergence condition is met, then based on w r+1 Determine the target model.

[0152] Figure 3 This is a schematic diagram of a computer device structure provided in an embodiment of this application. The computer device 300 may include one or more central processing units (CPUs) 301 and a memory 305, in which one or more application programs or data are stored.

[0153] The memory 305 can be volatile or persistent storage. The program stored in the memory 305 can include one or more modules, each module including a series of instruction operations on the computer device. Furthermore, the central processing unit 301 can be configured to communicate with the memory 305 and execute the series of instruction operations stored in the memory 305 on the computer device 300.

[0154] The computer device 300 may also include one or more power supplies 302, one or more wired or wireless network interfaces 303, one or more input / output interfaces 304, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0155] The central processing unit 301 can perform the aforementioned... Figures 1 to 2 The specific operations performed by the computer device in the illustrated embodiment will not be described in detail here.

[0156] It should be noted that although the steps in the flowcharts of the various embodiments are drawn sequentially according to the arrows, unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the various embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0159] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0161] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0162] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the unbiased federated learning incentive method based on Stackelberg game as described above.

Claims

1. A motivational method for unbiased federated learning based on Stackelberg games, characterized in that, Applied to an aggregation server, the method includes: The user utility function of each candidate participant and the service utility function of the aggregation server are determined based on the expected participation level of each candidate participant. The objective optimization problem corresponding to the service utility function and the user utility function is determined based on the Stackelberg game model, including: mathematical derivation based on the Stackelberg game model, the service utility function, and the user utility function of each candidate participant, resulting in the following objective optimization problem: ； Where N represents the total number of alternative participants, This represents the proportion of the training data of the nth candidate participant to the sum of the training data of all other candidate participants. This represents the expected participation level of the nth candidate participant. This represents the local gradient of the nth candidate participant. as well as For system parameters, The objective function is to minimize the global loss function; R represents the total number of training epochs. Solve the target optimization problem to obtain the target participation level of each candidate participant and the target incentive function corresponding to the target participation level. Under the target participation level, the user utility function and the service utility function reach Stackelberg equilibrium. Federated learning is performed based on the target participants in each round of training until the model converges to obtain the target model, including: Send the aggregated model parameters of round r to each target participant participating in round r+1 of training; The parameters of the aggregated model trained in the (r+1)th round are calculated using the following formula: ； in, This represents the aggregation model parameters in the (r+1)th round. This represents the aggregation model parameters in round r. Let r represent the set of target participants in round r. This represents the proportion of the training data of the nth candidate participant to the total training data. This represents the actual participation level of the nth candidate participant. Indicates that the nth alternative participant is based on The parameters of the model to be aggregated in the (r+1)th round obtained from local training; If the above If the preset convergence condition is met, then based on the above... Determine the target model; The target participants in each round of training are determined by random sampling of multiple candidate participants based on the target participation level, and the target model is obtained by aggregating the actual participation levels of the target participants in each round of training. The learning incentives for each target participant are calculated based on the corresponding target incentive function.

2. The method according to claim 1, characterized in that, The process of determining the user utility function of each candidate participant and the service utility function of the aggregation server based on the expected participation level of each candidate participant includes: The service utility function of the aggregation server is calculated based on the expected participation level of each candidate participant and the preset loss function. Send a preset incentive function to each of the candidate participants; The system receives a user utility function sent by each candidate participant. The user utility function is constructed by the candidate participant based on the expected participation level, a preset incentive function, and a preset cost function.

3. The method according to claim 2, characterized in that, The user utility function is constructed by the candidate participants based on their expected participation level, a preset incentive function, and a preset cost function, including: The user utility function for each candidate participant is constructed according to the following formula; ； in, Let n be the preset cost function for the nth candidate participant. Let n be the expected participation level of the nth candidate participant. This is the preset incentive function for the nth candidate participant. For the nth alternative participant as well as The user utility function below.

4. The method according to claim 3, characterized in that, The calculation of the service utility function of the aggregation server based on the expected participation level of each candidate participant and a preset loss function includes: Construct the service utility function of the aggregation server according to the following formula; ； Where P is the set of preset incentive functions for each target participant, q is the set of expected participation levels for each target participant, and F is the preset loss function. Let r be the aggregation model parameters. Indicates the expected value. Let be the service utility function of the aggregation server under P and q.

5. The method according to any one of claims 1 to 4, characterized in that, Solving the objective optimization problem to obtain the target participation level of each candidate participant and the target incentive function corresponding to the target participation level includes: Under the constraints of expected costs and the preset maximum participation level for each candidate participant, the objective optimization problem is solved to calculate the target participation level for each candidate participant and the target incentive function corresponding to the target participation level.

6. A computer device, characterized in that, include: The determining unit is used to determine the user utility function of each candidate participant and the service utility function of the aggregation server based on the expected participation level of each candidate participant; The determining unit is further configured to determine the objective optimization problem corresponding to the service utility function and the user utility function based on the Stackelberg game model, including: mathematical derivation based on the Stackelberg game model, the service utility function, and the user utility function of each candidate participant, to obtain the following objective optimization problem: ； Where N represents the total number of alternative participants, This represents the proportion of the training data of the nth candidate participant to the sum of the training data of all other candidate participants. This represents the expected participation level of the nth candidate participant. This represents the local gradient of the nth candidate participant. as well as For system parameters, The objective function is to minimize the global loss function; R represents the total number of training epochs. The solution unit is used to solve the target optimization problem, obtain the target participation level of each candidate participant and the target incentive function corresponding to the target participation level, and under the target participation level, the user utility function and the service utility function reach the Stackelberg equilibrium. Training units, used for federated learning based on the target participants in each training round until the model converges to obtain the target model, include: Send the aggregated model parameters of round r to each target participant participating in the (r+1)th round of training; The parameters of the aggregated model trained in the (r+1)th round are calculated using the following formula: ； in, This represents the aggregation model parameters in the (r+1)th round. This represents the aggregation model parameters in round r. Let r represent the set of target participants in round r. This represents the proportion of the training data of the nth candidate participant to the total training data. This represents the actual participation level of the nth candidate participant. Indicates that the nth alternative participant is based on The parameters of the model to be aggregated in the (r+1)th round obtained from local training; If the above If the preset convergence condition is met, then based on the above... Determine the target model; The target participants in each round of training are determined by random sampling of multiple candidate participants based on the target participation level, and the target model is obtained by aggregating the actual participation levels of the target participants in each round of training. The incentive unit is used to calculate the learning incentive for each of the target participants according to the corresponding target incentive function.

7. A computer device, characterized in that, include: Central processing unit, memory, and input / output interfaces; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method of any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed on the computer, cause the computer to perform the method as described in any one of claims 1 to 5.

Images