A method and apparatus for pricing and incentivizing federated learning computation tasks

Abstract

The application provides a pricing and incentive method and device for federated learning computing tasks, and relates to the technical field of federated learning. The method comprises the following steps: publishing a computing task to an edge node machine learning training model and paying a task reward; establishing an evaluation model of data contribution according to data collected by the edge node, calculating the data contribution of the edge node, and giving a corresponding reward according to the degree of data contribution; establishing a Stackelberg game model for the problem, establishing corresponding utility functions for the central parameter server and the edge node in a complete information scenario, and solving Nash equilibrium and Stackelberg equilibrium by using a backward induction method; and in an incomplete information scenario, using multi-agent deep reinforcement learning technology to solve the pricing strategy of the central parameter server and the training strategy of the edge node. The device executes the above method. The method and device provided in the embodiments of the application can improve the model training efficiency.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and more specifically to a pricing and incentive method and apparatus for federated learning computing tasks. Background Technology

[0002] With the rapid development of IoT technology and its widespread application, IoT data is growing rapidly. Machine learning models are built upon this data to predict unknown events. In recent years, federated learning, as a novel distributed machine learning model, has attracted widespread attention. Federated learning allows many edge nodes to collaboratively train a learning model using their own unique datasets, while ensuring that this data is not disclosed to others. Federated learning demonstrates strong advantages in supporting collaborative training and protecting privacy; ensuring user privacy while incentivizing more efficient model training is also crucial.

[0003] Edge-based federated learning systems offer significant advantages in protecting data privacy and collaborative training. Existing centralized model training consumes substantial resources and cannot guarantee data security. Federated learning emerged to address these issues. A typical federated learning system consists of a central parameter server and a set of edge nodes. The central parameter server determines the global model architecture and initializes its parameters, then sends these parameters to the edge nodes. Edge nodes collect data on IoT devices as local datasets and collaboratively train the model without sharing the raw data. They send updated model parameters to the central parameter server, which aggregates these parameters and then sends the updated model parameters back to the edge nodes.

[0004] Therefore, incentivizing edge nodes to participate in federated learning for efficient model training is a challenge. Throughout the training process, the central parameter server dominates the payment process for computational tasks, but edge nodes can also manipulate local training, thus influencing the central parameter server's decisions. Simultaneously, edge nodes compete with each other for rewards from the central parameter server. In federated learning, many data owners may not actively participate in training the shared model, especially when the data owner is a business rather than an individual. Participants often receive some income when using their local datasets in federated learning. Therefore, it is necessary to evaluate the contributions of different data providers to appropriately distribute the profits obtained from the learning system. Reasonable contribution evaluation criteria can further attract more participants to join the incentive mechanism. This requires accurately determining the data contributions of edge nodes, enabling them to more efficiently iterate and train the model locally to obtain corresponding rewards. Summary of the Invention

[0005] To address the problems in the prior art, embodiments of the present invention provide a pricing and incentive method and apparatus for federated learning computing tasks, which can at least partially solve the problems existing in the prior art.

[0006] On the one hand, this invention proposes a pricing and incentive method for federated learning computing tasks, including:

[0007] The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally.

[0008] An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution.

[0009] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction.

[0010] In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0011] The step of establishing a data contribution evaluation model based on data collected from edge nodes and calculating the data contribution of edge nodes includes:

[0012] Data quality is determined based on the number of iterations during local training and the local accuracy achieved after local training is completed.

[0013] The data contribution is determined based on the data quality and the number of data samples in the local dataset.

[0014] The utility function U(τ) for the central parameter server is established according to the following formula:

[0015]

[0016] Where ω>0 represents the conversion parameter from model performance to profit, in the three-parameter power-law model (a-bx). -c This represents the model's performance, defined as a value between 0 and 1, where a = 1, b, and c > 0. This represents the data contribution; x n ε represents the number of data samples in the local dataset of the nth edge node. n This represents the local accuracy achieved by the nth edge node after local training is complete. The coefficient representing the number of iterations of the local model affected by the accuracy of local data, N represents the total number of edge nodes, and τ represents the total reward given by the central parameter server.

[0017] The utility function for edge nodes is established according to the following formula.

[0018]

[0019] in, The computational cost required for an edge node to perform one iteration of training locally. The communication cost of uploading model parameters from edge nodes to the central parameter server.

[0020] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and edge nodes, respectively. The Nash equilibrium and Stackelberg equilibrium are solved using a method of back induction, including:

[0021] The problem is modeled as a Stackelberg game, where the central parameter server sets the pricing for the leader's tasks and incentivizes each edge node, while the edge nodes undertake the computational training tasks for federated learning for the followers.

[0022] Each edge node determines the local iteration count;

[0023] Given any incentive value, solve for the Nash equilibrium, find the first and second derivatives of the utility function corresponding to the edge node, prove that it is a convex function, and solve for the optimal training strategy when the utility of the edge node is maximized by setting the first derivative to 0.

[0024] Using backward induction, the optimal training strategy is substituted into the utility function corresponding to the central parameter server, and the first and second derivatives are calculated to prove that it is a convex function. The optimal payment strategy when the utility of the central parameter server is maximized is obtained by setting the first derivative to 0.

[0025] The method of using multi-agent deep reinforcement learning to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes in scenarios with incomplete information includes:

[0026] Initialize the Actor-Critic network parameters for the central parameter server and edge nodes, as well as the replay experience buffer pool;

[0027] Initialize the state of the central parameter server and edge nodes;

[0028] The central parameter server generates actions based on the current state using a policy function and receives rewards based on a reward function.

[0029] Edge nodes generate corresponding actions and receive rewards based on their current state;

[0030] If it is determined that the edge node has reached the corresponding first termination condition, the central parameter server generates an action based on the current state using the strategy function and obtains a reward based on the reward function; if it is determined that the edge node has not reached the corresponding first termination condition, the edge node generates a corresponding action based on the current state and obtains a reward.

[0031] If it is determined that the central parameter server has reached the corresponding second termination condition, then the initialization of the state of the central parameter server and the edge nodes is performed; if it is determined that the central parameter server has not reached the corresponding second termination condition, then the central parameter server generates an action based on the current state using the policy function and obtains a reward based on the reward function.

[0032] On one hand, the present invention proposes a pricing and incentive device for federated learning computing tasks, comprising:

[0033] The publishing unit is used to publish computing tasks to edge nodes for machine learning training models and pay task rewards, so that edge nodes can collect data and perform iterative training locally.

[0034] The computing unit is used to build an evaluation model of data contribution based on the data collected by the edge nodes, calculate the data contribution of the edge nodes, and give corresponding rewards according to the degree of data contribution.

[0035] The first solution unit is used to establish the problem as a Stackelberg game model. In the scenario of complete information, it establishes corresponding utility functions for the central parameter server and edge nodes respectively, and uses the method of backward induction to solve the Nash equilibrium and Stackelberg equilibrium.

[0036] The second solving unit is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes in scenarios with incomplete information using multi-agent deep reinforcement learning techniques.

[0037] In another aspect, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the following method:

[0038] The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally.

[0039] An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution.

[0040] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction.

[0041] In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0042] This invention provides a computer-readable storage medium, comprising:

[0043] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the following method:

[0044] The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally.

[0045] An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution.

[0046] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction.

[0047] In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0048] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the following method:

[0049] The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally.

[0050] An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution.

[0051] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction.

[0052] In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0053] The pricing and incentive method and apparatus for federated learning computing tasks provided in this invention publishes computing tasks and pays task rewards to edge node machine learning training models, enabling edge nodes to collect data and perform iterative training locally. An evaluation model for data contribution is established based on the data collected by the edge nodes, calculating the data contribution of the edge nodes and providing corresponding rewards based on the degree of data contribution. The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and edge nodes respectively, and the Nash equilibrium and Stackelberg equilibrium are solved using inverse induction. In the scenario of incomplete information, multi-agent deep reinforcement learning technology is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes, which can improve the model training efficiency. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0055] Figure 1 This is a flowchart illustrating a pricing and incentive method for federated learning computing tasks provided in an embodiment of the present invention.

[0056] Figure 2 This is a flowchart illustrating a pricing and incentive method for federated learning computing tasks provided in another embodiment of the present invention.

[0057] Figure 3 This is a schematic diagram of the pricing and incentive device for federated learning computing tasks provided in an embodiment of the present invention.

[0058] Figure 4 A schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0060] Figure 1 This is a flowchart illustrating a pricing and incentive method for federated learning computing tasks according to an embodiment of the present invention, as shown below. Figure 1 As shown, the pricing and incentive method for federated learning computing tasks provided in this embodiment of the invention includes:

[0061] Step S1: Issue computational tasks to the edge node machine learning training model and pay task rewards so that the edge node can collect data and perform iterative training locally.

[0062] Step S2: Establish a data contribution evaluation model based on the data collected by the edge nodes, calculate the data contribution of the edge nodes, and give corresponding rewards according to the degree of data contribution.

[0063] Step S3: Establish the problem as a Stackelberg game model. In the scenario of complete information, establish corresponding utility functions for the central parameter server and edge nodes respectively, and use the method of backward induction to solve for the Nash equilibrium and Stackelberg equilibrium.

[0064] Step S4: In scenarios with incomplete information, use multi-agent deep reinforcement learning techniques to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0065] In step S1 above, the device issues computational tasks and pays task rewards to the edge node machine learning training model, enabling the edge node to collect data and perform iterative training locally. The device can be a computer device, etc., executing this method. The acquisition, storage, use, and processing of data in this application's technical solution all comply with relevant regulations. Figure 2 As shown, in a federated learning environment, there is a central parameter server responsible for issuing computation tasks and determining task prices, and a set of edge nodes responsible for local training. This system considers a single computation task process, which is quasi-static, meaning no edge nodes join or leave.

[0066] Edge nodes are represented by a set N:

[0067] N = {e1,e2,…,e N}

[0068] Where N represents the total number of edge nodes in the system, e i Let i represent the i-th edge node in the system, satisfying 1≤i≤N.

[0069] The central parameter server resides in the cloud. It broadcasts computational tasks to the edge nodes in the system, which involve passing model parameters to a group of edge nodes and paying rewards τ to incentivize them to participate in model training. nData collected from IoT devices as a local dataset X n X n Includes x n Data samples, of which m≠n. Generally speaking, x n >0, if x n A value of 0 indicates that the edge node does not participate in the federated learning training process. Edge nodes use their local private dataset to perform iterative training locally, participate in federated learning, and receive rewards.

[0070] Edge nodes undergo iterative training locally. Due to the varying number of iterations, the data quality differs, resulting in varying model accuracy. Fewer local iterations lead to higher local accuracy ε. n That is, given the local precision of the target, the edge node e n The number of iterations for local training can be represented by the following formula:

[0071]

[0072] Where, ε n This represents the local accuracy achieved after local training is completed.

[0073] In step S2 above, the device establishes a data contribution evaluation model based on the data collected by the edge nodes, calculates the data contribution of the edge nodes, and gives corresponding rewards according to the degree of data contribution.

[0074] The step of establishing a data contribution evaluation model based on data collected from edge nodes and calculating the data contribution of edge nodes includes:

[0075] Data quality is determined based on the number of iterations during local training and the local accuracy achieved after local training; data quality θ n It can be expressed as follows:

[0076]

[0077] in It is a coefficient representing the number of iterations of the local model, which is affected by the accuracy of local data.

[0078] The data contribution is determined based on the data quality and the number of data samples in the local dataset. Data contribution φ n It can be expressed as follows:

[0079]

[0080] x n This indicates the number of data samples in the local dataset.

[0081] The more contributions an edge node makes, the higher its reward will be, and the reward is directly proportional to the payment τ from the central parameter server.

[0082] In step S3 above, the device establishes the problem as a Stackelberg game model. Under the scenario of complete information, it establishes corresponding utility functions for the central parameter server and edge nodes respectively, and uses the method of backward induction to solve for the Nash equilibrium and Stackelberg equilibrium.

[0083] The utility function U(τ) for the central parameter server is established according to the following formula:

[0084]

[0085] Where ω>0 represents the conversion parameter from model performance to profit, in the three-parameter power-law model (a-bx). -c This represents the model's performance, defined as a value between 0 and 1, where a = 1, b, and c > 0. This represents the data contribution; x n ε represents the number of data samples in the local dataset of the nth edge node. n This represents the local accuracy achieved by the nth edge node after local training is complete. The coefficient representing the number of iterations of the local model affected by the accuracy of local data, N represents the total number of edge nodes, and τ represents the total reward given by the central parameter server.

[0086] Establish a utility function for edge nodes based on the following formula.

[0087]

[0088] in, The computational cost required for an edge node to perform one iteration of training locally. The communication cost of uploading model parameters from edge nodes to the central parameter server.

[0089] Edge nodes require multiple iterations when training the model locally, and the computational resources consumed in one iteration, i.e., the CPU cycle frequency, are defined as f. n The number of CPU cycles required for edge nodes to train a local model using a single data sample is f. n Therefore, the time required for one local iteration at an edge node is:

[0090]

[0091] The computational cost of one local iteration is:

[0092]

[0093] Where κ represents the edge node e n The effective capacitance parameters of the chipset are calculated.

[0094] Therefore, edge node iteration I n (ε n The total computational cost for this iteration is:

[0095]

[0096] After local model training, all users upload their local models to the central parameter server via Frequency Division Multiple Access (FDMA). The location of the data owner remains constant when feeding back local model parameters. The transmission rate of the edge nodes is expressed as:

[0097]

[0098] Where B is the transmission bandwidth, ρ n It is the edge node e n The transmission power, h n It is the edge node e n The channel gain of the peer-to-peer link with the central parameter server, and N0 is the background noise.

[0099] The data for local model updates is a constant δ with the same value for all edge nodes. Therefore, the transmission time for local model updates is:

[0100]

[0101] Edge node e n The communication cost of transmitting local model parameters in global iteration is expressed as:

[0102]

[0103] The problem is modeled as a Stackelberg game. In a scenario with complete information, corresponding utility functions are established for the central parameter server and edge nodes. The Nash equilibrium and Stackelberg equilibrium are solved using inverse induction, including:

[0104] The problem is modeled as a Stackelberg game, where the central parameter server sets the pricing for the leader's tasks and incentivizes each edge node, while the edge nodes undertake the computational training tasks for federated learning for the followers.

[0105] Each edge node determines its local iteration count. The mechanism is divided into two phases: in the first phase, the central parameter server announces the total reward, and in the second phase, each edge node determines its own training strategy, i.e., determines its local iteration count, in order to maximize its own benefits.

[0106] Given arbitrary incentive values, solve for the Nash equilibrium, calculate the first and second derivatives of the utility function corresponding to the edge nodes, prove that it is a convex function, and solve for the optimal training strategy when the edge node utility is maximized by setting the first derivative to 0; given arbitrary τ values, solve for the Nash equilibrium in the second stage. Solve for the first and second derivatives of the utility function of edge nodes, prove that it is a convex function, and solve for the optimal training strategy when the utility of edge nodes is maximized by setting the first derivative to 0.

[0107] Using backward induction, the optimal training strategy is substituted into the utility function corresponding to the central parameter server, and the first and second derivatives are calculated to prove that it is a convex function. The optimal payment strategy when the utility of the central parameter server is maximized is obtained by setting the first derivative to 0.

[0108] To study the Nash equilibrium in the second phase of the game, the following derivation is needed. about First derivative:

[0109]

[0110] based on The first derivative is used to derive... about The second derivative:

[0111]

[0112] Since the data contribution from edge nodes is positive, the derivative can be directly observed. about The second derivative is less than 0, meaning the utility function is a continuous convex function.

[0113] Because it is necessary to guarantee U n ≥0, therefore: Therefore, the policy set is closed, bounded, and convex; and since the set of marginal nodes is finite, the utility function is a continuous convex function in the policy space, indicating that a Nash equilibrium exists in the second stage of the game. Let the first derivative... Right now:

[0114]

[0115] Solving the above equation, we get:

[0116]

[0117] Through a series of transformations, we can obtain...

[0118]

[0119] Where M is the number of edge nodes participating in the game.

[0120] The central parameter server understands that a unique Nash equilibrium exists among all edge nodes for any given value of τ. Therefore, it can choose to optimize τ to maximize its utility.

[0121] The first derivative with respect to is:

[0122]

[0123] Therefore, the second derivative of U(τ) is:

[0124]

[0125] It is obvious that the second derivative of U(τ) can be observed. Therefore, the utility function of the central parameter server is a strictly convex function on τ∈[0,∞). When τ=0, U(τ)=0; when τ→∞, U(τ→∞). Therefore, it has a unique decision τ. * To maximize U(τ), let the first derivative... Find τ * :

[0126]

[0127] In a scenario with complete information, after observing the optimal training strategy of each edge node in the second stage of a non-cooperative game, the central parameter server can determine the optimal payoff strategy to ensure that the game has a unique Stackelberg equilibrium.

[0128] <I * ,τ * >

[0129] In step S4 above, the device uses multi-agent deep reinforcement learning technology to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes in the case of incomplete information.

[0130] In scenarios with incomplete information, edge nodes do not share information with each other, including dataset size, local model training accuracy, training strategies, computational costs, and communication costs. Therefore, multi-agent deep reinforcement learning methods are used to address these issues, with the MADDPG algorithm being a suitable implementation.

[0131] The central parameter server and edge nodes act as agents. The central parameter server acts as a leader interacting with the environment. During each training process, the central parameter server agent observes the state and makes decisions. The central parameter server agent can only observe historical payment strategies and the past training strategy history of the edge nodes. The state space of the central parameter server is:

[0132] s t ={I t-L ,τ t-L ,I t-L+1 ,τ t-L+1 ,...,I t-1 ,τ t-1}

[0133] Where I t-L τ represents the training strategy history of the edge node in the previous L steps. t-L The payment strategy history represents the previous L steps of the central parameter server;

[0134] The action of the central parameter server is τ t ∈[0,+∞), the action space of the central parameter server is continuous, and its output of a specific floating-point number represents a deterministic strategy;

[0135] Central parameter server executes action τ t Afterwards, the central parameter server agent receives rewards r from the environment. τ :

[0136] r τ =U(τ) t )

[0137] That is, the central parameter server takes the current action τ. t The utility value at that time.

[0138] In each training time slot, the edge nodes determine their optimal training policy based on the current payoff determined by the central parameter server. However, due to a lack of information from other edge nodes, each edge node employs an offline training method, learning its optimal policy in a simulated game environment until a Nash equilibrium is reached. Each edge node agent can observe the training policy history of other nodes and the current payoff policy of the central parameter server. The state space of the edge nodes is:

[0139]

[0140] in τ represents the training policy history of other edge nodes L steps prior to this time slot. t This indicates the action of the current time slot central parameter server;

[0141] The actions of the edge nodes are:

[0142]

[0143] Where R n This indicates the upper limit of the number of local iterations, and the action space of edge nodes is also continuous.

[0144] In the current training process, during the k-th game iteration, the edge node executes an action. When that happens, it will receive a corresponding reward.

[0145]

[0146] That is, the current payment τ is made at the central parameter server. t Edge nodes perform actions The utility value at that time.

[0147] The method of using multi-agent deep reinforcement learning to solve the pricing strategy of the central parameter server and the training strategy of edge nodes in scenarios with incomplete information includes:

[0148] Initialize the Actor-Critic network parameters for the central parameter server and edge nodes, as well as the replay experience buffer pool;

[0149] Initialize the state of the central parameter server and edge nodes;

[0150] The central parameter server generates actions based on the current state using a policy function and receives rewards based on a reward function.

[0151] Edge nodes generate corresponding actions and receive rewards based on their current state;

[0152] If it is determined that the edge node has reached the corresponding first termination condition, the central parameter server generates an action based on the current state using a strategy function and obtains a reward based on the reward function; if it is determined that the edge node has not reached the corresponding first termination condition, the edge node generates a corresponding action based on the current state and obtains a reward; the first termination condition may be reaching a predetermined number of rounds.

[0153] If the central parameter server is determined to have met the corresponding second termination condition, then the initialization of the central parameter server and edge node states is performed; if the central parameter server is determined not to have met the corresponding second termination condition, then the central parameter server generates an action based on the current state using a policy function and receives a reward based on a reward function. The second termination condition is the same as the first termination condition, or it can be reaching a predetermined number of rounds.

[0154] The pricing and incentive method for federated learning computation tasks provided in this invention evaluates the data contribution of edge nodes and designs a reasonable and fair incentive mechanism for them. In scenarios with complete information, the pricing decisions of the central parameter server and the training decisions of the edge nodes are established as a Stackelberg game process, designing utility functions with Nash and Stackelberg equilibria. In scenarios with incomplete information, a Markov decision process is established, and the optimal strategy is solved using a multi-agent deep reinforcement learning algorithm. The central parameter server and edge nodes learn the optimal decisions from past historical experience without needing prior knowledge of other edge nodes. Therefore, while protecting the privacy of edge nodes, it can incentivize them to perform federated learning training more efficiently. This method is highly automated, easy to implement, and applicable to complex federated learning systems.

[0155] The pricing and incentive method for federated learning computing tasks provided in this invention publishes computing tasks and pays task rewards to edge node machine learning training models, enabling edge nodes to collect data and perform iterative training locally. An evaluation model for data contribution is established based on the data collected by the edge nodes, calculating the data contribution of each edge node and providing corresponding rewards based on the degree of contribution. The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and edge nodes respectively, and the Nash equilibrium and Stackelberg equilibrium are solved using inverse induction. In the scenario of incomplete information, multi-agent deep reinforcement learning technology is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes, which can improve the model training efficiency.

[0156] Furthermore, the step of establishing a data contribution evaluation model based on the data collected by the edge nodes and calculating the data contribution of the edge nodes includes:

[0157] Data quality is determined based on the number of iterations during local training and the local accuracy achieved after local training is completed; this can be referred to the above embodiments for explanation, and will not be repeated here.

[0158] The data contribution is determined based on the data quality and the number of data samples in the local dataset. This can be referred to the above embodiments for further explanation, and will not be repeated here.

[0159] Furthermore, a utility function U(τ) is established for the central parameter server according to the following formula:

[0160]

[0161] Where ω>0 represents the conversion parameter from model performance to profit, in the three-parameter power-law model (a-bx). -cThis represents the model's performance, defined as a value between 0 and 1, where a = 1, b, and c > 0. This represents the data contribution; x n ε represents the number of data samples in the local dataset of the nth edge node. n This represents the local accuracy achieved by the nth edge node after local training is complete. The coefficient represents the number of iterations of the local model affected by the accuracy of local data, N represents the total number of edge nodes, and τ represents the total reward given by the central parameter server. Refer to the above embodiment for further explanation; details will not be repeated here.

[0162] Furthermore, a utility function is established for the edge nodes according to the following formula.

[0163]

[0164] in, The computational cost required for an edge node to perform one iteration of training locally. The communication cost for edge nodes to upload model parameters to the central parameter server. This can be referred to the above embodiment for explanation, and will not be repeated here.

[0165] Furthermore, the problem is established as a Stackelberg game model. Under the scenario of complete information, corresponding utility functions are established for the central parameter server and edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved using a method of back induction, including:

[0166] The problem is modeled as a Stackelberg game, where the central parameter server sets the pricing for the leader's tasks and incentivizes each edge node, while the edge nodes undertake the computational training tasks for federated learning for the followers; the above embodiments can be referred to for further explanation, and will not be repeated here.

[0167] Each edge node determines the number of local iterations; this can be referred to the above embodiments for explanation, and will not be repeated here.

[0168] Given any incentive value, solve for the Nash equilibrium, calculate the first and second derivatives of the utility function corresponding to the edge node, prove that it is a convex function, and solve for the optimal training strategy when the utility of the edge node is maximized by setting the first derivative to 0; the above embodiments can be referred to for explanation, and will not be repeated here.

[0169] Using backward induction, the optimal training strategy is substituted into the utility function corresponding to the central parameter server, and its first and second derivatives are calculated to prove that it is a convex function. Setting the first derivative to zero yields the optimal payment strategy when the utility of the central parameter server is maximized. This can be referred to the above embodiments for further explanation and will not be repeated here.

[0170] Furthermore, the method of using multi-agent deep reinforcement learning to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes in scenarios with incomplete information includes:

[0171] Initialize the Actor-Critic network parameters of the central parameter server and edge nodes, as well as the replay experience buffer pool; refer to the above embodiments for explanation, and will not be repeated here.

[0172] Initialize the state of the central parameter server and edge nodes; refer to the above embodiments for details, which will not be repeated here.

[0173] The central parameter server generates actions based on the current state using a policy function and obtains rewards based on a reward function; this can be referred to the above embodiment for explanation, and will not be repeated here.

[0174] Edge nodes generate corresponding actions and receive rewards based on their current state; this can be referred to the above embodiments for explanation, and will not be repeated here.

[0175] If it is determined that the edge node has reached the corresponding first termination condition, the central parameter server generates an action based on the current state using the strategy function and obtains a reward based on the reward function; if it is determined that the edge node has not reached the corresponding first termination condition, the edge node generates a corresponding action based on the current state and obtains a reward; the above embodiments can be referred to for explanation, and will not be repeated here.

[0176] If it is determined that the central parameter server has met the corresponding second termination condition, then the initialization of the central parameter server and edge node states is performed; if it is determined that the central parameter server has not met the corresponding second termination condition, then the central parameter server generates an action based on the current state using a policy function and obtains a reward based on the reward function. This can be referred to the above embodiments for explanation, and will not be repeated here.

[0177] Figure 3 This is a schematic diagram of the pricing and incentive device for federated learning computing tasks provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the pricing and incentive device for federated learning computing tasks provided in this embodiment of the invention includes a publishing unit 301, a computing unit 302, a first solving unit 303, and a second solving unit 304, wherein:

[0178] The publishing unit 301 is used to publish computing tasks and pay task rewards to the machine learning training model of edge nodes, so that edge nodes can collect data and perform iterative training locally; the computing unit 302 is used to build an evaluation model of data contribution based on the data collected by edge nodes, calculate the data contribution of edge nodes, and give corresponding rewards according to the degree of data contribution; the first solving unit 303 is used to establish the problem as a Stackelberg game model, and in the scenario of complete information, establish corresponding utility functions for the central parameter server and edge nodes respectively, and solve the Nash equilibrium and Stackelberg equilibrium by using the method of inverse induction; the second solving unit 304 is used to solve the pricing strategy of the central parameter server and the training strategy of edge nodes by using multi-agent deep reinforcement learning technology in the scenario of incomplete information.

[0179] Specifically, the publishing unit 301 in the device is used to publish computing tasks and pay task rewards to the machine learning training model of the edge nodes, so that the edge nodes can collect data and perform iterative training locally; the computing unit 302 is used to build an evaluation model of data contribution based on the data collected by the edge nodes, calculate the data contribution of the edge nodes, and give corresponding rewards according to the degree of data contribution; the first solving unit 303 is used to build the problem into a Stackelberg game model, and in the scenario of complete information, establish corresponding utility functions for the central parameter server and the edge nodes respectively, and solve the Nash equilibrium and Stackelberg equilibrium by using the method of inverse induction; the second solving unit 304 is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes by using multi-agent deep reinforcement learning technology in the scenario of incomplete information.

[0180] The pricing and incentive device for federated learning computing tasks provided in this invention publishes computing tasks and pays task rewards to edge node machine learning training models, enabling edge nodes to collect data and perform iterative training locally. It establishes a data contribution evaluation model based on the data collected by edge nodes, calculates the data contribution of edge nodes, and provides corresponding rewards based on the degree of data contribution. The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and edge nodes respectively, and the Nash equilibrium and Stackelberg equilibrium are solved using inverse induction. In the scenario of incomplete information, multi-agent deep reinforcement learning technology is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes, which can improve the model training efficiency.

[0181] The embodiments of the present invention provide a pricing and incentive device for federated learning computing tasks, which can be used to execute the processing flow of the above-described method embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above-described method embodiments.

[0182] Figure 4 This is a schematic diagram of the physical structure of a computer device provided in an embodiment of the present invention, such as... Figure 4 As shown, the computer device includes: a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402. When the processor 402 executes the computer program, it implements the following method:

[0183] The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally.

[0184] An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution.

[0185] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction.

[0186] In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0187] This embodiment discloses a computer program product, which includes a computer program that, when executed by a processor, implements the following method:

[0188] The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally.

[0189] An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution.

[0190] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction.

[0191] In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0192] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the following method:

[0193] The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally.

[0194] An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution.

[0195] The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction.

[0196] In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes.

[0197] Compared with existing technical solutions, the pricing and incentive method for federated learning computing tasks provided in this invention publishes computing tasks and pays task rewards to edge node machine learning training models, enabling edge nodes to collect data and perform iterative training locally. It establishes a data contribution evaluation model based on the data collected by edge nodes, calculates the data contribution of edge nodes, and provides corresponding rewards based on the degree of data contribution. The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and edge nodes respectively, and the Nash equilibrium and Stackelberg equilibrium are solved using inverse induction. In the scenario of incomplete information, multi-agent deep reinforcement learning technology is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes, which can improve model training efficiency.

[0198] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0200] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0201] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0202] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0203] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A pricing and incentive method for federated learning computing tasks, characterized in that, include: The system issues computational tasks to edge nodes for training machine learning models and pays task rewards so that edge nodes can collect data and perform iterative training locally. An evaluation model for data contribution is established based on the data collected by edge nodes, the data contribution of edge nodes is calculated, and corresponding rewards are given according to the degree of data contribution. The problem is established as a Stackelberg game model. In the scenario of complete information, corresponding utility functions are established for the central parameter server and the edge nodes respectively. The Nash equilibrium and Stackelberg equilibrium are solved by the method of backward induction. In scenarios with incomplete information, a multi-agent deep reinforcement learning technique is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes. The step of establishing a data contribution evaluation model based on data collected from edge nodes and calculating the data contribution of edge nodes includes: Data quality is determined based on the number of iterations during local training and the local accuracy achieved after local training is completed. The data contribution is determined based on the data quality and the number of data samples in the local dataset; Establish a utility function for the central parameter server based on the following formula. : ； in, The parameters representing the transformation from model performance to profit are from a three-parameter power-law model. This represents the model performance, and performance is defined as follows: The middle value, , This refers to data contribution; This represents the number of data samples in the local dataset of the nth edge node. This represents the local accuracy achieved by the nth edge node after local training is complete. The coefficient representing the number of local model iterations affected by the accuracy of local data, where N represents the total number of edge nodes. This represents the total reward given by the central parameter server.

2. The pricing and incentive method for federated learning computing tasks according to claim 1, characterized in that, Establish a utility function for edge nodes based on the following formula. : ； in, The computational cost required for an edge node to perform one iteration of training locally. The communication cost of uploading model parameters from edge nodes to the central parameter server.

3. The pricing and incentive method for federated learning computing tasks according to claim 2, characterized in that, The problem is modeled as a Stackelberg game. In a scenario with complete information, corresponding utility functions are established for the central parameter server and edge nodes. The Nash equilibrium and Stackelberg equilibrium are solved using inverse induction, including: The problem is modeled as a Stackelberg game, where the central parameter server sets the pricing for the leader's tasks and incentivizes each edge node, while the edge nodes undertake the computational training tasks for federated learning for the followers. Each edge node determines the local iteration count; Given any incentive value, solve for the Nash equilibrium, find the first and second derivatives of the utility function corresponding to the edge node, prove that it is a convex function, and solve for the optimal training strategy when the utility of the edge node is maximized by setting the first derivative to 0. Using backward induction, the optimal training strategy is substituted into the utility function corresponding to the central parameter server, and the first and second derivatives are calculated to prove that it is a convex function. The optimal payment strategy when the utility of the central parameter server is maximized is obtained by setting the first derivative to 0.

4. The pricing and incentive method for federated learning computing tasks according to claim 3, characterized in that, The method of using multi-agent deep reinforcement learning to solve the pricing strategy of the central parameter server and the training strategy of edge nodes in scenarios with incomplete information includes: Initialize the Actor-Critic network parameters for the central parameter server and edge nodes, as well as the replay experience buffer pool; Initialize the state of the central parameter server and edge nodes; The central parameter server generates actions based on the current state using a policy function and receives rewards based on a reward function. Edge nodes generate corresponding actions and receive rewards based on their current state; If it is determined that the edge node has reached the corresponding first termination condition, the central parameter server generates an action based on the current state using a strategy function and obtains a reward based on the reward function; if it is determined that the edge node has not reached the corresponding first termination condition, the edge node generates a corresponding action based on the current state and obtains a reward. If it is determined that the central parameter server has reached the corresponding second termination condition, then the initialization of the state of the central parameter server and the edge nodes is performed; if it is determined that the central parameter server has not reached the corresponding second termination condition, then the central parameter server generates an action based on the current state using the policy function and obtains a reward based on the reward function.

5. A pricing and incentive mechanism for federated learning computing tasks, characterized in that, include: The publishing unit is used to publish computing tasks to edge nodes for machine learning training models and pay task rewards, so that edge nodes can collect data and perform iterative training locally. The computing unit is used to build an evaluation model of data contribution based on the data collected by the edge nodes, calculate the data contribution of the edge nodes, and give corresponding rewards according to the degree of data contribution. The first solution unit is used to establish the problem as a Stackelberg game model. In the scenario of complete information, it establishes corresponding utility functions for the central parameter server and edge nodes respectively, and uses the method of backward induction to solve the Nash equilibrium and Stackelberg equilibrium. The second solving unit is used to solve the pricing strategy of the central parameter server and the training strategy of the edge nodes in scenarios with incomplete information using multi-agent deep reinforcement learning techniques. The step of establishing a data contribution evaluation model based on data collected from edge nodes and calculating the data contribution of edge nodes includes: Data quality is determined based on the number of iterations during local training and the local accuracy achieved after local training is completed. The data contribution is determined based on the data quality and the number of data samples in the local dataset; Establish a utility function for the central parameter server based on the following formula. : ； in, The parameters representing the transformation from model performance to profit are from a three-parameter power-law model. This represents the model performance, and performance is defined as follows: The middle value, , This refers to data contribution; This represents the number of data samples in the local dataset of the nth edge node. This represents the local accuracy achieved by the nth edge node after local training is complete. The coefficient representing the number of local model iterations affected by the accuracy of local data, where N represents the total number of edge nodes. This represents the total reward given by the central parameter server.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 4.

Images