Layered federal model training method based on reputation awareness incentive mechanism

Abstract

The invention discloses a hierarchical federal model training method based on a reputation awareness incentive mechanism. The method comprises the steps that a cloud server CS sends a global training task to an edge server ES and transmits the global training task to a terminal device ED; calculating the reputation value of each ED according to the historical training record of each ED in the alliance block chain; the CS determines a unit service price P sent to the ES; according to the P and each ES, determining a total reward strategy; each ED selects an optimal ES according to the total reward strategy of the ES and the unit training cost and reputation value of the ED; the training data size of each ED is determined through a Stackelberg game, and the unit service price and the total reward strategy are adjusted through reverse derivation; the CS distributes rewards to the ES participating in model aggregation after model training is completed, and the ES distributes the rewards to the ED participating in the training task; the current training reputation value of each ED participating in the training task is calculated and recorded in the block chain. Therefore, the quantity and quality of data contributed by each participant in the training process are ensured, and the performance of model training is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a hierarchical federated model training method, computing device and computer storage medium based on reputation awareness incentive. Background Art

[0002] The goal of the federated learning (FL) task is to obtain a global model with the required accuracy within a certain period of time. Therefore, the model obtained by the cloud server depends on the training results of the subsequent devices.

[0003] However, in the prior art, nodes in the Hierarchical Federated Learning (HFL) platform usually only care about their own utility and training data, so it is impossible to obtain the precise accuracy of the training model. The quality of the local model uploaded by the FL participants will affect the performance of the global model. For example, in some cases, the data owner may perform malicious or unreliable local model updates to attack the platform or gain more benefits. On the one hand, malicious data owners will deliberately launch attacks, such as poisoning attacks. On the other hand, unreliable model updates are mainly caused by erroneous or forged data sets. The final global model accuracy and convergence speed will be affected by malicious or unreliable model updates. Summary of the invention

[0004] One of the purposes of the embodiments of the present invention is to provide a hierarchical federated model training method, computing device and computer storage medium based on a reputation awareness incentive mechanism, which ensures the quantity and quality of data that each participant can contribute during the model training process, thereby improving the performance and security of the model training. In addition, it can also prevent malicious participants from attacking the system.

[0005] In order to solve the above technical problems, in a first aspect, an embodiment of the present invention provides a hierarchical federated model training method based on a reputation awareness incentive mechanism, and the method specifically includes:

[0006] The cloud server CS sends the global training task to the edge server, and passes it from the edge server ES to the terminal device ED;

[0007] Based on the historical training records of each ED in the alliance blockchain, the current reputation value of each ED is calculated, and the cloud server determines the unit service price to be sent to the ES;

[0008] Based on the unit service price of the cloud server CS, each ES determines its own total reward strategy;

[0009] Each ED selects an optimal ES based on the total reward strategy released by the ES and the ED's own unit training cost and reputation value. Each cluster consists of an ES and the ED that selected it;

[0010] When all EDs have selected the ES to serve, each ED determines its own data training amount through the Stackelberg game, and uses reverse deduction to adjust the unit service price of CS and the total reward strategy of ES;

[0011] When the HFL global training task is completed, the cloud server CS distributes the reward to the edge server ES that participated in the model aggregation, and the ES distributes the reward to the terminal device ED that participated in the training task;

[0012] The reputation value of each ED participating in the training task is calculated and recorded in the alliance blockchain.

[0013] In a second aspect, an embodiment of the present invention further provides a computing device, comprising a processor or a calculator and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor or the calculator is configured to call the program instructions to execute the method as described above.

[0014] In a third aspect, the present invention is an embodiment that further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor or a computer, the processor or the computer executes the method as described above.

[0015] Compared with the prior art, the hierarchical federated model training method, computing device and computer storage medium based on the reputation awareness incentive mechanism provided by the embodiment of the present invention have at least the following beneficial effects:

[0016] The embodiments of the present invention provide a hierarchical federated model training method and system, a computing device, and a computer storage medium based on a reputation-aware incentive mechanism. The entire system can better manage participants, including cloud servers, edge servers, and terminal devices. Its incentive mechanism plays a vital role in the selection of participants (edge ​​servers and terminal devices). When there are large-scale terminal devices in the hierarchical federated learning HFL framework, the incentive and selection mechanism based on reputation value will select the terminal devices with the best performance to participate in the training, which greatly reduces the cost and risk of HFL and improves the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The preferred implementation modes will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present invention.

[0018] Figure 1 This is a flow chart of a hierarchical federated model training method based on a reputation awareness incentive mechanism according to an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a hierarchical federated model training system based on reputation awareness incentives according to an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a three-stage Stackelberg game process for training a hierarchical federated model based on a reputation awareness incentive mechanism according to an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of the computing device structure of a hierarchical federated model training method based on a reputation awareness incentive mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0023] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0024] The following mainly takes some specific embodiments as examples to describe in detail the implementation of the technical solution of the present invention.

[0025] like Figure 1 As shown, in order to achieve the purpose of the invention, a hierarchical federated model training method based on a reputation awareness incentive mechanism provided by an embodiment of the invention specifically includes:

[0026] S1, the cloud server sends the global training task to the edge server, and then passes it from the edge server to the terminal device;

[0027] S2. The platform calculates the current reputation value of each ED based on its historical training records in the alliance blockchain, and the cloud server CS determines the unit service price P to be sent to the ES;

[0028] S3. According to the unit service price P of the cloud server CS, each ES determines its own total reward strategy;

[0029] S4. Each ED selects an optimal ES based on the total reward strategy released by the ES and the ED's own unit training cost and reputation value. Each cluster consists of an ES and the ED that selected it;

[0030] S5. When all EDs have selected the ES to serve, each ED will determine its own data training amount through Stackelberg game, and use reverse deduction to adjust the unit service price of CS and the total reward strategy of ES;

[0031] S6. When the HFL global training task is completed, the cloud server CS distributes the reward to the edge server ES that participated in the model aggregation, and the ES distributes the reward to the terminal device ED that participated in the training task;

[0032] S7. The reputation value of each ED participating in the training task is calculated and recorded in the alliance blockchain.

[0033] like Figure 2 As shown, the HFL (Hierarchical Federated Learning) system of the embodiment of the present invention is composed of a cloud server (CS), M edge servers (ES j , Edge Devices), j = {1, 2, ..., M}, and N terminal devices (ED i , End Device), i is expressed as: i = {1, 2, …, N}.

[0034] The cloud server is used to publish federated learning FL tasks in the HFL system, so as to achieve optimal model performance and corresponding benefits. In an embodiment of the invention, the cloud server directly stimulates the edge server and indirectly stimulates the terminal device to participate in the model training process.

[0035] Assuming that each ED can freely choose any ES to serve, in the embodiment of the present invention, once each ED selects a certain FL task, it will not change. As an important role in HFL, the edge server is used to aggregate model parameters and ensure fair incentive distribution among terminal devices.

[0036] In fact, in the embodiment of the present invention, each ED can only select one ES to serve. j and all the terminal devices connected to it.

[0037] Here, each terminal device has its own characteristics, which are represented by a triple (D i ,C i ,R i), where D i It's ED i The total size of local data samples, C i It's ED i The unit training cost, R i It's ED i The reputation value of is in the range of (0,1). Each terminal device can freely choose which edge server to join.

[0038] To maximize utility, ED i Will select an optimal ES j and by changing the data plan α ij To adjust the amount of training data participating in the federated learning task.

[0039] Preferably, the HFL of the embodiment of the present invention can also add a consortium blockchain module, where the blockchain module provides a feasible solution to avoid unreliable participants and protect privacy, avoiding the computational overhead and system complexity brought by using a specific encryption algorithm. That is, by adopting a consortium blockchain to incorporate a reputation-aware incentive mechanism, public authority and fairness are guaranteed, and transactions are stored in blocks containing timestamps, reputation and other information to grow in the form of a chain.

[0040] like Figure 2 and 3 As shown, the training incentive process of the system in the embodiment of the present invention is as follows:

[0041] The cloud server sends the global training task to the edge server, which then passes it to the terminal device (corresponding to Figure 2 Step ①); the platform calculates the current reputation value of each ED (corresponding to Figure 2 In step ②), the cloud server CS determines the unit service price P (corresponding to Figure 2 Step ③); After all edge servers ES and terminal devices ED receive the global training task, according to the unit service price P of the cloud server CS, each ES determines its own total reward strategy to attract ED to participate in the training task (corresponding to Figure 2 Step ④ in ); Each ED selects an optimal ES to serve according to the total reward strategy issued by ES and the unit training cost and reputation value of ED itself (corresponding to Figure 2 Step ⑤ in the above example, the ES can maximize its utility; when all EDs select the ES to serve, each ED will determine its own data training amount and participate in the training (corresponding to Figure 2Step ⑥ in ); When the HFL global training task is completed, the cloud server CS distributes the reward to the edge server ES participating in the model aggregation, and ES distributes the reward to the terminal device ED participating in the training task (corresponding to Figure 2 Step ⑦);

[0042] S7, the reputation value of each ED participating in the training task (calculated based on the credibility of the submitted parameters ω l,i ) is recorded in the blockchain (corresponding to Figure 2 Step ⑧).

[0043] Preferably, in step S3, according to the historical training records of each ED in the alliance blockchain, the calculation formula for obtaining the reputation value of each ED is as follows:

[0044]

[0045] Where β is the time decay coefficient and t is the ED i The number of training sessions in a certain period of time, r i τ ∈(0, 1) represents the ED in the τth round of training task i The reputation value can be calculated after completing each training task.

[0046] Generally speaking, the quality of local models uploaded by federated learning FL participants will affect the performance of the global model. However, data owners may perform malicious or unreliable local model update operations to attack the platform or gain more benefits. The final global model accuracy and convergence speed will be affected by malicious attacks or unreliable model updates. This is because on the one hand, malicious data owners will deliberately launch attacks, such as poisoning attacks. On the other hand, unreliable model update operations may also be caused by incorrect or forged datasets.

[0047] Therefore, the embodiment of the present invention uses a reputation mechanism to prevent malicious or unreliable data owners. In addition, the reputation of the participants will also motivate the behavior of the participants. Participants with high reputation are more likely to complete the training tasks with high quality. We use R i To express ED i Reputation value. If ED i If it is the first time to participate in the federated learning platform, the reputation value is initialized to R0, which belongs to the (0,1) interval. Otherwise, its reputation value can be calculated based on its historical reputation record in the alliance blockchain.

[0048] In step S4, each ED selects an optimal ES to serve according to the total reward strategy published by the ES and its own unit training cost and reputation value. Specifically, it includes:

[0049] Reorder edge servers from high to low according to the total reward strategy of edge services, and reorder terminal devices from low to high according to the ratio of unit training cost to reputation value of terminal devices. The lower the ratio of unit training cost to reputation value of terminal devices, the higher its competitiveness. The higher the competitiveness, the more priority is given to edge servers with high rewards. In order of competitiveness from high to low, the first terminal device ED directly selects the first edge server ES with the highest total reward strategy. When selecting edge servers, the subsequent terminal devices all select the edge server that makes their utility function the highest. When the utility function is the highest, the edge server selected by the terminal device is the best. Each terminal device can only select one edge server, and one edge server may be selected by multiple terminal devices.

[0050] The algorithm by which each ED selects an optimal ES to serve based on the total reward strategy published by ES and its own unit training cost and reputation value can be expressed as follows:

[0051]

[0052]

[0053] In the embodiments of the present invention, unless otherwise specified, H j Yes All Select ES j The collection of terminal devices; H j+ Yes All Select ES j The set of terminal devices participating in model training, l represents the set H j Take the value inside.

[0054] Assuming the unit service price of the cloud server is P and the total reward of each edge server is γ j (j∈[1,M]) is determined, each terminal device ED i Select the optimal edge server and determine the data plan for local participation in training by maximizing its utility:

[0055]

[0056] Among them, U ij For ED i Select ES j The obtained utility function, U i is the maximum utility function obtained by the server selected by the terminal device among the M edge servers, α ij For ED i Connect to ES j The ratio of local training data to actual local data, D i It's ED i The total size of local data, Ci It's ED i The unit training cost, R i It's ED i Reputation value, γ j ES in cluster j j The total reward given to the terminal device, D l Is involved in model training and connected to ES j Terminal equipment ED l The total size of local data, R l Is involved in model training and connected to ES j Terminal equipment ED l Reputation value, H j Is connected to ES j A collection of terminal devices;

[0057] By solving the first-order and second-order derivatives, we can obtain: and

[0058] Obviously when γ j >0 o'clock,

[0059] Here, for ED i Select ES j The obtained utility function, α ij For ED i Connect to ES j The ratio of local training data to actual local data, D i For ED i The total size of local data, R i For ED i Reputation value, γ j ES in cluster j j The total reward given to the terminal device, C i For ED i The unit training cost, α lj To connect to ES j Terminal equipment ED l The ratio of local training data to actual local data, D l To connect to ES j Terminal equipment ED l The total size of local data, R l To connect to ES j Terminal equipment ED l Reputation value, H j To connect to ES j A collection of terminal devices.

[0060] So when the first-order derivative is equal to 0, it is α ijThe optimal solution is that the terminal device ED i The utility of is the largest, let Find:

[0061]

[0062] in, To participate in model training and connect to ES j A collection of terminal devices, To participate in model training and connect to ES j The number of terminal devices, γ j ES in cluster j j The total reward given to the terminal device, C i For ED i The unit training cost, D i For ED i The total size of local data, R i For ED i Reputation value, C l To participate in model training and connect to ES j Terminal equipment ED l The unit training cost, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data.

[0063] Under the condition that each terminal device adopts the optimal data plan, the optimal reward strategy of each edge server is γ j (j∈[1,M]) is also calculated by maximizing its own utility function. The utility function of the edge server is defined as:

[0064]

[0065] Among them, γ j ES in cluster j j The total reward given to the terminal device, α lj To participate in model training and connect to ES j Terminal equipment ED l The ratio of local training data to actual local data, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data, R i ES that participates in model training and connects to j Terminal Equipment l Reputation value, To participate in model training and connect to ES jThe number of terminal devices, risk aversion parameter δ j and the reward scaling factor θ j Each edge server ES j Decision, K j For ES j unit coordination and calculation costs;

[0066] Substitute the optimal training data ratio of all terminal devices participating in training in cluster j You can get:

[0067]

[0068] Among them, a is the system parameter, γ j ES in cluster j j The total reward given to the terminal device, P is the unit service price of the cloud server CS, and the risk aversion parameter δ j and the reward scaling factor θ j Each edge server ES j Decision, K j For ES j Unit coordination and calculation costs, To participate in model training and connect to ES j A collection of terminal devices, To participate in model training and connect to ES j The number of terminal devices, C l To participate in model training and connect to ES j Terminal equipment ED l The unit training cost, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data;

[0069] Similarly, by solving the first-order and second-order derivatives, we can get:

[0070] Therefore When γ j Will maximize the utility of edge server j, that is:

[0071]

[0072] Furthermore, we can also use the same method to maximize the utility of the cloud server, thereby obtaining the optimal strategy for the cloud server: unit service price P * The utility function of the cloud server is defined as:

[0073]

[0074] Among them, X j is the total amount of credible training data in cluster j, λ is its weighting parameter, and P is the unit service price of cloud server CS.

[0075] When utility is maximized, the unit service price is:

[0076]

[0077] Among them, the risk aversion parameter δ j and the reward scaling factor θ j Each edge server ES j is determined, and λ is its weighting parameter.

[0078] That is, in the embodiment of the present invention, the unit service price P of the cloud server and the edge server ES j The total reward strategy of γ j , terminal equipment ED i Data Plan α i , their decision-making process is simulated as a three-stage Stancolbeck game process and calculated using the reverse deduction method. Figure 3 This is a schematic diagram of a three-stage Stancolberk game process for training a hierarchical federated model based on a reputation awareness incentive mechanism according to an embodiment of the present invention.

[0079] During the game, each terminal device ED i Have data plan experience Each edge server has a total reward experience value The cloud server has a unit service price experience value P 0 Since the terminal device will solve the optimal value according to its utility function and adjust its data plan according to the optimal solution i (No adjustment is required within a certain error range); When the data plan of the terminal device changes, the cloud server will then solve the optimal value according to its utility function and adjust its unit service price P according to the optimal solution obtained; When the unit service price P of the cloud server changes, each edge server ES j The optimal value will be solved according to its utility function, and the total reward strategy γ will be adjusted according to the optimal solution obtained. j , cyclically correct until the data plan of all terminal devices is α i No more adjustments are made, equilibrium is eventually reached, and the model begins training.

[0080] The above game process can be implemented using the following algorithm:

[0081]

[0082] Preferably, in the embodiment of the present invention, in step S5, when all EDs have selected the ESs to be served, each ED will determine its own data training amount and participate in the training, which specifically includes:

[0083] Each ED attempts to use its local data set to collaboratively train the global model ω. The embodiment of the present invention uses the HierFAVG algorithm to update the model parameters. The typical training process can be summarized as follows:

[0084] First, the cloud server CS publishes a global training task and sends the initial global model Distribute to each ES, then each ES j send For each ED in cluster j.

[0085] The terminal devices participating in model training use their local data samples to train the model.

[0086] After the first iteration of the training process is completed, each terminal device participating in the training will use its local model Sent to its service ES j Each ES j Aggregate the received local models to generate a partial model Then part of the model All EDs in their clusters are transferred back for the next training iteration.

[0087] After every k rounds of partial model aggregation (k is specified by the cloud server), the cloud server aggregates all partial models returned by ES to obtain the global model

[0088] Until the training reaches the total number of iterations, the global training task is completed.

[0089] Preferably, in step 6, when the HFL global training task is completed, the cloud server CS distributes the reward to the edge server ES participating in the model aggregation, and the ES distributes the reward to the terminal device ED participating in the training task, specifically including:

[0090] The reward value obtained by the edge server can be obtained by multiplying the unit service price by the total amount of trusted data. For example, the reward obtained by each edge server j from the cloud server is ε j =X j *P, where

[0091]

[0092] Among them, γ j ES in cluster j j The total reward given to the terminal device, α lj To participate in model training and connect to ESj Terminal equipment ED l The ratio of local training data to actual local data, C l To participate in model training and connect to ES j Terminal equipment ED l The unit training cost, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data, R l ES that participates in model training and connects to j Terminal Equipment l Reputation value, To participate in model training and connect to ES j The number of terminal devices;

[0093] Each terminal device ED i From the edge server ES it selects j The rewards received are:

[0094]

[0095] Among them, α ij For ED i Connect to ES j The ratio of local training data to actual local data, D i For ED i The total size of local data, R i For ED i Reputation value, γ j ES in cluster j j The total reward given to the terminal device, α lj To participate in model training and connect to ES j Terminal equipment ED l The ratio of local training data to actual local data, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data, R l ES that participates in model training and connects to j Terminal Equipment l Reputation value.

[0096] Preferably, in step 7, the reputation value of each ED participating in this training task is calculated (based on the credibility calculation of the submitted parameters ω l,i ) and recorded in the blockchain specifically include:

[0097] First, the reputation value of this model training is calculated, and then the reputation value is stored in the blockchain.

[0098] The reputation value of this model training is calculated as:

[0099] r i ∈(0, 1) indicates that ED in this training task i The reputation value is obtained by averaging the local model l i The cosine similarity between and the corresponding global model g is calculated as follows:

[0100]

[0101] Among them, r i ∈(0, 1) indicates ED in the training task i Reputation value, where l i and g represent the locally averaged updated model parameters and the globally updated model parameters, respectively. w represents the total model parameters in model training.

[0102] The embodiment of the present invention is based on the hierarchical federated learning model training method of the reputation-aware incentive mechanism (RAIM). The entire RAIM system can better manage participants, including cloud servers, edge servers and terminal devices, to ensure the quantity and quality of data that each participant can contribute during the model training process, thereby improving the performance of model training. In addition, it can also prevent malicious participants from attacking the system.

[0103] Compared with the prior art, the hierarchical federated model training method based on reputation awareness incentive, computing device and computer storage medium provided by the embodiment of the present invention have at least the following beneficial effects:

[0104] The embodiments of the present invention provide a hierarchical federated model training method, a computing device, and a computer storage medium based on reputation awareness incentives. The entire RAIM system can better manage participants, including cloud servers, edge servers, and terminal devices. Its incentive mechanism plays a vital role in the selection of participants (edge ​​servers and terminal devices). When there are large-scale terminal devices in the hierarchical federated learning HFL framework, the incentive and selection mechanism based on reputation value will select the terminal devices with the best performance to participate in the training, which greatly reduces the cost and risk of HFL and improves the performance of the model.

[0105] In a second aspect, an embodiment of the present invention further provides a computing device, which includes a processor or a calculator and a memory, wherein the memory is used to store a computer program, and the computer program includes program instructions. The processor or the calculator is configured to call the program instructions to execute the hierarchical federated model training method based on the reputation-aware incentive mechanism as described above.

[0106] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor or a computer, the processor or the computer executes the hierarchical federated model training method based on the reputation-aware incentive mechanism as described above.

[0107] like Figure 4 As shown, a computing device provided in an embodiment of the present application, the computing device 1000 includes a processor or a calculator (not shown in the figure) 1001 and a memory 1002, and the processor or the calculator 1001 and the memory 1002 can be interconnected through a communication bus 1003. The communication bus 1003 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 1003 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the memory 1002 is used to store a computer program, and the computer program includes program instructions. The processor 1001 is configured to call the program instructions, and the above program includes a method for executing some or all of the steps in the aforementioned method.

[0108] The processor 1001 may be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program.

[0109] The memory 1002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0110] The electronic device 1000 may further include a communication module 1004 and a display 1005. The communication module 1004 may be connected to the optical tracking device for communication. The communication module 1004 may be a wireless communication module (eg, a WiFi module, a Bluetooth module, etc.) or a wired communication module.

[0111] In addition, the electronic device 1000 may also include common components such as a communication interface (eg, a USB interface, a microphone interface, etc.), an antenna, etc., which will not be described in detail here.

[0112] An embodiment of the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the hierarchical federated model training methods based on a reputation-aware incentive mechanism as recorded in the above method embodiments.

[0113] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0114] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0118] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0119] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0120] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0121] It should be noted that the above embodiments can be freely combined as needed. The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A hierarchical federated model training method based on reputation awareness incentive mechanism, characterized in that: The method comprises: The cloud server CS sends the global training task to the edge server, and passes it from the edge server ES to the terminal device ED; Based on the historical training records of each ED in the alliance blockchain, the current reputation value of each ED is calculated, and the cloud server determines the unit service price to be sent to the ES; Based on the unit service price of the cloud server CS, each ES determines its own total reward strategy; Each ED selects an optimal ES based on the total reward strategy released by the ES and the ED's own unit training cost and reputation value. Each cluster consists of an ES and the ED that selected it; When all EDs have selected the ES to serve, each ED determines its own data training amount through the Stackelberg game, and uses reverse deduction to adjust the unit service price of CS and the total reward strategy of ES; When the HFL global training task is completed, the cloud server CS distributes the reward to the edge server ES that participated in the model aggregation, and the ES distributes the reward to the terminal device ED that participated in the training task; The reputation value of each ED participating in the training task is calculated and recorded in the alliance blockchain.

2. The hierarchical federated model training method based on reputation awareness incentive mechanism as described in claim 1 is characterized in that: The calculation formula of the reputation value is as follows: Among them, β is the time decay coefficient, t is the number of times the i-th ED participates in model training within the preset time, represents the ED in the τth training task i Reputation value.

3. The hierarchical federated model training method based on reputation awareness incentive mechanism as described in claim 1 is characterized in that: Each ED selects an optimal ES based on the total reward strategy published by the ES and the ED's own unit training cost and reputation value. When all EDs have selected the ES to be served, each ED determines its own data training amount through the Stackelberg game and uses reverse deduction to adjust the unit service price of the CS and the total reward strategy of the ES. Specifically, it includes: Calculate ED for each terminal device i The first utility function of When the first utility function is optimal, calculate the ES of each edge server j The second utility function of When the second utility function is optimal, the third utility function of the cloud server is calculated to obtain the optimal unit service price of the cloud server.

4. The hierarchical federated model training method based on reputation awareness incentive mechanism as claimed in claim 3, characterized in that: The calculation of each terminal device ED i The first utility function specifically includes: Predefine each terminal device ED i The first utility function of is: Among them, U ij For ED i Select ES j The obtained utility function, U i For the terminal device ED in M ​​edge servers i The maximum utility function obtained by selecting a certain edge server, α ij For ED i Connect to ES j The ratio of local participating training data to actual local data, D i For ED i The total size of local data, R i For ED i Reputation value, γ j ES in cluster j j The total reward given to the terminal device, D l To connect to ES j Terminal equipment ED l The total size of local data, R l To connect to ES j Terminal equipment ED l Reputation value, H j To connect to ES j The terminal device collection, C i For ED i The unit training cost, α lj To connect to ES j Terminal equipment ED l The ratio of local training data to actual local data; By solving the first-order and second-order derivatives, we can obtain: Obviously when γ j >0 o'clock, Among them, U ij For ED i Select ES j The obtained utility function, α ij For ED i Connect to ES j The ratio of local training data to actual local data, D i For ED i The total size of local data, R i For ED i Reputation value, γ j ES in cluster j j The total reward given to the terminal device, C i For ED i The unit training cost, α lj To connect to ES j Terminal equipment ED l The ratio of local training data to actual local data, D l To connect to ES j Terminal equipment ED l The total size of local data, R l To connect to ES j Terminal equipment ED l Reputation value, H j To connect to ES j A collection of terminal devices; Then when the first-order derivative is equal to 0, it is α ij The optimal solution is that the terminal device ED i The utility of is the largest, that is, Find the optimal solution: in, To participate in model training and connect to ES j A collection of terminal devices, To participate in model training and connect to ES j The number of terminal devices, γ j ES in cluster j j The total reward given to the terminal device, C i For ED i The unit training cost, D i For ED i The total size of local data, R i For ED i Reputation value, C l To participate in model training and connect to ES j Terminal equipment ED l The unit training cost, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data.

5. The hierarchical federated model training method based on reputation awareness incentive as described in claim 3 is characterized in that: The calculating of the second utility function of each edge server specifically includes: The utility function of the edge server is predefined as: Among them, γ j ES in cluster j j The total reward given to the terminal device, α lj To participate in model training and connect to ES j Terminal equipment ED l The ratio of local training data to actual local data, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data, R i ES that participates in model training and connects to j Terminal Equipment l Reputation value, To participate in model training and connect to ES j The number of terminal devices, risk aversion parameter δ j and the reward scaling factor θ j Each edge server ES j Decision, K j For ES j unit coordination and calculation costs; Substitute the optimal training data ratio of all terminal devices participating in training in cluster j get: Among them, a is the system parameter, γ j ES in cluster j j The total reward given to the terminal device, P is the unit service price of the cloud server CS, and the risk aversion parameter δ j and the reward scaling factor θ j Each edge server ES j Decision, K j For ES j Unit coordination and calculation costs, To participate in model training and connect to ES j A collection of terminal devices, To participate in model training and connect to ES j The number of terminal devices, C l To participate in model training and connect to ES j Terminal equipment ED l The unit training cost, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data; By solving the first and second order derivatives we get: Therefore When , we get the optimal γ j Enable edge server ES j The utility is the largest, that is:

6. The hierarchical federated model training method based on reputation awareness incentive as described in claim 3 is characterized in that: When the second utility function is optimal, calculating the third utility function of the cloud server to obtain the optimal unit service price of the cloud server specifically includes: The utility function of the cloud server is predefined as: Among them, X j is the total amount of credible training data in cluster j, λ is the weighting parameter, and P is the unit service price of cloud server CS; When the utility of the cloud server is maximized, the unit service price is: Among them, the risk aversion parameter δ j and the reward scaling factor θ j It is determined by each edge server ESj, and λ is its weight parameter.

7. The hierarchical federated model training method based on reputation awareness incentive as claimed in claim 1, characterized in that: The training process of the HFL global training task specifically includes: The cloud server CS will initialize the global model Distribute to each ES, then ES j send For each ED in cluster j; The terminal devices participating in model training use their local data samples to train the model; After the first iteration of the training process is completed, each terminal device participating in the training will use its local model Sent to its service ES j ; ES j Aggregate the received local models to generate a partial model Then part of the model The ED is transferred back to its cluster for the next training iteration; After each k rounds of partial model aggregation, the k value is determined by the cloud server, which aggregates all partial models returned by ES to obtain the global model. Until the training reaches the total number of iterations, the global training task is completed.

8. The reputation-aware incentive-based hierarchical federated model training method of claim 1, characterized in that: The cloud server CS distributes the reward to the edge server ES participating in the model aggregation, and the ES distributes the reward to the terminal device ED participating in the training task, specifically including: The reward value obtained by the edge server is obtained by multiplying the unit service price by the total amount of trusted data. The reward that each edge server j receives from the cloud server is ε j =X j *P, where Among them, γ j ES in cluster j j The total reward given to the terminal device, α lj To participate in model training and connect to ES j Terminal equipment ED l The ratio of local training data to actual local data, C l To participate in model training and connect to ES j Terminal equipment ED l The unit training cost, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data, R l ES that participates in model training and connects to j Terminal Equipment l Reputation value, To participate in model training and connect to ES j The number of terminal devices; Each terminal device ED i From the edge server ES it selects j The rewards received are: Among them, α ij For ED i Connect to ES j The ratio of local training data to actual local data, D i For ED i The total size of local data, R i For ED i Reputation value, γ j ES in cluster j j The total reward given to the terminal device, α lj To participate in model training and connect to ES j Terminal equipment ED l The ratio of local training data to actual local data, D l To participate in model training and connect to ES j Terminal equipment ED l The total size of local data, R l ES that participates in model training and connects to j Terminal Equipment l Reputation value.

9. The hierarchical federated model training method based on reputation awareness incentive as claimed in claim 1, characterized in that: Terminal device ED after model training i The reputation value of this training is obtained by averaging the local model l i The cosine similarity between and the corresponding global model g is calculated as follows: Among them, r i ∈(0, 1) represents the ED calculated in the training task i The reputation value of this training, where l i and g represent the locally averaged updated model parameters and the globally updated model parameters, respectively, and w represents the total model parameters in model training.

10. A computing device, characterized in that The computing device comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 9.

Images