Federal learning method and device based on reputation mechanism, storage medium and equipment

Abstract

The invention provides a federal learning method and device based on a reputation mechanism, a storage medium and equipment. The method comprises the steps that a central server obtains a reputation value of each client according to a local model of the client and a training round behavior; the central server performs sorting according to the reputation values of the clients, and extracts local models of the clients according to a sorting result to obtain a to-be-clustered model list; calculating a quality index of each local model based on the loss value of each local model in the to-be-clustered model list; performing traversal detection on a local model in the to-be-clustered model list, and if the quality index of the local model meets a preset quality condition, adding the local model into a global model updating list; and carrying out aggregation processing on all local models in the global model updating list to obtain a global model. The federated learning method based on the reputation mechanism comprises multi-dimensional reputation evaluation of direct and indirect reputation components, and the personal contribution of the client and the overall influence on the federated learning system can be comprehensively evaluated.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and particularly to a federated learning method, device, storage medium and equipment based on a reputation mechanism. Background Art

[0002] Federated Learning (FL for short) is an emerging distributed machine learning method that allows multiple participants (such as mobile devices, organizations or data centers) to jointly train a model while maintaining data privacy. In federated learning, data does not need to be centrally stored or processed, but remains on local devices and is only transmitted to the central server for aggregation in the case of model updates (such as gradient information or parameter updates). The core advantage of federated learning is that it can protect user privacy, and sensitive data does not need to leave its original environment.

[0003] Under the federated learning framework, the reputation mechanism is a core technical means to ensure the stable operation of the system. Its core function is to quantify the behavioral characteristics and contribution values of clients through multiple dimensions, so as to drive benign cooperation and restrain potential malicious behaviors. Existing reputation mechanisms mainly focus on the direct contributions of clients, such as the prediction accuracy of local models or the quality level of training data. Typically, reputation assignment is performed through performance evaluation on the test set or subjective evaluation / trust transfer based on other participants. At the same time, existing reputation mechanisms focus on the direct contributions of clients and ignore other impacts of clients on the global model aggregation process, such as cooperation willingness and fairness, resulting in biased evaluation results and being unfavorable for building a fair and sustainable federated learning ecosystem. Summary of the Invention

[0004] Based on this, the present invention provides a federated learning method, device, storage medium and equipment based on a reputation mechanism, which can consider the direct model contributions of clients and comprehensively evaluate their indirect impacts, such as the behaviors of clients in promoting data sharing, maintaining data security and participation in aspects of federated learning, providing a more comprehensive and multi-dimensional reputation evaluation mechanism, which helps to improve the model learning efficiency and the model robustness.

[0005] In a first aspect, the present invention provides a federated learning method based on a reputation mechanism. The federated learning method based on the reputation mechanism is applied to a federated learning system. The federated learning system includes a central server and clients. The federated learning method based on the reputation mechanism includes the following steps:

[0006] The central server sends an initial global model to each client;

[0007] Each client trains the initial global model based on the local training dataset to obtain a local model until the convergence condition is met, and then sends the local model and the training round behavior to the central server;

[0008] The central server obtains the reputation values of each client according to the local models and training round behaviors of each client;

[0009] The central server sorts according to the reputation values of each client, and extracts the local models of the clients according to the sorting result to obtain a list of models to be clustered;

[0010] Based on the loss values of each local model in the list of models to be clustered, calculate the quality indicators of each local model;

[0011] Traverse and detect the local models in the list of models to be clustered. If the quality indicator of the local model meets the preset quality condition, add the local model to the global model update list;

[0012] Perform aggregation processing on all local models in the global model update list to obtain a global model;

[0013] Repeat the above steps until the global model meets the output condition, and save the updated global model.

[0014] Among them, the central server obtains the reputation values of each client according to the local models and training round behaviors of each client, including:

[0015] The central server obtains the direct reputation and indirect reputation of each client according to the local models and training round behaviors of each client;

[0016] Obtain the reputation value of each client according to the direct reputation and indirect reputation of each client;

[0017] Among them, the direct reputation of the client includes the local model quality indicator of the client, the data quality indicator of the client, the local model validity indicator of the client, the client honesty indicator, and the client reliability indicator. The indirect reputation of the client includes the multi-source information fusion indicator and the task publisher similarity indicator.

[0018] Furthermore, the expression of the direct reputation of the client is:

[0019] ,

[0020] Among them, is the direct reputation of client , is the local model quality indicator of client The local model quality indicator, is the client Data quality indicator For the client Local model effectiveness indicator For the client Honesty indicator For the client Reliability indicator;

[0021] The client The specific expression of the local model quality indicator is:

[0022] ,

[0023] For the client Regulation coefficient of the local model quality scale For In the number of training rounds without the client Temporary global model In the test set Loss function For Number of training rounds of the standard global model In the test set Loss function;

[0024] The client The specific expression of the local model quality indicator is:

[0025] ,

[0026] For the client Regulation coefficient of the local data quality scale Weight for regulating quantity and quality emphasis For the client Degree of data label skewness For the client Participation rate of the local dataset in this training round;

[0027] The client The specific expression of the local model effectiveness indicator is:

[0028] ,

[0029] For the client Adjustment coefficient of local model effectiveness For the client Probability of being selected to participate in training For the client Probability of submitting the local model on time For the client Probability of being selected to participate in the global model aggregation For the client Weight of being selected to participate in the training For the client Weight of submitting the local model on time For the client Weight of being selected to participate in the global model aggregation;

[0030] The client The specific expression of the honesty index is:

[0031] ,

[0032] For the client Adjustment coefficient of the honesty index For the client Lazy behavior score For the client Malicious behavior score;

[0033] The client The specific expression of the reliability index is:

[0034] ,

[0035] For the client Adjustment coefficient of the reliability index For the client Computing resources For the client Network resources Is the weight for regulating computing resources and network resources.

[0036] Furthermore, the indirect reputation expression of the client is:

[0037] ,

[0038] For the task publisher For the client Indirect reputation value For the task publisher For the client Direct reputation score For the task publisher And Similarity For the task publisher Provide a set of task publishers with indirect reputation worth.

[0039] Furthermore, the federated learning method based on the reputation mechanism further includes:

[0040] Obtain the historical reputation values of each client;

[0041] Correct the reputation values of each client according to the historical reputation values to obtain the corrected reputation values of each client.

[0042] Furthermore, in the step where the central server sorts according to the reputation values of each client and extracts the local models of the clients according to the sorting results to obtain a list of models to be clustered, the probability of each client being extracted is specifically:

[0043] The local models of the clients with the reputation value ranking greater than the first threshold are directly included in the list of models to be clustered;

[0044] For the clients with the reputation value ranking greater than the second threshold and less than the first threshold, extract the local models of the clients according to the first probability and include them in the list of models to be clustered;

[0045] For the clients with the reputation value ranking less than the second threshold, extract the local models of the clients according to the second probability and include them in the list of models to be clustered.

[0046] Furthermore, the specific expression of the preset quality condition is:

[0047] ,

[0048] where is a set of test sample batches extracted by the server from the test dataset it maintains, is the local model quality threshold.

[0049] Furthermore, the specific expression for aggregating all local models in the global model update list to obtain the global model is:

[0050] ,

[0051] where is the global model, are all local models in the global model update list, is the reputation value of the task publisher for when the client is the reputation value, is the client is the local model of the test accuracy on the server test set.

[0052] Second aspect, the present invention further provides a federated learning device based on a reputation mechanism. The federated learning device based on the reputation mechanism is applied to a federated learning system, and the federated learning system includes a central server and at least two clients. The federated learning device based on the reputation mechanism includes:

[0053] A model distribution module, configured to send an initial global model from the central server to each client;

[0054] A model training module, configured to train the initial global model by each client according to the local training data set to obtain a local model until the convergence condition is satisfied, and send the local model and the training round behavior to the central server;

[0055] A reputation evaluation module, configured to obtain the reputation value of each client by the central server according to the local models and training round behaviors of each client;

[0056] A reputation screening module, configured to sort the reputation values of each client by the central server, and extract the local models of the clients according to the sorting result to obtain a list of models to be clustered;

[0057] A model quality evaluation module, configured to calculate the quality index of each local model based on the loss value of each local model in the list of models to be clustered;

[0058] A quality screening module, configured to traverse and detect the local models in the list of models to be clustered. If the quality index of the local model meets the preset quality condition, add the local model to the global model update list;

[0059] A model aggregation module, configured to perform aggregation processing on all local models in the global model update list to obtain a global model;

[0060] A model output module, configured to repeat the above steps until the global model meets the output condition, and save the updated global model.

[0061] Third aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the federated learning methods based on the reputation mechanism in the first aspect are implemented.

[0062] Fourth aspect, the present invention further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it executes any one of the federated learning methods based on the reputation mechanism in the first aspect.

[0063] The beneficial effects of adopting the above technical solutions are as follows: In this embodiment, indirect reputation is integrated on the basis of the existing reputation mechanism to establish a comprehensive evaluation of the direct contributions of clients and their overall impact on the federated learning system. Further, based on the above reputation mechanism, the screening of local models during client selection and global model aggregation is improved, enhancing the overall performance and robustness of the federated learning global model. Historical reputation is introduced to correct the current reputation of clients, improving the accuracy of reputation evaluation and the trust of participants, avoiding the consideration of the long-term behavior of clients in the existing reputation evaluation mechanism, and preventing the phenomenon of malicious clients obtaining benefits in the short term. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for the description of the embodiments or the prior art.

[0065] Figure 1 Schematic diagram of a federated learning method based on a reputation mechanism in an embodiment of the present application;

[0066] Figure 2 Schematic flow diagram of a federated learning method based on a reputation mechanism in an embodiment of the present application;

[0067] Figure 3 Schematic diagram of a direct reputation component in an embodiment of the present application;

[0068] Figure 4 Five-dimensional radar diagram of a direct reputation component in an embodiment of the present application;

[0069] Figure 5 Schematic diagram of an indirect reputation component in an embodiment of the present application;

[0070] Figure 6 Schematic diagram of a federated learning device based on a reputation mechanism in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. To describe the present invention in more detail, the following specifically describes the federated learning method, device, storage medium, and equipment based on the reputation mechanism provided by the present invention with reference to the drawings.

[0072] Unless otherwise defined, the technical terms or scientific terms used in this application disclosure shall have the ordinary meaning as understood by those of ordinary skill in the field to which this invention pertains. The "first", "second" and similar terms used in this invention do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "a", "an" or "the" do not denote a quantity limitation, but mean that there is at least one. Words such as "comprising" or "including" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. Words such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Upper", "lower", "left", "right", etc. are only used to indicate relative position relationships, and when the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0073] Federated Learning (FL for short) is a distributed machine learning framework that allows multiple participants to collaboratively train a shared model while protecting data privacy. In this process, the reputation mechanism, as a core component, is used to encourage good collaboration, restrain malicious behavior, and optimize system efficiency.

[0074] The reputation mechanism avoids the emergence of "free-riding" behavior by quantifying the contributions of participants, ensuring that the resource input and benefits are matched; dynamically identifies malicious nodes, such as model tampering, and reduces the impact of malicious participant attacks on the global model; dynamically adjusts the federated learning resource allocation based on reputation, improving the convergence speed and communication efficiency of the global model.

[0075] However, existing reputation mechanisms mainly focus on the direct contributions of clients, such as the prediction accuracy of local models or the quality level of training data. Typically, reputation assignment is performed through performance evaluation on the test set or based on the subjective evaluation / trust transfer of other participants. At the same time, existing reputation mechanisms emphasize the direct contributions of clients and ignore other impacts of clients on the global model aggregation process, such as cooperation willingness and fairness, resulting in biased evaluation results and being unfavorable for building a fair and sustainable federated learning ecosystem.

[0076] In response to this, the present invention provides a federated learning method based on a reputation mechanism, which can consider the direct model contributions of clients and can also comprehensively evaluate their indirect impacts, such as the behaviors of clients in aspects of promoting data sharing, maintaining data security, and participation in federated learning, providing a more comprehensive and multi-dimensional reputation evaluation mechanism, which helps to improve the model learning efficiency and improve the robustness of the global model. This method is described by applying it to a federated learning system, in combination with the attached Figure 1Schematic diagram of the federated learning method based on the reputation mechanism and appendix Figure 2 Schematic diagram of the process of the federated learning method based on the reputation mechanism shown

[0077] The embodiment of the present application provides an application scenario of the federated learning method based on the reputation mechanism. This application scenario is a federated learning system, and the federated learning system includes a central server and at least two clients. It should be noted that the above central server and clients include a processor, a storage module, and a communication interface. Among them, the processor of the central server is used to meet the computing requirements for tasks such as processing a large amount of model parameter update information and collaborative communication, ensuring that the model uploaded by the client can be processed and analyzed quickly; the storage module of the central server is used to temporarily store and quickly access the model parameters being processed, intermediate calculation results, and communication data to improve the efficiency of data processing and model aggregation; further, the storage module of the central server can also be used to store the global model parameters, historical records of model training, configuration files, and other relevant data of the federated learning process for a long time for model backtracking, analysis, and optimization; the communication interface of the central server is used to achieve fast data transmission with a large number of clients, receiving and sending a large amount of model parameters and control information in a short time. The processor of the client is used to train and process the received model in combination with local data; the storage module of the client is used to store local data and model parameters for effective model training and update; the communication interface of the client is used to upload and download model parameters, and receive instructions and global model update information sent by the central server.

[0078] Based on the above architecture of federated learning, a role of a task publisher is also set in the federated learning system of this embodiment. The task publisher is not used to provide training data, but only to define task objectives and constraint conditions. The task publisher can be the central server, which acts as a central node for coordinating global training while publishing tasks, such as the server of an enterprise's self-built federated platform; the task publisher can also be a client, but this client does not participate in the training process of the global model or the local model and interacts with the central server through the API.

[0079] Based on the above federated learning system, the central server sends the initial global model to each client (except the task publisher) to obtain the trained local models, determines the reputation values of each client based on the reputation mechanism, and selects local models to cluster and generate the global model in combination with the model quality metrics. For the specific process, please refer to the embodiment of the federated learning method based on the reputation mechanism.

[0080] Step S101, the central server sends the initial global model to each client.

[0081] Specifically, the central server, acting as a task publisher, distributes the initial global model (equivalent to the training task of federated learning) to each client (equivalent to potential participants).

[0082] Step S102: Each client trains the initial global model based on its local training dataset to obtain a local model until the convergence condition is met, and then sends the local model and the training round behavior to the central server.

[0083] Specifically, each client trains the initial global model while protecting the privacy of its local training dataset to obtain a local model that conforms to the data characteristics of the local training dataset. In this embodiment, there is no limit on the convergence condition of the client. The convergence condition of the local model can be that the number of training rounds reaches a preset value, the loss function of the local model is less than a preset threshold, etc. When the local model of the client meets the convergence condition, it sends the local model and the training round behavior of this client to the central server. The above training round behavior includes, but is not limited to, the loss function of the local model on the test set, the labels of the local training dataset participating in the training rounds, whether the local model submits to the central server on time, etc.

[0084] Step S103: The central server obtains the reputation values of each client based on the local models and training round behaviors of each client.

[0085] Specifically, the above step S103 includes the following steps:

[0086] Step S201: The central server obtains the direct reputation and indirect reputation of each client based on the local models and training round behaviors of each client.

[0087] Among them, the direct reputation of the client includes the local model quality index, the client data quality index, the local model effectiveness index, the client honesty index, and the client reliability index. The indirect reputation of the client includes the multi-source information fusion index and the client similarity index.

[0088] Step S202: Obtain the reputation values of each client based on the direct reputation and indirect reputation of each client.

[0089] Specifically, the task publisher for the client reputation assessment, that is, the reputation value of each client, includes direct reputation and indirect reputation assessment. The specific expression of the reputation value is: ,

[0090] Among them, is the weight of the direct reputation component, is the weight of the indirect reputation component, The larger the value, the more important the direct reputation evaluation of the current task publisher is for the client's own direct reputation evaluation.

[0091] Furthermore, in this embodiment, the direct reputation expression of the client is:

[0092] ,

[0093] wherein, is the direct reputation of the client , is the local model quality index of the client , is the data quality index of the client , is the local model effectiveness index of the client , is the honesty index of the client , is the reliability index of the client .

[0094] In this embodiment, the indirect reputation of the client includes a multi-source information fusion index and a task publisher similarity index. Among them, the multi-source information fusion index refers to collecting and integrating evaluation information from different task publishers and model aggregators to obtain a comprehensive view of the client's performance and behavior in different environments. That is to say, by integrating the direct reputation value evaluation results of the same client's task publishing and model aggregation by different task publishers, the multi-source information fusion index is obtained. By analyzing this multi-source information, the credibility value of the client can be evaluated more accurately, and the bias that may be brought by a single evaluation source can be reduced. The task publisher similarity index refers to using a similarity metric (cosine similarity) to evaluate the similarity between different task publishers, so as to determine the relevance and importance of their evaluation results to the current task, which helps to identify evaluation opinions that are more valuable for reference to the current federated learning task. Therefore, the indirect reputation expression of the client is:

[0095] ,

[0096] wherein, is the indirect reputation value of the task publisher for the client , is the direct reputation score of the task publisher for the client , is the similarity between the task publisher and , is for the task publisher Provide a set of task publishers with indirect reputation worth.

[0097] Step S104, the central server sorts according to the reputation values of each client, and extracts the local models of the clients according to the sorting result to obtain a list of models to be clustered.

[0098] Specifically, to improve the fairness, accuracy, and robustness of the federated learning system, the central server constructs a reputation record list, starts reputation evaluation after each round of training, and records the reputation values of each client. Before aggregating the global model, the central server sorts the reputation values of each client from high to low, and extracts the local models according to the sorting result to obtain a list of models to be clustered.

[0099] Among them, the extraction probability of each client participating in model training is specifically:

[0100] Step S301, directly incorporate the local models of the clients whose reputation value ranking is greater than the first threshold into the list of models to be clustered.

[0101] Specifically, the first local models corresponding to the ranking are directly incorporated into the list of models to be clustered.

[0102] Step S302, for the clients whose reputation value is greater than the second threshold and whose ranking is less than the first threshold, extract the local models of the clients according to the first probability and incorporate them into the list of models to be clustered.

[0103] Specifically, in order to accommodate the clients with temporarily low reputation values, the clients whose reputation scores exceed and are ranked outside the are incorporated according to the first probability, and this first probability is represented by .

[0104] Step S303, for the clients whose reputation value is less than the second threshold, extract the local models of the clients according to the second probability and incorporate them into the list of models to be clustered.

[0105] Specifically, the clients with reputation values less than the second threshold can be regarded as malicious nodes or low-quality nodes. In order to balance incorporating some malicious nodes and updating the reputation value, and reduce the potential interference that may be brought by too many nodes in the federated learning process, the probability of incorporating malicious nodes or low-quality nodes, that is, the second probability in this embodiment, is associated with the specific reputation value.

[0106] Based on the above description, for the access probability of the client participating in model training, the specific expression is as follows:

[0107] ,

[0108] Among them, is the sorting number of the function, specifically represented by the product of the number of clients participating in the training and .

[0109] Step S105: Calculate the quality metrics of each local model based on the loss values of the local models in the list of models to be clustered.

[0110] Specifically, based on the list of models to be clustered, the central server further evaluates whether the quality of each local model saved in the list meets the criteria for global model aggregation, and uses a method based on loss value measurement to calculate the quality metrics of each local model.

[0111] Step S106: Traverse and detect the local models in the list of models to be clustered. If the quality metrics of the local model meet the preset quality conditions, add the local model to the global model update list.

[0112] Specifically, a method based on loss value measurement is used to calculate the quality of each local model. When the quality of the local model of the client meets the preset quality conditions, it is added to the global model update list. The expression of the preset quality conditions is:

[0113] ,

[0114] Among them, is a set of test samples extracted by the server from the test dataset it maintains in one batch, is the size of the set of test samples, is the local model quality threshold, which is set according to different tasks and can usually be set to 1, is a small test set maintained on the central server.

[0115] Step S107: Aggregate all the local models in the global model update list to obtain a global model.

[0116] Specifically, since the model quality obtained by testing with the server test set cannot fully represent the true quality of the local model, in this embodiment, the reputation value and the model quality are comprehensively considered, and an aggregation method of comprehensive weighted reputation value and model quality is designed. Therefore, the specific expression of the above global model is:

[0117] ,

[0118] Among them, is the global model, is all the local models in the global model update list, is for the task publisher At the client 's reputation value For the client 's local model The test accuracy on the server test set.

[0119] Step S108, repeat the above steps until the global model meets the output condition, and save the updated global model.

[0120] Specifically, this embodiment does not limit the output condition of the global model. The output condition of the global model can be that the number of repetitions of the above steps reaches the training times threshold, or the global model meets the preset convergence condition.

[0121] This embodiment integrates indirect reputation on the basis of the existing reputation mechanism to comprehensively evaluate the direct contribution of the client and its overall impact on the federated learning system; further, based on the above reputation mechanism, it improves the screening of local models in the process of client selection and global model aggregation, and improves the overall performance and robustness of the federated learning global model.

[0122] Further, in combination with the schematic diagram of the direct reputation component shown in the attached Figure 3 drawings, and the five-dimensional radar diagram of the direct reputation component shown in the attached Figure 4 drawings, each component of the direct reputation in the above step S201 is described in detail, specifically as follows:

[0123] (1) Client local model quality metric

[0124] The client local model quality metric evaluates the contribution of the local model submitted by the client to the performance of the global model, and is used to evaluate how the local model positively or negatively affects the accuracy, efficiency, and effectiveness of the model.

[0125] In the prior art, the test accuracy of the client local model is mostly used to evaluate the model quality. The model with the test accuracy meeting the requirements is defined as a benign model, otherwise it is defined as a malignant model. However, in the prior art, the calculation result of the test accuracy is related not only to factors such as the calculation input of the client local model and the quality of the data set, but also to the client list selected by the server in this round of training. Under certain data distributions or client lists selected by the server, the calculation of the test accuracy does not conform to the actual situation. Therefore, considering that a good local model should not cause the loss value of the global model to increase too much after aggregation, this embodiment selects to use the loss value to evaluate the model quality.

[0126] In the th training round, in order to Evaluate the local models, and aggregate all local models into a standard global model , while aggregating a temporary global model that only does not include the client , client The specific expression of the local model quality metric is:

[0127] ,

[0128] is the regulation coefficient of the local model quality scale of the client , is at training rounds without the client of the temporary global model on the test set of the loss function, is at training rounds of the standard global model on the test set of the loss function.

[0129] (2) Client data quality metric

[0130] The data of the federated learning client is the foundation of the global model. The quality of the client's local data affects the quality of the client's local model. The quality of the client's local model was measured in the above local model quality scale evaluation, but the local model quality cannot fully represent the data quality of the client. It is also related to the use of local computing resources, training strategies, etc. However, due to the limitations of user privacy protection and data security, directly accessing the data information of the client is not practically operable. Therefore, the local data quality of the client is measured from two aspects: the quantity and quality of the local data. The specific expression of the client data quality metric is:

[0131] ,

[0132] is the regulation coefficient of the local data quality scale of the client , is the weight for regulating the emphasis on quantity and quality, is the degree of label skewness of the client data, is the participation rate of the client local dataset in this training round.

[0133] For the quantity , in this embodiment, the number of clients participating in training in each training round is counted ​Local dataset, calculate the proportion of the client; at the same time, to prevent some clients from using too much data to gain an overwhelming advantage over other participants, the present invention normalizes the proportion result using a logarithmic function, client The specific expression of the participation degree of the local dataset in this training round is:

[0134] ,

[0135] where, is the total number of samples in the dataset used by client for training, is the maximum value of the total number of samples in the client datasets participating in training in this training round.

[0136] Regarding the local data quality of the client, the closer the label distribution of the local dataset is to the label distribution of the global dataset, the better the data quality of the client. In this embodiment, Jensen–Shannon (JS) divergence is used to measure the label skewness degree of the client, specifically:

[0137] ,

[0138] ,

[0139] where, is the distribution of the local dataset of client , is the distribution of the global dataset. Considering that it is difficult to collect the global data distribution at one time in some cases, at this time it can be collected in an accumulative manner. The expression of in the th training round is , where is the set of clients participating in training in the th training round. In this dimension, the central server requires the clients to report their training data and the label distribution of the local dataset.

[0140] (3) Client local model effectiveness index

[0141] The messages of the client local model give metrics for the client's willingness and ability to participate in the global model training. The quality metrics of the client local model and the data quality metrics can measure the potential of the client to contribute to federated learning, but this potential is not sufficient to describe the contribution made by the client to federated learning. Therefore, in this embodiment, a client local model effectiveness metric is introduced to measure the degree of effective participation of each client in federated learning. The effectiveness metric includes three quantifiable metrics, specifically the probability of the client being selected to participate in training, the success rate of the client local model training, and the success rate of the client local model aggregation; among them, for more intuitive calculation, the success rate of the client local model training is represented by the probability of the client submitting the local model on time, and the success rate of the client local model aggregation is represented by the probability of the client being selected to participate in the global model aggregation. It should be noted that in the stage of selecting the client local model, the central server tends to select excellent clients with a large amount of data or high-contribution models. Therefore, the frequent selection of the client's local model means that the client's local model has certain advantages. The probability of the client submitting the local model on time mainly evaluates whether the client continuously has sufficient computing resources and network bandwidth to complete the locally assigned model training tasks, and whether it subjectively prioritizes participating in these tasks. The probability of the client being selected to participate in the global model aggregation mainly evaluates whether the client's local model is always not inferior to the local models of other clients.

[0142] Thus, for the client The specific expression of the local model effectiveness metric is:

[0143] ,

[0144] where is the adjustment coefficient of the local model effectiveness for the client, is the probability of the client being selected to participate in training, is the probability of the client submitting the local model on time, is the probability of the client being selected to participate in the global model aggregation, is the weight of the client being selected to participate in training, is the weight of the client submitting the local model on time, is the weight of the client being selected to participate in the global model aggregation.

[0145] It should be noted that this embodiment uses a weighted geometric mean instead of a linear combination for the combination of the above three parameter indicators. This is because when any part of the weighted geometric mean is 0, the effectiveness indicator will be 0, which is impossible for a linear combination; and the weighted geometric mean changes more smoothly.

[0146] (4) Client Honesty Indicator

[0147] The client honesty index evaluates the degree of consistency between the client's claimed efforts and actual efforts. The behavior of the client's local model is divided into three categories: positive behavior, negative behavior, and malicious behavior. Among them, positive behavior refers to the client providing real and high-quality data, as well as high-configuration computing resources and network resources to train the local model; negative behavior refers to the client providing low-quality data, or low-configuration computing resources and network resources to train the local model, but hoping to obtain more benefits from the central server than paid; malicious behavior refers to the client launching poisoning attacks or deceptive operations on the central server by forging data, forging models, etc.

[0148] This embodiment uses the method of trimming the average to determine whether the client is a malicious client. Specifically, in the submitted model, the average is calculated for each dimension, and the maliciousness of the model is determined dimension by dimension. A portion of clients closest to the average are selected for each dimension, and the malicious factors of the unselected clients are accumulated. When the malicious factor of the client exceeds a certain threshold, it is considered that the model submitted by the client may be a malicious model. The specific expression of the malicious factor is:

[0149] ,

[0150] in, For Clients The malicious factor, represents the number of dimensions of the model parameters, For Clients The submitted local model has a parameter dimension The value of Indicates parameter dimensions The subset of clients closest to the mean, the size of this set is related to the number of malicious nodes estimated by the task publisher based on the number of clients submitting in each round. The malicious behavior score of each client The specific expression can be written as: .

[0151] Considering the non-linear relationship between the malicious factor and the malicious score of the model, a simple quadratic function is adopted in this embodiment to achieve the goal. Since there may be a small number of malicious factors in normal models, it is necessary to minimize the malicious scores of good clients while increasing the malicious scores of malicious clients. At the same time, in order to detect whether there are "free-riding" and other negative behaviors among the clients participating in model training, tests are conducted based on the clean dataset maintained on the central server. Considering that in the "free-riding" attack, the client does not use too much computing resources to update the global model, in this embodiment, a sample set of the central server test set is randomly selected to check the output difference between the local model and the global model to detect whether the client launches a "free-riding" attack. In this embodiment, the client lazy behavior score can be used to represent:

[0152] ,

[0153] where, is the client lazy behavior score, is a random subset of the central server test set.

[0154] According to the client's lazy score and malicious score, the specific expression of the client honesty index is:

[0155] ,

[0156] is the client adjustment coefficient of the honesty index, is the client lazy behavior score, is the client malicious behavior score.

[0157] (5) Client reliability index

[0158] The client reliability index measures whether the computing resources and network resources of the client are abundant and reliable, so that the local model can be uploaded in time when it is selected to participate in model training. The reliability scale can be quantified into two parts: computing resources and network resources. The client specific expression of the reliability index is:

[0159] ,

[0160] is the client adjustment coefficient of the reliability index, is the client computing resources, is the client Network resources, To adjust the weights of computing resources and network resources. The client computing resources can be determined by the time when the client submits the model, and the client network resources can be determined by the client's network transmission speed, packet loss rate, etc. The specific expression of the client's computing resources is: , where represents the client submission time of the model; the specific expression of the client's network resources is: , where represents the client network communication rate, represents the client packet loss rate.

[0161] Create a direct reputation component based on the above five indicators and use it as an overall assessment of each client's personal contribution and reputation. In this embodiment, a five-dimensional radar chart is used to represent the designed multi-dimensional reputation evaluation model. The value range of each dimension is normalized to . Calculate the normalized values of their five indicators for each client and record them in the corresponding coordinates on the radar chart. Then, further calculate the area enclosed by these five coordinates to output the reputation score of the current client. In this embodiment, the final task publisher obtains the direct reputation assessment of the client by calculating the area of the graph formed by the values of the five direct reputation assessment scales of the client . The calculation method of the pentagon area can effectively integrate the evaluation results of the five dimensions. The evaluation result of each dimension is regarded as a vertex of the pentagon, so that it can be ensured that the evaluation result of each dimension can be reflected in the final reputation evaluation. This method can not only reflect the advantages of the client in each dimension, but also reveal its deficiencies in some aspects. Secondly, the size of the pentagon area intuitively reflects the overall reputation level of the client. The larger the area, the better the performance of the client in the five dimensions and the higher the overall reputation; on the contrary, it means that the client's performance in some dimensions is poor and the overall reputation is low.

[0162] Combined with the appendix Figure 5The schematic diagram of the indirect reputation component shown is designed to comprehensively evaluate the client through multiple task publishers and model aggregators. The indirect reputation component can not only improve the diversity of evaluation perspectives, but also effectively ensure the quality and robustness of the reputation evaluation mechanism, thereby improving the fairness of the evaluation mechanism. In this embodiment, the direct publisher in the middle is the publisher of this task, and the others are other publishers who submit indirect reputation opinions. First, other publishers will provide the subjective reputation evaluation results of the subset, and the direct task publisher will calculate the similarity with a certain other publisher by combining its own subjective reputation evaluation results. Finally, the final reputation evaluation of the client is generated by combining the weighted evaluation results of the indirect task publisher with the subjective reputation evaluation results.

[0163] To avoid malicious publication of harmful reputation information by task publishers, the similarity between task publishers is measured based on cosine similarity and is as follows: The specific expression is:

[0164] ,

[0165] where, represents the score of task publisher for the client , ranges from 0 to 5, corresponding to the scores for model quality, data quality, reliability, and honesty respectively. represents 's average score. quantifies the similarity between task publisher and . This similarity score is obtained by calculating the cosine similarity of the direct reputation evaluations of two task publishers for the client . The cosine similarity score ranges from -1 (completely dissimilar) to 1 (identical), and the higher the score, the higher the similarity.

[0166] Furthermore, based on the above reputation evaluation part, the reputation value of the client is not only related to the current reputation but also to the historical reputation. The federated learning center server maintains a reputation record list for each client to store its historical reputation values. When calculating the reputation value of each client by weighting the reputation value of the client in the current training round and its historical reputation value, thus, the federated learning method of the reputation mechanism in the embodiment of the present invention further includes the following steps:

[0167] Step S401, obtain the historical reputation values of each client.

[0168] Step S402, correct the reputation values of each client according to the historical reputation values to obtain the corrected reputation values of each client.

[0169] The reputation values of each client are calibrated according to the historical reputation value, and the specific expression is:

[0170] ,

[0171] where, is the reputation value of the calibrated client , is the historical reputation value of the client , is the reputation value calibration weight. It should be noted that considering that the recent reputation value is closer to the actual situation, a greater weight is assigned to the recent reputation value of the client, that is . The above method of reputation value calibration is different from the method of simply multiplying by the decay coefficient. This calibration method also attaches importance to retaining the weight of some early reputations and avoids exponential decline to zero.

[0172] It should be understood that although each step in the attached Figure 1 flowchart is shown in sequence according to the arrow indication, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the attached figure may include multiple sub-steps or sub-stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0173] In the above embodiments disclosed in the present invention, the federated learning method based on the reputation mechanism is described in detail. The above method disclosed in the present invention can be implemented by devices in various forms. Therefore, the present invention also discloses a federated learning device based on the reputation mechanism. Combining the attached Figure 6 , specific embodiments are given below for detailed description.

[0174] The federated learning device based on the reputation mechanism is applied to a federated learning system. The federated learning system includes a central server and at least two clients. The federated learning device based on the reputation mechanism includes:

[0175] A model distribution module 501, configured to send an initial global model from the central server to each client;

[0176] A model training module 502, configured to train the initial global model by each client according to the local training data set to obtain a local model until the convergence condition is met, and send the local model and the training round behavior to the central server;

[0177] A reputation evaluation module 503, configured to enable the central server to obtain the reputation values of each client according to the local models of each client and the training round behavior;

[0178] A reputation screening module 504, configured to enable the central server to sort according to the reputation values of each client, and extract the local models of the clients according to the sorting result to obtain a list of models to be clustered;

[0179] A model quality evaluation module 505, configured to calculate the quality metrics of each local model based on the loss values of each local model in the list of models to be clustered;

[0180] A quality screening module 506, configured to traverse and detect the local models in the list of models to be clustered. If the quality metrics of the local model meet the preset quality conditions, add the local model to the global model update list;

[0181] A model aggregation module 507, configured to perform an aggregation process on all local models in the global model update list to obtain a global model;

[0182] A model output module 508, configured to repeat the above steps until the global model meets the output conditions, and save the updated global model.

[0183] For the federated learning device based on the reputation mechanism, all can refer to the above limitations on the method, and will not be elaborated here. Each module in the above device can be implemented in whole or in part by software, hardware and their combination. Each of the above modules can be embedded in the processor of the terminal device in hardware form or independent of it, or stored in the memory of the terminal device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0184] In one embodiment, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned federated learning method based on the reputation mechanism are implemented.

[0185] The computer-readable storage medium may be an electronic memory such as a flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), a hard disk, or a ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has a storage space for program codes for performing any of the method steps in the above method. These program codes may be read from or written into one or more computer program products, and the program codes may be compressed in a suitable form.

[0186] In one embodiment, the present invention provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it executes the above-described federated learning method based on a reputation mechanism.

[0187] The computer device includes a memory, a processor, and one or more computer programs. One or more of the computer programs may be stored in the memory and configured to be executed by one or more processors, and one or more application programs are configured to execute the above-described federated learning method based on a reputation mechanism.

[0188] The processor may include one or more processing cores. The processor connects various parts within the entire computer device using various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory, and by calling data stored in the memory, the processor executes various functions of the computer device and processes data. Optionally, the processor may be implemented in at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate a Central Processing Unit (CPU), a Graphics Processing Unit (GPU) for reporting and verifying buried point data, and a modem, etc., in a combination of one or more of them. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor and may be implemented separately by a communication chip.

[0189] The memory may include a Random Access Memory (RAM), and may also include a Read-Only Memory. The memory can be used to store instructions, programs, codes, code sets or instruction sets. The memory may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can also store data created during the use of the terminal device, etc.

[0190] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A federated learning method based on a reputation mechanism, the federated learning method based on the reputation mechanism being applied to a federated learning system, the federated learning system including a central server and clients, characterized in that, The federated learning method based on the reputation mechanism includes the following steps: The central server sends the initial global model to each client; Each client trains the initial global model according to the local training data set to obtain a local model until the convergence condition is met, and then sends the local model and the training round behavior to the central server; The central server obtains the reputation value of each client according to the local model and the training round behavior of each client; The central server sorts the reputation values of each client, and extracts the local models of the clients according to the sorting result to obtain a list of models to be clustered; Based on the loss values of each local model in the list of models to be clustered, calculate the quality index of each local model; Traverse and detect the local models in the list of models to be clustered. If the quality index of the local model meets the preset quality condition, add the local model to the global model update list; Perform an aggregation process on all local models in the global model update list to obtain a global model; Repeat the above steps until the global model meets the output condition, and save the updated global model.

2. The federated learning method based on a reputation mechanism according to claim 1, wherein The central server obtains the reputation value of each client according to the local model and the training round behavior of each client, including: The central server obtains the direct reputation and indirect reputation of each client according to the local model and the training round behavior of each client; Obtain the reputation value of each client according to the direct reputation and indirect reputation of each client; Among them, the direct reputation of the client includes the quality index of the client's local model, the data quality index of the client, the effectiveness index of the client's local model, the honesty index of the client, and the reliability index of the client. The indirect reputation of the client includes the multi-source information fusion index and the task publisher similarity index.

3. The federated learning method based on a reputation mechanism according to claim 2, wherein, The expression of the direct reputation of the client is: ， Among them, is the direct reputation of the client ; is the local model quality indicator of the client ; is the data quality indicator of the client ; is the local model effectiveness indicator of the client ; is the honesty indicator of the client ; is the reliability indicator of the client . Client The specific expression of the local model quality metric is as follows: ， For the client The regulation coefficient of the local model quality scale Is in Training rounds that do not include the client Of the temporary global model In the test set Of the loss function Is in Training rounds of the standard global model In the test set Of the loss function; The client The specific expression of the local model quality metric is as follows: ， For the client The regulation coefficient of the local data quality scale Is the weight for regulating the emphasis on quantity and quality For the client The degree of data label skew For the client The participation degree of the local dataset in this training round The client The specific expression of the local model validity index is as follows: ， For the client Adjustment coefficient for the validity of the local model For the client Probability of being selected to participate in training For the client Probability of submitting the local model on time For the client Probability of being selected to participate in the global model aggregation For the client Weight of being selected to participate in training For the client Weight of submitting the local model on time For the client Weight of being selected to participate in the global model aggregation The client The specific expression of the honesty index is: ， For the client Adjustment coefficient of the honesty index For the client Lazy behavior score For the client Malicious behavior score The client The specific expression of the reliability index is as follows: ， is the client reliability index adjustment coefficient is the client computing resources is the client network resources is the weight for regulating computing resources and network resources 4. The federated learning method based on a reputation mechanism according to claim 3, wherein, The expression of the indirect reputation of the client is: ， For the task publisher Regarding the indirect reputation value of the client For the task publisher Regarding the direct reputation score of the client For the task publisher And Regarding the similarity For the set of task publishers who provide indirect reputation values to the task publisher ​​​ 5. The federated learning method based on a reputation mechanism according to claim 4, wherein It also includes: Obtain the historical reputation value of each client; Correct the reputation value of each client according to the historical reputation value to obtain the corrected reputation value of each client.

6. The federated learning method based on a reputation mechanism according to claim 5, wherein In the step where the central server sorts the reputation values of each client and extracts the local models of the clients according to the sorting result to obtain a list of models to be clustered, the probability of each client being extracted is specifically: The local model of the client whose reputation value ranking is greater than the first threshold is directly included in the list of models to be clustered; For the client whose reputation value is greater than the second threshold and whose ranking is less than the first threshold, extract the local model of the client according to the first probability and include it in the list of models to be clustered; For the client whose reputation value is less than the second threshold, extract the local model of the client according to the second probability and include it in the list of models to be clustered.

7. The federated learning method based on a reputation mechanism according to claim 6, wherein The specific expression of the preset quality condition is: ， Among them, a batch of test sample sets are extracted by the server from the test dataset it maintains, is the local model quality threshold.

8. The federated learning method based on the reputation mechanism according to claim 7, characterized in that The specific expression of performing an aggregation process on all local models in the global model update list to obtain a global model is: ， Among them, is the global model, is for all local models in the global model update list, is for the task publisher when the client has a reputation value, is the local model of the client and is the test accuracy on the server test set.

9. A federated learning device based on a reputation mechanism, the federated learning device based on the reputation mechanism being applied to a federated learning system, the federated learning system including a central server and at least two clients, characterized in that, The federated learning device based on the reputation mechanism includes: A model distribution module for the central server to send the initial global model to each client; A model training module, which is used for each client to train the initial global model according to the local training dataset to obtain a local model until the convergence condition is met, and send the local model and the training round behavior to the central server; A reputation evaluation module, which is used for the central server to obtain the reputation values of each client according to the local models and training round behaviors of each client; A reputation screening module, which is used for the central server to sort according to the reputation values of each client, and extract the local models of the clients according to the sorting result to obtain a list of models to be clustered; A model quality evaluation module, which is used to calculate the quality indicators of each local model based on the loss values of each local model in the list of models to be clustered; A quality screening module, which is used to traverse and detect the local models in the list of models to be clustered. If the quality indicator of the local model meets the preset quality condition, the local model is added to the global model update list; A model aggregation module, which is used to perform aggregation processing on all local models in the global model update list to obtain a global model; A model output module, which is used to repeat the above steps until the global model meets the output condition, and save the updated global model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the federated learning method based on the reputation mechanism according to any one of claims 1-8.

11. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it executes the federated learning method based on the reputation mechanism according to any one of claims 1-9.

Images