Federal learning method, system and device based on third-party assistance and storage medium
By introducing a third-party auxiliary server for dynamic trustworthiness assessment and weighted aggregation, the problem of insufficient federated learning defense capabilities in the Internet of Things environment is solved, achieving low-overhead, high-efficiency malicious attack defense and privacy protection.
Patent Information
- Application Number
- CN202511824473.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-05
- Publication Date
- 2026-03-13
AI Technical Summary
In closed or restricted environments such as the Internet of Things, federated learning has insufficient defense capabilities, especially when facing malicious clients, it is difficult to maintain efficient model performance, and existing defense measures have high computational overhead and are difficult to update in a timely manner.
A third-party auxiliary server is introduced to dynamically select certain communication rounds for trust assessment, generate trust scores, and perform weighted aggregation through a central server. An external Byzantine robust defense mechanism is implemented, and noise addition and compression are used to protect privacy and reduce system resource consumption.
Without refactoring the existing framework, it enhances the defensive capabilities of federated learning, reduces computational and communication overhead, effectively resists malicious attacks, and ensures model performance and privacy protection.
Smart Images
Figure CN121664501A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of federated learning technology, and in particular to third-party assisted federated learning methods, systems, devices, and storage media. Background Technology
[0002] With the development of technologies such as the Internet of Things (IoT) and edge computing, more and more devices are acquiring computing capabilities and generating massive amounts of data. Traditional machine learning techniques rely on aggregating large amounts of data to a central server for centralized training. However, for applications involving sensitive user information, directly providing data to the server poses a privacy risk. Furthermore, uploading data to a central server consumes significant network bandwidth, making it difficult to meet the low-latency requirements of IoT edge devices. Due to the distributed nature of federated learning, its attack surface is expanded, and different nodes are vulnerable to model poisoning attacks. This significantly hinders the performance of federated models, leading to a sharp decline in their accuracy.
[0003] However, most methods deeply intervene in federated learning, making it difficult to update defenses in a timely manner, which is particularly challenging when federated learning is deployed in closed environments such as the Internet of Things (IoT). Furthermore, these defense mechanisms impose additional computational overhead on the central server, which may exacerbate resource consumption issues given the limited communication bandwidth in IoT. Moreover, most methods struggle to maintain efficient model performance when most clients are malicious.
[0004] Therefore, how to improve the defense capabilities of federated learning in closed or restricted environments such as the Internet of Things is an urgent problem to be solved. Summary of the Invention
[0005] The main purpose of this application is to provide a third-party assisted federated learning method, system, device, and storage medium, aiming to solve the technical problem of how to improve the defense capabilities of federated learning in closed or restricted environments such as the Internet of Things.
[0006] To achieve the above objectives, this application proposes a third-party-assisted federated learning method. This method is applied to a federated learning system, which includes a central server, auxiliary servers, and multiple clients. The method comprises: The central server obtains the initial global model parameters and distributes them to multiple clients respectively; Each client performs local training based on the initial global model parameters, generates a local model update, and uploads the local model update to the central server; The auxiliary server dynamically selects certain communication rounds as evaluation rounds, and performs a credibility assessment on the local model updates uploaded by the client in the evaluation rounds, generating corresponding trust scores. The central server performs weighted aggregation of the local model updates of each client based on the trust score, and updates the initial global model parameters to obtain the target global model.
[0007] In one embodiment, the auxiliary server dynamically selects certain communication rounds as evaluation rounds, and performs a credibility assessment on the local model updates uploaded by the client in the evaluation rounds, generating corresponding trust scores, including: The auxiliary server assigns a round weight to each communication round and dynamically samples the communication rounds based on the round weight to determine the evaluation round set; The auxiliary server sends the set of evaluation rounds to the central server; The central server performs security processing on the local model update based on the set of evaluation rounds to obtain a security-processed local model update, and then sends the security-processed local model update to the auxiliary server. The auxiliary server evaluates the credibility of the local model update after security processing based on the auxiliary verification data, and generates a corresponding trust score.
[0008] In one embodiment, the central server performs security processing on the local model update based on the evaluation round set to obtain a security-processed local model update, and sends the security-processed local model update to the auxiliary server, including: The central server performs noise addition and compression processing on the local model update based on the set of evaluation rounds to obtain a safe local model update. The local model update after security processing is packaged, an audit tuple is generated, and the audit tuple is sent to the auxiliary server.
[0009] In one embodiment, the central server, based on the set of evaluation rounds, performs noise addition and compression processing on the local model update to obtain a safe-processed local model update, including: The central server, based on the set of evaluation rounds, adds noise perturbation to the local model update according to the analytical Gaussian mechanism to obtain the first processing result; The first processing result is compressed using stochastic gradient descent with gradient sign bit to obtain a locally updated model with secure processing.
[0010] In one embodiment, the auxiliary server performs a credibility assessment on the security-processed local model update based on auxiliary verification data, generating a corresponding trust score, including: The auxiliary server trains a baseline model based on the auxiliary validation dataset and calculates the baseline model update; The baseline model update is privacy-processed based on the analytical Gaussian mechanism to obtain the privacy-processed baseline model update. Calculate the cosine similarity between the privacy-processed baseline model update and the security-processed local model update; The cosine similarity is cropped to generate a trust score corresponding to the client.
[0011] In one embodiment, the central server performs weighted aggregation of the local model updates of each client based on the trust score, and updates the initial global model parameters to obtain the target global model, including: For each evaluation round, the central server determines a first weight based on the trust score; For non-evaluation rounds, the central server calculates the second weight based on historical gradients; The local model updates of each client are weighted and aggregated based on the first weight and the second weight, and the initial global model parameters are updated to obtain the updated global model parameters, which are then distributed to multiple clients for iterative training. When the global model converges, the target global model is determined based on the updated global model parameters.
[0012] In one embodiment, for non-evaluation rounds, the central server calculates the second weight based on historical gradients, including: For non-evaluation rounds, the central server calculates the forgetting factor based on the relationship between the current round gradient and the nearest historical gradient to obtain the second weight.
[0013] In addition, to achieve the above objectives, this application also proposes a third-party assisted federated learning system, which includes: a central server, an auxiliary server, and multiple clients; The central server is used to obtain initial global model parameters and distribute them to multiple clients respectively. Each of the aforementioned clients is used to perform local training based on the initial global model parameters, generate local model updates, and upload the local model updates to the central server; The auxiliary server is used to dynamically select certain communication rounds as evaluation rounds, and to evaluate the credibility of the local model updates uploaded by the client in the evaluation rounds, generating corresponding trust scores. The central server is also used to perform weighted aggregation of the local model updates of each client based on the trust score, and update the initial global model parameters to obtain the target global model.
[0014] In addition, to achieve the above objectives, this application also proposes an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the third-party assisted federated learning method as described above.
[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the third-party assisted federated learning method described above.
[0016] One or more technical solutions proposed in this application have at least the following technical effects: The central server obtains initial global model parameters and distributes them to multiple clients. Each client trains locally based on these parameters, generates a local model update, and uploads it to the central server. An auxiliary server dynamically selects certain communication rounds as evaluation rounds and assesses the credibility of the local model updates uploaded by clients during these rounds, generating corresponding trust scores. The central server then performs weighted aggregation of the local model updates from each client based on these trust scores and updates the initial global model parameters to obtain the target global model. By introducing an independent auxiliary server, the Byzantine robustness defense mechanism is decoupled from the central server, externalizing the defense functionality. The central server can maintain its original model distribution and aggregation logic, while delegating the complex model update credibility assessment task to the auxiliary server. Without reconstructing the existing federated learning framework, this non-intrusive, low-overhead, and privacy-preserving approach empowers the system with strong resistance to most malicious client Byzantine attacks, while ensuring the sustainable upgrade potential of the defense mechanism itself.
[0017] The auxiliary server employs a dynamic sampling strategy, evaluating only a portion of the communication rounds rather than intervening throughout the entire process, significantly reducing the continuous consumption of system communication and computing resources. In the selected evaluation rounds, the central server first preprocesses the original model updates uploaded by the client by compressing and adding noise. This drastically reduces the amount of data that needs to be transmitted to the auxiliary server, improving communication efficiency, and effectively prevents potential privacy leaks during the evaluation process through differential privacy technology. Attached Figure Description
[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating the first embodiment of the third-party assisted federated learning method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the third-party assisted federated learning method of this application; Figure 3 This is a flowchart illustrating the third embodiment of the third-party assisted federated learning method of this application; Figure 4 This is a schematic diagram of the structure of the third-party assisted federated learning system in this application.
[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0022] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0023] This application provides a third-party assisted federated learning method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the third-party-assisted federated learning method of this application. It should be noted that the third-party-assisted federated learning method of this application can be applied to a federated learning system, which includes a central server, auxiliary servers, and multiple clients. It should also be noted that a federated learning system can be understood as a distributed machine learning framework. In this embodiment, the third-party-assisted federated learning method includes steps S10-S40:
[0024] In step S10, the central server obtains the initial global model parameters and distributes them to multiple clients.
[0025] It should be noted that the central server can be understood as the system's coordination center, responsible for maintaining and distributing the global model and aggregating model updates uploaded by various clients. Clients can be understood as participating devices with local data, such as mobile phones and IoT devices, responsible for using their local data to train the global model. The initial global model parameters are the initial state of the model parameters set by the central server or randomly generated before federated learning training begins.
[0026] For example, in the central server and client Establish a secure network link between them. The central server initializes global model parameters. Set other update parameters related to model training, such as the learning rate. and loss function And then distribute it to each client.
[0027] In step S20, each client performs local training based on the initial global model parameters, generates a local model update, and uploads the local model update to the central server.
[0028] It's important to note that local model updates refer to information about how the model parameters were adjusted after local training on the client side, such as gradients or parameter differences. These are data entities uploaded by the client to the server. For example, a local client... Using the collected global model parameters, in its private dataset Perform local model training. This includes local model updates. Through formula Calculation, where Representing the client In the communication round The next global model parameters received. (Client) Update the local model Send to the central server.
[0029] In step S30, the auxiliary server dynamically selects some communication rounds as evaluation rounds, and evaluates the credibility of the local model updates uploaded by the client in the evaluation rounds, generating corresponding trust scores.
[0030] It's important to note that a communication round can be understood as a complete iteration of federated learning, encompassing the server distributing the model, client-side local training, client-side update uploads, and server-side aggregation updates. An evaluation round can be understood as a specific communication round dynamically selected by the auxiliary server for credibility evaluation; evaluation is not performed in every round to reduce system overhead. Credibility evaluation can be understood as the core operation performed by the auxiliary server: determining whether each client's uploaded local model update is credible and beneficial for the correct training of the global model. For example, the auxiliary server can train a trust baseline model using a pre-collected clean dataset, and generate a trust score for each client by calculating the similarity between each client's local model update and this baseline model update, then feed this score back to the central server.
[0031] In step S40, the central server performs weighted aggregation of the local model updates of each client based on the trust score, and updates the initial global model parameters to obtain the target global model.
[0032] It should be noted that the trust score can be understood as the quantitative result generated by the credibility assessment process, and the target global model parameters can be understood as the global model parameters when the federated learning finally converges or reaches the expected performance after multiple rounds of iterative training. It is the output of the entire training process and is an optimized final model that can effectively complete the task and is robust against poisoning attacks.
[0033] In this embodiment, by introducing an independent auxiliary server, the Byzantine robustness defense mechanism is decoupled from the central server, thus externalizing the defense function. The central server can maintain its original model distribution and aggregation logic, while delegating the complex model update credibility evaluation task to the auxiliary server. The auxiliary server, through a dynamic sampling strategy, evaluates only a portion of the communication rounds, rather than intervening throughout, significantly reducing the continuous consumption of system communication and computing resources. In the selected evaluation rounds, the central server first preprocesses the original model updates uploaded by the client by compressing and adding noise. This significantly reduces the amount of data that needs to be transmitted to the auxiliary server, improving communication efficiency, and effectively prevents potential privacy leaks during the evaluation process through differential privacy technology. The auxiliary server then uses a baseline model trained on a clean and reliable auxiliary dataset as a root of trust. It generates a trust score by calculating the similarity between the client update and the baseline update, thereby identifying and suppressing poisoned attack updates submitted by malicious clients. Ultimately, the central server uses these objective trust scores from third parties as weights to weight and aggregate updates from all clients. This amplifies the contributions of honest clients while weakening the impact of malicious clients. Therefore, the method in this embodiment, without reconstructing the existing federated learning framework, provides the system with a strong ability to resist most malicious client Byzantine attacks in a non-intrusive, low-overhead, and privacy-preserving manner, while ensuring the sustainable upgrade potential of the defense mechanism itself.
[0034] Reference Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the third-party assisted federated learning method of this application. Based on the above... Figure 1 The first embodiment shown presents a second embodiment of the federated learning method based on third-party assistance.
[0035] In the second embodiment, step S30 includes: In step S301, the auxiliary server assigns round weights to each communication round and dynamically samples the communication rounds based on the round weights to determine the evaluation round set.
[0036] It's important to note that round weights assign a probability weight to each communication round being selected, typically based on the round number. For example, earlier rounds have higher weights, while later rounds have lower weights, prioritizing the evaluation of more important rounds. Dynamic sampling can be understood as randomly selecting a subset of rounds for evaluation based on their round weights, rather than evaluating all rounds, to reduce overhead. The evaluation round set is the same as the sampling round set, which is the set of communication rounds selected for evaluation.
[0037] In step S302, the auxiliary server sends the evaluation round set to the central server.
[0038] For example, the auxiliary server for each communication round Allocate a round weight to each The corresponding formula is: Then the first Round sampling probability It can be calculated using the following formula, which is the probability calculation formula: in, Given the sum of all communication rounds, it's understandable that rounds with higher weights are more likely to be sampled. Dynamic sampling is used to determine the communication rounds to be evaluated, and the set of sampled rounds is then compiled. The results are sent to the central server, where This represents the number of rounds selected.
[0039] In step S303, the central server performs security processing on the local model update based on the evaluation round set, obtains the security-processed local model update, and sends the security-processed local model update to the auxiliary server.
[0040] It should be noted that secure processing can protect local data privacy and reduce communication overhead. For example, secure processing can include gradient compression and differential privacy techniques. Securely processed local model updates can be smaller and contain noise, preventing information leakage.
[0041] In step S304, the auxiliary server performs a credibility assessment on the local model update after security processing based on the auxiliary verification data, and generates a corresponding trust score.
[0042] It should be noted that auxiliary validation data can be understood as a clean and reliable dataset used to train the baseline model and serve as a reference for credibility assessment. The auxiliary validation dataset collected by the auxiliary server can be used... express.
[0043] In this embodiment, the auxiliary server assigns round weights to each communication round and dynamically samples based on these weights, selecting only a subset of rounds as the evaluation round set. This solves the problem of high computational and communication overhead in traditional technologies, significantly reducing system load compared to full evaluation. It also prioritizes early rounds crucial for model convergence, improving evaluation efficiency. Next, the auxiliary server sends the evaluation round set to the central server, ensuring the coordination of the evaluation process. Upon receiving the evaluation round set, the central server performs secure processing on relevant local model updates, resolving privacy leaks and communication bandwidth waste issues in the original update transmission. Compared to directly transmitting the original update, this achieves data compression and privacy protection, reducing transmission overhead. Finally, the auxiliary server trains a baseline model based on auxiliary validation data and performs a credibility assessment on the computationally secure local model updates, generating a trust score. This solves the problem of difficult detection of malicious client poisoning attacks, enabling objective third-party evaluation. This allows the central server to effectively suppress malicious updates through weighted aggregation, improving the Byzantine robustness of the global model.
[0044] In one implementation, step S303 includes: the central server performs noise addition and compression processing on the local model update based on the evaluation round set to obtain a safe local model update; the safe local model update is packaged to generate an audit tuple, and the audit tuple is sent to the auxiliary server.
[0045] It should be noted that an audit tuple can be understood as a structured data packet, which may include the model structure of the current round, global model parameters, hyperparameters, and gradient parameters, etc. For example, the central server packages the model information for the selected round into audit tuples. And send it to the auxiliary server, where Represents a neural network model. Indicates that the current selection is made. Global model parameters of the wheel, Hyperparameters such as convergence measure This indicates that it represents the client. The gradient.
[0046] Specifically, the central server performs noise addition and compression processing on the local model update based on the evaluation round set to obtain a safe local model update, which includes: the central server adds noise perturbation to the local model update based on the evaluation round set according to the analytical Gaussian mechanism to obtain the first processing result; and compresses the first processing result according to the stochastic gradient descent method with gradient sign bit to obtain the safe local model update.
[0047] For example, the central server receives the sampling round set Then, the selected rounds are processed using an analytical Gaussian mechanism. Local model update under wheels By applying noise perturbation and combining it with the SGD method (gradient sign-taking), the gradient is further compressed to further limit the data transmission size. After compression and perturbation, a securely processed local model update is obtained. , Represented as:
[0048] in, The cumulative distribution function of a standard Gaussian distribution, and the noise standard deviation. It satisfies the analytic Gaussian mechanism, and DP(Sign(·)) is a differential privacy symbol encoder designed to compress the client-side local gradient.
[0049] In this implementation, the central server first adds noise and compresses the local model updates for selected evaluation rounds, then packages the results into audit tuples and sends them to the auxiliary server. The noise addition process masks potential client-side private data features in the model updates, resolving privacy leaks that may occur during gradient transmission and evaluation. Compared to directly transmitting the original gradients, this effectively protects client data privacy. Simultaneously, the compression process significantly simplifies the originally dense gradient vectors, addressing the issues of excessively large original update data volume and high transmission overhead. Compared to transmitting the complete update, this significantly reduces network bandwidth usage. Furthermore, by packaging the processed updates and related contextual information into structured audit tuples, the auxiliary server obtains complete and consistent evaluation background information, resolving evaluation bias issues caused by fragmented evaluation information. Compared to piecemeal transmission, this improves the reliability and efficiency of the credibility evaluation process.
[0050] In one implementation, step S304 includes: the auxiliary server training a benchmark model based on the auxiliary verification dataset and calculating the benchmark model update; performing privacy processing on the benchmark model update based on the analytic Gaussian mechanism to obtain the privacy-processed benchmark model update; calculating the cosine similarity between the privacy-processed benchmark model update and the security-processed local model update; and performing pruning processing on the cosine similarity to generate a trust score corresponding to the client.
[0051] It's important to note that the baseline model can be understood as the model trained by the auxiliary server on the auxiliary validation dataset using the global model parameters of the current round. The baseline model can be considered a trustworthy, uncontaminated reference model. The baseline model update represents the gradient or parameter change calculated during its training. Privacy processing adds noise to the baseline model update to further protect the information security of the auxiliary validation dataset, ensuring it meets differential privacy requirements. Cosine similarity is used to quantify the consistency between each client's local update direction and the trustworthy baseline update direction. Pruning can be understood as setting the less than 0 parts of the cosine similarity value to 0. Negative similarity implies an update direction completely opposite to the correct direction, potentially indicating malicious updates; setting its weight to 0 directly excludes its influence. After generating the trust score corresponding to each client, normalization can be performed to obtain normalization coefficients. The auxiliary server can then package the trust score and normalization coefficients together and send them to the central server.
[0052] For example, the auxiliary server receives the audit tuple. Then, in the auxiliary validation dataset Train a trust benchmark model And calculate model update The auxiliary server uses the parsed Gaussian mechanism to handle baseline model updates. get Then, the cosine similarity between the baseline model update and the local model update is calculated. , The calculation formula is:
[0053] in, This represents L2 regularization. The auxiliary server utilizes... Functions on similarity values Pruning is performed to obtain a trust score for each local model update. That is, when hour, ,otherwise The auxiliary server can obtain the normalization coefficients for each local client by calculating the normalization coefficients. Then and All data is sent to the central server. The normalization factor is calculated using the following formula:
[0054] In this implementation, the auxiliary server trains a baseline model and calculates its updates based on a clean auxiliary validation dataset, achieving a more reliable evaluation source using clean external data as a benchmark. Since the baseline model update also undergoes privacy processing based on an analytical Gaussian mechanism, the evaluation process not only protects client data privacy but also prevents potential leakage of auxiliary validation set information. Compared to an unprotected baseline, this ensures the security of the auxiliary server's own data and the privacy compliance of the process. By calculating the cosine similarity between the privacy-processed baseline update and the client update, the credibility of the client update is transformed into a measure of the consistency between its update direction and the credible direction, enabling more effective identification of directional poisoning attacks. Furthermore, by pruning the cosine similarity and setting negative values to zero, known malicious behaviors are filtered out. Finally, by introducing a normalization coefficient and sending it along with the trust score to the central server, a more refined weighted balance is achieved during aggregation, simultaneously considering the credibility of the update direction and the relative proportion of the magnitude.
[0055] Reference Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the third-party assisted federated learning method of this application, based on the above. Figure 2 The second embodiment shown presents a third embodiment of the federated learning method based on third-party assistance.
[0056] In the third embodiment, step S40 includes: In step S401, for each evaluation round, the central server determines the first weight based on the trust score.
[0057] It should be noted that the first weight refers to the trust score directly provided by the auxiliary server in the evaluation round, which serves as the weight of the client in that round of aggregation.
[0058] In step S402, for non-evaluation rounds, the central server calculates the second weight based on the historical gradient.
[0059] It should be noted that non-evaluation rounds can be understood as communication rounds that were not selected by the auxiliary server and did not undergo formal credibility evaluation. Historical gradients refer to the local model updates uploaded by the client in previous communication rounds. The second weight refers to a weight calculated by the central server in non-evaluation rounds; it is not a direct evaluation from a third party, but an estimation based on the client's own historical update behavior. Step S402 includes: for non-evaluation rounds, the central server calculates the forgetting factor based on the relationship between the current round's gradient and the nearest historical gradient to obtain the second weight.
[0060] For example, for communication rounds that were not selected for evaluation, a forgetting factor will be calculated. The weights of each local client in this round are obtained using the following formula: in, Indicates the current round The nearest historical gradient, where For the current round The gradient.
[0061] Step S403: Based on the first weight and the second weight, the local model updates of each client are weighted and aggregated, and the initial global model parameters are updated to obtain the updated global model parameters, which are then distributed to multiple clients for iterative training.
[0062] For example, the central server obtains a new round of global model updates by weighted averaging, as shown below: The updated global model parameters are used as the first... The global model is distributed to each local client in round +1.
[0063] Step S404: When the global model converges, determine the target global model based on the updated global model parameters.
[0064] For example, the central server distributes the latest global model update to each client, and each client performs a new round of training based on its local dataset. This process is repeated iteratively until the global model converges. Global model convergence can be understood as the training process reaching a stable state, manifested as the global model's performance no longer significantly improving or the loss function value no longer significantly decreasing. The target global model can be understood as the final global model that meets the expected performance requirements when the global model converges.
[0065] In this embodiment, by directly using the trust score provided by the third-party auxiliary server as the first weight for evaluation rounds, it is ensured that model aggregation is entirely based on objective and reliable credibility metrics in rounds with third-party evaluation, effectively suppressing malicious updates. For non-evaluation rounds, a forgetting factor calculated based on the relationship between the client's historical gradient and the current gradient magnitude is introduced to obtain the second weight, cleverly solving the problem of the lack of direct trust scores in non-evaluation rounds. The credibility of the current round is inferred by utilizing the continuity of the client's own update behavior, achieving continuity of the defense mechanism and preventing malicious clients from launching attacks in non-evaluation rounds. Furthermore, by integrating the first and second weights for weighted aggregation, the system can differentiate the weighting of client updates regardless of whether the rounds are supervised by a third party or maintained solely by itself. Finally, through iterative training until the model converges and the target global model parameters are obtained, it is ensured that the final output model not only meets the performance requirements but also possesses the anti-interference capability provided by the aforementioned dynamic weight mechanism.
[0066] Based on the same inventive concept, such as Figure 4 As shown, this application also provides a third-party assisted federated learning system, which includes a central server 10, an auxiliary server 20, and multiple clients 30. It should be noted that the central server 10 has a network connection channel with each of the multiple clients 30, and the auxiliary server 20 also has a network connection channel with each of the multiple clients 30. Similarly, there is a network connection channel between the central server 10 and the auxiliary server 20.
[0067] The central server 10 is used to obtain initial global model parameters and distribute them to multiple clients 30. Clients 30 are used to perform local training based on the initial global model parameters, generate local model updates, and upload the local model updates to the central server. The auxiliary server 20 is used to dynamically select some communication rounds as evaluation rounds, evaluate the credibility of the local model updates uploaded by the clients in the evaluation rounds, and generate corresponding trust scores. The central server 10 is also used to perform weighted aggregation of the local model updates of each client 30 based on the trust scores, and update the initial global model parameters to obtain the target global model.
[0068] It should be noted that the third-party-assisted federated learning system provided in this application embodiment and the third-party-assisted federated learning method provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned third-party-assisted federated learning method, and the repeated parts will not be described again.
[0069] In some embodiments, an electronic device provided in this application includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the above-described third-party assisted federated learning method.
[0070] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.
[0071] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.
[0072] This application also provides a storage medium, which is a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the aforementioned third-party-assisted federated learning method. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments or implementations of this application.
[0073] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0074] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A federated learning method based on third-party assistance, characterized in that, The third-party-assisted federated learning method is applied to a federated learning system, which includes a central server, auxiliary servers, and multiple clients. The method includes: The central server obtains the initial global model parameters and distributes them to multiple clients respectively; Each client performs local training based on the initial global model parameters, generates a local model update, and uploads the local model update to the central server; The auxiliary server dynamically selects certain communication rounds as evaluation rounds, and performs a credibility assessment on the local model updates uploaded by the client in the evaluation rounds, generating corresponding trust scores. The central server performs weighted aggregation of the local model updates of each client based on the trust score, and updates the initial global model parameters to obtain the target global model.
2. The method as described in claim 1, characterized in that, The auxiliary server dynamically selects certain communication rounds as evaluation rounds, and performs a credibility assessment on the local model updates uploaded by the client in the evaluation rounds, generating corresponding trust scores, including: The auxiliary server assigns a round weight to each communication round and dynamically samples the communication rounds based on the round weight to determine the evaluation round set; The auxiliary server sends the set of evaluation rounds to the central server; The central server performs security processing on the local model update based on the set of evaluation rounds to obtain a security-processed local model update, and then sends the security-processed local model update to the auxiliary server. The auxiliary server evaluates the credibility of the local model update after security processing based on the auxiliary verification data, and generates a corresponding trust score.
3. The method as described in claim 2, characterized in that, The central server performs security processing on the local model update based on the set of evaluation rounds to obtain a security-processed local model update, and then sends the security-processed local model update to the auxiliary server, including: The central server performs noise addition and compression processing on the local model update based on the set of evaluation rounds to obtain a safe local model update. The local model update after security processing is packaged, an audit tuple is generated, and the audit tuple is sent to the auxiliary server.
4. The method as described in claim 3, characterized in that, The central server, based on the set of evaluation rounds, performs noise addition and compression processing on the local model update to obtain a safe-processed local model update, including: The central server, based on the set of evaluation rounds, adds noise perturbation to the local model update according to the analytical Gaussian mechanism to obtain the first processing result; The first processing result is compressed using stochastic gradient descent with gradient sign bit to obtain a locally updated model with secure processing.
5. The method as described in claim 2, characterized in that, The auxiliary server, based on the auxiliary verification data, performs a credibility assessment on the security-processed local model update and generates a corresponding trust score, including: The auxiliary server trains a baseline model based on the auxiliary validation dataset and calculates the baseline model update; The baseline model update is privacy-processed based on the analytical Gaussian mechanism to obtain the privacy-processed baseline model update. Calculate the cosine similarity between the privacy-processed baseline model update and the security-processed local model update; The cosine similarity is cropped to generate a trust score corresponding to the client.
6. The method as described in claim 1, characterized in that, The central server performs weighted aggregation of the local model updates of each client based on the trust score, and updates the initial global model parameters to obtain the target global model, including: For each evaluation round, the central server determines a first weight based on the trust score; For non-evaluation rounds, the central server calculates the second weight based on historical gradients; The local model updates of each client are weighted and aggregated based on the first weight and the second weight, and the initial global model parameters are updated to obtain the updated global model parameters, which are then distributed to multiple clients for iterative training. When the global model converges, the target global model is determined based on the updated global model parameters.
7. The method as described in claim 6, characterized in that, For non-evaluation rounds, the central server calculates the second weight based on historical gradients, including: For non-evaluation rounds, the central server calculates the forgetting factor based on the relationship between the current round gradient and the nearest historical gradient to obtain the second weight.
8. A federated learning system based on third-party assistance, characterized in that, The third-party-assisted federated learning system includes: a central server, auxiliary servers, and multiple clients; The central server is used to obtain initial global model parameters and distribute them to multiple clients respectively. Each of the aforementioned clients is used to perform local training based on the initial global model parameters, generate local model updates, and upload the local model updates to the central server; The auxiliary server is used to dynamically select certain communication rounds as evaluation rounds, and to evaluate the credibility of the local model updates uploaded by the client in the evaluation rounds, generating corresponding trust scores. The central server is also used to perform weighted aggregation of the local model updates of each client based on the trust score, and update the initial global model parameters to obtain the target global model.
9. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the third-party assisted federated learning method as described in any one of claims 1 to 7.
10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the third-party assisted federated learning method as described in any one of claims 1 to 7.