Dual-layer filtering collaborative federated learning method, system, device and storage medium
By employing a two-layer filtering collaborative federated learning method, and utilizing dynamic norm boundaries and norm condensation clustering to identify malicious model updates, this approach addresses the vulnerabilities in privacy protection and poisoning detection inherent in existing federated learning technologies, achieving accurate identification of malicious attacks and strong privacy protection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUIZHOU UNIV
- Filing Date
- 2026-01-16
- Publication Date
- 2026-05-29
AI Technical Summary
Existing federated learning technologies are vulnerable in terms of privacy protection and poison detection, making it difficult to distinguish between normal updates and malicious attacks, and unable to effectively identify Sybil attacks in which multiple attackers coordinate, resulting in a decline in defense capabilities.
A two-layer filtering collaborative approach is adopted. First, malicious model updates with obvious anomalies are screened through dynamic norm boundaries. Then, norm agglomeration clustering is used to identify Sybil attacks. Finally, privacy-preserving aggregation is performed to generate a global update model.
It achieves accurate identification and effective defense against malicious attacks, while also possessing strong privacy protection capabilities, adapting to the security and confidentiality needs of industrial scenarios.
Smart Images
Figure CN122114079A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of federated learning technology, and in particular to a two-layer filtering collaborative federated learning method, system, device and storage medium. Background Technology
[0002] Industry 5.0 aims to create a highly flexible production model using information and intelligent technologies, and its success hinges on the integration of massive, multi-source production data. While strict privacy regulations have led to severe data silos, federated learning, as an emerging paradigm, offers the possibility of collaborative modeling without sharing raw data.
[0003] However, the inherent mechanisms of federated learning make it significantly vulnerable in both security and privacy dimensions, both of which are unacceptable for industrial applications. On the privacy front, attackers can exploit model updates to carry out reconstruction and attribute inference attacks, stealing critical trade secrets or production parameters. On the security front, poisoning attacks launched by malicious users can directly undermine the integrity and usability of the model, leading to incorrect production predictions or quality judgments, severely impacting system utility.
[0004] Therefore, privacy protection and poison detection are not two independent issues, but a core challenge that must be addressed collaboratively. Current research combining the two relies on simple outlier detection, a method that not only fails to defend against highly covert threats such as Sybil attacks, but may even introduce new risks, rendering it impractical in complex industrial scenarios.
[0005] In existing technologies, privacy protection methods that encrypt or add noise to the entire model severely disrupt the inherent characteristics of model updates, making it difficult for servers to distinguish between normal updates and malicious attacks, thus reducing defense capabilities. At the same time, defense methods that rely on single indicators such as norm pruning or distance metrics cannot cope with carefully crafted adaptive attacks by attackers that are statistically similar to benign users, and are even less able to identify sybil attacks where multiple attackers act in concert. In addition, in many privacy protection schemes based on secret sharing, if a user disconnects midway, the secret cannot be fully reconstructed, affecting the normal operation of the system. Summary of the Invention
[0006] This application provides a two-layer filtering collaborative federated learning method, system, device, and storage medium, which solves the technical problems of existing privacy protection methods that encrypt or add noise to the entire model, which seriously disrupt the inherent characteristics of model updates, making it difficult for servers to distinguish between normal updates and malicious attacks, resulting in a decrease in defense capabilities. At the same time, defense methods that rely on single indicators such as norm pruning or distance metrics cannot cope with carefully constructed adaptive attacks by attackers that are statistically similar to benign users, and it is even more difficult to identify sybil attacks in which multiple attackers act in concert.
[0007] In view of this, the first aspect of this application provides a two-layer filtering collaborative federated learning method, the method comprising: Step S1: Each client iteratively trains its local model based on the initial dataset to obtain a locally updated model, encrypts the locally updated model, and then uploads it to the server. Step S2: The server performs a malicious model update filtering based on the dynamic norm boundary on the encrypted local update models uploaded by each client. Step S3: The server performs a second filtering based on norm agglomerative clustering on the local update model after the first filtering to identify the local model update corresponding to the malicious client. Step S4: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. Step S5: The server sends a global update model to each client, causing each client to iteratively execute step S1.
[0008] Optionally, step S1 specifically includes: Step S11, each client Based on the initial dataset and the previous round of global update model Perform model updates to obtain the locally updated model. , .
[0009] Step S12, each client With server Seeds of negotiation Meanwhile, each client With another client Seeds of negotiation This makes each client The local update model is calculated. noise satisfy ,in, This represents the noise generated during seed negotiation between clients. This indicates the noise generated by the seed selected by the client; Step S13, each client Local update model based on noise Encrypt to obtain Among them, the local update model The model parameters are divided into two parts: the front and the tail. , This represents a position index of the gradient vector; Step S14, each client The local model is updated using the Shamir secret sharing algorithm. Two noises as well as After splitting the noise into n parts, a preset number of noise parts are retained and sent to other clients, where the server will handle the rest. The original noise is reconstructed by collecting the noise shares sent by each client, where the noise... satisfy ,noise satisfy .
[0010] Optionally, step S2 specifically includes: server Receive from each client Send local update model And calculate gradient update local update model L2 norm, local model update Gradient norm of each set of model parameters Take the first tail norm value ; Among them, the server The gradient calculation process is as follows , Indicates the initial iteration round, Indicates the subsequent iteration round, Represents any random vector. This represents the global update model from the previous round, and ; server Update local update model The L2 norm process is , ; server Based on dynamic norm boundary Statistics for each client Local update model The first tail norm value is greater than the dynamic norm boundary. The number of gradient layers is used to determine the local model update. Whether or not they have been infected with malware, thus filtering out all clients at once. Malicious clients in the system.
[0011] Optionally, step S3 specifically includes: server Calculate each client Local update model gradient norm And obtain the local update model. The second tail norm value ; server For all clients Agglomerative clustering is performed on the second tail norm value, and all locally updated models are updated. Divided into two clusters , among which cluster ,cluster ; server For the root dataset The root data model is obtained through local training. Then, calculate the L2 norm and the third tail norm of the root data model. ; server Calculate the similarity scores of the two clusters respectively. ,in, For clusters The similarity score; For clusters Similarity score, cluster size ; server By comparing the similarity scores of two clusters, it is determined whether the two clusters are malicious or benign. Then the cluster Determined to be malicious Otherwise, cluster Determined to be malicious .
[0012] Optionally, after step S3 and before step S4, the method further includes: The server trims the local update model corresponding to malicious clients.
[0013] Optionally, step S4 specifically includes: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. server The local update model corresponding to the malicious client is reconstructed by collecting the noise share sent by each client. noise ; server The local update model for non-malicious clients is reconstructed by collecting the noise share sent by each client. noise ; server Local update model for non-malicious clients Aggregation for privacy and security Then, noise is eliminated and the parameters of the globally updated model are updated. .
[0014] Optionally, step S4 further includes: server Update dynamic norm boundary .
[0015] The second aspect of this application provides a two-layer filtering collaborative federated learning system, the system comprising a plurality of clients and a server; Each client iteratively trains its local model based on the initial dataset to obtain a locally updated model, which is then encrypted and uploaded to the server. The server performs a malicious model update filtering based on the dynamic norm boundary on the encrypted local update models uploaded by each client. The server performs a second filtering based on norm agglomerative clustering on the local update model after the first filtering to identify the local model update corresponding to malicious clients. The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. The server sends a global model update to each client, enabling each client to iteratively execute the training of its local model.
[0016] A third aspect of this application provides a two-layer filtering collaborative federated learning device, the device comprising a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute, according to instructions in the program code, the steps of the two-layer filtering collaborative federated learning method as described in the first aspect above.
[0017] A fourth aspect of this application provides a computer-readable storage medium for storing program code for performing the method described in the first aspect above.
[0018] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: This application provides a two-layer filtering collaborative federated learning method, system, device, and storage medium. By splitting the model parameters into two, separate channels are opened for privacy protection and attack detection. The server uses clean and noise-free tail parameters for accurate and efficient first-layer malicious behavior detection, while applying strong privacy protection generated collaboratively by multiple parties to the more informative and privacy-leaking head parameters of the model. This allows the server to filter out regular malicious attacks based on dynamic norm boundaries in the first layer of filtering, and to focus on finding collaborative traces left by Sybil attacks in locally encrypted data in the second layer of filtering. This solves the problem that existing privacy protection methods that encrypt or add noise to the entire model severely disrupt the intrinsic characteristics of model updates. It enables the server to effectively distinguish between normal updates and malicious attacks, thus avoiding the decline in defense capabilities caused by relying on single indicators such as norm pruning or distance metrics, which are unable to cope with carefully constructed adaptive attacks that are statistically similar to benign users. It also better identifies Sybil attacks involving multiple attackers acting in concert. Ultimately, this forms a federated learning technology solution based on two-layer filtering collaboration, which combines strong privacy protection and accurate malicious attack identification capabilities. Attached Figure Description
[0019] Figure 1 This is a flowchart of the federated learning method with two-layer filtering collaboration in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the two-layer filtering collaborative federated learning system in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the federated learning device with dual-layer filtering collaboration in the embodiments of this application. Detailed Implementation
[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0021] For easier understanding, please refer to Figure 1 , Figure 1 This is a flowchart of the federated learning method for two-layer filtering collaboration in the embodiments of this application, as follows: Figure 1 As shown, specifically: Step S1: Each client iteratively trains its local model based on the initial dataset to obtain a locally updated model, encrypts the locally updated model, and then uploads it to the server. Step S2: The server performs a malicious model update filtering based on the dynamic norm boundary on the encrypted local update models uploaded by each client. Step S3: The server performs a second filtering based on norm agglomerative clustering on the local update model after the first filtering to identify the local model update corresponding to the malicious client. Step S4: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. Step S5: The server sends a global update model to each client, causing each client to iteratively execute step S1.
[0022] It should be noted that the core idea of this solution is to divide the model update uploaded by each client into two parts: the front (head) and the tail. The header includes noise generated collaboratively by multiple clients to protect privacy and prevent the leakage of raw data.
[0023] The tail section is kept clean and free of noise to detect abnormal behavior. Then, the server performs two layers of filtering: First layer: Use dynamic norm boundaries to quickly filter out obviously abnormal updates (such as poisoning attacks); The second layer: use clustering algorithms to analyze the similarity of the remaining updates and identify Sybil attackers who are colluding to cause harm; Finally, privacy and security are aggregated only for clients that have been confirmed as benign, the global model is updated, and the judgment criteria for the next round are adjusted.
[0024] The entire process protects privacy while identifying sophisticated attacks, adapting to the dual requirements of security and confidentiality in industrial scenarios.
[0025] Furthermore, step S1 specifically includes: Step S11, each client Based on the initial dataset and the previous round of global update model Perform model updates to obtain the locally updated model. , ; It should be noted that each client trains its own model using its own data, resulting in a new local model.
[0026] Then, it calculates the local update, which is the difference between the new model and the previous global model (which can be understood as "which parameters I changed this time").
[0027] Step S12, each client With server Seeds of negotiation Meanwhile, each client With another client Seeds of negotiation This makes each client The local update model is calculated. noise satisfy ,in, This represents the noise generated during seed negotiation between clients. This indicates the noise generated by the seed selected by the client; It should be noted that while ensuring the total noise can be reconstructed by the server (by collecting a sufficient share), individual clients cannot know information about others, so noise is generated using a negotiation seed.
[0028] It should be noted that, to protect privacy, the client needs to add noise to the model updates. However, this noise is not added arbitrarily, but must meet the following conditions: The client cannot control the total noise on its own; The server can eventually reconstruct the total noise so that it can be eliminated during aggregation.
[0029] Therefore, a dual seed negotiation mechanism is adopted: The client and server negotiate a seed and generate a portion of noise (called "self-selected noise"). The client also negotiates a seed with another random client to generate another part of the noise (called "cooperative noise"). Total noise = self-selected noise + cooperative noise.
[0030] Step S13, each client Local update model based on noise Encrypt to obtain Among them, the local update model The model parameters are divided into two parts: the front and the tail. , This represents a position index of the gradient vector; It should be noted that the client divides the local update of model parameters into two parts: Front (e.g., the first 40% of parameters): Noise can be eliminated and privacy can be protected by generating parameters through secret sharing; Tail (remaining parameters): Keep clean for malicious model update detection.
[0031] Step S14, each client The local model is updated using the Shamir secret sharing algorithm. Two noises as well as After splitting the noise into n parts, a preset number of noise parts are retained and sent to other clients, where the server will handle the rest. The original noise is reconstructed by collecting the noise shares sent by each client, where the noise... satisfy ,noise satisfy .
[0032] It should be noted that the client cannot send noise directly to the server, otherwise privacy will be compromised.
[0033] Therefore, it uses the Shamir secret sharing algorithm to split each piece of noise into multiple parts (e.g., 5 parts), keeping one part for itself and sending the others to different clients.
[0034] Once the server collects enough samples (e.g., 3 samples), it can reconstruct the original noise for subsequent elimination.
[0035] Furthermore, step S2 specifically includes: server Receive from each client Send local update model And calculate gradient update local update model L2 norm, local model update Gradient norm of each set of model parameters Take the first tail norm value ; Among them, the server The gradient calculation process is as follows , Indicates the initial iteration round, Indicates the subsequent iteration round, Represents any random vector. This represents the global update model from the previous round, and ; server Update local update model The L2 norm process is , ; server Based on dynamic norm boundary Statistics for each client Local update model The first tail norm value is greater than the dynamic norm boundary. The number of gradient layers is used to determine the local model update. Whether or not they have been infected with malware, thus filtering out all clients at once. Malicious clients in the system.
[0036] It should be noted that the server receives all updates uploaded by clients, and each update includes: The encrypted value with added noise at the beginning and the clean original value at the end.
[0037] The server only looks at the tail parameters of each layer (the parts that add noise), calculates their sum of squares and then takes the square root to get a number called the "first head norm".
[0038] This value reflects the “difference” or “distinctiveness” of this update.
[0039] Instead of using a fixed threshold to determine anomalies, the server calculates the mean (μ) and standard deviation (σ) based on the tail norm of all clients in the previous round, and then sets a dynamic boundary: mean + 2 times standard deviation (adjustable).
[0040] For each client, count how many layers of its tail norm exceed this boundary.
[0041] If the number of layers exceeds the limit by too much (e.g., more than 60%), the update is considered to have been poisoned and will be filtered out.
[0042] Furthermore, step S3 specifically includes: server Calculate each client Local update model gradient norm And obtain the local update model. The second tail norm value dots represents an ellipsis; server For all clients Agglomerative clustering is performed on the second tail norm value, and all locally updated models are updated. Divided into two clusters , among which cluster ,cluster ; server For the root dataset The root data model is obtained through local training. Then, calculate the L2 norm and the third tail norm of the root data model. ; server Calculate the similarity scores of the two clusters respectively. ,in, For clusters The similarity score; For clusters Similarity score, cluster size ; server By comparing the similarity scores of two clusters, it is determined whether the two clusters are malicious or benign. Then the cluster Determined to be malicious Otherwise, cluster Determined to be malicious .
[0043] It should be noted that for the clients remaining after the first filtering, their tail norms are recalculated (possibly in a more refined way), which is called the "second tail norm value".
[0044] The server uses an agglomerative clustering algorithm (a type of hierarchical clustering) to divide all clients into two clusters (groups) based on the similarity of these tail norms.
[0045] Assumption: One group consists of benign users (whose behavior is naturally dispersed); The other group consists of Sybil attackers (who act in concert and exhibit highly similar behaviors).
[0046] The server has a clean, small dataset (called the root dataset). It trains a model, obtains the "root model update", and calculates its tail norm as a benign reference standard.
[0047] For each cluster, calculate the similarity between the tail norm of all clients within it and the tail norm of the root model (e.g., using Gaussian similarity or inverse distance). The closer the tail norm is to the root model, the more likely it is to be benign.
[0048] Furthermore, the steps following step S3 and before step S4 include: The server trims the local update model corresponding to malicious clients.
[0049] It should be noted that in updates deemed malicious, you can choose to trim their size (e.g., scale them to a normal range) instead of discarding them completely, thus avoiding wasting computing resources.
[0050] Furthermore, step S4 specifically includes: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. server The local update model corresponding to the malicious client is reconstructed by collecting the noise share sent by each client. noise ; server The local update model for non-malicious clients is reconstructed by collecting the noise share sent by each client. noise ; server Local update model for non-malicious clients Aggregation for privacy and security Then, noise is eliminated and the parameters of the globally updated model are updated. .
[0051] It should be noted that the server now knows which clients are benign, and it has started collecting the noise shares previously emitted by these clients (shared via Shamir).
[0052] Once enough samples are collected (e.g., 3 samples), the total noise added by each client can be accurately reconstructed.
[0053] The server adds up the encrypted updates from all benign clients to get a total.
[0054] Then, it adds up the total noise from these clients and subtracts it from the sum.
[0055] The result is a completely clean, noise-free, and privacy-free global update.
[0056] Use this clean aggregation result to update the previous global model and obtain the new global model.
[0057] Furthermore, step S4 also includes: server Update dynamic norm boundary .
[0058] It should be noted that the mean and standard deviation are recalculated using the tail norms of all benign clients in this round to generate the dynamic boundary for the next round, thus achieving adaptive evolutionary defense.
[0059] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of the two-layer filtering collaborative federated learning system in the embodiments of this application, as shown below. Figure 2 As shown, specifically: Step S1: Each client iteratively trains its local model based on the initial dataset to obtain a locally updated model, encrypts the locally updated model, and then uploads it to the server. Step S2: The server performs a malicious model update filtering based on the dynamic norm boundary on the encrypted local update models uploaded by each client. Step S3: The server performs a second filtering based on norm agglomerative clustering on the local update model after the first filtering to identify the local model update corresponding to the malicious client. Step S4: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. Step S5: The server sends a global update model to each client, causing each client to iteratively execute step S1.
[0060] Optionally, step S1 specifically includes: Step S11, each client Based on the initial dataset and the previous round of global update model Perform model updates to obtain the locally updated model. , .
[0061] Step S12, each client With server Seeds of negotiation Meanwhile, each client With another client Seeds of negotiation This makes each client The local update model is calculated. noise satisfy ,in, This represents the noise generated during seed negotiation between clients. This indicates the noise generated by the seed selected by the client; Step S13, each client Local update model based on noise Encrypt to obtain Among them, the local update model The model parameters are divided into two parts: the front and the tail. , This represents a position index of the gradient vector; Step S14, each client The local model is updated using the Shamir secret sharing algorithm. Two noises as well as After splitting the noise into n parts, a preset number of noise parts are retained and sent to other clients, where the server will handle the rest. The original noise is reconstructed by collecting the noise shares sent by each client, where the noise... satisfy ,noise satisfy .
[0062] Optionally, step S2 specifically includes: server Receive from each client Send local update model And calculate gradient update local update model L2 norm, local model update Gradient norm of each set of model parameters Take the first tail norm value ; Among them, the server The gradient calculation process is as follows , Indicates the initial iteration round, Indicates the subsequent iteration round, Represents any random vector. This represents the global update model from the previous round, and ; server Update local update model The L2 norm process is , ; server Based on dynamic norm boundary Statistics for each client Local update model The first tail norm value is greater than the dynamic norm boundary. The number of gradient layers is used to determine the local model update. Whether or not they have been infected with malware, thus filtering out all clients at once. Malicious clients in the system.
[0063] Optionally, step S3 specifically includes: server Calculate each client Local update model gradient norm And obtain the local update model. The second tail norm value ; server For all clients Agglomerative clustering is performed on the second tail norm value, and all locally updated models are updated. Divided into two clusters , among which cluster ,cluster ; server For the root dataset The root data model is obtained through local training. Then, calculate the L2 norm and the third tail norm of the root data model. ; server Calculate the similarity scores of the two clusters respectively. ,in, For clusters The similarity score; For clusters Similarity score, cluster size ; server By comparing the similarity scores of two clusters, it is determined whether the two clusters are malicious or benign. Then the cluster Determined to be malicious Otherwise, cluster Determined to be malicious .
[0064] Optionally, after step S3 and before step S4, the following steps are also included: The server trims the local update model corresponding to malicious clients.
[0065] Optionally, step S4 specifically includes: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. server The local update model corresponding to the malicious client is reconstructed by collecting the noise share sent by each client. noise ; server The local update model for non-malicious clients is reconstructed by collecting the noise share sent by each client. noise ; server Local update model for non-malicious clients Aggregation for privacy and security Then, noise is eliminated and the parameters of the globally updated model are updated. .
[0066] Optionally, step S4 further includes: server Update dynamic norm boundary .
[0067] Another embodiment of the present invention provides a federated learning device with two-layer filtering collaboration, such as... Figure 3 As shown, device 10 includes: One or more processors 110 and memory 120, Figure 3 The following description uses a processor 110 as an example. The processor 110 and the memory 120 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.
[0068] Processor 110 is used to perform various control logics of device 10, and can be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microcontroller, ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. Furthermore, processor 110 can also be any conventional processor, microprocessor, or state machine. Processor 110 can also be implemented as a combination of computing devices, such as a combination of DSP and microprocessor, multiple microprocessors, one or more microprocessors combined with DSP and / or any other such configuration.
[0069] The memory 120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the method for constructing the multilingual phoneme representation model in this embodiment of the invention. The processor 110 executes various functional applications and data processing of the device 10 by running the non-volatile software programs, instructions, and units stored in the memory 120, thereby implementing the method for constructing the multilingual phoneme representation model in the above-described method embodiment.
[0070] The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created according to the use of the device 10. Furthermore, the memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include memory remotely located relative to the processor 110, and these remote memories may be connected to the device 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0071] One or more units are stored in memory 120, and when executed by one or more processors 110, perform the following steps: Each client iteratively trains its local model based on the initial dataset to obtain a locally updated model, which is then encrypted and uploaded to the server. The server performs a malicious model update filtering based on the dynamic norm boundary on the encrypted local update models uploaded by each client. The server performs a second filtering based on norm agglomerative clustering on the local update model after the first filtering to identify the local model update corresponding to malicious clients. The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. The server sends a global model update to each client, enabling each client to iteratively execute the training of its local model.
[0072] This invention provides a non-volatile computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by one or more processors, they implement any one of the embodiments of the two-layer filtering collaborative federated learning method described above.
[0073] As examples, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The memory components or memories disclosed in the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.
[0074] This application provides a two-layer filtering collaborative federated learning method, system, device, and storage medium. By dividing the model parameters into two, separate channels are opened for privacy protection and attack detection. The server uses clean and noise-free tail parameters for accurate and efficient first-layer malicious behavior detection, while applying strong privacy protection generated collaboratively by multiple parties to the more informative and privacy-leaking head parameters of the model. This allows the server to filter out regular malicious attacks based on dynamic norm boundaries in the first layer of filtering, and to focus on finding collaborative traces left by Sybil attacks in encrypted data in the second layer of filtering. This solves the problem that existing privacy protection methods that encrypt or add noise to the entire model severely disrupt the intrinsic characteristics of model updates. It enables the server to effectively distinguish between normal updates and malicious attacks, thus avoiding the decline in defense capabilities caused by relying on single indicators such as norm pruning or distance metrics, which are unable to cope with carefully constructed adaptive attacks that are statistically similar to benign users. It also better identifies Sybil attacks where multiple attackers coordinate their actions. Ultimately, this forms a federated learning technology solution based on two-layer filtering collaboration, which combines strong privacy protection and accurate malicious attack identification capabilities.
[0075] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0076] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0077] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0078] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.
[0082] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
[0083] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.
Claims
1. A two-layer filtering collaborative federated learning method, characterized in that, include: Step S1: Each client iteratively trains its local model based on the initial dataset to obtain a locally updated model, encrypts the locally updated model, and then uploads it to the server. Step S2: The server performs a malicious model update filtering based on the dynamic norm boundary on the encrypted local update models uploaded by each client. Step S3: The server performs a second filtering based on norm agglomerative clustering on the local update model after the first filtering to identify the local model update corresponding to the malicious client. Step S4: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. Step S5: The server sends a global update model to each client, causing each client to iteratively execute step S1.
2. The federated learning method with two-layer filtering collaboration according to claim 1, characterized in that, Step S1 specifically includes: Step S11: Each client, based on the initial dataset... and the previous round of global update model Perform model updates to obtain the locally updated model. , ; Step S12, each client With server Seeds of negotiation Meanwhile, each client With another client Seeds of negotiation This makes each client The noise of the local update model is calculated. satisfy ,in, This represents the noise generated during seed negotiation between clients. This indicates the noise generated by the seed selected by the client; Step S13, each client Local update model based on noise Encrypt to obtain Among them, the local update model The model parameters are divided into two parts: the front and the tail. , This represents a position index of the gradient vector; Step S14, each client The local model is updated using the Shamir secret sharing algorithm. Two noises as well as After splitting the noise into n parts, a preset number of noise parts are retained and sent to other clients, where the server will handle the rest. The original noise is reconstructed by collecting the noise shares sent by each client, where the noise... satisfy ,noise satisfy .
3. The federated learning method with two-layer filtering collaboration according to claim 2, characterized in that, Step S2 specifically includes: server Receive from each client Send local update model And calculate gradient update local update model L2 norm, local model update Gradient norm of each set of model parameters Take the first tail norm value ; Among them, the server The gradient calculation process is as follows , Indicates the initial iteration round, Indicates the subsequent iteration round, Represents any random vector. This represents the global update model from the previous round, and ; server Update local update model The L2 norm process is , ; server Based on dynamic norm boundary Statistics for each client Local update model The first tail norm value is greater than the dynamic norm boundary. The number of gradient layers is used to determine the local model update. Whether or not they have been infected with malware, thus filtering out all clients at once. Malicious clients in the system.
4. The federated learning method with two-layer filtering collaboration according to claim 3, characterized in that, Step S3 specifically includes: server Calculate each client Local update model gradient norm and obtain the local update model. The second tail norm value ; server For all clients Agglomerative clustering is performed on the second tail norm value, and all locally updated models are updated. Divided into two clusters , among which cluster ,cluster ; server For the root dataset The root data model is obtained through local training. Then, calculate the L2 norm and the third tail norm of the root data model. ; server Calculate the similarity scores of the two clusters respectively. ,in, For clusters The similarity score; For clusters Similarity score, cluster size ; server By comparing the similarity scores of two clusters, it is determined whether the two clusters are malicious or benign. Then the cluster Determined to be malicious Otherwise, cluster Determined to be malicious .
5. The federated learning method with two-layer filtering collaboration according to claim 1, characterized in that, The steps following step S3 and before step S4 include: The server trims the local update model corresponding to malicious clients.
6. The federated learning method with two-layer filtering collaboration according to claim 4, characterized in that, Step S4 specifically includes: The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. server The local update model corresponding to the malicious client is reconstructed by collecting the noise share sent by each client. noise ; server The local update model for non-malicious clients is reconstructed by collecting the noise share sent by each client. noise ; server Local update model for non-malicious clients Aggregation for privacy and security Then, noise is eliminated and the parameters of the globally updated model are updated. .
7. The federated learning method with two-layer filtering collaboration according to claim 6, characterized in that, Step S4 further includes: server Update dynamic norm boundary .
8. A two-layer filtering collaborative federated learning system, characterized in that, Includes several clients and servers; Each client iteratively trains its local model based on the initial dataset to obtain a locally updated model, which is then encrypted and uploaded to the server. The server performs a malicious model update filtering based on the dynamic norm boundary on the encrypted local update models uploaded by each client. The server performs a second filtering based on norm agglomerative clustering on the local update model after the first filtering to identify the local model update corresponding to malicious clients. The server aggregates the local update models corresponding to non-malicious clients for privacy and security, generates a global update model, and updates the norm boundary for the next round of malicious screening. The server sends a global model update to each client, enabling each client to iteratively execute the training of its local model.
9. A federated learning device with dual-layer filtering collaboration, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the two-layer filtering collaborative federated learning method according to any one of claims 1-7 according to the instructions in the program code.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the two-layer filtering collaborative federated learning method according to any one of claims 1-7.