Privacy leakage risk assessment method, system and device for federated learning

Abstract

This application provides a method, system, and device for assessing privacy leakage risks in federated learning. The method includes: setting one member client in the federated learning system as an attack client, and using the global model sequence to train the attack client's ability to predict the global model; enabling the attack client to select a target sample and inject a malicious gradient probe into the federated learning system; during updates to the federated learning system, extracting the differential residual signal between the perturbed global model and the predicted global model, and quantifying the rebound strength score based on the differential residual signal; accumulating the rebound strength scores of the target sample to generate a cumulative member confidence score; if the cumulative member confidence score is greater than a member confidence threshold, the result is determined that the target sample belongs to a private training sample; obtaining the true result and calculating the accuracy of the determination result to quantify the risk of privacy leakage. This solves the problem that existing federated learning models are easily attacked by member clients.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods, systems and devices for assessing privacy breach risks for federated learning. Background Technology

[0002] Federated learning is an emerging distributed collaborative framework. Its core definition is to allow multiple data holders (clients) to jointly model knowledge while protecting data ownership and not sharing their local raw data, thus addressing the "data silo" problem. However, federated learning is not absolutely secure. During the collaboration process, clients and the central server must interact regarding model parameters or gradients. This interaction provides "honest but curious" clients with a potential channel to spy on users' local privacy. Among the many potential threats, Membership Inference Attacks (MIAs) constitute one of the most direct challenges to user privacy. The goal of MIA is to determine whether a specific data sample (such as a medical record or a transaction) has been used in a client's local training process. Once such an attack succeeds, the information it exposes is highly sensitive. Attackers can use the conclusions of MIAs to maliciously reconstruct and exploit private information such as a user's medical records, spending habits, and geographical location.

[0003] Therefore, in order to promote the compliant implementation of federated learning, there is an urgent need for a technology that can scientifically, objectively and accurately measure and evaluate the privacy leakage risk of federated learning models in the face of severe threat environments, so as to provide a reference for subsequent targeted design and iterative defense technologies. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a method, system and device for assessing privacy leakage risks in federated learning, which solves the problem that models in existing federated learning are easily attacked by member clients.

[0005] To achieve one of the aforementioned objectives, this application provides a privacy breach risk assessment method for federated learning, the method comprising:

[0006] In a federated learning system, one member client is designated as the attacking client, while the other clients are designated as honest clients. The attacking client obtains the global model sequence during the federated learning process and uses this global model sequence to train its ability to predict new global models within the federated learning system. All member clients in the federated learning system other than the attacking client are designated as honest clients. The attack client uses a time-sharing polling strategy to select target samples sequentially and injects malicious gradient probes into each round of updates of the federated learning system. In the next update of the federated learning system, the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client is extracted, and the rebound strength score is quantized based on the differential residual signal. Under the time-sharing polling strategy, the rebound strength score of the target sample in multiple rounds of updates is accumulated to generate a cumulative member confidence score; Set a member confidence threshold. If the cumulative member confidence score is greater than the member confidence threshold, the result is that the target sample belongs to the private training sample of the honest client of the federated learning system. Obtain the actual results of the target sample, calculate the accuracy of the judgment result, and quantify the risk of privacy leakage using the accuracy.

[0007] As a further improvement to one embodiment of this application, the step of enabling the attacking client to obtain the global model sequence in the federated learning process and using the global model sequence to train the attacking client's ability to predict new global models in the federated learning system includes: The attack client continuously observes the parameters of the global model issued by the federated learning system, and uses the difference between the global model in the current round and the global model in the previous round to establish a global update vector sequence. A predictor based on a time series prediction architecture is trained using the global update vector sequence, and the predictor has the ability to capture gradient trajectory patterns; The predictor is used to establish a dynamic behavioral benchmark for the federated learning system under normal update conditions.

[0008] As a further improvement to one embodiment of this application, the step of having the attack client sequentially select target samples using a time-sharing polling strategy and injecting malicious gradient probes into each round of updates of the federated learning system includes: The attacking client selects a target sample, and the correct learning gradient vector of the target sample under the global model in the current round is calculated according to the following formula: ; in, The learning gradient vector of the target sample. Here, t represents the gradient operator of the loss function with respect to the model parameters, and t represents the current round number. This is the global model from the previous round. The target sample; The malicious gradient probe is constructed using the learned gradient vector and then injected into the federated learning system.

[0009] As a further improvement to one embodiment of this application, constructing the malicious gradient probe using the learned gradient vector and injecting the malicious gradient probe into the federated learning system includes: An adaptive adjustment coefficient is selected, which is used to ensure that the offset generated by the malicious gradient probe is less than the server anomaly detection threshold of the federated learning system. The malicious gradient probe is constructed according to the following formula: ; in, The malicious gradient probe at the t-th update. The legitimate cover gradient generated for the attacking client, is the adaptive adjustment coefficient.

[0010] As a further improvement to one embodiment of this application, the step of extracting the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client in the next round of updates of the federated learning system includes: In the next update of the federated learning system, under the action of the malicious gradient probe, honest clients within the target sample will generate a repair gradient. A differential residual signal is generated between the global model aggregated by the repaired gradient and the original global model predicted by the attacking client; The differential residual signal is extracted using the following formula: ; Specifically, Let be the differential residual signal of round t. For the global model in round t, For the global model in round t-1, This is the predicted normal update vector for the t-th update round.

[0011] As a further improvement to one embodiment of this application, the step of quantizing the rebound strength score based on the differential residual signal includes: The rebound strength score is calculated using the following formula: ; in, The rebound strength score is... The learning gradient vector of the target sample. This is a cosine similarity measure.

[0012] As a further improvement to one embodiment of this application, the step of accumulating the rebound strength scores of the target sample in multiple rounds of updates under the time-sharing polling strategy to generate a cumulative member confidence score includes: The rebound strength score is accumulated according to the following formula: ; in, The confidence score for the i-th target sample is given. This represents the rebound strength score for the current round.

[0013] As a further improvement to one embodiment of this application, a member confidence threshold is set. If the cumulative member confidence score is greater than the member confidence threshold, the result is determined to be that the target sample belongs to the private training sample of the honest client of the federated learning system, including: The determination result is generated according to the following formula: ; in, The determination result is as follows. The confidence threshold for the member.

[0014] Based on the same inventive concept, this application also provides a privacy breach risk assessment method system for federated learning, including: The basic prediction module is used to set one of the member clients in the federated learning system as an attack client, enabling the attack client to obtain the global model sequence in the federated learning process and use the global model sequence to train the attack client's ability to predict new global models in the federated learning system; wherein, the member clients in the federated learning system other than the attack client are honest clients. The injection module is used to enable the attack client to select target samples sequentially using a time-sharing polling strategy and inject malicious gradient probes into each round of updates of the federated learning system. The extraction module is used to extract the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client in the next round of updates of the federated learning system, and to quantize the rebound strength score based on the differential residual signal. The scoring module is used to accumulate the rebound strength score of the target sample in multiple rounds of updates under the time-sharing polling strategy, and generate a cumulative member confidence score. The determination module is used to set a member confidence threshold. If the cumulative member confidence score is greater than the member confidence threshold, the determination result is that the target sample belongs to the private training sample of the honest client of the federated learning system. An evaluation module is used to obtain the actual results of the target sample, calculate the accuracy of the judgment result, and quantify the risk of privacy leakage using the accuracy.

[0015] Based on the same inventive concept, this application also provides an electronic device, including: a processor and a memory; the memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the privacy breach risk assessment method for federated learning.

[0016] Compared to existing technologies, the technical advantages of this invention are as follows: First, it establishes the ability of the attacking client to predict the global model. Then, it injects highly covert malicious gradient probes into the federated learning system. The federated learning system generates cumulative member confidence scores based on the bounce-back correction signals from multiple rounds of updates to the malicious gradients. Finally, it determines the target sample attributes based on a threshold and the relationship between the generated cumulative member confidence scores, assessing member privacy risks. This "accumulating small gains into large ones" strategy enables the malicious gradient probes to effectively penetrate multiple security defenses such as robust aggregation and differential privacy while maintaining covertness. It also tests the ability of the federated learning system to resist such covert malicious gradient probes. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the implementation methods or related technologies will be briefly introduced below. Obviously, the drawings described below are only the implementation methods of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a privacy breach risk assessment method for federated learning, provided as an embodiment of this application; Figure 2 A schematic diagram of a privacy breach risk assessment method system for federated learning, provided as another embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided for another embodiment of this application. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0020] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by those skilled in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects.

[0021] Currently, assessment methods for privacy breaches initiated by attacking clients in federated learning are mainly divided into two categories based on the attack method: passive and active. Passive assessment methods follow an "honest but curious" model, where the attacker infers privacy solely by analyzing legally received information without tampering with the training process. However, as the scale of participants in federated learning increases, the impact of a single client on the global model is significantly diluted, leading to performance limitations of passive methods in environments with extremely low signal-to-noise ratios.

[0022] In proactive evaluation methods, a mainstream approach amplifies the performance differences between member and non-member samples by manipulating local data or model updates (i.e., poisoning attacks). For example, a proposal transforms Poisoned Member Inference (PMIA) into an optimization problem by injecting carefully crafted malicious updates to alter the global model's response to the target sample.

[0023] However, existing local membership reasoning evaluation techniques face several technical barriers in practical applications. First, most effective proactive attacks rely on strong prior knowledge assumptions, meaning the attacker can control multiple participating clients. Second, modern federated learning commonly deploys Byzantine-robust aggregation (RSA) mechanisms to detect and eliminate anomalous updates. These defenses make traditional brute-force poisoning probes easily identifiable and discarded as outliers.

[0024] In summary, designing an evaluation model that can both bypass RSA detection and efficiently extract privacy signals in real-world scenarios where the attacker controls only a single client is a technical challenge that urgently needs to be addressed in this field.

[0025] To address the aforementioned issues, this application provides a privacy breach risk assessment method for federated learning, such as... Figure 1 As shown, the method includes the following steps: Step S100: Set one member client in the federated learning system as an attack client, so that the attack client can obtain the global model sequence in the federated learning process and use the global model sequence to train the attack client's ability to predict new global models in the federated learning system; wherein, the member clients in the federated learning system other than the attack client are honest clients.

[0026] Specifically, this step aims to leverage the attacker's status as an honest client within the federated learning system to obtain the global model from the first t rounds. A time series predictor is then built using an LSTM neural network model to acquire a dynamic behavioral baseline. This stage serves as the perceptual foundation of this method, its primary responsibility being to establish a dynamic behavioral benchmark for the federated learning global model updates under a "no-intervention, normal training" state. In the initial stage of the evaluation task, i.e., the baseline period (K rounds) defined by the algorithm flow, the attacker accesses the system in a completely honest manner, aiming to reveal the inherent dynamic distribution characteristics of the deep learning system without triggering any security alerts.

[0027] In one possible implementation of this application, step S100 includes: Step S110: The attacking client continuously observes the parameters of the global model issued by the federated learning system, and uses the difference between the global model in the current round and the global model in the previous round to establish a global update vector sequence.

[0028] Step S120: Train a predictor based on a time series prediction architecture using a global update vector sequence. The predictor has the ability to capture gradient trajectory patterns.

[0029] Step S130: Use the predictor to establish a dynamic behavior benchmark for the federated learning system under normal update conditions.

[0030] Specifically, the first step is to continuously observe the global model sequence during the baseline modeling phase. This involves utilizing the dynamic evolution of the predictor's learning system under undisturbed conditions within the time series prediction architecture to establish a prediction benchmark that eliminates background noise. During this period, the attacking client continuously observes and records the changes in global model parameters for each round. By calculating the difference between the current round's model and the previous round's model, the true global update vector sequence is established. The specific calculation process is shown in the following formula. ; in, Representing the The real global model updates the vector; Representing the The global model parameters are aggregated in rounds. Subsequently, the federated learning system uses these high-dimensional vector sequences to train a predictor based on a deep time series prediction architecture, enabling it to capture the gradient trajectory patterns of federated learning in complex environments. The core function of this predictor is to predict the theoretically expected update direction of the system in the next round, excluding attack interference, when gradient probing is initiated. The specific prediction process is shown in the following formula: ; in, For the first The normal update vector for round prediction; A pre-trained time series prediction model used to infer future evolutionary trends based on historical update trajectories; This represents the true update vector observed in the previous round. By establishing this dynamic baseline, the evaluation system can abstract the complex collective dynamic trajectory, thereby providing a precise benchmark for subsequently extracting the weak micro-correction signals caused by specific member samples from the aggregate update.

[0031] Step S200 involves the attacking client using a time-sharing polling strategy to sequentially select target samples and inject malicious gradient probes into each round of updates of the federated learning system.

[0032] Specifically, in this stage, the attacking client uses a time-sharing polling strategy (modN) to select target samples in sequence, construct and upload malicious gradient probes hidden in legitimate cover updates, and in subsequent steps use gradient ascent to induce honest clients that possess the member samples to generate an instinctive corrective bounce.

[0033] In one possible implementation of this application, step S200 includes: Step S210: The attacking client selects a target sample and calculates the correct learning gradient vector of the target sample under the global model in the current round according to the following formula: ; in, The learning gradient vector of the target sample. Here, t represents the gradient operator of the loss function with respect to the model parameters, and t represents the current round number. This is the global model from the previous round. For the target sample.

[0034] Step S220: Construct a malicious gradient probe using the learned gradient vector and inject the malicious gradient probe into the federated learning system.

[0035] Specifically, in each probing round, the attacker selects a sample from the target dataset using a time-sharing polling mechanism and calculates the correct learning gradient vector for that sample under the current model. Subsequently, the attacker constructs a malicious update vector.

[0036] In one possible implementation of this application, step S220 includes: Step S221: Select an adaptive adjustment coefficient. The adaptive adjustment coefficient is used to ensure that the offset generated by the malicious gradient probe is less than the server anomaly detection threshold of the federated learning system.

[0037] Step S222: Construct a malicious gradient probe according to the following formula: ; in, This is a malicious gradient probe during the t-th update. Generating legitimate cover gradients for attacking clients, This is the adaptive adjustment coefficient.

[0038] Specifically, in order to ensure that the attack behavior is not detected by the robust aggregation mechanism deployed on the server side, this invention introduces subthreshold control logic. By setting an adaptive adjustment coefficient, the strength of the malicious gradient probe is controlled. The adaptive adjustment coefficient ensures that the offset generated by the malicious gradient probe is always strictly below the server's anomaly detection threshold, thereby ensuring that the attack command can enter the aggregation bucket as a legitimate update, and realizing penetrating detection of the defense mechanism.

[0039] It should be noted that the core of the technique of covertly injecting malicious gradient probes is to adaptively adjust the detection coefficients so that the attack updates always conform to the geometric characteristics of the honest update cluster in statistical distribution. This avoids all mainstream Byzantine robust aggregation defenses based on norm detection or distance filtering in modern federated learning, and realizes privacy risk assessment in a strong security defense environment.

[0040] Step S300: In the next update of the federated learning system, extract the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client, and quantize the rebound strength score based on the differential residual signal.

[0041] Specifically, after injecting a malicious gradient probe, evidence of privacy breaches is captured by measuring the corrective response of the system to the malicious gradient probe. This step utilizes the differential residual signal between the perturbed global model and the predicted original global model to represent the corrective response.

[0042] It should also be noted that by constructing a predictor in step S100 to eliminate the complex environmental background in the federated learning evolution process, the unexpected residuals caused by the error correction behavior of the attacked client can be accurately separated at the gradient level, which significantly improves the detection sensitivity in a single attack point environment.

[0043] In one possible implementation of this application, step S300, extracting the differential residual signal, includes: In step S310, during the next update of the federated learning system, honest clients, including those with target samples, will generate repair gradients under the action of malicious gradient probes.

[0044] In step S320, a differential residual signal is generated between the global model whose gradient has been repaired and the original global model predicted by the attacking client.

[0045] Step S330: Extract the differential residual signal according to the following formula: ; Specifically, Let be the differential residual signal of round t. For the global model in round t, For the global model in round t-1, This is the predicted normal update vector for the t-th update round.

[0046] Specifically, if the target sample does indeed belong to the private training set of an honest client, that honest client will spontaneously generate a repair gradient. After the server completes this round of aggregation, the auditor downloads the new model and, combined with the aforementioned benchmark, extracts the abnormal response signal through differencing. Conversely, if the target sample does not belong to the private training set of an honest client, the honest client will not generate a repair gradient, and after differencing the new model with the aforementioned benchmark, there will be no abnormal response signal. In this case, the privacy theft attempt by the attacking client fails, and privacy in federated learning is not compromised.

[0047] In one possible implementation of this application, step S300, quantifying the rebound strength score, includes: Step S340: Calculate the rebound strength score according to the following formula: ; in, The rebound strength score is the score for the rebound strength. The learning gradient vector of the target sample. This is a cosine similarity measure.

[0048] Specifically, the attacking client uses cosine similarity to measure the spatial consistency between the residual signal and the gradient direction of the target sample, i.e., to quantify the rebound strength.

[0049] Steps S310-S340 successfully simplified member inference into a weak signal detection problem. Even if the victim's repair efforts are diluted by a large number of client updates, a non-random bias can still be shown in a specific gradient dimension through differential operations.

[0050] Step S400: Under the time-sharing polling strategy, accumulate the rebound strength scores of the target samples in multiple rounds of updates to generate the cumulative member confidence score.

[0051] Specifically, this phase is responsible for aggregating fragmented evidence accumulated over a long period into conclusive criteria for privacy breaches. The system maintains a persistent confidence score for each sample in the target dataset, and due to the use of a time-sharing polling strategy, the bounce measurements for specific samples are continuously accumulated. This cross-period evidence accumulation mechanism effectively counteracts the influence of random noise, allowing definite membership characteristics to gradually emerge on the timeline.

[0052] In one possible implementation of this application, step S400 includes: Step S410: Accumulate the rebound strength score according to the following formula: ; in, The confidence score for the i-th target sample is given. This represents the rebound strength score for the current round.

[0053] It should be noted that by performing time-sharing scheduling on the target training set, which includes target samples, the detection pressure is distributed across the time axis. Furthermore, by using long-term confidence integration to offset the effects of random noise and differential privacy perturbations, the technical challenge of extracting deterministic member features under conditions where the control of a single attack evaluator is extremely low is solved.

[0054] Step S500: Set a member confidence threshold. If the cumulative member confidence score is greater than the member confidence threshold, the result is that the target sample belongs to the private training sample of the honest client of the federated learning system.

[0055] In one possible implementation of this application, step S500 further includes: Step S510: Generate the determination result according to the following formula: ; in, To determine the result, This is the confidence threshold for members. 1 indicates that the target sample belongs to the training set of honest clients, and 0 indicates that the target sample does not belong to the training set of honest clients.

[0056] It should be noted that by leveraging the inherent interactive permissions of the attack client, the previously difficult-to-observe privacy leaks are transformed into a visualized confidence evolution curve, providing a scientific basis for quantifying risks in building a secure, compliant, and auditable distributed collaboration framework.

[0057] Step S600: Obtain the actual results of the target sample, calculate the accuracy of the judgment results, and quantify the risk of privacy leakage using the accuracy.

[0058] Specifically, steps S100-S500 simulate attacker behavior, attempting to obtain private samples from other honest clients in the federated learning system. The accuracy of the guesses is verified in this step. Therefore, a higher accuracy rate of the simulated attack client indicates a higher risk of privacy breaches on the central server of this federated learning system. Conversely, a lower accuracy rate indicates a lower risk of privacy breaches on the central server of this federated learning system.

[0059] The beneficial effects of the privacy leakage risk assessment method for federated learning disclosed in the embodiments of this application are as follows: This method achieves accurate quantification of privacy risks in federated learning under single-point attack scenarios by constructing a closed-loop audit logic of "modeling-detection-response-accumulation". First, during the baseline modeling stage, the global model sequence is continuously observed, and the dynamic evolution law of the system under undisturbed state is learned using the time series predictor to establish a prediction benchmark that eliminates background noise. After entering the adaptive tracking stage, the attacking client adopts a time-sharing polling strategy to select target samples sequentially, constructs and uploads malicious gradient probes hidden in legitimate cover updates, and uses gradient ascent to induce honest clients with the target samples to generate an instinctive corrective rebound. When a new round of global model is distributed, the algorithm extracts the micro-response signal by calculating the difference residual between the actual update amount and the baseline prediction value, and uses cosine similarity to measure the consistency between the signal and the gradient direction of the audit target. Addressing the weakness of single-client attack signals, the algorithm accumulates confidence levels for bounced evidence across multiple time cycles. By canceling out random noise, it gradually reveals the identifiable characteristics of members. Finally, it determines sample attributes based on accumulated scores and quantifies the accuracy of the system's privacy leakage risk. This "accumulating small gains into large ones" strategy allows it to effectively penetrate multiple security defenses, such as robust aggregation and differential privacy, while maintaining stealth.

[0060] Another embodiment of this application discloses a privacy breach risk assessment method system for federated learning, such as... Figure 2 As shown, it includes: The basic prediction module is used to set up a member client in the federated learning system as an attack client, enabling the attack client to obtain the global model sequence in the federated learning process and use the global model sequence to train the attack client's ability to predict new global models in the federated learning system; among them, the member clients in the federated learning system other than the attack client are honest clients.

[0061] The injection module enables the attacking client to select target samples sequentially using a time-sharing polling strategy and inject malicious gradient probes into each round of updates of the federated learning system. The extraction module is used to extract the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client in the next round of updates of the federated learning system, and to quantize the rebound strength score based on the differential residual signal. The scoring module is used to accumulate the rebound strength score of the target sample in multiple rounds of updates under the time-sharing polling strategy, and generate the cumulative member confidence score. The judgment module is used to set the member confidence threshold. If the cumulative member confidence score is greater than the member confidence threshold, the judgment result is that the target sample belongs to the private training sample of the honest client of the federated learning system. The evaluation module is used to obtain the actual results of the target samples, calculate the accuracy of the judgment results, and quantify the risk of privacy leakage based on the accuracy.

[0062] Figure 3 This diagram illustrates a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0063] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0064] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0065] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0066] The communication interface 1040 is used to connect the communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, radio (shortwave / ultra-shortwave) communication, satellite communication, data link communication, etc.).

[0067] Bus 1050 includes pathways for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0068] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments described in this specification, and need not include all the components shown in the figures.

[0069] The electronic devices described above are used to implement the corresponding privacy leakage risk assessment method for federated learning in any of the foregoing embodiments, and have the beneficial effects of the corresponding method implementation methods, which will not be elaborated here.

[0070] Based on the same inventive concept, corresponding to any of the above-described embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the privacy leakage risk assessment method for federated learning as described in any of the above embodiments.

[0071] The computer-readable medium in this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0072] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the privacy leakage risk assessment method for federated learning as described in any of the above embodiments, and have the beneficial effects of the corresponding method implementation, which will not be repeated here.

[0073] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; this manner of description is merely for clarity, and those skilled in the art should consider the specification as a whole. Within the framework of this application, the above embodiments or the technical features of different embodiments can also be appropriately combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0074] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be entirely within the understanding of those skilled in the art). While specific details (e.g., circuits) are set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0075] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0076] The embodiments described herein are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and principles of the embodiments described herein should be included within the protection scope of this application.

Claims

1. A privacy leakage risk assessment method for federated learning, characterized in that, The method includes: One member client in the federated learning system is set as an attacking client, enabling the attacking client to obtain the global model sequence during the federated learning process. The attacking client is then used to train its ability to predict new global models in the federated learning system. All other member clients in the federated learning system besides the attacking client are considered honest clients. The attack client uses a time-sharing polling strategy to select target samples sequentially and injects malicious gradient probes into each round of updates of the federated learning system. In the next update of the federated learning system, the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client is extracted, and the rebound strength score is quantized based on the differential residual signal. Under the time-sharing polling strategy, the rebound strength score of the target sample in multiple rounds of updates is accumulated to generate a cumulative member confidence score; Set a member confidence threshold. If the cumulative member confidence score is greater than the member confidence threshold, the result is that the target sample belongs to the private training sample of the honest client of the federated learning system. Obtain the actual results of the target sample, calculate the accuracy of the judgment result, and quantify the risk of privacy leakage using the accuracy.

2. The method of privacy leakage risk assessment for federated learning according to claim 1, wherein, The step of enabling the attacking client to obtain the global model sequence in the federated learning process and using the global model sequence to train the attacking client's ability to predict new global models in the federated learning system includes: The attack client continuously observes the parameters of the global model issued by the federated learning system, and uses the difference between the global model in the current round and the global model in the previous round to establish a global update vector sequence. A predictor based on a time series prediction architecture is trained using the global update vector sequence, and the predictor has the ability to capture gradient trajectory patterns; The predictor is used to establish a dynamic behavioral benchmark for the federated learning system under normal update conditions.

3. The privacy leakage risk assessment method for federated learning according to claim 1, characterized in that, The step of having the attacking client sequentially select target samples using a time-sharing polling strategy and injecting malicious gradient probes into each round of updates of the federated learning system includes: The attacking client selects a target sample, and the correct learning gradient vector of the target sample under the global model in the current round is calculated according to the following formula: ； in, The learning gradient vector of the target sample. Here, t represents the gradient operator of the loss function with respect to the model parameters, and t represents the current round number. This is the global model from the previous round. The target sample; The malicious gradient probe is constructed using the learned gradient vector and then injected into the federated learning system.

4. The privacy leakage risk assessment method for federated learning according to claim 3, characterized in that, Constructing the malicious gradient probe using the learned gradient vector and injecting the malicious gradient probe into the federated learning system includes: An adaptive adjustment coefficient is selected, which is used to ensure that the offset generated by the malicious gradient probe is less than the server anomaly detection threshold of the federated learning system. The malicious gradient probe is constructed according to the following formula: ； in, The malicious gradient probe at the t-th update. The legitimate cover gradient generated for the attacking client, is the adaptive adjustment coefficient.

5. The privacy leakage risk assessment method for federated learning according to claim 1, characterized in that, In the next update of the federated learning system, the extraction of the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client includes: In the next update of the federated learning system, under the action of the malicious gradient probe, honest clients within the target sample will generate a repair gradient. A differential residual signal is generated between the global model aggregated by the repaired gradient and the original global model predicted by the attacking client; The differential residual signal is extracted using the following formula: ； Specifically, Let be the differential residual signal of round t. For the global model in round t, For the global model in round t-1, This is the predicted normal update vector for the t-th update round.

6. The privacy leakage risk assessment method for federated learning according to claim 5, characterized in that, The step of quantizing the rebound strength score based on the differential residual signal includes: The rebound strength score is calculated using the following formula: ； in, The rebound strength score is... The learning gradient vector of the target sample. This is a cosine similarity measure.

7. The privacy leakage risk assessment method for federated learning according to claim 1, characterized in that, Under the time-sharing polling strategy, the rebound strength score of the target sample in multiple rounds of updates is accumulated to generate a cumulative member confidence score, including: The rebound strength score is accumulated according to the following formula: ； in, The confidence score for the i-th target sample is given. This represents the rebound strength score for the current round.

8. The privacy leakage risk assessment method for federated learning according to claim 7, characterized in that, A member confidence threshold is set. If the cumulative member confidence score is greater than the member confidence threshold, the result is that the target sample belongs to the private training sample of the honest client of the federated learning system, including: The determination result is generated according to the following formula: ； in, The determination result is as follows. The confidence threshold for the member.

9. A privacy breach risk assessment method system for federated learning, characterized in that, The system includes: The basic prediction module is used to set one of the member clients in the federated learning system as an attack client, enabling the attack client to obtain the global model sequence in the federated learning process and use the global model sequence to train the attack client's ability to predict new global models in the federated learning system; wherein, the member clients in the federated learning system other than the attack client are honest clients. The injection module is used to enable the attack client to select target samples sequentially using a time-sharing polling strategy and inject malicious gradient probes into each round of updates of the federated learning system. The extraction module is used to extract the differential residual signal between the global model generated after being interfered with by the malicious gradient probe and the original global model predicted by the attacking client in the next round of updates of the federated learning system, and to quantize the rebound strength score based on the differential residual signal. The scoring module is used to accumulate the rebound strength score of the target sample in multiple rounds of updates under the time-sharing polling strategy, and generate a cumulative member confidence score. The determination module is used to set a member confidence threshold. If the cumulative member confidence score is greater than the member confidence threshold, the determination result is that the target sample belongs to the private training sample of the honest client of the federated learning system. An evaluation module is used to obtain the actual results of the target sample, calculate the accuracy of the judgment result, and quantify the risk of privacy leakage using the accuracy.

10. An electronic device, characterized in that, include: Processor and memory; The memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the privacy breach risk assessment method for federated learning as described in any one of claims 1 to 8.

Images