Risk-driven adaptive differential privacy protection method and system based on member inference
By introducing generalization gap, personalization ratio, and sharing direction difference features into federated learning, and dynamically adjusting pruning and noise, the dynamic risk mismatch and heterogeneity problem of static differential privacy protection in federated learning is solved, achieving efficient privacy protection and model accuracy improvement, and is suitable for complex heterogeneous environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 湖南工商大学
- Filing Date
- 2026-03-24
- Publication Date
- 2026-07-31
AI Technical Summary
Existing federated learning suffers from several problems, including a mismatch between static differential privacy protection schemes and dynamic risks during the training process, unfair protection due to heterogeneity, a lack of personalized risk characterization mechanisms, and insufficient privacy constraints due to the lack of budget for adaptive parameter tuning. These issues make it difficult to effectively defend against member inference attacks.
An adaptive differential privacy protection method based on member inference risk is adopted. By calculating the generalization gap feature, the personalization proportion feature, and the sharing direction difference feature, the pruning threshold and noise intensity are dynamically adjusted to achieve computable risk characterization and adaptive differential privacy protection. Combined with the Reni differential privacy accountant, the privacy budget is tracked throughout the process.
In a dishonest central server environment, risk-driven adaptive differential privacy protection is achieved, which improves privacy security and model accuracy, adapts to complex heterogeneous environments, and provides an auditable and controllable long-term training mechanism.
Smart Images
Figure CN122490569A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data security protection technology in federated learning, specifically to an adaptive differential privacy protection method and system based on member inference risk. Background Technology
[0002] Within the current technical framework of federated learning, even if the client only uploads model updates rather than the original data, existing protective measures still have obvious logical flaws and utility bottlenecks when facing a central server with powerful computing capabilities that intends to steal privacy through member inference attacks: (1) The mismatch between static differential privacy and the dynamic risks of the training process. Existing technologies usually use fixed pruning thresholds, noise standard deviations, or fixed budget allocation ratios to perform differential privacy protection. However, this static configuration ignores the fact that the federated learning training process is essentially a non-stationary dynamic process. The model's memory of data and its tendency to overfit will fluctuate and increase with the increase of training rounds, resulting in a significant increase in the signal strength that member inference attacks can exploit (such as gradient distribution differences and confidence extremes) in the later stages of training. It can be deduced that if the static parameters are set according to the highest risk in the later stages of training, it will cause excessive perturbation to the update of general features that do not yet contain sensitive details in the early stages of training, which will seriously slow down the convergence speed; conversely, if the average risk is set, it will be unable to resist high-intensity member inference attacks in the later stages of training. This mismatch in mechanisms directly leads to a serious imbalance between privacy protection and model utility over long time scales, resulting in either model unusability or collapse of privacy defenses in the later stages. (2) Heterogeneity leads to unfair protection and the barrel effect. Existing personalized federated learning privacy schemes often apply the same differential privacy parameters (such as uniform pruning thresholds and noise multipliers) to all clients in the same round. The technical fallacy of this approach is that it does not consider the statistical heterogeneity of data distribution among clients. The difference in data distribution among different clients will lead to huge differences in the gradient norm, sensitivity, and directional features of their model updates. The updates of some clients holding rare or outlier data are often more "unique" and easier for members to infer that attackers can separate and identify them from the aggregated results. Therefore, under the mandatory constraint of uniform parameters, the system will have serious unfair protection: high-risk (unique data) clients become the privacy bottleneck of the entire system due to insufficient protection strength, while low-risk clients suffer from unnecessary excessive noise. This directly causes the "barrel effect" of the system, that is, the overall privacy security is determined by the most vulnerable client, while the overall model accuracy is dragged down by the overprotected client. (3) Lack of calculable metrics to characterize the sources of risk in personalized mechanisms. Existing privacy protection schemes mainly rely on coarse-grained external signals such as gradient norm or training epochs when adjusting parameters, lacking a quantitative characterization of the micro-mechanism of "how personalized structures specifically change leakage risk". In personalized federated learning, the internal states such as the update intensity of the shared part and the personalized part, the deviation of the update direction from the global direction, and the generalization gap of the local model are the fundamental factors determining the separability of members.Because of the lack of an interpretable and implementable risk indicator system to transform these mechanistic factors into numerical signals, existing protection strategies can only degenerate into blindly increasing noise or random pruning. This not only makes it difficult to accurately carry out "surgical" key protection for high-risk updates, but also makes it difficult for researchers to prove "why" a certain moment is dangerous through reproducible experiments or ablation analysis, directly hindering the refined evolution of defense mechanisms. (4) The privacy constraints are not rigorous due to the lack of budget in the adaptive parameter tuning process. Many existing methods that claim to be "adaptive DP" only record the privacy accounting for the final noisy uploaded update vector, while artificially ignoring the "parameter tuning process" such as quantile estimation, threshold adaptive calculation, noise intensity selection or branch strategy determination. From the perspective of information theory, the input of the above parameter tuning process depends on the statistical characteristics of private data (such as the gradient distribution histogram). Therefore, parameter selection itself is a query mechanism for private data, which will inevitably consume the privacy budget and affect the final output distribution. Ignoring this process will lead to a serious underestimation of the privacy budget, making the so-called satisfaction of ( ). The differential privacy guarantee (δ) is mathematically invalid. The direct consequence is that the privacy constraints of the entire system not only have logical flaws in their formal proof, but may also completely fail in the face of advanced attacks targeting auxiliary information, thus failing to meet stringent privacy audit standards. Summary of the Invention
[0003] The technical problem to be solved by this invention is as follows: In view of the above-mentioned problems of the prior art, this invention provides an adaptive differential privacy protection method and system based on member inference risk-driven approach. This invention aims to solve the key technical defects of existing privacy protection schemes in personalized federated learning when facing dishonest central servers, in terms of dynamic risk adaptation, non-independent and identically distributed fairness, and provable privacy budget throughout the process. In personalized federated learning scenarios where dishonest central servers may launch member inference attacks, this invention achieves a closed loop of computable risk characterization, risk-driven adaptive differential privacy protection, and unified tracking of provable budget.
[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An adaptive differential privacy protection method based on member inference risk-driven approach includes the following steps: S101, The client node receives the shared parameters for this round from the central server; S102, the client node uses the shared part parameters and its own personalized part parameters to build a federated model composed of the shared part and personalized part, and uses the local private dataset to train the federated model for one round to obtain the updated parameters of the shared part and personalized part in this round. S103, the client node calculates the inferred risk score based on multiple risk features trained by the federated model; based on the inferred risk score, noise is injected into the shared part update parameters of this round, and the noise-injected shared part update parameters are sent to the central server for alignment and aggregation to generate new shared part parameters for the next round.
[0005] Optionally, the various risk features in step S103 include generalization gap feature, personalization proportion feature, and shared direction difference feature, wherein the generalization gap feature is used to characterize the local overfitting degree of the federated model, the personalization proportion feature is used to characterize the proportion of personalized updates in the total layer updates of the federated model, and the shared direction difference feature is used to characterize the degree of deviation between the shared update direction of this client and the group reference direction.
[0006] Optionally, the calculation function expression for the generalization gap feature is: ; in, To generalize the gap feature, To perform the maximum value operation, and Client nodes Use the Round using local private dataset The validation loss and training loss obtained during training of the federated model; the calculation function expression for the personalized proportion feature is: ; in, For personalized proportion characteristics, For client nodes No. Wheel personalization update parameters of Norm, For client nodes No. Update parameters of the shared part of the wheel of Norm, To prevent positive numbers from being divided by zero; the calculation function expression for the shared direction difference feature is: ; in, To share directional differences, for and The dot product of vectors, The first one issued by the central server The group reference direction of the wheel, wherein the group reference direction is the direction of the central server for the first wheel. The total vector obtained by aggregating the shared update parameters after injecting noise uploaded by all client nodes according to their weights is then processed. The unit vector obtained by norm calculation; for of Norm.
[0007] Optionally, in step S103, the client node calculates the inferred risk score based on multiple risk features trained by the federated model, which includes: weighting and summing multiple risk features, adding a preset bias term to the result of the weighted summation, and then multiplying it by a preset inferred risk score to obtain the inferred risk score.
[0008] Optionally, the perturbation injection of the shared portion update parameters in this round based on the inferred risk score in step S103 includes: S201, based on the inferred risk score, adaptively hierarchically prune the update parameters of the shared part and the personalized part in this round to obtain the pruned update parameters of the shared part and the personalized part; allocate the total privacy budget of this round between the shared part and the personalized part to obtain the single-round privacy budget of the shared part and the personalized part. S202, based on the single-round privacy budget of the shared part and the personalized part, determine the noise multiplier and noise standard deviation of the shared part and the personalized part respectively, and generate and inject noise into the trimmed update parameters of the shared part and the personalized part to obtain the updated parameters of the shared part and the personalized part after injecting noise. S203, based on the single-round privacy budget and noise multipliers, calculate the cumulative Reni privacy for each order in the given set of orders, and calculate the cumulative privacy consumption of the client node based on the cumulative Reni privacy for each order: ; in, For client nodes Cumulative privacy consumption, To perform the minimum value operation, For client nodes The The cumulative amount of privacy data for each level. Given a set of orders, The default failure probability parameter for single-round privacy consumption; S204, determine whether the cumulative privacy consumption of the client node exceeds the preset total budget. If it does not exceed the total budget, send the shared part update parameters after injecting noise to the central server for alignment and aggregation to generate new shared part parameters for the next round of distribution; otherwise, do not send the shared part update parameters after injecting noise to the central server in this round. Alternatively, step S103, which involves injecting a perturbation into the shared portion of the update parameters based on the inferred risk score, includes: S301, based on the inferred risk score, adaptively hierarchically prune the update parameters of the shared portion in this round to obtain the pruned update parameters of the shared portion; allocate the total privacy budget of this round to the privacy budget of the shared portion; S302, noise is generated and injected into the shared part update parameters after clipping to obtain the shared part update parameters after noise injection; S303, calculate the cumulative Reni privacy amount for each order in a given set of orders based on the noise multiplier used when generating the noise, and calculate the cumulative privacy consumption of the client node based on the cumulative Reni privacy amount of each order. S304, determine whether the cumulative privacy consumption of the client node exceeds the preset total budget. If it does not exceed the total budget, send the shared part update parameters after injecting noise to the central server for alignment and aggregation to generate new shared part parameters for the next round of distribution; otherwise, do not send the shared part update parameters after injecting noise to the central server in this round.
[0009] Optionally, when the shared part update parameters after injecting noise are not sent to the central server in this round, it also includes triggering at least one of the following control actions: Control action ①: The client node increases the noise multiplier used when generating noise and waits to enter the next round of training; Control action ②: The client node decreases the adaptive pruning threshold when performing adaptive layer pruning and waits to enter the next round of training; Control action ③: The client node terminates its participation in the next round of training.
[0010] Optionally, step S201, which involves adaptively tiered pruning of the shared and personalized update parameters for this round based on the inferred risk score, includes: S401 calculates the update parameters for the shared portion and the personalized portion of this round, respectively. After applying the norm, the quantile operator is used to calculate the base threshold for adaptive hierarchical pruning of the shared and personalized update parameters: S402, calculate the adaptive pruning thresholds for both the shared partial update parameters and the personalized partial update parameters according to the following formula: ; ; in, and Client nodes No. The adaptive pruning threshold for adaptive layered pruning of both shared and personalized update parameters. To perform the maximum value operation, and These are the lower limits of the clipping thresholds for adaptive hierarchical clipping of both shared and personalized update parameters. and These are the adjustment coefficients for the shared portion of the update parameters and the personalized portion of the update parameters, respectively. For client nodes No. The inferred risk coefficient of the wheel; S403, adaptively prunes the shared part and the personalized part according to the adaptive pruning threshold, respectively, for the updated parameters of the shared part and the updated parameters of the personalized part in this round; In step S301, the risk score is inferred. For the first Update parameters of the shared part of the wheel Compute client node No. Adaptive pruning threshold for shared wheel components And adaptive layered cropping includes: S501, Calculation of updated parameters for the shared portion in this round. After obtaining the norm, the quantile operator is used to calculate the basic threshold for adaptive hierarchical pruning of the shared part update parameters; S502, calculate the adaptive pruning threshold for adaptive layered pruning of shared part update parameters according to the following formula: ;
[0011] in, For client nodes No. The adaptive pruning threshold for adaptive layered pruning is updated in the shared portion of the round. To perform the maximum value operation, The lower limit of the clipping threshold for adaptive layered clipping with shared update parameters. The adjustment coefficient for the shared portion. For client nodes No. The inferred risk coefficient of the wheel; S503 performs adaptive layered pruning on the shared portion update parameters of this round based on the adaptive pruning threshold.
[0012] Optionally, the function expression for allocating the total privacy budget for this round between the sharing and personalization parts in step S201 is as follows: ; ; in, and Client nodes No. Budget for shared and personalized components. To perform the maximum value operation, and These are the minimum budgets for the shared portion and the personalized portion, respectively. and These are the default budgets for the shared portion and the personalized portion, respectively. and These are the adjustment coefficients for the shared portion and the personalized portion, respectively. For client nodes No. Risk factor of the wheel.
[0013] Optionally, in step S103, the central server performs alignment aggregation to generate a function expression for the shared partial parameters used in the next round of distribution as follows: ; in, For the first Shared parameters distributed in rounds, For the first Shared parameters distributed in rounds, For the first The set of client nodes in the wheel, For client nodes No. Alignment and aggregation weights of wheels For client nodes No. Update parameters for the shared portion after injecting noise in the round of submissions.
[0014] The present invention also provides an adaptive differential privacy protection system based on member inference risk, comprising an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the adaptive differential privacy protection method based on member inference risk.
[0015] Compared with existing technologies, the present invention mainly achieves the following beneficial effects: The method of the present invention includes a client node receiving the shared partial parameters of the current round issued by the central server; constructing a federated model using the shared partial parameters and personalized partial parameters of the current round; training the federated model using a local private dataset to obtain the updated parameters of the shared partial parameters and the updated parameters of the personalized partial parameters; calculating the inferred risk score based on various risk features of the training; injecting perturbation into the updated parameters of the shared partial parameters based on the inferred risk score and sending them to the central server for alignment and aggregation to generate the next round of shared parameters. The present invention achieves a closed loop of computable risk characterization, risk-driven adaptive differential privacy protection, and provable budget unified tracking in personalized federated learning scenarios where dishonest central servers may launch member inference attacks. This solves the key technical defects of existing privacy protection schemes in terms of dynamic risk adaptation, non-independent identically distributed fairness, and provable privacy budget throughout the entire process when facing dishonest central servers. It ensures that even in the case of malicious servers, uploaded data still meets differential privacy constraints. While improving privacy security, the present invention provides an auditable and controllable long-term training mechanism, which is particularly suitable for engineering deployment in complex heterogeneous environments. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.
[0017] Figure 2 This is a schematic diagram of the topology of the client node and the central server in an embodiment of the present invention.
[0018] Figure 3 This is a timing diagram of risk-driven differential privacy processing in an embodiment of the present invention.
[0019] Figure 4 This is a schematic diagram of the risk-driven differential privacy processing flow in an embodiment of the present invention.
[0020] Figure 5 This is a schematic diagram illustrating the division of software modules and their working principles in an embodiment of the present invention. Detailed Implementation
[0021] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.
[0022] Example 1: like Figure 1 As shown, the adaptive differential privacy protection method based on member inference risk driven in this embodiment includes the following steps: S101, The client node receives the shared parameters for this round from the central server; S102, the client node uses the shared part parameters and its own personalized part parameters to build a federated model composed of the shared part and personalized part, and uses the local private dataset to train the federated model for one round to obtain the updated parameters of the shared part and personalized part in this round. S103, the client node calculates the inferred risk score based on multiple risk features trained by the federated model; based on the inferred risk score, noise is injected into the shared part update parameters of this round, and the noise-injected shared part update parameters are sent to the central server for alignment and aggregation to generate new shared part parameters for the next round.
[0023] like Figure 2As shown, this embodiment's client nodes include a client node cluster consisting of clients 1 to K, each connected to a central server. The central server is configured to be dishonest, potentially recording and analyzing model updates uploaded by clients and launching membership inference attacks against them. Therefore, in this embodiment, risk assessment and risk-driven adaptive differential privacy protection are preferably performed on the client node side, while the central server only receives perturbation updates and performs alignment aggregation. Client nodes can be smartphones, wearable devices, edge gateways, workstations, or in-vehicle computing units, etc. Each client node includes at least: a processor: CPU and / or a dedicated accelerator (such as a GPU, NPU) for performing local training, update calculations, risk assessment, and differential privacy processing. A memory: for storing the local private dataset D_i, local model parameters, historical statistics of risk features (such as sliding window statistics required for quantile estimation), etc. The memory may employ encrypted storage and access control to reduce the risk of local data being read by unauthorized processes. A communication interface: for establishing an encrypted communication channel with the central server to transmit shared parameter distribution and perturbation update uploads. Trusted Execution Environment: In a preferred embodiment, the client may include a trusted execution environment to protect random number seed management, budget reduction logic, or accountant state from local system tampering. The central server may be a local server or a cloud instance, including at least a processor, memory, network interface, and logging module, used for: distributing shared model parameters and necessary auxiliary information; receiving perturbation updates uploaded by the client and performing alignment aggregation; and maintaining the shared model version history and aggregation metadata (such as participant sets, aggregation weights, etc.). In the threat model, the central server may attempt to implement membership inference attacks based on client-uploaded updates. However, in this embodiment, through a client-side perturbation mechanism, the central server only accesses perturbation updates that satisfy differential privacy constraints, thereby reducing its ability to recover / infer membership information from updates. After completing local training, each client node performs risk assessment and risk-driven adaptive differential privacy processing (including pruning, budget allocation, noise injection, privacy accounting, and budget control) locally to generate shared perturbation update parameters for this round; subsequently, it uploads these parameters to the central server via an encrypted channel. After receiving the shared perturbation update parameters for this round, the central server performs alignment and aggregation based on the shared perturbation update parameters sent by each client node to obtain new shared parameters for the next round of training.
[0024] The various risk features in step S103 of this embodiment include generalization gap feature, personalization proportion feature and shared direction difference feature. The generalization gap feature is used to characterize the local overfitting degree of the federated model. The personalization proportion feature is used to characterize the proportion of personalized updates in the overall layer updates of the federated model. The shared direction difference feature is used to characterize the degree of deviation between the shared update direction of this client and the group reference direction.
[0025] The generalization gap feature is used to characterize the degree of local overfitting. The stronger the overfitting, the easier it is for member inference attacks to exploit the separability differences between training and non-training samples. In this embodiment, the calculation function expression for the generalization gap feature is: ; in, To generalize the gap feature, To perform the maximum value operation, and Client nodes Use the Round using local private dataset The validation loss and training loss obtained when training the federated model.
[0026] The personalization percentage feature is used to characterize the "proportion of personalized updates in total updates." When personalization dominates the update behavior, the update is more likely to exhibit a client-specific pattern, increasing the risk of discriminability. In this embodiment, the calculation function expression for the personalization percentage feature is: ; in, For personalized proportion characteristics, For client nodes No. Wheel personalization update parameters of Norm, For client nodes No. Update parameters of the shared part of the wheel of Norm, This is used to prevent positive numbers from being divided by zero.
[0027] The shared direction difference feature is used to characterize the degree of deviation between the client's shared update direction and the group reference direction. The greater the deviation, the more "unique" the update, and the more likely it is to be exploited by attackers. Assume the central server distributed the group reference direction in the previous round... (This can be obtained by normalizing the direction of the previous round of shared aggregation update, specifically the central server's update direction for the first...) The total vector obtained by aggregating the shared update parameters after injecting noise uploaded by all client nodes according to their weights is then processed. The unit vector obtained from norm calculation), the expression for the calculation function of shared directional difference features is: ; in, To share directional differences, for and The dot product of vectors, The first one issued by the central server The group reference direction of the wheels, for of Norm. The cosine similarity ranges from [-1, 1], therefore the range of shared directional difference features is... When the shared direction difference feature is closer to 0, it means that the shared update direction is closer to the group reference direction; when the shared direction difference feature is closer to 1, it means that the shared update direction deviates more from the group direction.
[0028] The risk assessment in this embodiment is used to quantify the risk of member inference leakage that may result from a client uploading an update in a certain communication round, and outputs an inference risk score. The risk assessment is performed locally by the client node and does not rely on the trustworthiness of the central server. Step S103 in this embodiment, where the client node calculates the inference risk score based on multiple risk features trained by the federated model, includes: weighted summation of multiple risk features, adding a preset bias term to the weighted summation result, and then multiplying by a preset inference risk score to obtain the inference risk score. Specifically, the function expression for calculating the inference risk score is: ; in, To infer the risk score, To be used to infer risk scores Compress to range Monotonic compression functions within the range, such as the Sigmoid function, can be used. ~ Weighting coefficients that are greater than 0 To generalize the gap feature The normalization result, Personalized proportion features The normalization result, For shared directional differences The normalization result, This is the bias term. The inferred risk score can be divided into low / medium / high risk levels based on a threshold range, as needed. Optionally, the inferred risk score can be discretized into risk levels for strategy switching. Let the first... The set of client nodes selected in the wheel is Then any client node No. The updated parameters generated after training rounds include: superscript Indicates the shared portion. This refers to the personalized part.
[0029] like Figure 3 and Figure 4 As shown, at the beginning of each round, the central server selects participating clients and distributes shared parameters and reference directions through an encrypted channel. The clients perform local training and parameter alignment, followed by risk assessment. In the "Risk-Driven Differential Privacy Processing" module, they sequentially complete layered pruning, budget allocation, noise injection, and privacy accounting and budget control. If the cumulative privacy consumption does not exceed the budget limit, or if uploading is still allowed after triggering the control policy, the client uploads a perturbation-based shared update through the encrypted channel. After receiving and verifying the updates, the server aggregates and updates the updates from each client to obtain and distribute the new round of shared parameters, thus entering the next round of training. Specifically, in this embodiment, step S103, perturbation injection of the shared update parameters for this round based on the inferred risk score includes: S201, based on the inferred risk score, adaptively hierarchically prune the update parameters of the shared part and the personalized part in this round, respectively, to obtain the pruned update parameters of the shared part and the personalized part; allocate the total privacy budget of this round between the shared part and the personalized part to obtain the single-round privacy budget of the shared part and the personalized part; specifically, based on the inferred risk score... For the first Update parameters of the shared part of the wheel And personalization update parameters Calculate client nodes separately No. Wheel section Adaptive cropping threshold And perform adaptive layered cropping to obtain part Update parameters after cropping ,part For shared or personalized parts, , These are divided into shared and personalized components; based on inferred risk scores. Client node No. Total budget for the wheel Budget allocation is performed between the shared and personalized portions, with each portion going to the respective client nodes. No. Single-round privacy budget for shared and personalized components and Among them, the first Update parameters of the shared part of the wheel For objects that must be cropped; update parameters for personalized parts. The cropping object is optional, or it can be applied only to the first one. Update parameters of the shared part of the wheel Perform cropping (Example 2 involves updating parameters for the personalized portion without cropping) The purpose of pruning is to limit the maximum impact of a single round of updates on the output, establish an upper bound on sensitivity, and provide a controllable scale for subsequent Gaussian noise injection. S202, based on the single-round privacy budget of the shared part and the personalized part, determine the noise multiplier and noise standard deviation of the shared part and the personalized part respectively, and generate and inject noise into the trimmed update parameters of the shared part and the personalized part to obtain the updated parameters of the shared part and the personalized part after injecting noise. The functional expressions for the noise multipliers and noise standard deviation of the shared and personalized portions are as follows: ; ; in, For client nodes No. Wheel section The noise standard deviation For client nodes No. Wheel section noise multipliers, For client nodes No. Wheel section Adaptive cropping threshold, For client nodes No. Wheel section The fundamental noise multiplier (greater than 0). For layer The adjustment coefficient, For client nodes No. Risk factor of the wheel; The function expression for noise generation and injection is: ; in, For client nodes No. Wheel section Update parameters after noise injection It is Gaussian distributed noise. It is the identity matrix; S203, based on a single-round privacy budget and noise multipliers, for a given set of orders Each order in Calculate the cumulative amount of Reni's privacy: ; ; in, For the first The cumulative amount of privacy data for each level, "This is an accumulation operation." and Client nodes No. The cumulative Reni privacy amount for both the shared and personalized portions of the round is converted into privacy intensity based on the results of Reni differential privacy accounting. And the preset failure probability parameter for single-round privacy consumption And record it in the privacy mileage table of this client; calculate the cumulative privacy consumption of the client node based on the cumulative Reni privacy amount at each level: ; in, For client nodes The cumulative privacy consumption; S204, Determine whether the cumulative privacy consumption of the client node exceeds the preset total budget. If it does not exceed the total budget If the noise-injected shared portion update parameters are successfully executed, they will be sent to the central server for alignment and aggregation to generate new shared portion parameters for the next round of distribution; otherwise, the noise-injected shared portion update parameters will not be sent to the central server in this round.
[0030] To avoid the loss of control over cumulative privacy constraints caused by only accounting for the final uploaded noise while ignoring dynamic parameter tuning, this embodiment maintains a privacy accountant on the client side. This accountant uniformly records the noise multipliers and mechanism parameters actually used in each round and provides a budget control strategy. The client node maintains a set of orders. (e.g., several integer orders), and for each Maintaining cumulative Reni privacy data For the Gaussian mechanism, under the condition that the sensitivity is normalized to the noise ratio, the first... Wheel, Part The increment can be written as: ; The client node adds and accumulates the increments of the shared and personalized parts, which can be represented as: ; If client-side sampling is adopted (i.e., only a portion of clients participate in each round), the accounting unit can further use the upper bound of the sampling combination for bookkeeping; this is a well-known implementation method, and in this embodiment, it does not change the core idea of "bookkeeping with actual noise multipliers and accumulating control budgets". Based on the results of Reni differential privacy accounting, the privacy consumption in this round is converted into privacy strength. and failure probability This information is recorded in the client's privacy mileage table as a basis for subsequent budget control.
[0031] In this embodiment, when the shared part update parameters after injecting noise are not sent to the central server in this round, at least one of the following control actions is also triggered: Control action ①: Increase the number of client nodes No. Wheel section noise multiplier (thereby reducing) ), waiting to enter the first Round training; Control action ②: Reduce client node No. Wheel section Adaptive cropping threshold Waiting to enter the first Round training; Control movement ③: Terminate participation in the first round of training; Rounds of training. The above strategy ensures that even during risk-driven adaptive hyperparameter tuning, the system remains under provably cumulative privacy constraints.
[0032] Because a small number of oscillations or anomalous updates may occur during training, direct linear scaling would cause the risk score to be dominated by extreme values. Therefore, this embodiment maintains the quantile statistics of the features on the client side. The statistical method can be a sliding window quantile (e.g., the most recent quantile). (Round) or online quantile estimation. For any feature sequence ,maintain , Quantiles. For any feature The value of this round Define robust normalization results : ; in, To truncate the operator, This is a cutoff threshold used to suppress the impact of extreme values.
[0033] In step S201 of this embodiment, the adaptive hierarchical pruning of the shared and personalized update parameters for this round based on the inferred risk score includes: S401 calculates the update parameters for the shared portion and the personalized portion of this round, respectively. After applying the norm, the quantile operator is used to calculate the basic threshold for adaptive hierarchical pruning of the shared and personalized update parameters, which can be expressed as: ; ; in, and These are the base thresholds for the shared portion and the personalized portion, respectively. For quantile operators, Set a preset hyperparameter (e.g., a value of 0.9, used to obtain a clipping baseline that is "insensitive to outliers"). For client nodes No. Wheel personalization update parameters of Norm, For client nodes No. Update parameters of the shared part of the wheel of Norm; S402, calculate the adaptive pruning thresholds for both the shared partial update parameters and the personalized partial update parameters according to the following formula: ; ; in, and Client nodes No. The adaptive pruning threshold for adaptive layered pruning of both shared and personalized update parameters. To perform the maximum value operation, and These are the lower bounds of the pruning thresholds for adaptive hierarchical pruning of both the shared part and the personalized part of the parameter update (to prevent over-pruning from rendering the training unusable). and These are the adjustment coefficients for the shared portion of the update parameters and the personalized portion of the update parameters, respectively. For client nodes No. The inferred risk coefficient of the wheel; S403, based on the adaptive pruning threshold, adaptively prunes the shared and personalized update parameters for this round, respectively, for the shared and personalized parts. This can be expressed as: ; ; in, and Client nodes No. Update parameters after trimming the shared and personalized parts. To perform the minimum value operation, This is used to prevent positive numbers from being divided by zero.
[0034] This embodiment sets a privacy budget for the client in each round and differentiates it between the shared and personalized portions to match the differences in sensitivity, leakage risk, and utility contribution between the two portions. Let the client be... The total budget for the round is And allocate between the shared / personalized parts; the budget can be shrunk under risk-driven conditions (high risk → smaller). →Stronger protection). Specifically, in step S201 of this embodiment, the risk score is inferred. Client node No. Total budget for the wheel The function expression for allocating the budget between the shared and personalized portions is: ; ; in, and Client nodes No. Budget for shared and personalized components. To perform the maximum value operation, and These are the minimum budgets for the shared portion and the personalized portion, respectively. and These are the default budgets for the shared portion and the personalized portion, respectively. and These are the adjustment coefficients for the shared portion and the personalized portion, respectively. For client nodes No. The risk factor of the wheel. If the personalized part is not updated, then ( =0), differential privacy processing is performed only on the shared portion; this processing is an equivalent implementation of this embodiment and can expand the scope of application.
[0035] Each client node Client node No. Wheel disturbance shared parameters The data is uploaded to the server, and if optional perturbation / personalization information exists, it is also uploaded synchronously. In step S103 of this embodiment, the central server performs alignment aggregation to generate a function expression for the shared partial parameters used in the next round of distribution, as follows: ; in, For the first Shared parameters distributed in rounds, For the first Shared parameters distributed in rounds, For the first The set of client nodes in the wheel, For client nodes No. Alignment and aggregation weights of wheels For client nodes No. The shared portion of the parameters is updated after the noise injection in the first round of submissions. The central server then updates the parameters for the second round. Shared parameters distributed in round Issued to the Once a client node is selected in the round, it can proceed to the next stage. Rotational training.
[0036] As an optional implementation method, such as Figure 5 As shown, in this embodiment, the software modules of the client node and the central server are logically divided into independent yet collaborative modules. The software modules on the client node side include: a local training and parameter alignment module: receiving the shared parameters for this round from the server. And maintain personalized parameters. Before local training, perform parameter alignment constraints or initial alignment operations to match the data interface required for alignment aggregation; then perform local training to obtain the shared portion of updated parameters. And personalization update parameters Risk assessment module: Based on the statistical and update behavior during local training, it calculates risk characteristics and generates an inferred risk score. This serves as the driving signal for subsequent adaptive privacy protection. The adaptive differential privacy protection module includes sub-modules for "adaptive hierarchical pruning—hierarchical budget allocation—adaptive noise injection—privacy accounting and budget control," which performs adaptive differential privacy protection on the shared portion update parameters to obtain the shared portion update parameters after noise injection (optionally, perturbation-injected personalized portion update parameters are also obtained). The upload module uploads the noise-injected shared portion update parameters. Uploaded to the server. The software modules on the central server side include: a parameter distribution module: distributing parameters to the selected client set. Distribute the shared parameters for this round Along with necessary auxiliary information (such as group reference direction, etc.). Alignment and aggregation module: Performs alignment and aggregation on perturbation updates from clients, updates shared parameters, and distributes the next round of shared models. Evaluation module (optional): Used to calculate performance indicators, attack indicators, etc. during development and experimentation; not a required deployment module. The client node-side software module and the central server-side software module conduct bidirectional interaction of "shared parameter distribution" and "perturbation update upload" through an encrypted communication channel. The "perturbation update" is the result generated by the client node side after completing risk assessment and adaptive differential privacy processing. The central server side only receives and aggregates perturbation updates and cannot directly obtain the undisturbed original update, reducing the amount of information members can use to infer attacks.
[0037] In summary, the adaptive differential privacy protection method based on member inference risk in this embodiment includes: (1) Constructing a risk-driven dynamic protection mechanism: quantifying the risk of each round by calculating the generalization gap, personalization ratio, and directional difference in real time, dynamically adjusting the pruning threshold and noise intensity, achieving the effect of less disturbance when the risk is low (preserving accuracy) and strong protection when the risk is high (preventing attacks), thus solving the problem of temporal imbalance of static strategies. (2) Achieving differentiated fair protection under heterogeneity: allocating privacy budget and pruning strategy in a personalized manner according to the real-time risk score of each client. Providing stricter protection for clients with unique data distribution, and retaining more utility for regular clients, thus solving the barrel effect caused by non-independent identically distributed (Non-IID). (3) Establishing a computable and interpretable multidimensional risk feature system: no longer relying solely on the gradient norm, but introducing three orthogonal dimensions of indicators: generalization gap feature, personalization ratio feature, and shared directional difference feature, to accurately locate the source of risk. (4) Introduction of a full-process Raney differential privacy accountant and privacy filter: All steps involving private data, such as risk assessment, parameter selection, and update release, are incorporated into a unified differential privacy budget tracking system. The budget is automatically truncated before it is exhausted by a privacy filter, ensuring that the scheme has a mathematically rigorous upper bound proof on privacy. This embodiment addresses the problem of severe data heterogeneity and large differences in client behavior in personalized federated learning, where existing static differential privacy schemes struggle to balance privacy and performance. It introduces a risk assessment based on three types of features and uses risk scores to drive hierarchical pruning, hierarchical budgeting, and noise intensity adjustment. This allows for dynamic matching of protection strength at each round, for each client, and between the shared and personalized layers. Sensitivity is automatically tightened and noise is increased for high-risk updates, while sufficient effective signals are maintained for low-risk updates. This better maintains model convergence and personalization effects under the same privacy constraints, outperforming differential privacy methods using uniform fixed parameters. Meanwhile, this embodiment deploys the entire privacy protection process on the client side, uploading only perturbed updates, making it difficult for the central server to directly exploit the original gradients to launch membership inference attacks, even if it is dishonest. Furthermore, it employs a Raney differential privacy accountant to continuously track and constrain cumulative privacy consumption, automatically triggering protection measures or stopping uploads when the budget limit is approached. Therefore, this embodiment improves privacy security while providing an auditable and controllable long-term training mechanism, making it more suitable for engineering deployment in complex heterogeneous environments.
[0038] Example 2: This embodiment is basically the same as Embodiment 1, with the main difference being that in step S103, when injecting perturbation into the shared portion update parameters of this round based on the inferred risk score, the personalized portion update parameters are not processed, which can improve the efficiency of data processing. Step S103, injecting perturbation into the shared portion update parameters of this round based on the inferred risk score, includes: S301, based on the inferred risk score, adaptively hierarchically prune the update parameters of the shared portion in this round to obtain the pruned update parameters of the shared portion; allocate the total privacy budget of this round to the single-round privacy budget of the shared portion; S302, noise is generated and injected into the shared part update parameters after clipping to obtain the shared part update parameters after noise injection. The method is the same as step S202 in Embodiment 1, except that it does not include personalized part update parameters. S303, calculate the cumulative Reni privacy amount for each order in a given set of orders based on the noise multiplier used when generating the noise, and calculate the cumulative privacy consumption of the client node based on the cumulative Reni privacy amount of each order; the method is the same as step S302 in Embodiment 1, except that it does not include personalized part update parameters. S304, determine whether the cumulative privacy consumption of the client node exceeds the preset total budget. If it does not exceed the total budget, send the shared part update parameters after injecting noise to the central server for alignment and aggregation to generate new shared part parameters for the next round of distribution; otherwise, do not send the shared part update parameters after injecting noise to the central server in this round.
[0039] Step S301, which involves adaptively tiered pruning of the shared portion update parameters for this round based on the inferred risk score, includes: S501, Calculation of updated parameters for the shared portion in this round. After the norm is calculated, the quantile operator is used to calculate the basic threshold for adaptive hierarchical pruning of the shared part update parameters. The method is the same as step S401 in Example 1, except that it does not include personalized part update parameters. S502, calculate the client node according to the following formula. No. Risk-driven adaptive pruning threshold for wheels: ;
[0040] in, For client nodes No. Adaptive pruning threshold for shared components in round trips. To perform the maximum value operation, This is the lower limit of the pruning threshold for the shared portion. The adjustment coefficient for the shared portion. For client nodes No. The inferred risk coefficient of the wheel; S503, adaptively prunes the shared portion according to the adaptive pruning threshold: ;
[0041] in, For client nodes No. Update parameters after cropping the shared portion. To perform the minimum value operation, This is used to prevent positive numbers from being divided by zero.
[0042] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. An adaptive differential privacy protection method based on member inference risk-driven approach, characterized in that, Includes the following steps: S101, The client node receives the shared parameters for this round from the central server; S102, the client node uses the shared part parameters and its own personalized part parameters to build a federated model composed of the shared part and personalized part, and uses the local private dataset to train the federated model for one round to obtain the updated parameters of the shared part and personalized part in this round. S103, the client node calculates and infers a risk score based on multiple risk features trained by the federated model; Noise is injected into the shared part update parameters of this round based on the inferred risk score, and the noise-injected shared part update parameters are sent to the central server for alignment and aggregation to generate new shared part parameters for the next round.
2. The adaptive differential privacy protection method based on member inference risk-driven approach according to claim 1, characterized in that, The various risk features in step S103 include generalization gap feature, personalization proportion feature, and shared direction difference feature. The generalization gap feature is used to characterize the local overfitting degree of the federated model. The personalization proportion feature is used to characterize the proportion of personalized updates in the total layer updates of the federated model. The shared direction difference feature is used to characterize the degree of deviation between the shared update direction of this client and the group reference direction.
3. The adaptive differential privacy protection method based on member inference risk-driven approach according to claim 2, characterized in that, The expression for the calculation function of the generalization gap feature is: ； in, To generalize the gap feature, To perform the maximum value operation, and Client nodes Use the Round using local private dataset The validation loss and training loss obtained during training of the federated model; the calculation function expression for the personalized proportion feature is: ； in, For personalized proportion characteristics, For client nodes No. Wheel personalization update parameters of Norm, For client nodes No. Update parameters of the shared part of the wheel of Norm, To prevent positive numbers from being divided by zero; the calculation function expression for the shared direction difference feature is: ； in, To share directional differences, for and The dot product of vectors, The first one issued by the central server The group reference direction of the wheel, wherein the group reference direction is the direction of the central server for the first wheel. The total vector obtained by aggregating the shared update parameters after injecting noise uploaded by all client nodes according to their weights is then processed. The unit vector obtained by norm calculation; for of Norm.
4. The adaptive differential privacy protection method based on member inference risk-driven approach according to claim 2, characterized in that, In step S103, the client node calculates the inferred risk score based on multiple risk features trained by the federated model, which includes: weighting and summing multiple risk features, adding a preset bias term to the result of the weighted summation, and then multiplying it by a preset inferred risk score to obtain the inferred risk score.
5. The adaptive differential privacy protection method based on member inference risk-driven approach according to claim 1, characterized in that, Step S103, which involves injecting perturbations into the shared portion of the update parameters based on the inferred risk score, includes: S201, based on the inferred risk score, adaptively hierarchically prune the update parameters of the shared part and the personalized part in this round to obtain the pruned update parameters of the shared part and the personalized part; allocate the total privacy budget of this round between the shared part and the personalized part to obtain the single-round privacy budget of the shared part and the personalized part. S202, based on the single-round privacy budget of the shared part and the personalized part, determine the noise multiplier and noise standard deviation of the shared part and the personalized part respectively, and generate and inject noise into the trimmed update parameters of the shared part and the personalized part to obtain the updated parameters of the shared part and the personalized part after injecting noise. S203, based on the single-round privacy budget and noise multipliers, calculate the cumulative Reni privacy for each order in the given set of orders, and calculate the cumulative privacy consumption of the client node based on the cumulative Reni privacy for each order: ； in, For client nodes Cumulative privacy consumption, To perform the minimum value operation, For client nodes The The cumulative amount of privacy data for each level. Given a set of orders, The default failure probability parameter for single-round privacy consumption; S204, determine whether the cumulative privacy consumption of the client node exceeds the preset total budget. If it does not exceed the total budget, send the shared part update parameters after injecting noise to the central server for alignment and aggregation to generate new shared part parameters for the next round of distribution; otherwise, do not send the shared part update parameters after injecting noise to the central server in this round. Alternatively, step S103, which involves injecting a perturbation into the shared portion of the update parameters based on the inferred risk score, includes: S301, based on the inferred risk score, adaptively hierarchically prune the update parameters of the shared portion in this round to obtain the pruned update parameters of the shared portion; allocate the total privacy budget of this round to the single-round privacy budget of the shared portion; S302, noise is generated and injected into the update parameters of the clipped shared part to obtain the update parameters of the shared part after noise injection; S303, calculate the cumulative Reni privacy amount for each order in a given set of orders based on the noise multiplier used when generating the noise, and calculate the cumulative privacy consumption of the client node based on the cumulative Reni privacy amount of each order. S304, determine whether the cumulative privacy consumption of the client node exceeds the preset total budget. If it does not exceed the total budget, send the shared part update parameters after injecting noise to the central server for alignment and aggregation to generate new shared part parameters for the next round of distribution; otherwise, do not send the shared part update parameters after injecting noise to the central server in this round.
6. The adaptive differential privacy protection method based on member inference risk-driven approach according to claim 5, characterized in that, The step of not sending the updated parameters of the shared part after injecting noise to the central server in this round also includes triggering at least one of the following control actions: Control action ①: The client node increases the noise multiplier used when generating noise and waits to enter the next round of training; Control action ②: The client node decreases the adaptive pruning threshold when performing adaptive layer pruning and waits to enter the next round of training; Control action ③: The client node terminates its participation in the next round of training.
7. The adaptive differential privacy protection method based on member inference risk-driven approach according to claim 5, characterized in that, Step S201 involves adaptive hierarchical pruning of the shared and personalized update parameters for this round based on the inferred risk score, including: S401 calculates the update parameters for the shared portion and the personalized portion of this round, respectively. After applying the norm, the quantile operator is used to calculate the base threshold for adaptive hierarchical pruning of the shared and personalized update parameters: S402, calculate the adaptive pruning thresholds for both the shared partial update parameters and the personalized partial update parameters according to the following formula: ； ； in, and Client nodes No. The adaptive pruning threshold for adaptive layered pruning of both shared and personalized update parameters. To perform the maximum value operation, and These are the lower limits of the clipping thresholds for adaptive hierarchical clipping of both shared and personalized update parameters. and These are the adjustment coefficients for the shared portion of the update parameters and the personalized portion of the update parameters, respectively. For client nodes No. The inferred risk coefficient of the wheel; S403, adaptively prunes the shared part and the personalized part according to the adaptive pruning threshold, respectively, for the updated parameters of the shared part and the updated parameters of the personalized part in this round; Step S301, which involves adaptively tiered pruning of the shared portion update parameters for this round based on the inferred risk score, includes: S501, Calculation of updated parameters for the shared portion in this round. After obtaining the norm, the quantile operator is used to calculate the basic threshold for adaptive hierarchical pruning of the shared part update parameters; S502, calculate the adaptive pruning threshold for adaptive layered pruning of shared part update parameters according to the following formula: ； in, For client nodes No. The adaptive pruning threshold for adaptive layered pruning is updated in the shared portion of the round. To perform the maximum value operation, The lower limit of the clipping threshold for adaptive layered clipping with shared update parameters. The adjustment coefficient for the shared portion. For client nodes No. The inferred risk coefficient of the wheel; S503 performs adaptive layered pruning on the shared portion update parameters of this round based on the adaptive pruning threshold.
8. The adaptive differential privacy protection method based on member inference risk-driven method according to claim 5, characterized in that, The function expression for allocating the total privacy budget for this round between the sharing and personalization parts in step S201 is as follows: ； ； in, and Client nodes No. Budget for shared and personalized components. To perform the maximum value operation, and These are the minimum budgets for the shared portion and the personalized portion, respectively. and These are the default budgets for the shared portion and the personalized portion, respectively. and These are the adjustment coefficients for the shared portion and the personalized portion, respectively. For client nodes No. Risk factor of the wheel.
9. The adaptive differential privacy protection method based on member inference risk-driven approach according to claim 1, characterized in that, In step S103, the central server performs alignment aggregation to generate a function expression for the shared partial parameters used in the next round of distribution: ； in, For the first Shared parameters distributed in round 1 For the first Shared parameters distributed in round 1 For the first The set of client nodes in the wheel, For client nodes No. Alignment and aggregation weights of wheels For client nodes No. Update parameters for the shared portion after injecting noise in the round of submissions.
10. An adaptive differential privacy protection system based on member inference risk, comprising interconnected microprocessors and memory, characterized in that, The microprocessor is programmed or configured to execute the adaptive differential privacy protection method based on member inference risk as described in any one of claims 1 to 9.