A federated learning backdoor risk detection method and system based on composite disturbance response residual

Abstract

The application discloses a kind of federal learning backdoor risk detection method and system based on composite disturbance response residual, center server receives each client model update before aggregation, with the global model that is not updated as detection benchmark;Select detection probe and two semantic keeping disturbance operators, construct undisturbed, two single disturbance and composite disturbance input, calculate the deviation of composite disturbance actual response relative to reference response after inputting model in specified response layer, and obtain composite disturbance response residual.Estimating the influence of each client update on residual change by residual parameter gradient, screening the client to be reviewed, applying its update to benchmark model to generate candidate model and calculate actual residual change, while checking candidate global model residual before model release.According to the review result, the client update is executed normal aggregation, weight reduction, delay or isolation and other processing.This method realizes the backdoor risk detection of client update granularity, can reduce review overhead, and improves risk identification interpretability.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence security and network security technology, specifically to a federated learning backdoor risk detection method and system based on composite perturbation response residuals. Background Technology

[0002] Federated learning allows multiple clients to collaboratively train a global model using their own data without uploading their local raw data. The central server typically only receives model parameters, updates, or gradient information uploaded by clients and aggregates them accordingly. Therefore, it's difficult to directly verify the reliability of the client's local training data, training process, and optimization objective. Malicious clients might embed backdoor behavior into the global model by modifying local training samples, adjusting training objectives, or tampering with uploaded model updates. This allows the model to maintain high predictive performance with normal inputs but produce attacker-predicted outputs when the input contains specific triggering patterns.

[0003] Existing backdoor defense methods for federated learning mainly include anomaly detection methods based on update magnitude, update direction, or distance relationship, robust aggregation methods, and detection methods based on synthetic samples, model fingerprints, and neuron response features.

[0004] In their paper "DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection," P. Rieger et al. clustered and filtered client updates by analyzing the output layer parameters, predicted outputs, and internal structural features of the client model.

[0005] In "FedDefender: Backdoor Attack Defense in Federated Learning", W. Gill et al. tested multiple client models using the same synthetic input, extracted their neuron activation states as model fingerprints, and used differential testing to identify client models that may contain backdoors.

[0006] Chinese patent application No. 202311734020 discloses a "Method and System for Detecting Poisoning in Bidirectional Models in Federated Learning". This method involves a validator generating a verification vector based on the global gradient, calculating the local gradient on the client side and the dot product between the global gradient and the verification vector, identifying and removing harmful gradients based on the verification results, and simultaneously verifying whether the central server is engaging in poisoning behavior.

[0007] Chinese patent application No. 202510068019 discloses "A Federated Learning Backdoor Attack Defense Method Based on Model Fingerprints". This method generates fingerprint samples on the server side that can cause normal models and backdoor models to produce different responses. The fingerprint samples are input into the model submitted by the client, and the output category, confidence level, or loss value are recorded. The actual response is compared with the expected standard response. The server adjusts the aggregation weights based on the number of client participations and historical detection results, rejecting or restricting subsequent participation of clients with continuously abnormal updates.

[0008] However, the aforementioned existing technologies still have the following shortcomings: Existing model fingerprinting or differential testing methods typically compare the actual response to a single detection input with the expected response, or compare the activation differences between different client models on the same input. They fail to further distinguish between the response changes caused by two semantically preserving perturbations individually and the response changes caused by their combined effect. Therefore, it is difficult to extract the non-superimposed response generated by composite perturbations from general perturbation sensitivity. While screening based solely on update vectors, gradient directions, or dot product relationships has low overhead, the resulting anomalies mainly reflect differences in the parameter space or gradient space, and cannot directly indicate how the corresponding update will change the behavior of a specific model. Loading and running all client models one by one can obtain actual behavioral results, but the computational cost of detection typically increases with the number of clients.

[0009] If approximate calculations are used to predict the impact of client updates on model behavior without calibrating the approximation error using actual candidate model results, it becomes difficult to determine the reliable boundaries of the screening results. If the detection samples, perturbation configurations, and response judgment rules are fixed for a long time or known before the client generates the current update, there is a possibility of targeted adaptation by attackers. If client results that have not been adequately verified are used to update normal historical benchmarks, subsequent judgment benchmarks may also be affected by abnormal updates. Furthermore, multiple client updates that do not meet the high-risk conditions in individual checks may still produce superimposed anomalies after aggregation, meaning that performing only single-client checks cannot fully guarantee the security of the global model to be released. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides a federated learning backdoor risk detection method and system based on composite perturbation response residuals. It uses specific model behavior statistics as the detection object, performs low-cost impact screening on all client updates, verifies the actual candidate models only for necessary clients, and uses the verification results to calibrate screening errors. At the same time, it performs behavior checks on the aggregated candidate global models again before official release, thereby achieving backdoor risk location and aggregation at the client update granularity while controlling detection overhead.

[0011] This invention discloses a backdoor risk detection method for federated learning based on composite perturbation response residuals, applied to a federated learning system including a central server and multiple clients. The method is characterized in that the central server receives shared model updates uploaded by each client based on the same current global model before the formal aggregation round; the current global model, as described below, is the baseline model before aggregation that has not yet been incorporated into any client's update in this round. The central server performs the following steps: Step 1: Establish the detection configuration; the detection configuration includes a probe base sample pool, a semantic preservation perturbation operator library, response acquisition location, response pooling rules, response scalar projection rules, historical normal residual benchmark, upper bound of estimation error, candidate model verification calculation budget, various screening, judgment and release thresholds, and aggregation processing rules; among them, semantic preservation perturbation operators refer to transformation methods that do not change the detection probe task label or main semantics; Step 2: Receive the shared model updates uploaded by each client in this round based on the current global model, and form a set of client updates to be detected: wherein each record in the set of client updates to be detected includes the client identifier, the current global model version, and the shared model update; Step 3: After forming the client update set to be detected, select or generate the current round detection probe, and select two semantic preservation perturbation operators from the semantic preservation perturbation operator library. Based on the same detection probe, construct an unperturbed input, two single perturbation inputs, and a composite perturbation input. Step 4: Input the unperturbed input, the two single perturbation inputs and the composite perturbation input into the current global model respectively, obtain the pooled response at the response acquisition location, calculate the deviation of the actual response of the composite perturbation input from the reference response formed by the unperturbed input and the two single perturbation inputs, and use it as the composite perturbation response residual of the current global model; Step 5: Convert the composite disturbance response residuals described in Step 4 into scalar detection quantities, and calculate the residual parameter gradient of the scalar detection quantities relative to the current global model parameters; based on the inner product of the residual parameter gradients and the shared model updates for each client, obtain the estimated impact of each client update on the composite disturbance response residuals, which is used to characterize the expected extent to which the corresponding client update increases, decreases, or deviates from the historical normal residual range of the composite disturbance response residuals; Step 6: Based on the estimated impact, historical normal residual benchmark, upper bound of estimation error, client identity status and candidate model review budget described in Step 5, filter the set of clients to be reviewed; Step 7: For each client to be reviewed identified in Step 6, copy the current global model and apply the shared model update only to that client to construct a single-client candidate model; input the same set of inputs from Step 3 into the candidate model, calculate the actual residual change and actual residual risk brought about by the client update; calibrate the upper bound of the estimation error based on the difference between the actual residual change and the estimated impact in Step 5, and obtain the backdoor risk review result of the single-client candidate model by comparing the actual residual risk with the risk threshold; where the backdoor risk review result is low risk, uncertain risk, or high risk. Step 8: Based on the single-client candidate model review results determined in Step 7, perform preliminary processing on client updates and form a set of clients to be adopted: low-risk updates are aggregated normally, updates with uncertain risks are aggregated with reduced weight, updates that have not completed review are aggregated with delayed weighting, and updates that meet high-risk conditions are isolated; a candidate global model is formed based on the set of clients to be adopted, and its residual release risk is calculated; when the residual release risk does not exceed the threshold, the candidate global model is released according to the aforementioned processing; when the residual release risk exceeds the threshold, release is paused, and the review is expanded, re-aggregated, the aggregation step size is reduced, or a rollback is performed. The detection results include at least the client identifier, disturbance combination, response acquisition location, abnormal residual projection components, residual risk level, and aggregation processing.

[0012] As a further improvement of the present invention, in step 3, for the detection probe The central server selects two semantically preserving perturbation operators. and The central server constructs the following input:

[0013] in, For unperturbed input, and For two single perturbation inputs, and The input is a composite perturbation with two different application orders; the central server further calculates the order difference. :

[0014] in, The allowed input distance or reference feature distance for the task; when ≤ hour, To preset the order difference threshold, the central server adopts... Or adopt and The mean of the response is taken as the composite perturbation response; when > At that time, the central server retains respectively and The corresponding residuals do not require averaging the response to eliminate order differences.

[0015] As a further improvement of the present invention, in step 4, the composite perturbation response residual is equivalent to extracting the interaction term of two semantically preserving perturbations on the model response layer, and its abnormal deviation relative to the historical normal residual benchmark is used to indicate that the model’s response sensitivity to local feature combinations or composite perturbation combinations has changed abnormally. Let the current global model parameters be denoted as The model will input at the response acquisition location. The pooling response is denoted as ;in, express , , , or Any input in; when using When used as a composite disturbance input, the composite disturbance response residual for:

[0016] in, This represents the deviation of the actual response to the composite perturbation from the reference response formed by the unperturbed input and the two single perturbation inputs. It is used to describe the anomaly in the interactive response of the current global model under the combined effect of two semantically preserving perturbations. > At that time, the central server used replace Using the same calculation formula, the residual of the composite disturbance response corresponding to the order in which another disturbance is applied is calculated.

[0017] As a further improvement of the present invention, in step 5, the central server adopts a fixed response scalar projection function. vector residuals Convert to scalar detection quantity And calculate the gradient of the residual parameters. and the Estimated impact of client updates :

[0018] in, Used for comparison with historical normal residual benchmarks; Scalar detection quantity Relative to the current global model parameters The gradient; For the first Updates to the shared model uploaded by each client; Update the first-order estimate of the impact of residual changes on the client. This represents the inner product of parameters at the corresponding model parameter positions; the estimated impact is used for full screening and is not directly used as the final anomaly conclusion.

[0019] As a further improvement of the present invention, in step 6, the upper bound of the estimation error is determined based on the deviation between the estimated impact and the actual residual change in the historical low-risk review samples. The conservative screening risk is calculated and determined based on the estimated impact of each client, the historical normal residual benchmark, and the upper bound of the estimation error. Clients that are expected to cause the composite disturbance response residual to deviate from the historical normal range or whose estimation results are uncertain are screened. If one of the following conditions is met: the conservative screening risk reaches the screening threshold, the upper bound of the estimation error reaches the uncertainty threshold, or the client's identity status meets the mandatory review conditions specified by the server, then the client is included in the set to be reviewed. If the number of clients to be reviewed exceeds the review budget, the review priority is determined comprehensively based on the conservative screening risk, the upper bound of the estimation error, the proposed aggregation weight, and the identity status.

[0020] As a further improvement of the present invention, in step 6, the central server performs conservative screening using the historical normal residual benchmark and the upper bound of the estimation error; the historical normal residual benchmark is denoted as... ,in The central value of the low-risk historical residual scalar sample. The normal fluctuation scale for the low-risk historical residual scalar sample; for the first Each client, the central server calculates conservative screening risks. :

[0021] in, In order to be with the first The upper bound of the estimation error is obtained by updating historical review samples with similar amplitudes for each client. Additional risk items introduced due to first-time participation, inability to continuously associate identities, prolonged lack of review, abnormal update amplitude, or unreproducible inference state. To prevent positive numbers with a denominator of zero; when Exceeding the screening threshold ,or Exceeding the uncertainty threshold At that time, the central server will add the corresponding client to the set of clients to be reviewed.

[0022] As a further improvement of the present invention, in step 7, the central server selects the first client from the set of clients to be reviewed. Each client only applies the client-shared model update to construct candidate models. And calculate the actual residual change. and actual residual risk :

[0023]

[0024]

[0025] in, The composite perturbation response residuals obtained by the candidate model under the same detection probe, the same perturbation combination, and the same response acquisition location; This represents the actual change in residuals caused by the client update. This indicates the degree to which the residual scalar deviates from the historical normal residual baseline after the application of this client for updating.

[0026] As a further improvement of the present invention, in step 7, the central server calibrates the upper bound of the estimation error using client samples that have passed the candidate model review or have been determined to be non-abnormal; the estimation error samples for:

[0027] in, Indicates the first The difference between the actual residual change and the estimated impact for each client. Samples that fail the candidate model review or are deemed anomalous are not included in the set of normal samples at the upper bound of the estimation error, nor are they used to update the historical normal residual benchmark.

[0028] As a further improvement of the present invention, in step 8, the central server first determines the set of clients to be adopted to form a candidate global model based on the single-client candidate model review results. And calculate the release risk of candidate global models. ,

[0029]

[0030] in, For the first Each client updates the actual aggregate weight after aggregation processing. This represents the degree to which the residual scalar of the candidate global model deviates from the historical normal residual baseline.

[0031] The central server processes client updates according to the following rules, where, This is the normal residual risk threshold. The high-risk threshold Threshold for publishing candidate global models: If the client update passes the candidate model review, it satisfies... And the candidate global model satisfies Then it will participate in the aggregation normally; If the client update passes the candidate model review, it satisfies... If the client participates for the first time, its identity cannot be continuously associated, or the review interval exceeds the preset period but does not trigger high-risk conditions, its participation in aggregation will be downgraded or delayed. like If so, the corresponding client is isolated from updating and the high-risk detection result is output; like If so, the release of the current aggregation model will be suspended, the set of clients to be reviewed will be expanded, and aggregation will be re-aggregated, the aggregation step size will be reduced, or the model will be rolled back.

[0032] This invention also discloses a federated learning backdoor risk detection system that implements the above detection method, applied to a federated learning system including a central server and multiple clients, wherein the central server includes: The detection configuration module is used to execute step 1 of the detection method; The client updates the receiving module to execute step 2 of the detection method; The input detection module is used to execute step 3 of the detection method. The composite disturbance response residual calculation module is used to perform step 4 of the detection method; The impact estimation module is used to execute step 5 of the detection method. The review and screening module is used to execute step 6 of the detection method; The candidate model verification module is used to perform step 7 of the detection method; The aggregation processing and release inspection module is used to execute step 8 of the detection method.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows: First, in this invention, the central server internally selects the detection probe and perturbation configuration after receiving the client update, without relying on the client to honestly execute the test sequence or upload the real detection response. Second, this invention uses a composite perturbation response residual characterization model to reflect the abnormal response interaction between two semantically preserving perturbations, which can reflect backdoor risks from a behavioral perspective, rather than just observing update norms, directions, or clustering relationships. Third, this invention utilizes the inner product of the residual parameter gradient and the client update to perform low-cost estimation of the impact of all client updates for screening, while constructing candidate models only for high-risk, high-uncertainty, new identity, or rotating sampled clients, thereby reducing server computational overhead. Fourth, this invention calibrates the upper bound of the estimation error by verifying the actual residual change obtained from the candidate model, thus avoiding taking the first-order estimate directly as a deterministic conclusion. Fifth, this invention restricts the updating of historical normal residual benchmarks to samples that have passed review, have low risk, and are normally aggregated, and checks candidate global models before release, reducing the risk of abnormal updates polluting the benchmark and multiple low-risk updates causing global anomalies. Sixth, the backdoor risk detection results output by this invention to the client at the shared model update granularity include high-risk client identifiers, anomaly detection combinations, anomaly residual projection components, risk levels, and aggregation handling, making the detection purpose, location granularity, and handling results clear. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall process of the federated learning backdoor risk detection method based on composite perturbation response residuals of the present invention; Figure 2 This is a schematic diagram of the composite disturbance input structure and response residual calculation of the present invention; Figure 3 This is a schematic diagram illustrating the aggregation processing and candidate global model release check of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] The present invention will now be described in further detail with reference to the accompanying drawings: This invention provides a method and system for detecting backdoor risks in federated learning based on composite perturbation response residuals. The core technical problems to be solved include: First, avoid relying on the client to honestly perform tests and upload real responses in the detection process; Second, the impact of client updates on model behavior statistics should be included in the screening, rather than just using update norm, direction, or clustering relationship; Third, cover all client updates in a low-cost manner, while only performing high-cost candidate model verification on necessary clients; Fourth, it makes it difficult for the detection probes, perturbation combinations, and response projection rules to be used in advance when the client generates the current update, thus reducing the risk of pre-adaptation; Fifth, the upper bound of the estimation error is verified and calibrated through candidate models, and unverified or abnormal samples are restricted from entering the historical normal residual benchmark, thereby reducing the risk of benchmark contamination.

[0037] The design basis of this invention is as follows: when semantic preservation perturbation is small and model behavior is normal, the response changes caused by two single perturbations and the response changes when they act together should generally be within a stable historical interaction range. If a client update causes the response interaction term under composite perturbation to deviate significantly from this range, it indicates that the update may have changed the model's sensitivity to local input patterns, triggering similar patterns, or combined perturbations. Backdoor updates, in order to maintain normal input performance, may not be significantly abnormal in the update norm, but may cause abnormal response changes under conditions that trigger similar composite perturbations. Therefore, this invention uses the composite perturbation response residual as a behavioral-level risk signal. The scalar detection quantity corresponding to the residual is differentiable with respect to the model parameters, and its parameter gradient can be used to approximate the impact of each client update on the residual, thereby screening all clients at low cost. Since this estimation is a first-order approximation, this invention only constructs candidate models for suspicious clients for actual verification and calibrates subsequent estimation errors with measured deviations; finally, the candidate global model to be released is checked again to identify the combined risks generated by the aggregation of multiple client updates.

[0038] The specific detection method of the present invention includes: S1. Establish detection configuration: The central server establishes a detection configuration, which includes a probe-based sample pool, a semantically preserved perturbation operator library, response acquisition locations, response pooling rules, response scalar projection rules, historical normal residual benchmarks, upper bounds of estimation errors, candidate model verification calculation budgets, various screening and release thresholds, and aggregation and processing rules.

[0039] The probe-based sample pool can come from validation samples retained on the server, authorized and desensitized samples, rule-generated synthetic samples, or a combination of the above sources.

[0040] Semantic-preserving perturbation operators refer to input transformations that do not alter the task label or primary semantics. For example, in image tasks, these might include minor brightness adjustments, minor contrast adjustments, minor blurring, minor scale changes, or minor noise reduction; in text tasks, they might include equivalent substitutions, punctuation normalization, or minor formatting transformations that do not change the label semantics. The central server can determine whether the perturbation still preserves the semantics based on task rules, a frozen reference model, the previously published global model, or reference feature distance. If the input after perturbation does not meet the category preservation or reference feature distance constraints, the perturbation intensity is reduced, the detection probe is replaced, or the perturbation combination is abandoned.

[0041] The historical normal residual benchmark is used to represent the statistical range of residuals generated by normal models or low-risk client updates under similar detection combinations. For a detection combination, the historical normal residual benchmark is denoted as... ,in The center value of historical residual scalar samples that have passed candidate model verification, have low risk, have actually participated in normal aggregation, and have not triggered anomalies in candidate global model checks. This represents the normal fluctuation scale of the historical residual scalar samples. When historical samples are insufficient, the central server uses a preset temporary benchmark, a neighboring model version window benchmark, or a robust population benchmark.

[0042] S2. Receive client updates and form a set to be detected: No. During each training round, the central server sends the current global model parameters to each client. . No. After each client completes local training, it uploads the shared model for updates. ,in Indicates the client number. This indicates that the client wants to incorporate parameter updates into the global model. The central server obtains each client's data before the actual aggregation. And the corresponding client identifier, forming a set of client updates to be detected.

[0043] Each record in the client update set to be detected includes at least the client identifier, the current global model version, and the shared model update.

[0044] S3. Construct the input for this round of detection: After forming the set of client updates to be detected in this round, the central server internally selects or generates detection probes for this round. The detection probe Samples can be drawn from a probe-based sample pool or generated by a synthetic sample generator. Subsequently, the central server selects two semantically preserving perturbation operators. and .

[0045] The central server targets the same detection probe. Construct input:

[0046] in, For unperturbed input, To apply only the perturbation operator A single perturbation input, To apply only the perturbation operator A single perturbation input, Apply first Apply after The resulting composite perturbation input Apply first Apply after The resulting composite perturbation input. The purpose of the above construction is not to find real attack triggers, but to observe whether the model's response interaction is abnormal when two perturbations are applied together, while keeping the task semantics basically unchanged.

[0047] The central server calculates the sequential difference between the two composite perturbation inputs:

[0048] in, The input can be spatial distance, reference feature distance, or a distance allowed by the task rules. Used to determine whether the order in which two perturbation operators are applied significantly affects the detected input. ≤ hour, To preset the order difference threshold, the central server adopts... , or use and The mean response decreases numerical fluctuation; when > At that time, the central server retains respectively and The corresponding residuals are used as one of the risk clues.

[0049] S4. Calculate the residuals and gradients of the residual parameters in the combined disturbance response: The central server sends the input obtained in step S3 into the current global model. The model parameters are... At the time of response acquisition location, the input is... The pooling response is denoted as ,in It can be , , , or The response acquisition location can be a specified network layer, feature extraction location, or feature location before the classification head. The central server calculates the composite perturbation response residual:

[0050] in, The vector residual represents the deviation of the actual network response to the composite perturbation input from the reference response formed by the undisturbed input and the two single perturbation inputs. This formula is equivalent to extracting the interaction term of the two semantically preserving perturbations at the model response layer: if the model's response to two small perturbations is approximately superimposed, then this residual should fall within the historical normal residual baseline range; if a client update causes this residual to deviate significantly, it indicates that the update may have changed the model's sensitivity to local input patterns, triggering similar patterns, or combinations of composite perturbations, thus indicating a backdoor risk. > At that time, the central server also used replace Calculate the residuals for the other order using the same formula.

[0051] To convert the vector residuals into screenable scalar detections, the central server employs a fixed-response scalar projection function. calculate:

[0052] in, This is a projection of the residual scalar. Projection function. It can be a fixed random projection, a fixed channel weighted sum, a response norm, a response mean, or a combination of the above. The projection rule is determined by the central server and is used to observe residual anomalies from different response directions.

[0053] Subsequently, the central server performs backpropagation calculations at the current global model:

[0054] in, for Relative to the current global model parameters The gradient of the residual parameters. The technical function of this gradient is to identify which parameter directions increase or decrease the composite perturbation response residual; it is only used to estimate the functional impact of client updates on the residual, not as an input gradient, not used to generate triggers, and not used as an optimization gradient for training model parameters.

[0055] S5. Calculate the estimated impact of updating all clients: For the Shared model update for each client The central server combines it with the gradient of the residual parameters. By performing the inner product, we can obtain the estimated influence. :

[0056] in, For the first The estimated impact of each client update on the projection of the residual scalar. This represents the inner product of parameters at the corresponding parameter positions. In actual implementation, the inner product can be calculated in blocks according to network layers or parameter tensors and then added together.

[0057] This estimated impact is derived from a first-order approximation of the current global model, and therefore is not a final anomaly determination, but rather a low-cost screening measure. Its advantage is that the central server only needs to perform one or a small amount of residual backpropagation on the current global model to estimate the functional impact of all client updates; if a client update has a large projection along the gradient-sensitive direction of the residual parameters, then the update is expected to significantly alter the composite perturbation response residual and should be given higher review priority.

[0058] S6. Filter the set of clients to be reviewed: The central server filters clients to be reviewed based on the estimated impact, historical normal residual benchmarks, and the upper bound of the estimated error. For the first... For each client, the central server first uses an estimated residual scalar to represent the estimated value of the residual scalar projection after the client's update:

[0059] in, This is the residual scalar projection of the current global model. For the first Estimated impact of client updates.

[0060] The current round of screening uses the upper bound of the estimated error obtained from the previous round or the previous calibration cycle. This indicates that, under the same detection combination and similar update amplitude, the first-order estimated influence may deviate from the conservative upper bound of the actual residual change. The central server employs the following screening rules:

[0061] in, and From historical normal residual benchmark , The central value of the normal residual. This is the normal residual fluctuation scale; To prevent positive numbers with a denominator of zero; This is an additional risk item used to indicate non-residual signals such as first-time participation, inability to continuously associate identity, long period without review, abnormal update amplitude, or unreproducible inference state; To conservatively screen for risks. If Exceeding the screening threshold ,or Exceeding the uncertainty threshold Then the central server will add the corresponding client to the client set to be reviewed.

[0062] The set of clients to be reviewed includes at least one of the following: clients that meet the above screening rules; clients whose estimation error upper bound is too large, leading to uncertainty in judgment; clients participating for the first time, re-registering, or whose identities cannot be continuously associated and require normal aggregation weights; clients that meet the conditions for rotation review; and randomly selected clients by the central server. When the number of clients to be reviewed exceeds the review calculation budget for this round of candidate models, the central server will proceed according to... , The client intends to use aggregate weight, identity status, and random sampling requirements to determine the review priority.

[0063] S7. Perform candidate model verification and calibrate the upper bound of the estimation error: For the first batch of clients entering the set of clients awaiting review For each client, the central server applies the shared model update of that client to the current global model, resulting in candidate models:

[0064] in, For the first The candidate model parameters are for each client. This step is not retraining and does not require local client data.

[0065] The central server sends the same set of inputs from step S3 into the candidate model and calculates the actual residual change:

[0066] in, For the first The actual residual change of each client candidate model relative to the current global model. The composite perturbation response residual is calculated for the candidate model under the same detection probe, the same perturbation combination, and the same response acquisition location.

[0067] The central server further calculates the actual residual risk:

[0068] in, This indicates the degree to which the candidate model's residual scalar deviates from the historical normal residual baseline after the client update is applied. The judgment logic for candidate model review is as follows: if... If the client update passes the review, then the update is approved. The client update is then marked as high risk; if or If so, the client update is marked as an uncertain risk. This is the normal threshold. This is a high-risk threshold.

[0069] For samples that pass the candidate model review or are determined to be non-abnormal, the central server calculates the estimation error:

[0070] in, Indicates the first Actual residual change for each client Compared with the estimated impact The differences between them. The central server will Samples are categorized into normal error samples based on the detection combination and updated amplitude range, and the highest quantile is used to form the upper bound of the estimated error for subsequent rounds. Samples that fail the candidate model verification or are judged as abnormal are not included in the normal error sample set, nor are they used to update the historical normal residual benchmark. When the number of samples is insufficient or the updated amplitude exceeds the historical calibration range, a preset conservative error upper bound, an adjacent range error upper bound, or a global error upper bound is used, and the proportion of subsequent verifications is increased.

[0071] S8. Perform candidate global model checks and aggregation control: In step S8, the central server determines the aggregation processing method in the order of "candidate model verification results, pre-release global check, and contribution budget constraints", instead of using fuzzy high-low judgment.

[0072] The central server updates the collection based on the clients to be adopted. Forming candidate global models:

[0073] in, This is a candidate global model to be released. For the set of clients to be adopted, For the first Each client aggregates weights in this round. The central server performs the same composite perturbation response residual check on the candidate global models and calculates the release risk:

[0074] in, This represents the degree to which the residual scalar of the candidate global model deviates from the historical normal residual baseline.

[0075] The central server processes client updates according to the following rules, where, This is the normal residual risk threshold. The high-risk threshold Threshold for publishing candidate global models: If the client update passes the candidate model review, it satisfies... And the candidate global model satisfies Then it will participate in the aggregation normally; like If the client participates for the first time, its identity cannot be continuously associated, or the review interval exceeds the preset period but does not trigger high-risk conditions, its participation in aggregation will be downgraded or delayed. like If so, the corresponding client is isolated from updating and the high-risk detection result is output; like If so, the release of the current aggregation model will be suspended, the set of clients to be reviewed will be expanded, and aggregation will be re-aggregated, the aggregation step size will be reduced, or the model will be rolled back.

[0076] For clients participating for the first time, whose identities cannot be continuously linked, who have not completed candidate model review, or whose review interval exceeds a preset period, the central server will limit their cumulative valid contributions:

[0077] in, For the first The cumulative effective contribution of each client during the period when it was not fully reviewed. This refers to the aggregate weight actually used by the client in this round. The norm is the amplitude of the client update. If Reaching or exceeding the budget limit The central server reduces the client's aggregation weight, delays its participation in aggregation, or adds it to the client set awaiting review. Only samples that pass the candidate model review, have low risk, actually participate in normal aggregation, and whose candidate global model check in their round did not trigger anomalies are used to update the historical normal residual benchmark and the upper bound of normal estimation error.

[0078] Example: Federated Training for Multi-Hospital Medical Image Classification In a multi-hospital medical image classification system, 12 hospitals jointly train a chest X-ray image classification model. Each hospital retains its local image data and does not upload raw images to the central server. The central server can obtain the shared model updates and hospital client identifiers submitted by each hospital before formal aggregation, but cannot obtain the hospitals' local training data. The detection objective of this embodiment is to locate high-risk hospital updates that may cause abnormal residuals in composite perturbation responses, and based on this, decide whether to aggregate them normally, downweight them, delay them, or isolate them.

[0079] S1. Establish detection configuration. The central server establishes a probe-based sample pool, which includes publicly available validation chest X-rays, a small number of authorized and desensitized validation images, and rule-generated low-risk synthetic chest X-ray samples. The semantically preserving perturbation operator library includes minor brightness adjustments, minor contrast adjustments, minor blurring, and minor scale changes. For chest X-ray classification tasks, the perturbation intensity is limited to not changing the lesion category judgment, and the output difference of the frozen reference model or the previous round of the global model is required to not exceed a preset range.

[0080] The central server selects the penultimate feature extraction layer and the layer before the classification head as the response acquisition location, configuring response pooling rules and fixed scalar projection rules. The candidate model review calculation budget is set to review a maximum of 4 hospitals per round. Hospitals joining for the first time or with insufficient identity records enter a cold start state, with aggregated weights not exceeding 30% of the normal weights. Historical normal residual baselines are maintained separately according to model version window, chest X-ray probe type, brightness / contrast perturbation combination, response layer, and scalar projection component.

[0081] S2. Receive updates from hospitals and form a set to be tested. In the t-th round of training, the central server distributes the current global model parameters to the 12 hospitals. .Hospital After training on local chest X-ray data, upload the shared model for updates. ,in This indicates the hospital client ID. The central server generates a record for each hospital to be tested, which includes the hospital client identifier, the current global model version, and shared model updates. .

[0082] S3. Construct the input for this round of detection. After obtaining the shared model update from 12 hospitals, the central server extracts probe chest X-rays from the probe base sample pool. The class preservation condition is verified using a frozen reference model and task rule constraints. If a candidate probe causes an excessive drop in the confidence of the reference model after perturbation, the central server reduces the perturbation intensity or replaces the probe.

[0083] This round selects the perturbation operator. For slight brightness adjustment, the perturbation operator Slight contrast adjustment. For probe chest X-rays. Central server structure:

[0084] in, To perturb the chest X-ray for brightness, To perturb the contrast of the chest X-ray, A composite perturbation chest radiograph with brightness first and then contrast. This is a composite perturbation chest radiograph with contrast first and brightness second. The central server calculates:

[0085] like ≤ This indicates that the order in which the brightness and contrast perturbations are applied has little impact on the chest X-ray input, and can be used... As a composite disturbance input; if > Then retain both and Residuals under two orders. This embodiment assumes... ≤ Therefore, adopt Perform subsequent calculations. The hospital client was unaware of the current round of testing probes before uploading this update. Disturbance combination and The response layer and projection rules make it difficult to pre-adapt to this specific detection combination.

[0086] S4, such as Figure 2 As shown, the residuals and residual parameter gradients of the combined perturbation response are calculated. The central server will display the unperturbed chest X-ray. Brightness disturbance chest X-ray Contrast-perturbed chest X-ray and composite perturbation chest X-ray Input the current global model Pooling responses are collected in the penultimate feature extraction layer. And calculate:

[0087] This residual indicates whether the actual intermediate layer response of the composite perturbation chest radiograph deviates from the reference relationship formed by the single perturbation response of luminance, the single perturbation response of contrast, and the unperturbed response. The central server performs a fixed scalar projection on this residual to obtain:

[0088] The gradient of the residual parameters is then obtained through backpropagation:

[0089] This gradient is used only for risk screening and not for training or updating model parameters.

[0090] S5. Perform a full estimate of the impact of updates for 12 hospitals. The central server reads the shared model updates from each hospital. The update will be based on the gradient of the residual parameters. By performing the inner product, we obtain the estimated influence:

[0091] For example, if the updates for Hospital 3 are mainly concentrated on the feature extraction layers with large residual parameter gradients, then... The update for Hospital 8 is relatively large; the updates are mainly distributed in the parameter region that is not sensitive to the current residual. The size is relatively small. This step does not require constructing candidate models for all 12 hospitals, nor does it require hospitals to upload any test responses.

[0092] S6. Determine the set of hospitals to be reviewed. The central server calculates the updated estimated residual scalar for each hospital:

[0093] Combined with historical normal residual benchmark Upper bound of estimation error Risks associated with identity Calculate the risk of conservative screening:

[0094] like ≥ or ≥ Then the hospital The hospital is added to the list of hospitals awaiting review. Assuming that hospitals 3 and 7 meet the above conditions in this round, hospital 5 is a newly added hospital, and hospital 10, although... Not exceeding However, the conditions for rotation review have been met. Since the budget for this round of candidate model review is updated for 4 hospitals, the central server adds Hospital 3, Hospital 7, Hospital 5, and Hospital 10 to the set of hospitals to be reviewed.

[0095] S7. Perform candidate model verification and calibrate the upper bound of the estimation error. For Hospital 3, the central server applies its shared model update to the current global model to obtain the candidate model:

[0096] The central server will send the same set of chest X-rays input from step S3. Calculate the actual change in residual:

[0097] And calculate the actual residual risk:

[0098] like If Hospital 3 is updated, it is marked as high-risk. The central server performs the same steps for Hospitals 7, 5, and 10. If Hospital 10 passes the review, the estimation error is calculated:

[0099] The error sample is added to the normal error sample set according to the detection combination and the updated amplitude range; if Hospital 3 is judged to be abnormal, the error of Hospital 3 will not be included in the normal error sample set, nor will the historical normal residual benchmark be updated.

[0100] S8, such as Figure 3 As shown, aggregation control and candidate global model checks are performed. The central server performs aggregation control based on the review results. If Hospital 3 meets the requirements... If so, isolate its update and output Hospital 3, brightness / contrast detection combination, abnormal residual projection component, and high-risk level in the detection results. If Hospital 7 meets the requirements... If Hospital 7's participation in aggregation is reduced in weight or delayed, then Hospital 5, being a new addition in a cold start state, can only participate in aggregation with a weight not exceeding 30% of its normal weight, even if it does not trigger high-risk conditions, until it passes the candidate model review and accumulates sufficient historical data. Hospital 10 meets the requirements. Then it will participate in the aggregation normally.

[0101] The central server updates the collection based on the hospitals to be adopted. Forming candidate global models:

[0102] The same composite perturbation response residual check is then performed on the candidate global model, and the following calculations are performed:

[0103] like The central server will pause the release of the current aggregation model, expand the set of hospitals awaiting review, and re-aggregate; if If a candidate global model is selected, it will be published. Only hospital updates that pass the candidate model review, have low risk, and actually participate in normal aggregation will be used to update the historical normal residual benchmark and the upper bound of normal estimation error. Hospital 5 will not be directly written into the formal historical normal residual benchmark even if it is temporarily not abnormal during the observation period.

[0104] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A backdoor risk detection method for federated learning based on composite perturbation response residuals, applied to a federated learning system including a central server and multiple clients, characterized in that, Before the formal aggregation in this round, the central server receives shared model updates uploaded by each client based on the same current global model; the current global model mentioned below is the baseline model before aggregation that has not yet been incorporated into any client update in this round. The central server performs the following steps: Step 1: Establish the detection configuration; the detection configuration includes a probe base sample pool, a semantic preservation perturbation operator library, response acquisition location, response pooling rules, response scalar projection rules, historical normal residual benchmark, upper bound of estimation error, candidate model verification calculation budget, various screening, judgment and release thresholds, and aggregation processing rules; among them, semantic preservation perturbation operators refer to transformation methods that do not change the detection probe task label or main semantics; Step 2: Receive the shared model updates uploaded by each client in this round based on the current global model, and form a set of client updates to be detected: wherein each record in the set of client updates to be detected includes the client identifier, the current global model version, and the shared model update; Step 3: After forming the client update set to be detected, select or generate the current round detection probe, and select two semantic preservation perturbation operators from the semantic preservation perturbation operator library. Based on the same detection probe, construct an unperturbed input, two single perturbation inputs, and a composite perturbation input. Step 4: Input the unperturbed input, the two single perturbation inputs and the composite perturbation input into the current global model respectively, obtain the pooled response at the response acquisition location, calculate the deviation of the actual response of the composite perturbation input from the reference response formed by the unperturbed input and the two single perturbation inputs, and use it as the composite perturbation response residual of the current global model; Step 5: Convert the composite disturbance response residuals described in Step 4 into scalar detection quantities, and calculate the residual parameter gradient of the scalar detection quantities relative to the current global model parameters; based on the inner product of the residual parameter gradients and the shared model updates for each client, obtain the estimated impact of each client update on the composite disturbance response residuals, which is used to characterize the expected extent to which the corresponding client update increases, decreases, or deviates from the historical normal residual range of the composite disturbance response residuals; Step 6: Based on the estimated impact, historical normal residual benchmark, upper bound of estimation error, client identity status and candidate model review budget described in Step 5, filter the set of clients to be reviewed; Step 7: For each client to be reviewed identified in Step 6, copy the current global model and apply the shared model update only to that client to construct a single-client candidate model; input the same set of inputs from Step 3 into the candidate model, calculate the actual residual change and actual residual risk brought about by the client update; calibrate the upper bound of the estimation error based on the difference between the actual residual change and the estimated impact in Step 5, and obtain the backdoor risk review result of the single-client candidate model by comparing the actual residual risk with the risk threshold; where the backdoor risk review result is low risk, uncertain risk, or high risk. Step 8: Based on the single-client candidate model review results determined in Step 7, perform preliminary processing on client updates and form a set of clients to be adopted: low-risk updates are aggregated normally, updates with uncertain risks are aggregated with reduced weight, updates that have not completed review are aggregated with delayed weight, and updates that meet high-risk conditions are isolated; a candidate global model is formed based on the set of clients to be adopted and its residual release risk is calculated; when the residual release risk does not exceed the threshold, the candidate global model is released according to the aforementioned processing; when the residual release risk exceeds the threshold, release is suspended, and the review is expanded, re-aggregated, the aggregation step size is reduced, or a rollback is performed.

2. The detection method according to claim 1, characterized in that, In step 3, for the detection probe The central server selects two semantically preserving perturbation operators. and The central server constructs the following input: ； in, For unperturbed input, and For two single perturbation inputs, and The input is a composite perturbation with two different application orders; the central server further calculates the order difference. : ； in, The allowed input distance or reference feature distance for the task; when ≤ hour, To preset the order difference threshold, the central server adopts... Or adopt and The mean of the response is taken as the composite perturbation response; when > At that time, the central server retains respectively and The corresponding residuals do not require averaging the response to eliminate order differences.

3. The detection method according to claim 2, characterized in that, In step 4, the composite perturbation response residual is equivalent to extracting the interaction term of the two semantically preserving perturbations on the model response layer, and its abnormal deviation relative to the historical normal residual benchmark is used to indicate that the model’s response sensitivity to local feature combinations or composite perturbation combinations has changed abnormally. Let the current global model parameters be denoted as The model will input at the response acquisition location. The pooling response is denoted as ;in, express , , , or Any input in; when using When used as a composite disturbance input, the composite disturbance response residual for: ； in, This represents the deviation of the actual response to the composite perturbation from the reference response formed by the unperturbed input and the two single perturbation inputs. It is used to describe the anomaly in the interactive response of the current global model under the combined effect of two semantically preserving perturbations. > At that time, the central server used replace Using the same calculation formula, the residual of the composite disturbance response corresponding to the order in which another disturbance is applied is calculated.

4. The detection method according to claim 3, characterized in that, In step 5, the central server uses a fixed-response scalar projection function. vector residuals Convert to scalar detection quantity And calculate the gradient of the residual parameters. and the Estimated impact of client updates : ； in, Used for comparison with historical normal residual benchmarks; Scalar detection quantity Relative to the current global model parameters The gradient; For the first Updates to the shared model uploaded by each client; Update the first-order estimate of the impact of residual changes on the client. This represents the inner product of parameters at the corresponding model parameter positions; the estimated impact is used for full screening and is not directly used as the final anomaly conclusion.

5. The detection method according to claim 4, characterized in that, In step 6, the upper bound of the estimation error is determined based on the deviation between the estimated impact and the actual residual change in the historical low-risk review samples. The conservative screening risk is calculated and determined based on the estimated impact of each client, the historical normal residual benchmark, and the upper bound of the estimation error. Clients that are expected to cause the composite disturbance response residual to deviate from the historical normal range or whose estimation results are uncertain are screened. If one of the following conditions is met: the conservative screening risk reaches the screening threshold, the upper bound of the estimation error reaches the uncertainty threshold, or the client's identity status meets the mandatory review conditions specified by the server, then the client is included in the set to be reviewed. If the number of clients to be reviewed exceeds the review budget, the review priority is determined comprehensively based on the conservative screening risk, the upper bound of the estimation error, the proposed aggregation weight, and the identity status.

6. The detection method according to claim 5, characterized in that, In step 6, the central server performs a conservative screening based on the historical normal residual benchmark and the upper bound of the estimation error; the historical normal residual benchmark is denoted as... ,in The central value of the low-risk historical residual scalar sample. This represents the normal fluctuation scale of the low-risk historical residual scalar sample; For the Each client, the central server calculates conservative screening risks. : ； in, In order to be with the first The upper bound of the estimation error is obtained by updating historical review samples with similar amplitudes for each client. Additional risk items introduced due to first-time participation, inability to continuously associate identities, prolonged lack of review, abnormal update amplitude, or unreproducible inference state. To prevent positive numbers with a denominator of zero; when Exceeding the screening threshold ,or Exceeding the uncertainty threshold At that time, the central server will add the corresponding client to the set of clients to be reviewed.

7. The detection method according to claim 6, characterized in that, In step 7, the central server selects the first client from the set of clients to be reviewed. Each client only applies the client-shared model update to construct candidate models. And calculate the actual residual change. and actual residual risk : ； ； ； in, The composite perturbation response residuals obtained by the candidate model under the same detection probe, the same perturbation combination, and the same response acquisition location; This represents the actual change in residuals caused by the client update. This indicates the degree to which the residual scalar deviates from the historical normal residual baseline after the application of this client for updating.

8. The detection method according to claim 7, characterized in that, In step 7, the central server calibrates the upper bound of the estimation error using client samples that have passed the candidate model review or have been determined to be non-abnormal; the estimation error samples... for: ； in, Indicates the first The difference between the actual residual change and the estimated impact for each client. Samples that fail the candidate model review or are deemed anomalous are not included in the set of normal samples at the upper bound of the estimation error, nor are they used to update the historical normal residual benchmark.

9. The detection method according to claim 8, characterized in that, In step 8, the central server first determines the set of clients to be adopted based on the review results of the single-client candidate model, forming a candidate global model. And calculate the release risk of candidate global models. , ； ； in, For the first Each client updates the actual aggregate weight after aggregation processing. This represents the degree to which the residual scalar of the candidate global model deviates from the historical normal residual baseline; The central server processes client updates according to the following rules, where, This is the normal residual risk threshold. The high-risk threshold Threshold for publishing candidate global models: If the client update passes the candidate model review, it satisfies... And the candidate global model satisfies Then it will participate in the aggregation normally; If the client update passes the candidate model review, it satisfies... If the client participates for the first time, its identity cannot be continuously associated, or the review interval exceeds the preset period but does not trigger high-risk conditions, its participation in aggregation will be downgraded or delayed. like If so, the corresponding client is isolated from updating and the high-risk detection result is output; like If so, the release of the current aggregation model will be suspended, the set of clients to be reviewed will be expanded, and aggregation will be re-aggregated, the aggregation step size will be reduced, or the model will be rolled back.

10. A federated learning backdoor risk detection system implementing the detection method of any one of claims 1 to 9, applied to a federated learning system including a central server and multiple clients, characterized in that, The central server includes: The detection configuration module is used to execute step 1 of the detection method; The client updates the receiving module to execute step 2 of the detection method; The input detection module is used to execute step 3 of the detection method. The composite disturbance response residual calculation module is used to perform step 4 of the detection method; The impact estimation module is used to execute step 5 of the detection method. The review and screening module is used to execute step 6 of the detection method; The candidate model verification module is used to perform step 7 of the detection method; The aggregation processing and release inspection module is used to execute step 8 of the detection method.

Images