Defense method and system for perceptual poisoning

Abstract

The invention discloses a defense method for perceptual poisoning, which is used for coping with perceptual poisoning attacks in federal learning target detection tasks by introducing suspicious client detection based on cosine similarity, self-adaptive learning rate adjustment and a dynamic defense interval mechanism, and realizing robustness training and stable defense of a system. The method can solve the technical problems that an existing defense method based on statistical anomaly detection depends on single-round gradient statistical characteristics, it is difficult to keep stable performance under the non-IID data condition, and the method is prone to being affected by noise fluctuation, so that the misjudgment rate is high. And the technical problems that the existing defense method based on trust scoring and weighted aggregation generally adopts a fixed learning rate or a linear attenuation strategy in a trust degree updating process, cannot flexibly adapt to a dynamically changing attack mode, easily causes defense response delay and causes insufficient system robustness are solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of federated learning and visual perception, and more particularly relates to a defense method and system for perception poisoning. BACKGROUND

[0002] With the increasing demand for data privacy protection and the popularity of distributed intelligent devices, federated learning has gradually become an important paradigm for training deep neural networks in a distributed data environment. Federated learning allows multiple clients to retain private data locally and only share model update parameters, thereby obtaining a high-performance global model while protecting data privacy. In recent years, federated learning has been widely used in sensitive data fields such as medical diagnosis, financial prediction, and intelligent transportation. However, as its application scope expands, potential security threats have gradually emerged, especially in complex visual perception tasks, where models are vulnerable to attacks by malicious clients. Among them, perception poisoning attacks are a new form of threat, in which attackers tamper with class labels, bounding box annotations, or target existence information in training data to cause the global model to produce false perception on specific targets.

[0003] There are three main existing defense methods against perception poisoning: the first method is a defense method based on statistical anomaly detection, which analyzes the statistical characteristics (such as mean, variance, cosine similarity, etc.) of the client gradient upload gradient or model update parameters to identify suspicious clients that deviate from the overall distribution pattern; the second method is a defense method based on trust scoring and weighted aggregation, which assigns a trust weight to each client and dynamically adjusts the aggregation weight based on its historical performance or the stability of the uploaded parameters to reduce the impact of abnormal updates; the third method is a defense method based on model consistency verification, which introduces an additional verification mechanism, such as using auxiliary data sets, global verification models, or cross-validation strategies, to compare the prediction results of the client uploaded model on public samples or verification tasks for consistency, thereby identifying potential malicious models.

[0004] However, the above-mentioned several existing perception poisoning defense methods have some non-negligible defects: First, the defense method based on statistical anomaly detection relies on the statistical characteristics of single-round gradients, which is difficult to maintain stable performance under non-independent and identically distributed (Non-Independent and Identically Distributed, abbreviated as non-IID) data conditions, and is easily affected by noise fluctuations, resulting in a high false positive rate; Second, the defense method based on trust scoring and weighted aggregation usually uses a fixed learning rate or a linear decay strategy in the trust update process, which cannot adapt to dynamic changes in attack patterns, causing defense reflection lag and insufficient system robustness; Thirdly, the defense method based on model consistency verification needs to introduce additional verification data or external models, which significantly increases the computing and communication costs, and has data security and implementability problems in scenarios involving privacy protection. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a defense method and system for perception poisoning. The purpose is to solve the technical problems that the existing defense method based on statistical anomaly detection relies on the statistical characteristics of single round gradient, is difficult to maintain stable performance under non-IID data conditions, and is easily affected by noise fluctuations, resulting in a high misjudgment rate, and the existing defense method based on trust score and weighted aggregation usually adopts a fixed learning rate or a linear decay strategy in the process of trust degree updating, which cannot flexibly adapt to dynamically changing attack modes, easily causes defense reflection lag, and leads to insufficient system robustness, and the existing defense method based on model consistency verification needs to introduce additional verification data or external models, which significantly increases the computing and communication costs, and has data security and implementability problems in scenarios involving privacy protection.

[0006] To achieve the above purpose, according to one aspect of the present application, a defense method for perception poisoning is provided, which is applied in a federated learning system including a plurality of clients and a server, the defense method comprising the following steps: (1) initializing global configuration parameters and state variables by the server to obtain initialized global configuration parameters and initialized state variables; (2) the server obtains a first data set and a server model, pre-processes the first data set to obtain a pre-processed first data set , expands the number of classifications of the obtained server model to obtain an expanded server model , and sends the pre-processed first data set and the expanded server model to the client; (3) the client obtains a second data set different from the first data set and a client model from the server, divides the second data set into a training set and a test set according to a ratio of 8:2, pre-processes the second data set, i.e. adjusts the image size of each sample in the data set to a size matching the input requirements of the client model, to obtain a pre-processed data set , expands the number of classifications of the client model from L to L+K to obtain an expanded client model, and uniformly mixes the pre-processed first data set and the training set in the second data set to obtain a mixed training set , and the extended server model according to step (2) , the parameter weights of the extended client model are initialized according to the parameter weights of the parameter of step (2) to obtain an initialized client model ; (4) The client uses the initialized client model in step (3) to perform local training on the mixed training set obtained in step (3) to obtain a client gradient set G composed of multiple client gradients , and sends the client gradient set G to the server, wherein m represents the number of clients participating in the current round of training, and the range is , n is the total number of clients in the federated learning system, represents the client gradient of the i-th client among all clients participating in the current round of local training, and , and ∈[1,m]; (5) The server iteratively trains the server model using the client gradient set G from the client to obtain the final server model.

[0007] Preferably, the global configuration parameters include the number of warm-up rounds, the dynamic cosine similarity adjustment coefficient, the static cosine similarity threshold, the smoothing coefficient, the multiplication penalty coefficient, the addition penalty coefficient, the learning rate benchmark, the defense interval, the upper limit constraint of the defense interval, and the defense interval coefficient, and the state variables include the initial trust score and the corresponding learning rate of each client, the historical gradient cache, and the global gradient reference vector. In step (1), the process of initializing the global configuration parameters and state variables is as follows: the number of warm-up rounds is initialized to 100, the dynamic cosine similarity adjustment coefficient is initialized to 1.5, the static cosine similarity threshold is initialized to 0.8, the smoothing coefficient is initialized to 0.5, the multiplication penalty coefficient is initialized to 0.9, the addition penalty coefficient is initialized to 0.05, the learning rate benchmark is initialized to 0.01, the defense interval is initialized to 1, the upper limit constraint of the defense interval is initialized to 5, the defense interval coefficient is initialized to 1.2, the initial trust score of each client is initialized to 1.0, the corresponding learning rate is initialized to the learning rate benchmark, the historical gradient cache is initialized to an empty set, and the global gradient reference vector is initialized to a 0 vector.

[0008] Preferably, step (2) is specifically as follows: first, the server performs preprocessing operations on the obtained first data set , that is, adjusts the image size of each sample in the first data set to a size that matches the input requirements of the server model, so that the preprocessed data set Subsequently, the server expands the number of categories in the server-side model from L to L+K, that is, it adds K new categories to the server-side model, with the new category labels numbered sequentially as L, L+1, ..., L+K-1, to obtain the expanded server-side model. K is set to 1.

[0009] Preferably, step (5) includes the following sub-steps: (5-1) Server-side counter settings =1; (5-2) Server-side judgment If the number of training rounds is greater than the preset number of training rounds T, the process ends; otherwise, proceed to step (5-3). The preset number of training rounds T ranges from 100 to 500. (5-3) Server-side judgment Is it greater than the number of preheating wheels in the global configuration parameters after initialization in step (1)? If so, initialize the defense round counter r and set r= If yes, then proceed to step (5-4); otherwise, proceed to step (5-11). (5-4) Judgment ( Is -r equal to the defense interval? If yes, proceed to step (5-5); otherwise, proceed to step (5-11). (5-5) Set r= and the client gradient set G from the client Preprocessing is performed to obtain the preprocessed client gradient set. ; (5-6) Based on the preprocessed client gradient set obtained in step (5-5) Construct a global reference gradient vector for defense determination. (It is used to reflect the general trend of gradients from all clients involved in training), and based on this global reference gradient vector. Obtain the cosine similarity set SIM ; (5-7) Based on the cosine similarity set SIM obtained in step (5-6) Obtain the cosine similarity threshold ; (5-8) Based on the cosine similarity threshold obtained in step (5-7) and the cosine similarity set SIM obtained in steps (5-6) Get the set of suspicious counts ; (5-9) The set of suspicious occurrences obtained from step (5-8-5) obtain the learning rate of each client participating in the training, and all the learning rates of the clients constitute a learning rate set ; (5-10) obtain the number of warm-up rounds in the global configuration parameter initialized in step (1) obtain the poisoning defense interval ; (5-11) the server aggregates the preprocessed client gradient set obtained in step (5-5) and the learning rate set obtained in step (5-9-4) to obtain the latest server model , and broadcasts the latest server model to all the clients (5-12) set = + 1, and return to step (5-2).

[0010] Preferably, step (5-6) comprises the following sub-steps: (5-6-1) obtain the preprocessed client gradient set obtained in step (5-5) , and obtain the reference vector of the current round according to the client gradient set : ; wherein represents the value of the client gradient in the i-th dimension, ∈ [1, d], d represents the dimension of the gradient, represents the median of all elements in the set (5-6-2) calculate the global reference vector of the r-th defense round according to the reference vector of the current round obtained in step (5-6-1) : ; ; wherein is a smoothing coefficient set in step (1) and is used to balance the global reference vector of the previous defense round and the current round, represents the previous defense round (5-6-3) set the counter = 1, and initialize the cosine similarity set SIM to be empty (5-6-4) judge ​whether it is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; (5-6-5) the preprocessed client gradient set obtained from step (5-5) obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; obtained from step (5-6-2) is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-6-5), otherwise the process ends; ; ;

[0011] Preferably, step (5-7) comprises the following sub-steps: (5-7-1) calculate the dynamic cosine similarity threshold value according to the cosine similarity set SIM obtained from step (5-6): ; , and ε is the dynamic cosine similarity adjustment coefficient set in step (1); (5-7-2) obtain the cosine similarity threshold value according to the dynamic cosine similarity threshold value obtained from step (5-7-1): ; is the static cosine similarity threshold value set in step (1), used to prevent the cosine similarity threshold value from being too high, represents the minimum value of the elements.

[0012] Preferably, step (5-8) comprises the following sub-steps: (5-8-1) set the counter = 1, and initialize the suspicious number set S to empty; (5-8-2) judge whether it is less than or equal to the number m of clients participating in the current round of training, if yes, go to step (5-8-3), otherwise go to step (5-8-5); (5-8-3) from the cosine similarity set SIM​​​​​ Obtain the first [number] participants in the current training round. Cosine similarity of each client and judge Is it less than or equal to the cosine similarity threshold? If yes, proceed to step (5-8-4); otherwise, proceed to step (5-9-4). (5-8-4) Settings = +1; (5-8-5) will Add to suspicious count set middle; (5-8-6) Settings = +1, and return to step (3-7-2).

[0013] Preferably, step (5-9) includes the following sub-steps: (5-9-1) Set the counter =1, and initialize the learning rate set. Empty; (5-9-2) Judgment Is it less than or equal to the number of clients m participating in the current training round, and If the value is greater than 0, proceed to step (5-9-3); otherwise, proceed to step (5-9-4). (5-9-3) From the cosine similarity set SIM Obtain the number of participants in the current training round. Cosine similarity of each client And based on the cosine similarity Get the first The trust score of each client in the r-th defense round : ; in, It is the multiplication penalty coefficient set in step (1), used to control the scaling amplitude. It is the addition penalty coefficient set in step (1), used to control additional deductions; (5-9-4) Based on step (5-9-3), the first... The trust score of each client in the r-th defense round Get the first Learning rate per client and put it into the learning rate set. In the middle; among which the learning rate The calculation formula is: ; wherein, is the learning rate baseline set in step (1); (5-9-5) setting = +1, and returning to step (5-9-2).

[0014] Preferably, the calculation formula of the poisoning defense interval in step (5-10) is: ; wherein, is the upper limit constraint of the defense interval set in step (1), is the defense interval coefficient set in step (1), used to control the dynamic increase of the defense interval, denotes the floor operation; The aggregation formula used in step (5-11) is as follows: .

[0015] According to another aspect of the present application, a defense system against poisoning in perception is provided, which is applied in a federated learning system including a plurality of clients and a server. The defense system includes the following modules: A first module is arranged on the server, used for initializing global configuration parameters and state variables, to obtain initialized global configuration parameters and initialized state variables; A second module is arranged on the server, used for obtaining a first data set and a server model, pre-processing the first data set to obtain a pre-processed first data set , expanding the number of classifications of the obtained server model to obtain an expanded server model , and sending the pre-processed first data set and the expanded server model to the client; A third module is arranged on the client, used for obtaining a second data set different from the first data set and a client model from the server, dividing the second data set into a training set and a test set according to a ratio of 8:2, pre-processing the second data set, i.e. adjusting the image size of each sample in the data set to a size matching the input requirements of the client model, to obtain a pre-processed data set , expanding the number of classifications of the client model from L to L+K to obtain an expanded client model, and uniformly mixing the pre-processed first data set and the training set in the second data set to obtain a mixed training set And based on the server-side model expanded from the second module The parameter weights are initialized for the extended client model to obtain the initialized client model. ; The fourth module, located on the client side, is used to initialize the client model using the third module. The mixed training set obtained in the third module Local training is performed on the client to obtain a client gradient set G consisting of multiple client gradients. The client gradient set G is then sent to the server, where m represents the number of clients participating in the current training round, and its range is... , where n is the total number of clients in the federated learning system. This indicates the number of clients participating in the current round of local training. The client gradient of each client, and has ∈[1, m]; The fifth module, located on the server side, is used to iteratively train the server-side model using the client gradient set G from the client to obtain the final server-side model.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: (1) Since the present invention adopts steps (5-5) to (5-6), it can effectively reduce the interference of outliers on the global gradient trend through the standardization preprocessing and median reference vector construction method, so that the global reference gradient can accurately reflect the overall update trend of most normal clients. Therefore, it can solve the technical problem that the existing defense method based on statistical anomaly detection relies on the statistical characteristics of single-round gradient, is difficult to maintain stable performance under non-IID data conditions, and is easily affected by noise fluctuations, resulting in a high misjudgment rate. (2) Since the present invention adopts steps (5-8) to (5-9), it uses an adaptive adjustment mechanism of trust score and learning rate to make the client's trust score and learning rate change dynamically according to its historical performance, thereby suppressing the influence of abnormal clients on the aggregation result. Therefore, it can solve the technical problem that the existing defense methods based on trust score and weighted aggregation usually adopt a fixed learning rate or linear decay strategy in the process of trust update, which cannot flexibly adapt to the dynamically changing attack mode, and is prone to causing the defense response to be slow, resulting in insufficient system robustness. (3) The present application adopts step (5-10), which significantly reduces the calculation and communication overhead while ensuring security through a dynamic defense interval adjustment mechanism, thereby solving the technical problem of the defense method based on model consistency verification that significantly increases the calculation and communication cost due to the need to introduce additional verification data or external models; (4) The present application adopts steps (5-6-2) to (5-7), which adopts a dynamic cosine similarity threshold updating mechanism, and through the joint action of the smoothing coefficient and the dynamic adjustment parameter, the defense judgment threshold can adapt to the gradient distribution change of different rounds, thereby solving the data security and implementability problems existing in the existing defense method based on model consistency verification in the scene involving privacy protection. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of the defense method against perception poisoning of the present application. DETAILED DESCRIPTION

[0018] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0019] The basic idea of the present application is to provide a defense method against perception poisoning, which introduces suspicious client detection based on cosine similarity, adaptive learning rate adjustment and dynamic defense interval mechanism to cope with perception poisoning attacks in federated learning target detection tasks, realizes robust training and stable defense of the system, As Figure 1 shown, the present application provides a defense method against perception poisoning, which is applied in a federated learning system including multiple clients and a server, the defense method comprising the following steps: (1) The server initializes the global configuration parameters and state variables to obtain the initialized global configuration parameters and the initialized state variables; Specifically, the global configuration parameters include the warm-up round number, the dynamic cosine similarity adjustment coefficient, the static cosine similarity threshold, the smoothing coefficient, the multiplication penalty coefficient, the addition penalty coefficient, the learning rate benchmark, the defense interval, the upper limit constraint of the defense interval and the defense interval coefficient, and the state variables include the initial trust score and the corresponding learning rate of each client, the historical gradient cache, and the global gradient reference vector. In this step, the process of initializing global configuration parameters and state variables is specifically initializing the preheating wheel number to 100, initializing the dynamic cosine similarity adjustment coefficient to 1.5, initializing the static cosine similarity threshold to 0.8, initializing the smoothing coefficient to 0.5, initializing the multiplication penalty coefficient to 0.9, initializing the addition penalty coefficient to 0.05, initializing the learning rate benchmark to 0.01, initializing the defense interval to 1, initializing the upper limit constraint of the defense interval to 5, initializing the defense interval coefficient to 1.2, initializing the initial trust score of each client to 1.0, initializing the corresponding learning rate to the learning rate benchmark, initializing the history gradient cache to an empty set, and initializing the global gradient reference vector to a 0 vector.

[0020] It should be noted that the specific initial values of the above global configuration parameters and state variables can be appropriately adjusted according to specific application scenarios, model sizes, and system resource constraints.

[0021] (2) The server obtains a first data set (where the label numbers of the sample categories are 0, 1, 2,..., L-1, and L is the total number of categories of samples in the first data set) and a server model (whose initial classification number is also L, and the label numbers of the classification categories are also 0, 1, 2,..., L-1), and pre-processes the first data set to obtain a pre-processed first data set , expands the classification number of the obtained server model to obtain an expanded server model , and sends the pre-processed first data set and the expanded server model to the client. This step specifically includes the following steps. First, the server pre-processes the obtained first data set , that is, adjusts the image size of each sample in the first data set to a size that matches the input requirements of the server model, so that the pre-processed data set (which can be directly used for model training) is obtained. Subsequently, the server expands the classification number of the server model from L to L+K (that is, adds K classification categories to the server model, and the label numbers of the added classification categories are L, L+1,..., L+K-1 in turn), to obtain an expanded server model (In the present application, K is set to 1, that is, only one new category is expanded, which can effectively detect malicious client models).

[0022] In order to enable the expanded model to have the ability to identify new categories, the server can re-label part of the samples in the pre-processed data set so that they belong to the new category.

[0023] The server model in this step is a deep learning model, such as CNN, ResNet, etc.

[0024] (3) The client obtains a second data set completely different from the first data set from the server and the client model (whose initial classification number is also L, and the label number of the classification category corresponds to 0, 1, 2, …, L-1), divides the second data set into a training set and a test set in a ratio of 8:2, and pre-processes the second data set, that is, adjusts the image size of each sample in the data set to a size matching the input requirements of the client model, to obtain a pre-processed data set , expands the classification number of the client model from L to L+K (the label number of the newly added classification category is L, L+1, …, L+K-1 in turn), to obtain an expanded client model, and uniformly mixes the pre-processed first data set and the training set of the second data set to obtain a mixed training set (which is used for subsequent local training), and initializes the parameter weights of the expanded client model according to the parameter weights of the expanded server model in step (2) to obtain an initialized client model .

[0025] The client model in this step is a deep learning model completely identical to the server model, such as CNN, ResNet, etc.

[0026] (4) The client uses the initialized client model in step (3) to perform local training on the mixed training set obtained in step (3) to obtain a client gradient set G composed of multiple client gradients , and sends the client gradient set G to the server, where m represents the number of clients participating in the current round of training, and its range is (n is the total number of clients in the federated learning system), and is preferably n (corresponding to the case where all clients participate), represents the client gradient of the th client among all clients participating in the current round of local training, and there is ∈[1,m]。

[0027] (5) The server iteratively trains the server model using the client gradient set G from the client to obtain the final server model; Specifically, step (5) includes the following sub-steps: (5-1) The server sets a counter =1; (5-2) Server judgment whether greater than the pre-set training rounds T, if yes, the process ends, otherwise go to step (5-3); Specifically, the pre-set training rounds T is in the range of 100 to 500, preferably 200, to balance the model performance and system resource consumption, and provide enough rounds for the defense mechanism to identify and suppress malicious clients.

[0028] (5-3) Server judgment whether greater than the pre-warm-up rounds in the global configuration parameters after initialization in step (1) , if yes, initialize the defense round counter r and set r= , then go to step (5-4), otherwise go to step (5-11).

[0029] The advantage of this sub-step (5-3) is that by setting the pre-warm-up rounds , the server only performs basic gradient aggregation and model updating in the early stage of federated learning training, so that the model reaches a certain accuracy and reduces the consumption of computing resources.

[0030] (5-4) Judgment whether r is equal to the defense interval , if yes, go to step (5-5), otherwise go to step (5-11); The advantage of this sub-step (5-4) is that the defense interval is used to control the execution frequency of the defense mechanism, which ensures security while avoiding excessive computation and communication overhead caused by executing the defense every round.

[0031] (5-5) Set r= (which indicates that the current round needs to execute the defense mechanism), and preprocess the client gradient set G from the client (including flattening and standardizing, so as to eliminate the influence of different layer parameter dimensions and scales on the detection result), to obtain the preprocessed client gradient set ; (5-6) According to the preprocessed client gradient set obtained in step (5-5), construct a global reference gradient vector for defense judgment (which is used to reflect the general trend of all participating training clients), and obtain a cosine similarity set SIM according to the global reference gradient vector ; Specifically, this step includes the following sub-steps: (5-6-1) Obtain the preprocessed client gradient set obtained in step (5-5) , and according to the client gradient set , obtain the reference vector of the current round ; Specifically, the calculation formula of the reference vector of the current round in this step is: ; Wherein, represents the value of the client gradient in the first dimension, ∈[1, d], d represents the dimension of the gradient, represents the median of all elements in the set.

[0032] The advantage of this sub-step (5-6-1) is that the median is selected to calculate the reference vector of the current round, which is not easily disturbed by extreme values, and can accurately reflect the feature trend of each dimension of the majority of normal client gradients, providing reliable data support for subsequent global reference vector construction.

[0033] (5-6-2) According to the reference vector of the current round obtained in step (5-6-1), calculate the global reference vector of the rth defense round ; Specifically, the calculation formula of the global reference vector of the rth defense round in this step is: ; Wherein, is the smoothing coefficient set in step (1), which is used to balance the global reference vector of the previous defense round and the current round, represents the previous defense round.

[0034] The advantage of this sub-step (5-6-2) is that the global reference vector of the current defense round can not only inherit historical information, but also adapt to current gradient changes, avoiding large deviation of the global reference vector caused by single round gradient fluctuation, thereby improving the stability and accuracy of subsequent cosine similarity calculation.

[0035] (5-6-3) Set the counter = 1, and initialize the cosine similarity set SIM to empty; (5-6-4) Determine whether is less than or equal to the number of clients participating in the current round of training m, if so, go to step (5-6-5), otherwise the process ends; (5-6-5) From the preprocessed client gradient set obtained in step (5-5)​ the rth client gradient participating in the current round of training is obtained , and the cosine similarity between the rth client gradient and the global reference vector of the rth defense round obtained in step (5-6-2) is calculated , and the cosine similarity is placed in the cosine similarity set ; Specifically, the calculation formula of the cosine similarity in this step is: ; ; (5-6-6) sets = = + 1, and returns to step (5-6-4); (5-7) obtains the cosine similarity threshold value from the cosine similarity set SIM obtained in step (5-6); ; Specifically, this step includes the following sub-steps: (5-7-1) calculates the dynamic cosine similarity threshold value from the cosine similarity set SIM obtained in step (5-6); Specifically, the calculation formula of the dynamic cosine similarity threshold value in this step is: ; wherein, , , and ε is the dynamic cosine similarity adjustment coefficient set in step (1).

[0036] (5-7-2) obtains the cosine similarity threshold value from the dynamic cosine similarity threshold value obtained in step (5-7-1); Specifically, the calculation formula of the cosine similarity threshold value in this step is: ; wherein is the static cosine similarity threshold value set in step (1), which is used to prevent the cosine similarity threshold value from being too high, represents the minimum value of the elements.

[0037] The advantages of the above sub-steps (5-7-1) to (5-7-2) are that the dynamic and static threshold values are combined, which can adaptively adjust the judgment standard according to the gradient fluctuation, thereby improving the sensitivity and robustness of the defense mechanism. ​

[0038] (5-8) Based on the cosine similarity threshold obtained in step (5-7) and the cosine similarity set SIM obtained in steps (5-6) Get the set of suspicious counts ; Specifically, this step includes the following sub-steps: (5-8-1) Set the counter =1, and initialize the suspicious count set S to be empty; (5-8-2) Judgment Is it less than or equal to the number of clients m participating in the current training round? If so, proceed to step (5-8-3); otherwise, proceed to step (5-8-5). (5-8-3) From the cosine similarity set SIM Obtain the first [number] participants in the current training round. Cosine similarity of each client and judge Is it less than or equal to the cosine similarity threshold? If yes, proceed to step (5-8-4); otherwise, proceed to step (5-9-4). (5-8-4) Settings = +1; (5-8-5) will Add to suspicious count set middle; (5-8-6) Settings = +1, and return to step (3-7-2); The advantage of the above sub-steps (5-8-1) to (5-8-6) is that by calculating the set of suspicious occurrences, the abnormal behavior of the client in multiple communication rounds can be quantified and accumulated, thereby effectively capturing the persistence and regularity of the client's abnormal behavior.

[0039] (5-9) The set of suspicious occurrences obtained from step (5-8-5) Obtain the learning rate for each client participating in training; the learning rates of all clients constitute a learning rate set. ; Specifically, this step includes the following sub-steps: (5-9-1) Set the counter =1, and initialize the learning rate set. Empty; (5-9-2) Judgment Is it less than or equal to the number of clients m participating in the current training round, and If the value is greater than 0, proceed to step (5-9-3); otherwise, proceed to step (5-9-4). (5-9-3) From the cosine similarity set SIM Obtain the first [number] participants in the current training round. Cosine similarity of each client And based on the cosine similarity Get the first The trust score of each client in the r-th defense round ; Specifically, the trust score in this step The calculation formula is: ; in, It is the multiplication penalty coefficient set in step (1), used to control the scaling amplitude. It is the addition penalty coefficient set in step (1), used to control additional deductions; (5-9-4) Based on step (5-9-3), the first... The trust score of each client in the r-th defense round Get the first Learning rate per client and put it into the learning rate set. middle; Specifically, the learning rate in this step The calculation formula is: ; in, The learning rate benchmark set in step (1).

[0040] (5-9-5) Settings = +1, and return to step (5-9-2); The advantage of the sub-steps (5-9-3) to (5-9-4) above lies in the fact that, through a dynamic calculation mechanism based on trust scores using cosine similarity, the client's learning rate can adaptively adjust with the trust score, thus forming a differentiated weight allocation strategy during the aggregation process. This mechanism can significantly suppress the influence of malicious clients in model updates while strengthening the contribution of highly trustworthy clients. It enables the system to maintain training stability and convergence performance even when facing non-IID data and malicious perturbations, ultimately balancing the security and robustness of global model updates.

[0041] (5-10) Based on the number of preheating wheels in the global configuration parameters after initialization in step (1) Get poison defense interval ; Specifically, the calculation formula of the poisoning defense interval in this step is as follows: wherein, is the upper limit constraint of the defense interval set in step (1), is the defense interval coefficient set in step (1) and used for controlling the dynamic increase of the defense interval, denotes the floor operation.

[0042] The advantage of this sub-step (5-10) is that, by introducing the adaptive dynamic updating mechanism of the defense interval, the defense execution frequency can be automatically adjusted according to the training phase and the system state, so as to avoid the waste of communication and computing resources caused by the excessive frequent execution of the defense mechanism, and also to prevent the security response delay caused by the too long defense interval.

[0043] (5-11) The server aggregates the preprocessed client gradient set obtained in step (5-5) and the learning rate set obtained in step (5-9-4) to obtain the latest server model , and broadcasts the latest server model to all clients; Specifically, the aggregation formula used in this step is as follows: (5-12) Set = +1, and return to step (5-2); Through the above steps, the present application can effectively identify and suppress the poisoning attack in the perception in the federated learning target detection scene, and realize the high robustness and dynamic scalability while ensuring the global model performance.

[0044] Those skilled in the art will readily understand that the above description is only the preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.​​​

Claims

1. A defense method for perception poisoning, applied in a federated learning system comprising a plurality of clients and a server, characterized in that, The defense method comprises the following steps: (1) the server initializes the global configuration parameters and state variables to obtain the initialized global configuration parameters and the initialized state variables; (2) The server obtains a first data set and a server model, and pre-processes the first data set to obtain a pre-processed first data set , expands the number of classifications of the obtained server model to obtain an expanded server model , and sends the pre-processed first data set and the expanded server model to the client. (3) The client obtains a second data set completely different from the first data set from the server And the client model divides the second data set into a training set and a test set in a ratio of 8:2, preprocesses the second data set, that is, adjusts the image size of each sample in the data set to a size matching the input requirements of the client model, to obtain a preprocessed data set , expands the number of classifications of the client model from L to L+K to obtain an expanded client model, and uniformly mixes the preprocessed first data set and the training set in the second data set to obtain a mixed training set , and initializes the parameter weights of the expanded client model according to the parameter weights of the expanded server model in step (2) to obtain an initialized client model ; (4) Client uses the client model after initialization in step (3). The mixed training set obtained in step (3) Local training is performed on the client to obtain a client gradient set G consisting of multiple client gradients. The client gradient set G is then sent to the server, where m represents the number of clients participating in the current training round, and its range is... , where n is the total number of clients in the federated learning system. This indicates the number of clients participating in the current round of local training. The client gradient of each client, and has ∈[1, m]; (5) the server uses the client gradient set G from the client to iteratively train the server model to obtain the final server model.

2. The defense method against perception poisoning according to claim 1, wherein The global configuration parameters comprise a preheating round number, a dynamic cosine similarity adjustment coefficient, a static cosine similarity threshold, a smoothing coefficient, a multiplication penalty coefficient, an addition penalty coefficient, a learning rate benchmark, a defense interval, a defense interval upper limit constraint, and a defense interval coefficient; and the state variables comprise initial trust scores and corresponding learning rates of the clients, a historical gradient cache, and a global gradient reference vector. In step (1), the process of initializing the global configuration parameters and the state variables is specifically: the preheating round number is initialized to 100, the dynamic cosine similarity adjustment coefficient is initialized to 1.5, the static cosine similarity threshold is initialized to 0.8, the smoothing coefficient is initialized to 0.5, the multiplication penalty coefficient is initialized to 0.9, the addition penalty coefficient is initialized to 0.05, the learning rate benchmark is initialized to 0.01, the defense interval is initialized to 1, the defense interval upper limit constraint is initialized to 5, the defense interval coefficient is initialized to 1.2, the initial trust scores of the clients are initialized to 1.0, the corresponding learning rates are initialized to the learning rate benchmark, the historical gradient cache is initialized to an empty set, and the global gradient reference vector is initialized to a 0 vector.

3. The defense method against perception poisoning according to claim 1 or 2, wherein, The step (2) specifically comprises the following steps. Firstly, the server performs a preprocessing operation on the obtained first data set, i.e., adjusting the image size of each sample in the first data set to a size matching the input requirement of the server model, so that the preprocessed data set Secondly, the server expands the classification number of the server model from L to L+K, i.e., adding K classification categories in the server model, and the label numbers of the added classification categories are L, L+1,..., L+K-1 in sequence, so as to obtain an expanded server model K is set to 1.

4. The defense method against perception poisoning according to any one of claims 1 to 3, wherein, Step (5) comprises the following sub-steps: (5-1) Server sets counter = 1; (5-2) Server judgment whether greater than the pre-set training rounds T, if yes, the process is ended, otherwise turn into step (5-3); wherein the pre-set training rounds T value range is 100 to 500; (5-3) Server judgment whether greater than the preheating round number in the global configuration parameter after initialization in step (1) if yes, initialize the defense round counter r and set r= then go to step (5-4), otherwise go to step (5-11); (5-4) judging whether or not r) is equal to the defense interval if so, proceeding to step (5-5), otherwise proceeding to step (5-11); (5-5) setting r = 1 , and preprocessing the client gradient set G from the client to obtain a preprocessed client gradient set ; (5-6) the pre-processed client gradient set obtained according to step (5-5) , a global reference gradient vector for defense decision is constructed (which is used to reflect the general trend of all the client gradients participating in training), and a cosine similarity set SIM is obtained according to the global reference gradient vector ;​ (5-7) A cosine similarity set SIM is obtained according to step (5-6) A cosine similarity threshold is obtained ; (5-8) Cosine similarity threshold value obtained according to step (5-7) and the cosine similarity set SIM obtained according to step (5-6) , obtain a suspicious number set ; (5-9) The suspicious number set obtained according to step (5-8-5) Obtain the learning rate of each client participating in training, and the learning rates of all clients constitute a learning rate set ; (5-10) the number of warm-up rounds in the global configuration parameters initialized according to step (1) acquiring a poisoning defense interval ; (5-11) The server aggregates the preprocessed client gradient set obtained in step (5-5) and the learning rate set obtained in step (5-9-4) to obtain a latest server model broadcasts the latest server model to all the clients;​​​ (5-12) Setting +1, and returns to step (5-2).​ 5. The defense method against perception poisoning according to claim 4, wherein, Step (5-6) comprises the following sub-steps: (5-6-1) Obtain the pre-processed client gradient set obtained in step (5-5) , and according to the client gradient set Obtain the reference vector of this round : ； wherein, denotes the client gradient In the first dimension, ∈ [1, d], d denotes the dimension of the gradient , denotes taking the median over all elements in the set; (5-6-2) The current reference vector according to step (5-6-1) , the global reference vector of the rth defense round is calculated : ； wherein, a smoothing coefficient set for step (1) for balancing the global reference vector of the previous defense round with the current round, denotes the previous defense round; (5-6-3) Setting a counter = 1, and initialize the cosine similarity set SIM to empty; (5-6-4) determining whether it is less than or equal to the number m of clients participating in the current round of training, and if so, proceeding to step (5-6-5), otherwise the process ends; (5-6-5) the pre-processed client gradient set obtained from step (5-5) the rth client gradient participating in the current round of training is obtained , and the cosine similarity between the rth client gradient and the global reference vector of the rth defense round obtained in step (5-6-2) is calculated , and the cosine similarity is put into the cosine similarity set ; wherein the calculation formula of the cosine similarity is: ； (5-6-6) setting +1, and return to step (5-6-4).​ 6. The defense method against perception poisoning according to claim 5, wherein, Step (5-7) comprises the following sub-steps: (5-7-1) The cosine similarity set SIM obtained according to step (5-6) Computing the dynamic cosine similarity threshold : ； wherein, , ε is the dynamic cosine similarity adjustment coefficient set in step (1); (5-7-2) The dynamic cosine similarity threshold value obtained according to step (5-7-1) , obtaining a cosine similarity threshold value : ； wherein is a static cosine similarity threshold value set in step (1) to prevent the cosine similarity threshold value from being too high, denotes taking the minimum value over the elements.

7. The defense method against perception poisoning, according to claim 6, wherein, Step (5-8) comprises the following sub-steps: (5-8-1) Setting a counter = 1, and initializes the suspicious number set S as empty; (5-8-2) judging whether it is less than or equal to the number m of clients participating in the current round of training, and if so, proceeding to step (5-8-3), otherwise proceeding to step (5-8-5); (5-8-3) From the cosine similarity set SIM the cosine similarity of the jth client participating in the current round of training is obtained whether it is less than or equal to the cosine similarity threshold value If yes, go to step (5-8-4), otherwise go to step (5-9-4);​​ (5-8-4) Setting +1;​ (5-8-5) to put into the suspicious number set collection (5-8-6) Setting +1, and return to step (3-7-2).​ 8. The defense method against perception poisoning, according to claim 7, wherein, Step (5-9) comprises the following sub-steps: (5-9-1) Setting a counter = 1, and initialize the learning rate set to empty; (5-9-2) judging whether it is less than or equal to the number m of clients participating in the current round of training, and whether it is greater than 0, and if so, proceeding to step (5-9-3), otherwise proceeding to step (5-9-4); (5-9-3) obtaining the cosine similarity of the rth client participating in the current round of training from the cosine similarity set SIM :​​​​​ ； wherein, is a multiplication penalty factor set in step (1) for controlling the scaling magnitude, is an addition penalty factor set in step (1) for controlling the additional deduction; (5-9-4) The trust score of the rth client in the rth defense round according to step (5-9-3) Obtain the learning rate of the rth client and put it into the learning rate set ; wherein the calculation formula of the learning rate is:​​ ； wherein, is the learning rate set in step (1); (5-9-5) Setting = +1, and returns to step (5-9-2).​ 9. The defense method against perception poisoning according to claim 8, wherein Poisoning defense interval in step (5-10) The calculation formula is: ； wherein is an upper limit of the defense interval set in step (1), is a coefficient of the defense interval set in step (1) for controlling the dynamic increase of the defense interval, denotes a floor operation; The aggregation formula used in step (5-11) is as follows: 。 10. A defense system against perception poisoning, applied in a federated learning system comprising a plurality of clients and a server, characterized in that, The defense system comprises the following modules: A first module, arranged on the server, is configured to initialize the global configuration parameters and the state variables to obtain the initialized global configuration parameters and the initialized state variables; A second module is arranged on the server, configured to acquire a first data set and a server model, and pre-process the first data set to obtain a pre-processed first data set , expand the number of categories of the acquired server model to obtain an expanded server model , and send the pre-processed first data set and the expanded server model to the client. A third module configured to obtain, from the server, a second data set that is completely different from the first data set and a client model, divide the second data set into a training set and a test set in a ratio of 8:2, preprocess the second data set, i.e., adjust the image size of each sample in the data set to a size that matches the input requirement of the client model, to obtain a preprocessed data set , expand the number of classifications of the client model from L to L+K to obtain an expanded client model, and uniformly mix the preprocessed first data set and the training set of the second data set to obtain a mixed training set , and initialize the parameter weights of the expanded client model according to the parameter weights of the expanded server model of the second module to obtain an initialized client model ; The fourth module, located on the client side, is used to initialize the client model using the third module. The mixed training set obtained in the third module Local training is performed on the client to obtain a client gradient set G consisting of multiple client gradients. The client gradient set G is then sent to the server, where m represents the number of clients participating in the current training round, and its range is... , where n is the total number of clients in the federated learning system. This indicates the number of clients participating in the current round of local training. The client gradient of each client, and has ∈[1, m]; A fifth module, arranged on the server, is configured to use the client gradient set G from the client to iteratively train the server model to obtain the final server model.

Images