A defense method, electronic device, and storage medium for multimodal poisoning attacks based on generative adversarial networks.
By constructing a multimodal poisoning attack defense method using generative adversarial networks, and combining image and text data poisoning processing with third-party parameter server verification, this method overcomes the limitations of single-modal datasets in existing technologies, achieves effective defense against multimodal poisoning attacks in federated learning, and improves the security of training data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2023-11-20
- Publication Date
- 2026-06-30
AI Technical Summary
Existing poisoning attack defense techniques are mainly designed for single-modality datasets and are ineffective in complex federated learning scenarios. Furthermore, existing techniques are not good at protecting the privacy of training data.
A multimodal poisoning attack defense method is constructed using generative adversarial networks. This method involves poisoning training data of images and text, verifying model discrepancies using a third-party parameter server, and employing Euclidean distance and PCA dimensionality reduction decomposition for defense to prevent poisoning attacks.
This improves the privacy protection level of federated learning, effectively defends against multimodal poisoning attacks, and enhances the security of training data.
Smart Images
Figure CN117494208B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence security technology, specifically relating to a defense method, electronic device and storage medium for multimodal poisoning attacks based on generative adversarial networks. Background Technology
[0002] Federated learning is a distributed machine learning framework for collaborative model training among multiple clients. It allows clients to store their sensitive data locally and share hyperparameter information only with a central server, thus protecting training data privacy. However, recent research has shown that even if clients share training information with the central server, they may still be vulnerable to hyperparameter leakage attacks, infringing on the privacy of their training data.
[0003] Poisoning attacks refer to attackers disrupting the federated learning process by controlling and manipulating portions of the training data or models. It is the most widely used and extensively studied attack in federated learning. Currently, poisoning attacks both domestically and internationally target single-modality datasets. However, in real-world federated learning scenarios, the local training data of each participant may contain multiple modalities, thus limiting existing technologies. Furthermore, existing attack and defense techniques are primarily discussed under specific assumptions, making it difficult to achieve satisfactory results in complex application scenarios. Summary of the Invention
[0004] The problem this invention aims to solve is to enhance the security of characteristic data of poisoning attacks during federated learning, and proposes a defense method, electronic device and storage medium for multimodal poisoning attacks based on generative adversarial networks.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A defense method against multimodal poisoning attacks based on generative adversarial networks includes the following steps:
[0007] S1. The attacker poisons the image training data to obtain poisoned image sample data;
[0008] S2. The attacker poisons the text training data to obtain poisoned text sample data;
[0009] S3. Construct a poisoning model. The attacker mixes the image poisoning sample data obtained in step S1 or the text poisoning sample data obtained in step S2 into the training dataset to obtain a poisoning training dataset for training the constructed poisoning model.
[0010] S4. Based on the poisoning model obtained in step S3, construct a defense method for multi-modal poisoning attacks, including establishing a third-party parameter server Server to verify the gap between the local model and the global model, measuring the gap between the local model and the global model using the Euclidean distance, and the central server performing data processing on the uploaded model parameters.
[0011] Further, in step S1, the attacker poisons the image training dataset it owns locally, randomly selecting some images of different categories for poisoning, including the following steps:
[0012] S1.1. The attacker first converts the image training data to be poisoned into a two-dimensional N×N pixel matrix, and modifies the pixel values of an m×m size as a pattern trigger, where m << N. The pattern trigger associates the features of the image with the label, introducing incorrect feature-label pairs. When the neural network is trained, the pattern trigger in the image will affect the convergence of the neural network training as a special feature;
[0013] S1.2. For the case where no pattern trigger is set, shuffle the image training data x i the corresponding label tag i , construct an incorrect mapping such that x i →tag i becomes x i →tag j , where i ≠ j, resulting in incorrect classification ability of the local neural network classifier.
[0014] Further, the specific implementation method of step S2 includes the following steps:
[0015] S2.1. The attacker first performs word segmentation on the text training data to be poisoned, and calculates the most important words in the text training data through the Jacobian matrix The calculation expression is:
[0016]
[0017] where, x i represents the i-th word after word segmentation of the text training data, N represents the total number of words in the text training data, represents the text classifier, and there are K categories in the classification results, and j is any one of K;
[0018] S2.2. Calculate the importance weight of the i-th word after word segmentation of the text training data The calculation expression is:
[0019]
[0020] Where y represents the i-th word x after word segmentation based on the text training data. i The class obtained from classification prediction;
[0021] S2.3. Sort the text words based on the importance weights calculated in step S2.2, and select the k most important words for poisoning. The poisoning methods include inserting a space into the word, deleting any number of characters in the word except the first and last characters, replacing O with 0 in the word, and replacing I with l in the word, or any combination of these methods.
[0022] Furthermore, the specific implementation method of step S3 includes the following steps:
[0023] S3.1. The attacker mixes the image poisoning sample data obtained in step S1 or the text poisoning sample data obtained in step S2 into the training dataset to obtain the poisoned training dataset.
[0024] S3.2. During the i-th round of training communication in federated learning, the global model parameters are downloaded to update the local model. The calculation expression is:
[0025]
[0026] Among them, Global i Represents the global model in the i-th round. Let represent the local model of the j-th participant in the i-th round of training. The scaling factor is updated locally;
[0027] Then, the updated local model is trained using the poisoned training dataset obtained in step S3.1, and the parameters of the trained local model are uploaded to the central server. The calculation expression is:
[0028]
[0029] Attack is the attacker's model;
[0030] S3.3. During the (i+1)th round of training communication in federated learning, the central server performs federated averaging based on the latest uploaded local training model parameters from the poisoned dataset obtained in step S3.2, utilizing global model convergence and fitting the local model with the global model. The calculation expression is as follows:
[0031]
[0032] S3.4. Calculate the parameters of the local poisoning model submitted by the attacker. The calculation expression is:
[0033]
[0034] Then the relationship between the m-th local model and the final global model is calculated as follows:
[0035]
[0036] Scale factor updated locally The attacker's poisoning parameters are amplified. During the global model aggregation phase, the poisoning parameters are retained, and the global model is eventually replaced by the local poisoning model.
[0037] Furthermore, the specific implementation method of step S4 includes the following steps:
[0038] S4.1. In federated learning training, a third-party parameter server (Server) is established to verify the difference between the local model and the global model. The Euclidean distance Dis(Locali,Global) is used to measure the difference, and the calculation expression is:
[0039]
[0040] S4.2. Before each round of federated training is initiated, a third-party parameter server calculates the difference Dis between the local model parameters of each participant in this task and the global model in turn, and sets a threshold ε to judge the difference: when the difference Dis exceeds the threshold ε, it is considered that the local training model parameters have a poisoning threat, and the local training model parameters in this round will no longer be used for federated aggregation.
[0041] S4.3. Adjust the threshold. If the number of participants exceeding or falling below the threshold ε in each training round exceeds... The threshold is then adjusted, and the updated threshold is used in the middle of this round. The formula for calculating the mean of the parameters and the updated threshold ε' is as follows:
[0042]
[0043] Where ζ is a preset scaling parameter;
[0044] S4.4. Considering the overhead of third-party parameter server verification, if no anomalies are found in k consecutive checks, the check frequency is halved; if anomalies are found, the check frequency is increased.
[0045] S4.5. Based on the verification in step S4.4, the central server performs PCA dimensionality reduction decomposition and then k-means clustering on the uploaded local model parameters to defend against multimodal poisoning attacks.
[0046] For each round of uploaded parameter information, a matrix M is formed by columnar arrangement. i×j ∈{R n ×R mTo normalize the mean of each row of data in the matrix, the expression is:
[0047]
[0048] Calculate the covariance matrix M cov The calculation expression is:
[0049]
[0050] Then extract M cov The eigenvalues and eigenvectors are used to form a matrix. The eigenvectors are arranged in rows from top to bottom according to the size of the corresponding eigenvalues. The first k rows are taken to form a matrix P, where P·M is the parameter information for dimensionality reduction to k dimensions. The parameter information after dimensionality reduction is then subjected to k-means clustering to obtain different parameter classes. The update corresponding to the largest class is regarded as a normal update, and the updates corresponding to other classes are regarded as malicious updates, thus completing the defense method against multimodal poisoning attacks.
[0051] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a defense method for multimodal poisoning attacks based on generative adversarial networks.
[0052] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for defending against multimodal poisoning attacks based on generative adversarial networks.
[0053] The beneficial effects of this invention are:
[0054] The present invention describes a defense method for multimodal poisoning attacks based on generative adversarial networks. This method improves upon the currently proposed single-modal poisoning attack methods and proposes a multimodal poisoning attack framework for text and image training data. This novel attack framework will further enhance research on federated learning poisoning attacks.
[0055] The present invention describes a defense method against multimodal poisoning attacks based on generative adversarial networks. It proposes multiple defense methods against poisoning attacks, thereby effectively improving the privacy protection level of federated learning in practical application scenarios. This method has practical significance and good application prospects. Attached Figure Description
[0056] Figure 1 This is a flowchart illustrating a defense method against multimodal poisoning attacks based on generative adversarial networks, as described in this invention. Detailed Implementation
[0057] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the specific embodiments described are only part of the embodiments of the present invention, rather than all of the specific embodiments. The components of the specific embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations, and the present invention can also have other embodiments.
[0058] Therefore, the detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents the selected specific embodiments of the present invention. All other specific embodiments obtained by those skilled in the art based on the specific embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.
[0059] To further understand the content, features and effects of the present invention, the following specific embodiments are exemplified and described in detail in conjunction with the attached Figure 1 as follows: Specific Embodiment 1:
[0061] A defense method against multi-modal poisoning attacks based on a generative adversarial network, comprising the following steps:
[0062] S1. The attacker conducts a poisoning attack on the image training data to obtain poisoned sample data of the image;
[0063] Further, in step S1, the attacker poisons the image training dataset it owns locally, randomly selects some images of different categories for poisoning processing, including the following steps:
[0064] S1.1. The attacker first converts the image training data to be poisoned into a two-dimensional N×N pixel matrix, and modifies the pixel values of the m×m size as a pattern trigger, where m << N. The pattern trigger associates the features of the image with the label, introducing an incorrect feature-label pair. When the neural network is trained, the pattern trigger in the image will affect the convergence of the neural network training as a special feature;
[0065] S1.2. In the case where no pattern trigger is set, shuffle the image training data x i corresponding label tag i , construct an incorrect mapping, so that x i →tag i becomes x i →tag j , i≠j, resulting in an error in the classification ability of the local neural network classifier;
[0066] S2. The attacker poisons the text training data to obtain poisoned text sample data;
[0067] Furthermore, the specific implementation method of step S2 includes the following steps:
[0068] S2.1. The attacker first performs word segmentation on the text training data to be poisoned, and then calculates the most important words in the text training data using the Jacobian matrix. The calculation expression is:
[0069]
[0070] Where, x i Let N represent the i-th word after word segmentation of the text training data, and N represent the total number of words in the text training data. This represents a text classifier, with K possible classification classes, where j is any one of the K classes.
[0071] S2.2. Calculate the importance weight of the i-th word after word segmentation of the text training data. The calculation expression is:
[0072]
[0073] Where y represents the i-th word x after word segmentation based on the text training data. i The class obtained from classification prediction;
[0074] S2.3. Sort the words in the text based on the importance weights calculated in step S2.2, and select the k most important words for poisoning. The poisoning methods include inserting a space into the word, deleting any number of characters in the word except the first and last characters, replacing O with 0 in the word, and replacing I with l in the word, or any combination of these methods.
[0075] S3. Construct a poisoning model. The attacker mixes the image poisoning sample data obtained in step S1 or the text poisoning sample data obtained in step S2 into the training dataset to obtain a poisoning training dataset for training the constructed poisoning model.
[0076] Furthermore, the specific implementation method of step S3 includes the following steps:
[0077] S3.1. The attacker mixes the image poisoning sample data obtained in step S1 or the text poisoning sample data obtained in step S2 into the training dataset to obtain the poisoned training dataset.
[0078] S3.2. During the i-th round of training communication in federated learning, the global model parameters are downloaded to update the local model. The calculation expression is:
[0079]
[0080] Among them, Global i Represents the global model in the i-th round. Let represent the local model of the j-th participant in the i-th round of training. The scaling factor is updated locally;
[0081] Then, the updated local model is trained using the poisoned training dataset obtained in step S3.1, and the parameters of the trained local model are uploaded to the central server. The calculation expression is:
[0082]
[0083] Attack is the attacker's model;
[0084] S3.3. During the (i+1)th round of training communication in federated learning, the central server performs federated averaging based on the latest uploaded local training model parameters from the poisoned dataset obtained in step S3.2, utilizing global model convergence and fitting the local model with the global model. The calculation expression is as follows:
[0085]
[0086] S3.4. Calculate the parameters of the local poisoning model submitted by the attacker. The calculation expression is:
[0087]
[0088] Then the relationship between the m-th local model and the final global model is calculated as follows:
[0089]
[0090] Scale factor updated locally The attacker's poisoning parameters are amplified. During the global model aggregation phase, the poisoning parameters are retained, and the global model is eventually replaced by the local poisoning model.
[0091] S4. Based on the poisoning model obtained in step S3, construct a defense method for multimodal poisoning attacks, including establishing a third-party parameter server Server to verify the difference between the local model and the global model, using Euclidean distance to measure the difference between the local model and the global model, and the central server to process the uploaded model parameters.
[0092] Furthermore, the specific implementation method of step S4 includes the following steps:
[0093] S4.1. In federated learning training, establish a third-party parameter server (Server) to verify the difference between the local model and the global model, using Euclidean distance (Dis(Local)). i The global metric for the gap is calculated using the following formula:
[0094]
[0095] S4.2. Before each round of federated training is initiated, a third-party parameter server calculates the difference Dis between the local model parameters of each participant in this task and the global model in turn, and sets a threshold ε to judge the difference: when the difference Dis exceeds the threshold ε, it is considered that the local training model parameters have a poisoning threat, and the local training model parameters in this round will no longer be used for federated aggregation.
[0096] S4.3. Adjust the threshold. If the number of participants exceeding or falling below the threshold ε in each training round exceeds... The threshold is then adjusted, and the updated threshold is used in the middle of this round. The formula for calculating the mean of the parameters and the updated threshold ε' is as follows:
[0097]
[0098] Where ζ is a preset scaling parameter;
[0099] S4.4. Considering the overhead of third-party parameter server verification, if no anomalies are found in k consecutive checks, the check frequency is halved; if anomalies are found, the check frequency is increased.
[0100] S4.5. Based on the verification in step S4.4, the central server performs PCA dimensionality reduction decomposition and then k-means clustering on the uploaded local model parameters to defend against multimodal poisoning attacks.
[0101] For each round of uploaded parameter information, a matrix M is formed by columnar arrangement. i×j ∈{R n ×R m To normalize the mean of each row of data in the matrix, the expression is:
[0102]
[0103] Calculate the covariance matrix M cov The calculation expression is:
[0104]
[0105] Then extract M covThe eigenvalues and eigenvectors are used to form a matrix. The eigenvectors are arranged in rows from top to bottom according to the size of the corresponding eigenvalues. The first k rows are taken to form a matrix P, where P·M is the parameter information for dimensionality reduction to k dimensions. The parameter information after dimensionality reduction is then subjected to k-means clustering to obtain different parameter classes. The update corresponding to the largest class is regarded as a normal update, and the updates corresponding to other classes are regarded as malicious updates, thus completing the defense method against multimodal poisoning attacks. Specific Implementation Method Two:
[0107] An electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of a defense method against multimodal poisoning attacks based on generative adversarial networks, as described in Specific Embodiment 1.
[0108] The computer device of the present invention may include a processor and a memory, such as a microcontroller containing a central processing unit. Furthermore, when the processor executes the computer program stored in the memory, it implements the steps of the aforementioned method for defending against multimodal poisoning attacks based on generative adversarial networks.
[0109] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.
[0110] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices. Specific implementation method three:
[0112] A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements a defense method against multimodal poisoning attacks based on generative adversarial networks as described in Specific Embodiment 1.
[0113] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned defense method against multimodal poisoning attacks based on generative adversarial networks can be implemented.
[0114] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0115] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0116] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.
Claims
1. A defense method against multimodal poisoning attacks based on generative adversarial networks, characterized in that, Includes the following steps: S1. The attacker poisons the image training data to obtain poisoned image sample data; S2. The attacker poisons the text training data to obtain poisoned text sample data; S3. Construct a poisoning model. The attacker mixes the image poisoning sample data obtained in step S1 or the text poisoning sample data obtained in step S2 into the training dataset to obtain a poisoning training dataset for training the constructed poisoning model. The specific implementation method of step S3 includes the following steps: S3.
1. The attacker mixes the image poisoning sample data obtained in step S1 or the text poisoning sample data obtained in step S2 into the training dataset to obtain a poisoned training dataset. S3.
2. In Federated Learning During round training communication, global model parameters are downloaded to update the local model. The calculation expression is: in, Indicates the first The global model of the wheel Indicates the first In the first round of training Local models for each participant The scaling factor is updated locally; Then, the updated local model is trained using the poisoned training dataset obtained in step S3.1, and the trained local model parameters are uploaded to the central server. The calculation expression is: in, For the attacker's model; S3.
3. In the Federal Learning Chapter During round training communication, the central server performs federated averaging based on the latest uploaded local training model parameters from the poisoned dataset obtained in step S3.2, utilizes global model convergence, and fits the local model with the global model. The calculation expression is: S3.
4. Calculate the parameters of the local poisoning model submitted by the attacker. The calculation expression is: Then calculate the first... The relationship between the local models and the final global model is as follows: Scale factor updated locally The attacker's poisoning parameters are amplified. During the global model aggregation phase, the poisoning parameters are retained, and the global model is eventually replaced by the local poisoning model. S4. Based on the poisoning model obtained in step S3, construct a defense method against multimodal poisoning attacks, including establishing a third-party parameter server. It is used to verify the difference between the local model and the global model, and uses Euclidean distance to measure the difference between the local model and the global model. The central server processes the uploaded model parameters.
2. The defense method against multimodal poisoning attacks based on generative adversarial networks according to claim 1, characterized in that, In step S1, the attacker poisons its own image training dataset locally by randomly selecting images of different categories for poisoning, including the following steps: S1.
1. The attacker first transforms the image training data to be poisoned into a two-dimensional... Pixel matrix, and modify it. The size of the pixel value serves as the mode trigger. By associating image features with labels through pattern triggers, incorrect feature-label pairs are introduced. When the neural network is trained, the pattern triggers in the image will act as special features and affect the convergence of the neural network training. S1.
2. For cases where no mode trigger is set, shuffle the image training data. Corresponding tags Build error mapping so that Become This causes the local neural network classifier to produce errors in its classification capabilities.
3. A method for defending against multimodal poisoning attacks based on generative adversarial networks according to claim 1 or 2, characterized in that, The specific implementation method of step S2 includes the following steps: S2.
1. The attacker first performs word segmentation on the text training data to be poisoned, and then calculates the most important words in the text training data using the Jacobian matrix. The calculation expression is: in, This represents the first word segmentation of the text training data. One word, This represents the total number of words in the text training data. This represents a text classifier, and the classification results are: Class, j is Any one of them; S2.
2. Calculate the first word segmentation result of the text training data. Importance weight of each word The calculation expression is: in, This indicates the word segmentation process based on the text training data. one word The class obtained from classification prediction; S2.
3. Rank the words in the text based on the importance weights calculated in step S2.2, and select the most important ones. The word is poisoned using methods including inserting a space into the word, deleting any number of characters except the first and last characters of the word, replacing O with 0, and replacing I with l, or any combination of these methods.
4. The defense method against multimodal poisoning attacks based on generative adversarial networks according to claim 3, characterized in that, The specific implementation method of step S4 includes the following steps: S4.
1. Establishing a third-party parameter server in federated learning training It is used to verify the difference between the local model and the global model, using Euclidean distance. The difference is measured and calculated using the following expression: S4.
2. Before each round of federated training begins, a third-party parameter server sequentially calculates the difference between the local model parameters of each participant and the global model. Set threshold Judge the gap: When the gap Exceeding the threshold The local training model parameters are deemed to pose a poisoning threat, and will no longer be used for federated aggregation in this round of local training model parameters. S4.
3. Adjust the threshold if the value exceeds or falls below the threshold in each training round. The number of participants exceeds If so, the threshold is adjusted, and the updated threshold is used in the middle of this round. Average of each parameter, update threshold The calculation expression is: in, This is a preset proportionality coefficient; S4.
4. Consider the overhead of third-party parameter server verification; if continuous... If no abnormalities are found in the tests, the testing frequency is halved; if an abnormality is found, the testing frequency is increased. S4.
5. Based on the verification in step S4.4, the central server performs PCA dimensionality reduction decomposition and then k-means clustering on the uploaded local model parameters to defend against multimodal poisoning attacks. The parameter information uploaded in each round is arranged into a matrix according to its columns. To normalize the mean of each row of data in the matrix, the expression is: Calculate the covariance matrix The calculation expression is: （11） Then extract Given the eigenvalues and eigenvectors, arrange the eigenvectors into a matrix from top to bottom according to the magnitude of their corresponding eigenvalues, and take the first row. Rows form a matrix , To reduce dimensionality to The parameter information of the dimension reduction is used to perform k-means clustering to obtain different parameter classes. The update corresponding to the largest class is regarded as a normal update, and the updates corresponding to other classes are regarded as malicious updates, thus completing the defense method against multimodal poisoning attacks.
5. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the defense method against multimodal poisoning attacks based on generative adversarial networks as described in any one of claims 1-4.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the defense method against multimodal poisoning attacks based on generative adversarial networks as described in any one of claims 1-4.