317 results about "Backdoor" patented technology

The invention discloses a federated learning backdoor defense method based on pruning and fine tuning in the technical field of artificial intelligence and network security, the method realizes defense through two core mechanisms of dynamic pruning and gradient constraint fine tuning, and the method comprises the following steps: firstly, calculating a sensitivity score based on a neuron activation frequency and a weight outlier degree; dynamically identifying and cutting redundant neurons utilized by a backdoor, and blocking an abnormal activation path; secondly, gradient direction consistency detection and amplitude constraint are introduced in the fine tuning stage, and a malicious client is inhibited from reconstructing a back door through an abnormal gradient; the server continuously purifies model parameters and enhances robustness by cyclically executing pruning, fine tuning and aggregation operations; the method does not need to depend on an extra clean data set, strictly follows a federated learning privacy protection principle, reduces communication overhead through lightweight pruning, maintains main task performance in combination with gradient constraint, is suitable for a federated learning scene in which edge equipment participates, and effectively balances a defense effect and model stability.

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

The invention discloses a federated learning backdoor attack defense method based on layer perception detection, which relates to the technical field of artificial intelligence security, and comprises the steps of client training and uploading, malicious client identification, backdoor key layer detection and deletion, robust aggregation updating and adversarial training for enhancing robustness. According to the method, the model parameters uploaded by the clients are subjected to clustering analysis, and the maximum benign cluster is identified by adopting an unsupervised clustering method, so that instability caused by single threshold judgment is avoided, and the benign client and the malicious client can be distinguished; and meanwhile, layer perception detection and elimination are used in the scheme, so that the backdoor introduced by a malicious client can be effectively identified and eliminated on the premise of ensuring the global model precision, and the influence of backdoor attack on the model is inhibited.

The invention relates to the technical field of network security, in particular to a defense method and system for federated learning backdoor attacks, a server initializes a federated learning global model, and identifies at least one key layer easy to implant a backdoor in the global model; distributing the current global model parameters to a plurality of clients, receiving model update information uploaded by each client, extracting a multi-dimensional gradient feature vector of each client based on gradient information of each client in a key layer, calculating a mahalanobis distance anomaly score of each client relative to the gradient distribution of the whole client, and calculating the mahalanobis distance anomaly score of each client according to the mahalanobis distance anomaly score; and screening and rejecting the clients which are judged to be abnormal, aggregating model updates uploaded by the clients which are judged to be benign, and generating a new global model. Malicious updating is effectively isolated while the performance of the main task is ensured, and the robustness and safety of a global model are improved.

The invention discloses a backdoor attack detection method, device and equipment, a medium and a program product. The method comprises the following steps: acquiring a benign sample set and input data; transplanting the local features of the input data to each benign sample in the benign sample set to obtain a disturbance sample set; determining a prediction category of the disturbance sample set and a prediction category of the benign sample set based on the target classification model; and according to the prediction category of the benign sample set and the prediction category of the disturbance sample set, determining whether the input data is backdoor data. Local features of input data are transplanted to a benign sample to obtain a disturbance sample set by utilizing the strong correlation characteristic of model implanted backdoor attack and disturbance styles, backdoor attack detection is performed on the consistency of prediction categories of the benign sample set and the disturbance sample set based on a target classification model, the universality of a backdoor attack mode is high, and the accuracy of backdoor attack detection is high. The method is not constrained by an attack mode, and has relatively high detection accuracy for back door attacks irrelevant to input and back door attacks relevant to input.

The invention discloses a graph backdoor encoder defense method and device, and a readable storage medium. The method comprises the steps that at least two label-free enhanced data sets are generated based on a preset training data set, a to-be-processed encoder is trained based on the enhanced data sets to obtain a teacher encoder, and the training data set is a downstream data set or a subset of the downstream data set; defining identical attention operators in corresponding layers of the teacher encoder and the to-be-processed encoder, wherein the attention operators are used for outputting layer attention maps of corresponding encoder layers; calculating the distribution offset of the layer attention map of the teacher encoder and the layer attention map of the encoder to be processed in the same encoder layer; and fixing the parameters of the teacher encoder, and updating the parameters of the to-be-processed encoder based on the distribution offset. Compared with the prior art, through lightweight deployment, on the premise that the model precision is not reduced, the attack success rate of backdoor attack on graph self-supervised learning is effectively reduced, and the backdoor defense capability of the system is improved.

The invention discloses a Non-IID federated learning backdoor attack defense method and system and a medium, and relates to the field of federated learning and network security. In order to solve the problems that an existing method depends on model parameters and is easy to avoid by a malicious client, and adaptive Non-IID scenes are poor, truncated singular value decomposition is executed through the client to extract first p left singular vectors, and the first p left singular vectors are uploaded to a server; the server constructs a matrix based on left singular vector cosine similarity and performs hierarchical clustering, and calculates a client similarity score and an aggregation weight by using a zoom dot product attention mechanism in combination with a left singular vector of the clean reference data set; the server distributes a global model, the client uploads parameters after local training, and the server weights and aggregates the model parameters in the cluster according to the weight and iteratively optimizes the model parameters. According to the method, the malicious client is identified from the data essential features, the malicious proportion does not need to be preset, the good client contribution and the data privacy are guaranteed while the backdoor attack is inhibited, the method is suitable for a Non-IID scene, and the model robustness and the main task performance are improved.

The invention provides a watermark processing method and device, watermark processing equipment, a program product and a medium, and relates to the technical field of privacy computing. The method comprises the following steps: acquiring first data which is requested to be input into a target model by a user through an application programming interface (API); aiming at the first data, judging whether watermarking processing needs to be carried out or not; when it is determined that watermark processing needs to be carried out, based on a backdoor function corresponding to the target model, an API response corresponding to the API request is generated according to the first data, and the backdoor function is constructed according to a decision function of the target model; the API response is output to the user, the first data is added to a trigger set corresponding to the target model, and the trigger set is used for verifying an alternative model of the target model. According to the scheme of the invention, the problem that the existing digital watermarking technology is difficult to effectively defend model extraction attacks initiated through an API (Application Program Interface) is solved.

Disclosure are a backdoor attack method and system for a classification task in a code model, the method includes: S1. collecting and preprocessing clean samples to obtain importance variable names; S2. classifying the variable names of the clean samples according to label categories to obtain a plurality of trigger sets; and selecting target labels from the clean samples; S3. performing score calculation on the variable names in the trigger sets corresponding to the target labels; replacing one importance variable name with the variable name having a maximum C score in the clean samples to obtain poisoned samples, and repeating the above process until the labels are changed into the target labels; and S4. randomly inserting the triggers in the poisoned samples into the clean samples to form negative samples; and performing an attack by using an attack model obtained based on the negative, poisoned and clean samples.

A black box code search model backdoor attack method based on learnable discrete code transformation comprises the steps that a learnable backdoor generator is constructed, and malicious codes with backdoors are generated on the premise that internal parameters of a damaged model are not accessed through the learnable discrete code transformation. Firstly, an agent model capable of simulating victim model behaviors is trained through query-response data; secondly, on the proxy model, a discrete selection process is differentiable by using a re-parameterization sampling technology, and a backdoor generator is trained in combination with a multi-objective loss function so as to realize effectiveness and concealment of backdoor implantation; according to the method, the limitation of a black box is successfully bypassed by constructing the proxy model, and a new possibility is provided for an attacker. A backdoor generation process is integrated into a differentiable training framework through learnable discrete transformation, so that end-to-end learnability is realized, and an attack strategy can be automatically optimized.

The invention discloses a BDDR backdoor detection and data recovery method and system oriented to a large model, and belongs to the field of backdoor defense. Comprising the following steps: constructing a knowledge distillation architecture under federal learning, including an edge server and a plurality of clients, and obtaining distillation data; inputting the distillation data into a randomly initialized model for training, recording the loss change of each batch of data, and screening out abnormal batches to form a backdoor data set; the edge server initializes two independent models, respectively uses a distillation data set and a backdoor data set for training, and guides learning of backdoor features; using probability distribution to calculate and correct a backdoor label, generating a clean data set by adding noise, finely adjusting a large model, detecting residual backdoor feature intensity, and adjusting probability distribution calculation parameters to further weaken backdoor features according to the residual backdoor feature intensity so as to obtain a final repaired data set; while the generalization ability of the large model is improved, backdoor attacks can be effectively identified and defended, the data privacy of the client is protected, and the model security is ensured.

The invention discloses a defense method and device for defending federated learning backdoor attack and a medium, and the method comprises the steps: a client global model is randomly distributed to a client set, and is trained and updated based on client local data; after model update quantity is collected, hierarchical clustering is carried out based on update direction similarity; for clusters obtained by clustering in each direction in hierarchical clustering, extracting a corresponding L2 norm as an amplitude feature, determining the optimal number of sub-clusters by adopting a contour coefficient, and performing secondary clustering; for each clustered sub-cluster, taking the median of the updating amplitude of the updating quantity of a plurality of groups of models as a clipping threshold, calculating the scaling of each node, and scaling the sub-cluster with the amplitude exceeding the threshold; aggregating the clipped model update quantity, carrying out weighted average to generate a new generation of global model, and distributing the new generation of global model to a client group for iteration; and repeating the steps until the model converges or reaches a preset training round. According to the invention, high-precision identification and isolation of malicious clients are realized.

The invention relates to the technical field of machine learning, in particular to a defense method and device for graph neural network backdoor attacks, equipment and a medium, when an open domain node classification task is received, the open domain node classification task is input into a preset trigger detection model, the model can determine unknown class nodes and cut edges of the unknown class nodes to obtain an initial defense sub-graph, and the initial defense sub-graph is used for defending the open domain node classification task. And performing importance score calculation on the target defense nodes in the initial defense subgraph to form a final defense subgraph, inputting the final defense subgraph into a preset dynamic classifier, and outputting a classification result of the target defense nodes. According to the method, the backdoor attack problem faced by the graph neural network in an open domain scene is effectively solved, and the classification accuracy and security are improved.

The invention relates to a power grid security defense system based on artificial intelligence and a block chain, and the system comprises an AI defense layer, a block chain trust layer, a cooperative control layer, an attack traceability and attribution layer, a physical-information fusion verification layer, a scene adaptive configuration layer and a risk prediction module. Data credibility is guaranteed by reputation weighted consensus, dynamic defense is completed by means of intelligent contracts and cross-layer cooperation, and beforehand early warning and computing power adaptation are realized through a risk prediction module. The system effectively solves the problems that in the prior art, an AI model is prone to being attacked, block chain response lags behind, virtual and real data are not synchronous, multi-scene adaptability is poor, pre-risk early warning is lacked, and computing power dispatching lags behind, can be widely applied to scenes such as a power transmission network, a power distribution network, a micro-grid and energy storage grid connection, and has high practicability. The anti-attack ability of the power grid to false data injection, equipment backdoor attack, collaborative DDoS and other threats is significantly improved, and the operation stability and safety of the power grid are guaranteed.

The invention discloses a federated backdoor defense method based on decoupling contrast learning, and the method comprises the steps: training a backdoor model based on a backdoor sample, and immediately stopping training after the backdoor model converges on the backdoor sample; respectively extracting a penultimate layer vector of the backdoor model and the local model from a sample pair held by the malicious client as a backdoor feature and a clean feature; comparing and learning the separated back door features and the clean features, and learning the clean features for the local model by using a sample weighting strategy to train the local model to obtain a trained local model; and sending local model parameters of the trained local model to a global server, and generating model parameters of a new global model based on the local model parameters through an aggregation function. The method aims at reducing information dependence between backdoor features and clean features through comparative learning, so that local model learning is free of backdoor representation, and the robustness of a global model is improved.

The invention discloses a privacy protection and robustness test method and system for large model fine tuning, and belongs to the technical field of machine learning security. The method comprises the steps that a three-layer distributed architecture comprising an edge server, a cloud server and a plurality of edge clients is constructed, the edge clients distill local privacy data and cooperate with the edge server to train a global model, and a candidate detection sample set is formed; screening a sample set based on the potential feature deviation evaluation index, and sending the sample set to a cloud server for vulnerability detection to obtain an optimal backdoor detection candidate sample set; and multi-trigger parallel and progressive trigger sequence backdoor implantation is respectively used for scenes of single fine tuning and multiple fine tuning of the large model, an optimal backdoor detection candidate sample set is combined with a preset trigger to generate a backdoor test sample set, the backdoor test sample set is mixed with a clean data set, and then the robustness of the backdoor test sample set is tested through fine tuning of the large model. Large model fine tuning and robustness testing of privacy protection can be realized in a heterogeneous model cooperative training environment.

Disclosed is a cybertwin-based method for constructing a two-layer federated learning framework for internet of vehicles, including constructing a two-layer federated learning system for internet of vehicles, dividing a cybertwin network framework into three layers: a central cloud server, an edge cloud server j and a vehicle device i, with the central cloud server attached to a server preset in the central cloud server, and the edge cloud server j attached to a roadside unit preset in the vehicle device i. According to the method, federated learning scenarios are expanded, and more applicable and capable of resisting more backdoor attacks.

The invention belongs to the technical field of network space security, and discloses a federated learning backdoor attack defense method, system and device, a medium and a program product thereof, the method comprises the following steps: generating an initialized global model through an aggregation server and issuing the initialized global model to a user, and training by the user by using a local data set and the latest global model issued by the aggregation server, generating an updated local model, uploading the updated local model to an aggregation server, then performing anomaly detection on local model gradient information uploaded by a user by the aggregation server, aggregating model gradients of normal users according to a predefined aggregation rule to obtain an encrypted global model, and then issuing the global model to the users in the system; the system, the equipment and the medium are used for implementing the method. A program product comprising a computer program of the method; according to the method, the negative influence of backdoor attack on the federated learning system is avoided, and the robustness of the system is improved.

The invention discloses a structure-preserving heterogeneous graph backdoor attack method. The method comprises the following steps: preprocessing a clean graph; calculating a composite score according to the multiple scores of each node, and selecting poisoning nodes according to the coincidence scores to obtain a poisoning map; meanwhile, performing global importance evaluation on a feature dimension, transmitting features which can be used as a trigger to a trigger generator, and generating the features by the trigger to disturb the clean graph; the poisoning map is subjected to quasi-classification through an agent model, a trigger generator is optimized according to parameters, double-layer optimization is formed according to the parameters of the trigger generator and a new agent model, and finally hidden and efficient backdoor attacks are completed; wherein the multiple scores comprise one or more of an uncertainty score, a relation weight representative score, a meta-path score and a detectability score. According to the method, through fine node / feature selection and heterogeneous perception trigger generation, the target label attack success rate is remarkably improved.

The invention discloses a natural language model-oriented backdoor sample detection method, which comprises the following steps of: retraining a target language model to enable the target language model to comprise a self-defined backdoor; generating N mutants of the target language model by using a depth model mutation technology; constructing a backdoor sample detector: detecting backdoor samples by utilizing the target language model and the predicted change of the mutant thereof, and representing the predicted change of each input sample as an N-dimensional vector; selecting a back door sample and a clean sample, and calculating a prediction change vector as a training set training detector; using the trained detector to distinguish clean samples and back door samples; according to the method, the security threat of backdoor attack is relieved more effectively.

The invention relates to an implicit black box watermarking method for copyright protection of a multi-modal model, and belongs to the technical field of artificial intelligence safety. The method specifically comprises the following steps: 1, generating a multi-modal model black box backdoor trigger; 2, completing model training of a conversion module according to the backdoor trigger set in the step 1; and 3, performing ownership verification and infringement model judgment on the suspicious model according to the backdoor trigger set in the step 1 and the conversion module in the step 2. According to the method, the resistance trigger is generated by adopting the original samples in the original data set, the limit of an existing distributed external trigger is broken, the method can adapt to a real application scene of a multi-modal model under the condition of small overhead, and the method has good practicability and expandability. In addition, two-stage watermark verification based on a conversion module is adopted, so that the infringement problem caused by hostile attacks can be effectively avoided.

The invention relates to a backdoor attack method and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a text set; inputting the text set into a backdoor discriminator; under the condition that the backdoor judgment result indicates that the corresponding text is a clean text, clean semantic features of the text are extracted; otherwise, extracting a pollution semantic feature of the text, and performing weighted summation on the pollution semantic feature and a preset target category feature; and inputting the clean semantic features, the fused semantic features and the image samples into a multi-modal retrieval system to perform cross-modal retrieval. Therefore, the polluted text, namely the backdoor text, is obtained by performing style migration on the original text, so that the phenomenon that the text fluency and semantic coherence are damaged due to insertion of obvious trigger vocabularies can be avoided, the risk found by a defensive mechanism and manual inspection can be effectively reduced, and the user experience is improved. Therefore, the concealment and naturalness of the backdoor attack can be improved.

The invention discloses a privacy protection fine tuning and security testing method for a large model, and belongs to the technical field of artificial intelligence, and the method comprises the steps: 1, enabling a data provider to cooperate with an edge coordinator to generate a feature extraction edge auxiliary model based on edge-end federal pre-training, and carrying out the feature matching of distillation privacy data based on multiple spatial data; 2, the data provider generates distillation data with an invisible backdoor; and step 3, the task initiator finely adjusts the cloud target large model and performs a security test. The method aims at protecting the privacy of private data of the data provider, reducing the scale of fine adjustment data and realizing the security test of the fine adjustment process of the large model.

The invention provides a large model backdoor attack detection and automatic tracing method, and aims to solve the problems of poor accuracy, high false alarm rate and difficulty in hidden backdoor recognition in large-scale pre-training model backdoor attack detection in the prior art. The method comprises the following steps: firstly, extracting parameter distribution characteristics and reasoning behavior characteristics of a target large model, and identifying potential abnormal clusters by utilizing clustering analysis; then, whether the model is affected by backdoor attacks or not is verified by analyzing deviation output by the model and generating an adversarial sample, and through a feature inversion backtracking technology, a mapping relation between input and output is tracked, and backdoor trigger features are positioned; and finally, identifying a backdoor attack source and a propagation path thereof through traceability analysis, and eliminating the backdoor influence by correcting abnormal parameters and retraining. The method has the advantages of high efficiency, low false alarm, automatic traceability and the like, the safety of a large-scale model can be effectively improved, and the credibility and reliability of the model in practical application are ensured.

The invention discloses a back door defense method and system based on adaptive feature blocking, and the method comprises the steps: 1, constructing a blocking module, and embedding the blocking module into an image classification model; the blocking module comprises a self-adaptive instance statistical calibration layer and a dynamic channel suppression layer, the self-adaptive instance statistical calibration layer is used for eliminating cross-sample statistical offset caused by backdoor attack, and the dynamic channel suppression layer is used for suppressing abnormal activation of a polluted channel; and 2, training the embedded classification network to obtain a final classification network, and completing backdoor attack defense according to the final classification network. Through lightweight modular design and a fine adjustment mechanism, the bottleneck problem of a traditional model defense method in the aspects of calculation cost and flexibility is effectively broken through.

The present application relates to the technical field of computer vision and natural language processing, and provides a cross-modal transferable backdoor attack method and device, comprising: constructing a backdoor dataset containing a trigger based on an unlabeled original dataset; inputting the original dataset and the backdoor dataset containing the trigger into a pre-constructed hacker network, and respectively calculating clean data silence loss and backdoor toxicity loss with the trigger; pre-training the pre-constructed hacker network by minimizing the clean data silence loss and the backdoor toxicity loss; calculating feature representation of a target category through image data and text data of the target category, generating a target CLIP model with a backdoor by using the pre-trained hacker network and the feature representation of the target category; and attacking by using the generated model. The backdoor trigger can be embedded in multi-modal data at the same time, has strong cross-modal transferability and concealment, and does not depend on large-scale labeled data.

The application discloses a federated learning persistence backdoor attack method based on dynamic optimization trigger, first, the attack feedback index is read from the second attack round, and the attack parameters of the round are adjusted; secondly, the backdoor trigger is dynamically optimized, the attacker uses the local data to simulate the benign global model after the attack stops while updating the global data dynamically, minimizes the difference between the trigger feature and the potential representation before and after the attack, and trains the memory ability of the global model to the trigger feature; then, noise is adaptively added to the model full connection layer, and the effectiveness of the attack is enhanced; finally, the attack feedback index is updated, and the redundant neurons are used as the feedback index of whether the attack is successful, which is initialized in the first attack round and updated in the remaining attack rounds. The application adopts the strategy of dynamically optimizing the trigger and the attack feedback index based on the redundant neurons to realize the persistence backdoor attack against the federated learning framework.

The application discloses a backdoor attack method based on color frequency injection and adaptive local enhancement, and relates to the technical field of machine learning and artificial intelligence security. The method comprises the following steps: introducing low-frequency color offset and weak high-frequency signal into an image in a CIELAB color space to perform global color-frequency injection; using a pre-trained proxy model to locate a high-sensitive perception domain of the model through mixed evaluation of gradients and class activation maps, and generating a binary mask; in an HSV color space, respectively applying nonlinear stretching factors to saturation and brightness of the sensitive domain based on the mask to perform adaptive local enhancement; using Gaussian smoothing, adaptive noise and histogram matching to eliminate edges and statistical abnormalities caused by local enhancement, completing compensation color enhancement to generate a poisoned image; and modeling a trigger core parameter as a constrained optimization problem, and using a particle swarm optimization algorithm to jointly dynamically update the trigger core parameter to obtain an optimal strategy. The application anchors the trigger feature depth in the core semantic area of the model and lurks in the normal data manifold, guarantees a high attack success rate, realizes extreme visual and feature concealment, and has strong anti-defense robustness.

The embodiment of the invention discloses an evidence obtaining method and system for detecting back door attack for deep malicious codes, and the method comprises the steps: obtaining an attacked data set of back door attack data to be subjected to evidence obtaining and a known back door attack data set, carrying out the preprocessing, and generating a corresponding feature vector; for the feature vector of the back door attack data to be obtained, performing similarity calculation on the feature vector of the back door attack data to be obtained and the feature vector of attack data belonging to each malicious code category in a known back door attack data set; and according to the similarity value, the evaluation value and the probability value of each malicious code category to which the evidence obtaining backdoor attack data belongs are calculated to determine the backdoor attack data of final evidence obtaining in the evidence obtaining backdoor attack data set, so that the accuracy and the efficiency of digital evidence obtaining are greatly improved, and the safety of a deep malicious code detection system is powerfully ensured.

The application belongs to the field of artificial intelligence security, and discloses a training method for physical light backdoor attack for artificial intelligence security, comprising the following steps: performing light backdoor attack on a target object, generating corresponding light triggers on the target object according to light colors, and generating backdoor image data based on the light triggers; obtaining clean image data, and respectively constructing training sets based on the backdoor image data and the clean image data; the clean image data is original image data without light triggers; constructing a backdoor model, the backdoor model is a deep learning model, training the backdoor model based on the training sets to obtain a trained backdoor model; constructing a test set, evaluating the trained backdoor model based on the test set to obtain attack success rate data and clean accuracy rate data of the light backdoor attack. The technical scheme disclosed by the application realizes more covert physical backdoor attack while having a higher attack success rate.