449 results about "Backdoor" patented technology

The invention discloses a federated learning backdoor defense method based on pruning and fine tuning in the technical field of artificial intelligence and network security, the method realizes defense through two core mechanisms of dynamic pruning and gradient constraint fine tuning, and the method comprises the following steps: firstly, calculating a sensitivity score based on a neuron activation frequency and a weight outlier degree; dynamically identifying and cutting redundant neurons utilized by a backdoor, and blocking an abnormal activation path; secondly, gradient direction consistency detection and amplitude constraint are introduced in the fine tuning stage, and a malicious client is inhibited from reconstructing a back door through an abnormal gradient; the server continuously purifies model parameters and enhances robustness by cyclically executing pruning, fine tuning and aggregation operations; the method does not need to depend on an extra clean data set, strictly follows a federated learning privacy protection principle, reduces communication overhead through lightweight pruning, maintains main task performance in combination with gradient constraint, is suitable for a federated learning scene in which edge equipment participates, and effectively balances a defense effect and model stability.

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

The invention relates to the field of software security, and particularly discloses a software supply chain security analysis method which comprises the following steps: S1, collecting data of components of a software supply chain, and calculating a historical security risk weight for each component in the supply chain; s2, collecting behavior data of the software during operation, and performing association analysis in combination with the supply chain data; s3, preprocessing the collected behavior data and supply chain data, and extracting key features; according to the software supply chain security analysis method, through dynamic behavior analysis, runtime threats, such as zero-day vulnerabilities and hidden backdoor threats, which cannot be detected by static analysis can be found, the coverage rate of threat detection is remarkably improved, meanwhile, historical security risk weights are introduced, historical security problems of supply chain components are quantified, and the security of the software supply chain is improved. And intelligent risk assessment and threat traceability are realized by combining a knowledge graph and a graph neural network technology.

The invention relates to the field of personal intelligent security evaluation, and particularly discloses a chain reasoning hidden backdoor vulnerability detection method of a vision-language-action model, which comprises the following steps of: respectively injecting micro pixel disturbance and rare character marks into vision and language input; on the basis of model autoregression prediction characteristics, designing a hybrid reasoning chain fusing normal reasoning steps and abnormal backdoor branches; adopting prefix tuning to take the hybrid reasoning sequence as a pluggable prefix injection model; and generating the vulnerability sensitivity of the abnormal action instruction through the systematic verification process detection model. Compared with an existing method, the method has the advantages that a nondestructive testing mechanism based on prefix adjustment and optimization does not need to modify model parameters or depend on training data, and the safety and reproducibility of detection are guaranteed; a multi-mode triggering mechanism is constructed, and the hidden vulnerability of the model in a complex scene is effectively revealed; the abnormal branches and the normal process are fused in a chain mode, and the defense capability of the model for the concealment logic offset can be systematically evaluated.

The invention discloses a frequency domain stealth backdoor attack method based on a generative adversarial network, and belongs to the technical field of image data processing, and the method comprises the steps: obtaining a clean sample set D1 and a classification model; constructing a generative adversarial network, and constructing a loss function based on picture similarity loss, frequency domain consistency loss and adversarial loss; training the generative adversarial network by using the D1 to obtain a generative model; constructing a clean data set and a backdoor data set based on the D1, a target label t of the backdoor attack and the generative model; constructing multi-layer MMD loss; and constructing total loss based on MMD and MSE, and performing poison training on the classification model to obtain a backdoor model. According to the method, a multi-domain disturbance generation network is constructed, pixel-level disturbance is generated in a spatial domain, discrete wavelet transform is introduced into a frequency domain to constrain frequency domain characteristics of back door disturbance, poisoning samples have concealment in the spatial domain and the frequency domain, multi-layer MMD loss is introduced, the deep characteristic distribution difference between clean samples and back door samples is reduced, and high-concealment attack is achieved.

The invention discloses a federated learning backdoor attack defense method based on layer perception detection, which relates to the technical field of artificial intelligence security, and comprises the steps of client training and uploading, malicious client identification, backdoor key layer detection and deletion, robust aggregation updating and adversarial training for enhancing robustness. According to the method, the model parameters uploaded by the clients are subjected to clustering analysis, and the maximum benign cluster is identified by adopting an unsupervised clustering method, so that instability caused by single threshold judgment is avoided, and the benign client and the malicious client can be distinguished; and meanwhile, layer perception detection and elimination are used in the scheme, so that the backdoor introduced by a malicious client can be effectively identified and eliminated on the premise of ensuring the global model precision, and the influence of backdoor attack on the model is inhibited.

The invention provides a federal learning backdoor attack defense method based on adaptive clustering, and aims to solve the problems that the defense capability is insufficient and backdoor attacks with the malicious client proportion exceeding 50% cannot be effectively defended under the condition of data non-independent identical distribution. An existing federal learning backdoor attack defense method based on clustering generally adopts a diclustering strategy, and has the problems that the malicious client identification accuracy is not high, and defense fails to the malicious client in a proportion of more than 50%. In order to solve the problems, a Louvain community discovery algorithm is applied to perform adaptive clustering on model parameters, and a historical cumulative credibility updating mechanism and an abnormal value cutting strategy are introduced, so that malicious clients are accurately identified and effective defense is realized on the premise of ensuring the main task precision of a global model.

The present invention discloses a deep learning backdoor attack method and device based on an ordinal network and a medium, which belongs to the technical field of neural network security. The method comprises: obtaining a training sample image; generating an ordinal network based on the training sample image, wherein the ordinal network is used for representing an exact structure of an image; and generating a trigger by using the ordinal network. Through the above method, and the device and the medium for realizing the above method, the present invention uses the ordinal network to generate the trigger, improves the concealability of a poisoning sample compared with poisoning samples generated by other attack methods, and can promote the further research of a hidden backdoor attack defense method in the academic circle.

The invention belongs to the field of agent security, and discloses a multi-agent application program attack assessment method and a related device, and the method comprises the steps: firstly, constructing a poisoning attack prompt word and a backdoor trigger through obtaining an attack motivator and an attack constraint, and combining to generate a backdoor attack sample; then identifying a potential easily-poisoned agent in the multi-agent application program, and inserting a backdoor attack sample into a specified site to generate a poisoned agent; carrying out multiple rounds of agent communication in the multi-agent application program based on the poisoning agent, and simulating an interaction process in a real attack scene to obtain a poisoning multi-agent application program; and finally, a statement with a backdoor trigger is input, and an attack evaluation report is generated according to an output result of the poisoning multi-agent application program, so that the comprehensiveness of poisoning and backdoor attack robustness evaluation of the multi-agent application program is improved, and a powerful basis is provided for optimization of the multi-agent application program. And the agents with excellent performance in the multi-agent application program can be mined.

The invention provides a federal learning backdoor attack defense method based on a multi-layer cooperative defense strategy, and belongs to the technical field of network data security. Aiming at the defects in the prior art that only single-point protection at a certain stage is covered in federated learning, systematic robust defense for a whole process is lacked, backdoor attack detection capability is weak, model recovery efficiency is low and the like, the invention provides a multi-layer defense framework penetrating through a whole period (training, aggregation and deployment) of federated learning. A special information flow channel, a global security evaluation system and a self-adaptive resource allocation strategy are established among the layers, and the three defense layers are organically combined into a cooperative combat whole. The cooperation mechanism not only improves the efficiency of a single defense layer, but also realizes' 1 + 1 + 1gt '; and 3 ''through the synergistic effect, an omnibearing and intelligent backdoor attack defense solution is provided for a federated learning system.

Various embodiments provide a robust backdoor attack on machine learning (ML)-based detection systems that can be applied to demonstrate and identify vulnerabilities thereof. In various embodiments, an artificial intelligence (AI)-based Trojan attack is generated and implanted inside a ML model trained for classification and / or detection tasks, and the AI-based Trojan attack can be triggered by specific inputs to manipulate the expected outputs of the ML model. Analysis of the behavior of an ML model having the AI-based Trojan implanted (and / or triggered) then enables identification of vulnerabilities of the ML model and further enables the design of ML models with improved security. Various embodiments of the present disclosure provide a fast and cost-effective solution in achieving 100% attack success rate that significantly outperforms adversarial attacks on ML models, thereby improving applicability and depth in testing ML-based detection systems.

The invention relates to the technical field of network security, in particular to a defense method and system for federated learning backdoor attacks, a server initializes a federated learning global model, and identifies at least one key layer easy to implant a backdoor in the global model; distributing the current global model parameters to a plurality of clients, receiving model update information uploaded by each client, extracting a multi-dimensional gradient feature vector of each client based on gradient information of each client in a key layer, calculating a mahalanobis distance anomaly score of each client relative to the gradient distribution of the whole client, and calculating the mahalanobis distance anomaly score of each client according to the mahalanobis distance anomaly score; and screening and rejecting the clients which are judged to be abnormal, aggregating model updates uploaded by the clients which are judged to be benign, and generating a new global model. Malicious updating is effectively isolated while the performance of the main task is ensured, and the robustness and safety of a global model are improved.

The invention discloses a text style backdoor defense method based on multi-granularity variant generation and style immunization. The method comprises the following steps: capturing a text style and content by combining explicit and implicit features; generating high-quality text variants on a plurality of granularities such as lexical, syntactic, style, context and the like by utilizing a large language model; carrying out label correction on the suspicious samples based on multi-dimensional risk assessment, and carrying out voting decision making by utilizing style neutralization variants and the like; and finally, through style invariant representation learning, style separation and style contrast training, the robustness of the model to style change is improved. According to the method, an explicit detection trigger is not needed, various attacks including style backdoors can be effectively defended, and through a systematic framework and a self-adaptive strategy, the safety and robustness of the model are remarkably improved while the normal performance of the model is ensured.

The invention discloses a backdoor attack detection method, device and equipment, a medium and a program product. The method comprises the following steps: acquiring a benign sample set and input data; transplanting the local features of the input data to each benign sample in the benign sample set to obtain a disturbance sample set; determining a prediction category of the disturbance sample set and a prediction category of the benign sample set based on the target classification model; and according to the prediction category of the benign sample set and the prediction category of the disturbance sample set, determining whether the input data is backdoor data. Local features of input data are transplanted to a benign sample to obtain a disturbance sample set by utilizing the strong correlation characteristic of model implanted backdoor attack and disturbance styles, backdoor attack detection is performed on the consistency of prediction categories of the benign sample set and the disturbance sample set based on a target classification model, the universality of a backdoor attack mode is high, and the accuracy of backdoor attack detection is high. The method is not constrained by an attack mode, and has relatively high detection accuracy for back door attacks irrelevant to input and back door attacks relevant to input.

The invention discloses a graph backdoor encoder defense method and device, and a readable storage medium. The method comprises the steps that at least two label-free enhanced data sets are generated based on a preset training data set, a to-be-processed encoder is trained based on the enhanced data sets to obtain a teacher encoder, and the training data set is a downstream data set or a subset of the downstream data set; defining identical attention operators in corresponding layers of the teacher encoder and the to-be-processed encoder, wherein the attention operators are used for outputting layer attention maps of corresponding encoder layers; calculating the distribution offset of the layer attention map of the teacher encoder and the layer attention map of the encoder to be processed in the same encoder layer; and fixing the parameters of the teacher encoder, and updating the parameters of the to-be-processed encoder based on the distribution offset. Compared with the prior art, through lightweight deployment, on the premise that the model precision is not reduced, the attack success rate of backdoor attack on graph self-supervised learning is effectively reduced, and the backdoor defense capability of the system is improved.

The invention provides a 3D point cloud defense method, device and equipment based on CLIP guidance, and a medium, relates to the technical field of point cloud defense, and aims to enhance the robustness and security of 3D point cloud data in the face of adversarial attack by using a contrast language-image pre-training model. According to the method, a new defense framework is provided by combining geometric and semantic features, triggers in backdoor attacks can be effectively identified and inhibited, meanwhile, the geometric structure of point cloud data is repaired, and the classification performance is improved. Specifically, the method includes projecting a 3D point cloud into a multi-view 2D depth map, extracting cross-modal features using a CLIP model, and performing semantically guided 3D reconstruction through a variational auto-encoder (VAE) to generate a repaired point cloud that retains key information and neutralizes perturbations. The method shows excellent performance in various complex scenes, and provides powerful support for the safety of the 3D vision technology in key application.

The invention discloses a Non-IID federated learning backdoor attack defense method and system and a medium, and relates to the field of federated learning and network security. In order to solve the problems that an existing method depends on model parameters and is easy to avoid by a malicious client, and adaptive Non-IID scenes are poor, truncated singular value decomposition is executed through the client to extract first p left singular vectors, and the first p left singular vectors are uploaded to a server; the server constructs a matrix based on left singular vector cosine similarity and performs hierarchical clustering, and calculates a client similarity score and an aggregation weight by using a zoom dot product attention mechanism in combination with a left singular vector of the clean reference data set; the server distributes a global model, the client uploads parameters after local training, and the server weights and aggregates the model parameters in the cluster according to the weight and iteratively optimizes the model parameters. According to the method, the malicious client is identified from the data essential features, the malicious proportion does not need to be preset, the good client contribution and the data privacy are guaranteed while the backdoor attack is inhibited, the method is suitable for a Non-IID scene, and the model robustness and the main task performance are improved.

The invention provides a watermark processing method and device, watermark processing equipment, a program product and a medium, and relates to the technical field of privacy computing. The method comprises the following steps: acquiring first data which is requested to be input into a target model by a user through an application programming interface (API); aiming at the first data, judging whether watermarking processing needs to be carried out or not; when it is determined that watermark processing needs to be carried out, based on a backdoor function corresponding to the target model, an API response corresponding to the API request is generated according to the first data, and the backdoor function is constructed according to a decision function of the target model; the API response is output to the user, the first data is added to a trigger set corresponding to the target model, and the trigger set is used for verifying an alternative model of the target model. According to the scheme of the invention, the problem that the existing digital watermarking technology is difficult to effectively defend model extraction attacks initiated through an API (Application Program Interface) is solved.

Disclosure are a backdoor attack method and system for a classification task in a code model, the method includes: S1. collecting and preprocessing clean samples to obtain importance variable names; S2. classifying the variable names of the clean samples according to label categories to obtain a plurality of trigger sets; and selecting target labels from the clean samples; S3. performing score calculation on the variable names in the trigger sets corresponding to the target labels; replacing one importance variable name with the variable name having a maximum C score in the clean samples to obtain poisoned samples, and repeating the above process until the labels are changed into the target labels; and S4. randomly inserting the triggers in the poisoned samples into the clean samples to form negative samples; and performing an attack by using an attack model obtained based on the negative, poisoned and clean samples.

The invention provides a defense device and method for a physical backdoor attack of a biological information identification system, and belongs to the field of information security. The biological information identification system is used for collecting and authenticating a multi-mode biological signal. The defense device comprises a coupling path blocking module which is used for cutting off or weakening the conduction of malicious noise in a physical connection path from three aspects of active shielding, passive filtering and voltage stabilization so as to guarantee the stability of a front-end physical environment. And the physical domain signal purification module forms a closed loop from real-time signal monitoring, feature analysis and interference suppression, and reduces noise transmitted from a power supply circuit to the acquisition device. And the numeric field semantic verification module is used for verifying the collected multi-mode biological signal and judging whether an abnormity or a backdoor injection attack sign exists or not. And the system control and management module is used for realizing communication between external equipment and each module and making a decision according to a numeric field semantic verification result. According to the invention, the physical interference signal injected into the back door can be effectively blocked, and the safety and reliability of the biological information identification system are guaranteed.

A black box code search model backdoor attack method based on learnable discrete code transformation comprises the steps that a learnable backdoor generator is constructed, and malicious codes with backdoors are generated on the premise that internal parameters of a damaged model are not accessed through the learnable discrete code transformation. Firstly, an agent model capable of simulating victim model behaviors is trained through query-response data; secondly, on the proxy model, a discrete selection process is differentiable by using a re-parameterization sampling technology, and a backdoor generator is trained in combination with a multi-objective loss function so as to realize effectiveness and concealment of backdoor implantation; according to the method, the limitation of a black box is successfully bypassed by constructing the proxy model, and a new possibility is provided for an attacker. A backdoor generation process is integrated into a differentiable training framework through learnable discrete transformation, so that end-to-end learnability is realized, and an attack strategy can be automatically optimized.

The invention discloses a BDDR backdoor detection and data recovery method and system oriented to a large model, and belongs to the field of backdoor defense. Comprising the following steps: constructing a knowledge distillation architecture under federal learning, including an edge server and a plurality of clients, and obtaining distillation data; inputting the distillation data into a randomly initialized model for training, recording the loss change of each batch of data, and screening out abnormal batches to form a backdoor data set; the edge server initializes two independent models, respectively uses a distillation data set and a backdoor data set for training, and guides learning of backdoor features; using probability distribution to calculate and correct a backdoor label, generating a clean data set by adding noise, finely adjusting a large model, detecting residual backdoor feature intensity, and adjusting probability distribution calculation parameters to further weaken backdoor features according to the residual backdoor feature intensity so as to obtain a final repaired data set; while the generalization ability of the large model is improved, backdoor attacks can be effectively identified and defended, the data privacy of the client is protected, and the model security is ensured.

The invention discloses a defense method and device for defending federated learning backdoor attack and a medium, and the method comprises the steps: a client global model is randomly distributed to a client set, and is trained and updated based on client local data; after model update quantity is collected, hierarchical clustering is carried out based on update direction similarity; for clusters obtained by clustering in each direction in hierarchical clustering, extracting a corresponding L2 norm as an amplitude feature, determining the optimal number of sub-clusters by adopting a contour coefficient, and performing secondary clustering; for each clustered sub-cluster, taking the median of the updating amplitude of the updating quantity of a plurality of groups of models as a clipping threshold, calculating the scaling of each node, and scaling the sub-cluster with the amplitude exceeding the threshold; aggregating the clipped model update quantity, carrying out weighted average to generate a new generation of global model, and distributing the new generation of global model to a client group for iteration; and repeating the steps until the model converges or reaches a preset training round. According to the invention, high-precision identification and isolation of malicious clients are realized.

The invention relates to the technical field of machine learning, in particular to a defense method and device for graph neural network backdoor attacks, equipment and a medium, when an open domain node classification task is received, the open domain node classification task is input into a preset trigger detection model, the model can determine unknown class nodes and cut edges of the unknown class nodes to obtain an initial defense sub-graph, and the initial defense sub-graph is used for defending the open domain node classification task. And performing importance score calculation on the target defense nodes in the initial defense subgraph to form a final defense subgraph, inputting the final defense subgraph into a preset dynamic classifier, and outputting a classification result of the target defense nodes. According to the method, the backdoor attack problem faced by the graph neural network in an open domain scene is effectively solved, and the classification accuracy and security are improved.

The invention relates to a power grid security defense system based on artificial intelligence and a block chain, and the system comprises an AI defense layer, a block chain trust layer, a cooperative control layer, an attack traceability and attribution layer, a physical-information fusion verification layer, a scene adaptive configuration layer and a risk prediction module. Data credibility is guaranteed by reputation weighted consensus, dynamic defense is completed by means of intelligent contracts and cross-layer cooperation, and beforehand early warning and computing power adaptation are realized through a risk prediction module. The system effectively solves the problems that in the prior art, an AI model is prone to being attacked, block chain response lags behind, virtual and real data are not synchronous, multi-scene adaptability is poor, pre-risk early warning is lacked, and computing power dispatching lags behind, can be widely applied to scenes such as a power transmission network, a power distribution network, a micro-grid and energy storage grid connection, and has high practicability. The anti-attack ability of the power grid to false data injection, equipment backdoor attack, collaborative DDoS and other threats is significantly improved, and the operation stability and safety of the power grid are guaranteed.

The invention relates to the technical field of information security, and discloses a Minkowski distance-based federated learning backdoor defense method, which comprises the following steps of S1, constructing a federated learning adaptive backdoor defense framework based on sparse training and Minkowski distance detection, and acquiring image data from an image public data set; adding a backdoor trigger to the image data to obtain poisoning image data injected into a backdoor, and dividing an image data set and a normal image data set into a training set, a verification set and a test set; according to the Minkowski distance-based federated learning backdoor defense method, the training burden of a client model is reduced by using a sparse training mode, the transmission pressure of excessive model parameters between the server and the client is relieved, the score is updated by using the Minkowski distance-based model, and the defensive performance of the model is improved. And possible malicious model updating is eliminated, so that backdoor attacks are effectively relieved.

The invention relates to the field of machine learning, in particular to a model assimilation-based suspicious model backdoor category positioning method, which comprises the following steps of: performing assimilation degree calculation on a model by utilizing an image data set, and judging whether the model is attacked by a backdoor or not; when the model is subjected to backdoor attack, covariance discriminant analysis is utilized to calculate a covariance index of each category so as to locate a suspicious category; performing data cleaning according to the suspicious category to obtain a clean data set; generating a balanced data set according to the clean data set; and retraining the model according to the balanced data set. According to the method, the suspicious poisoning type in the data set can be positioned by measuring the attention mode difference of different types in the data set, the backdoor model and the poisoning type are effectively detected in various backdoor attack scenes without depending on known backdoor information, and the backdoor attack detection accuracy and robustness are improved.

The invention belongs to the technical field of security, and particularly relates to an image enhancement method and system for hidden backdoor attack based on a denoising model, and the method comprises the steps: obtaining an image frequency feature according to a training noise image, and obtaining a disturbance feature through a disturbance function; obtaining a spatial domain signal according to the disturbance characteristic, and defining a trigger according to the spatial domain signal and the image frequency characteristic; implanting a trigger into the diffusion model to obtain a modified diffusion model, a forward process and a posterior process; obtaining clean loss and poisoning loss according to the forward process, the posterior process and the loss function, and obtaining total loss according to the clean loss and the poisoning loss; adjusting parameters of the disturbance function according to the total loss to obtain an adjustment diffusion model; and inputting the actual noise image into the adjustment diffusion model to obtain an enhanced image. From the perspective of a frequency domain, a diffusion model can output a desired feature image through a backdoor attack mode, texture features of required features are enhanced directionally, and the accuracy of judging the condition of a patient is improved.

The invention provides a node category prediction method, device and equipment for a backdoor attack and a medium, and relates to the technical field of deep learning, and the method comprises the steps: obtaining a disturbance diagram of the backdoor attack; determining at least one sub-graph corresponding to the perturbation graph based on the adjacency matrix of the perturbation graph and the similarity of each node in the perturbation graph; inputting each sub-graph into an integrated model to obtain a prediction category corresponding to each node in a disturbance graph output by the integrated model; the integrated model is obtained by training a plurality of sub-sample graphs corresponding to the sample graph attacked by the backdoor. According to the method, the accuracy of predicting each node category in the disturbance graph of the backdoor attack by the integrated model is improved.

The invention discloses a federated backdoor defense method based on decoupling contrast learning, and the method comprises the steps: training a backdoor model based on a backdoor sample, and immediately stopping training after the backdoor model converges on the backdoor sample; respectively extracting a penultimate layer vector of the backdoor model and the local model from a sample pair held by the malicious client as a backdoor feature and a clean feature; comparing and learning the separated back door features and the clean features, and learning the clean features for the local model by using a sample weighting strategy to train the local model to obtain a trained local model; and sending local model parameters of the trained local model to a global server, and generating model parameters of a new global model based on the local model parameters through an aggregation function. The method aims at reducing information dependence between backdoor features and clean features through comparative learning, so that local model learning is free of backdoor representation, and the robustness of a global model is improved.