523 results about "Backdoor" patented technology

The invention discloses a federated learning backdoor defense method based on pruning and fine tuning in the technical field of artificial intelligence and network security, the method realizes defense through two core mechanisms of dynamic pruning and gradient constraint fine tuning, and the method comprises the following steps: firstly, calculating a sensitivity score based on a neuron activation frequency and a weight outlier degree; dynamically identifying and cutting redundant neurons utilized by a backdoor, and blocking an abnormal activation path; secondly, gradient direction consistency detection and amplitude constraint are introduced in the fine tuning stage, and a malicious client is inhibited from reconstructing a back door through an abnormal gradient; the server continuously purifies model parameters and enhances robustness by cyclically executing pruning, fine tuning and aggregation operations; the method does not need to depend on an extra clean data set, strictly follows a federated learning privacy protection principle, reduces communication overhead through lightweight pruning, maintains main task performance in combination with gradient constraint, is suitable for a federated learning scene in which edge equipment participates, and effectively balances a defense effect and model stability.

Disclosed in the present invention is a federated learning method against a backdoor attack. Firstly, a few benign model updates having a relatively high confidence are selected by means of clustering-based voting, then the selected models are considered as benign modes of model updates, and on the basis of an anomaly detection method of a variational auto-encoder, more benign model updates are progressively selected from among the remaining candidate model updates, so that the scale of a group of the selected benign model updates is continuously expanded. The present invention does not depend on differential privacy, weight pruning, and learning rate adjustment and merely sightly changes a protocol of original federated learning, and therefore, on one hand, the impact on the accuracy of a global model is relatively small, and on the other hand, integration with existing federated learning systems is facilitated. Compared with existing federated learning methods against backdoor attacks, the present invention has relatively high adaptability.

The invention provides a privacy protection federated distillation and backdoor defense method for large model fine tuning, and belongs to the technical field of artificial intelligence security and federated learning, and the method comprises the steps: 1, carrying out the distillation and core representation extraction of a data set based on contribution degree weighted federated pre-training and local neural feature function matching; step 2, self-adaptive noise back door defense processing based on multi-feature fusion; according to the method, a dataset distillation mechanism based on neural feature function matching and a self-adaptive noise defense strategy are adopted, so that effective balance of the large model among data simplification, privacy protection and backdoor defense robustness is realized; the method is of great significance in improving the safety and reliability of an artificial intelligence system in a distributed environment.

The invention relates to the field of software security, and particularly discloses a software supply chain security analysis method which comprises the following steps: S1, collecting data of components of a software supply chain, and calculating a historical security risk weight for each component in the supply chain; s2, collecting behavior data of the software during operation, and performing association analysis in combination with the supply chain data; s3, preprocessing the collected behavior data and supply chain data, and extracting key features; according to the software supply chain security analysis method, through dynamic behavior analysis, runtime threats, such as zero-day vulnerabilities and hidden backdoor threats, which cannot be detected by static analysis can be found, the coverage rate of threat detection is remarkably improved, meanwhile, historical security risk weights are introduced, historical security problems of supply chain components are quantified, and the security of the software supply chain is improved. And intelligent risk assessment and threat traceability are realized by combining a knowledge graph and a graph neural network technology.

The invention provides a federated learning backdoor defense method based on singular value decomposition and model weight amplification, and the method comprises the steps: carrying out the normalization processing of a model updating parameter locally trained by a client, and obtaining a normalized model updating parameter; on the basis of the normalized model updating parameters, model updating parameters after dimension reduction are obtained; performing clustering algorithm processing on the model updating parameters after dimension reduction to obtain clustered clusters; obtaining cluster model parameters based on the clustered clusters; combining the cluster model parameters into a cluster model parameter matrix; performing singular value decomposition on the cluster model parameter matrix to obtain a singular vector; obtaining a trust score through the singular vector; obtaining global model update parameters based on the trust score; calculating by utilizing the global model updating parameters to obtain a global model; and adding Gaussian noise to the global model through a differential privacy mechanism to obtain a final global model. According to the method, backdoor attacks can still be effectively resisted under the scene that the client data sets are non-independent and identically distributed.

The invention relates to the field of personal intelligent security evaluation, and particularly discloses a chain reasoning hidden backdoor vulnerability detection method of a vision-language-action model, which comprises the following steps of: respectively injecting micro pixel disturbance and rare character marks into vision and language input; on the basis of model autoregression prediction characteristics, designing a hybrid reasoning chain fusing normal reasoning steps and abnormal backdoor branches; adopting prefix tuning to take the hybrid reasoning sequence as a pluggable prefix injection model; and generating the vulnerability sensitivity of the abnormal action instruction through the systematic verification process detection model. Compared with an existing method, the method has the advantages that a nondestructive testing mechanism based on prefix adjustment and optimization does not need to modify model parameters or depend on training data, and the safety and reproducibility of detection are guaranteed; a multi-mode triggering mechanism is constructed, and the hidden vulnerability of the model in a complex scene is effectively revealed; the abnormal branches and the normal process are fused in a chain mode, and the defense capability of the model for the concealment logic offset can be systematically evaluated.

The invention provides a parameter negotiation method for a high-speed serial interface, which is applied to an adaptation layer of any chip in a chip pair for data interaction based on UCIE, and comprises the following steps: requesting a home terminal physical layer to enter an active state under the condition that the home terminal physical layer completes link training; under the condition that a home terminal physical layer enters an active state and a backdoor selection bit is not set, determining a home terminal performance index based on a link training result and home terminal performance parameters pre-configured by a user; determining a target protocol stack to be subjected to parameter negotiation and a to-be-negotiated protocol type of the target protocol stack based on the home terminal performance index, and performing parameter negotiation handshake with the opposite terminal based on the to-be-negotiated protocol type of the target protocol stack; under the condition that the parameter negotiation is successful, the target protocol and the target data format are determined based on the result of the parameter negotiation handshake, and the indication information of the target protocol and the target data format is sent to a home terminal protocol layer, so that the parameter negotiation efficiency of the high-speed serial interface can be improved.

Systems, computer program products, and methods are described herein for detecting manipulation of training data used for machine learning models. The present disclosure is configured to receive an interaction originating from an end-point device; validate the interaction via a primary model; identify a set of triggers within a backdoor model, where the backdoor model is modeled off the primary model capable of undergoing stress testing associated with a set of triggers; pause the interaction upon identification of the set of triggers within the backdoor model; transmit the identified set of triggers to an autonomous artificial general intelligence (AGI) agent, generate a set of synthetic data via the autonomous AGI agent, where the set of synthetic data removes the set of triggers from the primary model; and distribute the set of synthetic data to the primary model to correct the set of triggers within the primary model.

The invention discloses a frequency domain stealth backdoor attack method based on a generative adversarial network, and belongs to the technical field of image data processing, and the method comprises the steps: obtaining a clean sample set D1 and a classification model; constructing a generative adversarial network, and constructing a loss function based on picture similarity loss, frequency domain consistency loss and adversarial loss; training the generative adversarial network by using the D1 to obtain a generative model; constructing a clean data set and a backdoor data set based on the D1, a target label t of the backdoor attack and the generative model; constructing multi-layer MMD loss; and constructing total loss based on MMD and MSE, and performing poison training on the classification model to obtain a backdoor model. According to the method, a multi-domain disturbance generation network is constructed, pixel-level disturbance is generated in a spatial domain, discrete wavelet transform is introduced into a frequency domain to constrain frequency domain characteristics of back door disturbance, poisoning samples have concealment in the spatial domain and the frequency domain, multi-layer MMD loss is introduced, the deep characteristic distribution difference between clean samples and back door samples is reduced, and high-concealment attack is achieved.

The invention discloses a federated learning backdoor attack defense method based on layer perception detection, which relates to the technical field of artificial intelligence security, and comprises the steps of client training and uploading, malicious client identification, backdoor key layer detection and deletion, robust aggregation updating and adversarial training for enhancing robustness. According to the method, the model parameters uploaded by the clients are subjected to clustering analysis, and the maximum benign cluster is identified by adopting an unsupervised clustering method, so that instability caused by single threshold judgment is avoided, and the benign client and the malicious client can be distinguished; and meanwhile, layer perception detection and elimination are used in the scheme, so that the backdoor introduced by a malicious client can be effectively identified and eliminated on the premise of ensuring the global model precision, and the influence of backdoor attack on the model is inhibited.

The invention provides a federal learning backdoor attack defense method based on adaptive clustering, and aims to solve the problems that the defense capability is insufficient and backdoor attacks with the malicious client proportion exceeding 50% cannot be effectively defended under the condition of data non-independent identical distribution. An existing federal learning backdoor attack defense method based on clustering generally adopts a diclustering strategy, and has the problems that the malicious client identification accuracy is not high, and defense fails to the malicious client in a proportion of more than 50%. In order to solve the problems, a Louvain community discovery algorithm is applied to perform adaptive clustering on model parameters, and a historical cumulative credibility updating mechanism and an abnormal value cutting strategy are introduced, so that malicious clients are accurately identified and effective defense is realized on the premise of ensuring the main task precision of a global model.

The present invention discloses a deep learning backdoor attack method and device based on an ordinal network and a medium, which belongs to the technical field of neural network security. The method comprises: obtaining a training sample image; generating an ordinal network based on the training sample image, wherein the ordinal network is used for representing an exact structure of an image; and generating a trigger by using the ordinal network. Through the above method, and the device and the medium for realizing the above method, the present invention uses the ordinal network to generate the trigger, improves the concealability of a poisoning sample compared with poisoning samples generated by other attack methods, and can promote the further research of a hidden backdoor attack defense method in the academic circle.

The invention discloses a federal self-supervised learning backdoor attack method and system based on illusion positive example entanglement, and the method comprises the steps: setting a target of an attacker to embed a backdoor into an image encoder during a pre-training period, and guaranteeing that a downstream classifier trained on the image encoder predicts an image with a trigger as a target class; the limited poisoning data is fully utilized, and the feature similarity of the same trigger sample is improved through the illusion enhancement module, so that the malicious client can effectively embed the backdoor into the global encoder. In the training process, a forged positive sample is generated, learning of the model in a feature space is misguided intentionally, poisoning data is utilized to the maximum extent, an entanglement loss function is designed to optimize a target, the attack efficiency is kept in the whole model aggregation process, and the attenuation risk is reduced. According to the invention, a new perspective and important challenge can be provided for the security research of the federated learning system, and the improvement and development of defense capability are promoted.

The invention belongs to the field of agent security, and discloses a multi-agent application program attack assessment method and a related device, and the method comprises the steps: firstly, constructing a poisoning attack prompt word and a backdoor trigger through obtaining an attack motivator and an attack constraint, and combining to generate a backdoor attack sample; then identifying a potential easily-poisoned agent in the multi-agent application program, and inserting a backdoor attack sample into a specified site to generate a poisoned agent; carrying out multiple rounds of agent communication in the multi-agent application program based on the poisoning agent, and simulating an interaction process in a real attack scene to obtain a poisoning multi-agent application program; and finally, a statement with a backdoor trigger is input, and an attack evaluation report is generated according to an output result of the poisoning multi-agent application program, so that the comprehensiveness of poisoning and backdoor attack robustness evaluation of the multi-agent application program is improved, and a powerful basis is provided for optimization of the multi-agent application program. And the agents with excellent performance in the multi-agent application program can be mined.

The invention belongs to the technical field of artificial intelligence security, and relates to a defense method for model backdoor attacks. The invention aims to solve the problem that the accuracy of the model is remarkably reduced due to the existing defense technology. The method combines a backdoor label detection method and a model fine tuning method, and is realized by the following steps of: firstly, detecting a backdoor label by utilizing a nerve cleaning method, and identifying a potential backdoor trigger; secondly, generating an adversarial sample by using a projection gradient descent algorithm; then, carrying out first-stage adversarial fine tuning training to reduce the sensitivity of the model to a backdoor trigger; and finally, performing second-stage fine tuning training through the clean sample, and gradually recovering the classification performance of the model on the original data.

The invention discloses a rumor detector-oriented backdoor attack generation method, system and device, and relates to the technical field of rumor detection and backdoor attack.The method comprises the steps of obtaining a rumor propagation graph; respectively generating importance scores of node structures and features of the propagation graph by adopting degree centrality and similarity between nodes and root nodes, and screening out attached nodes according to the importance scores; constructing a self-adaptive feature generator, and generating an aggressive self-adaptive trigger node by taking the node feature of the attached node as input; meanwhile, similarity constraints are added to the node features, so that the trigger node features can be adaptively close to the features of the attached nodes; a self-adaptive feature generator is used for self-adaptively generating a corresponding node feature for each trigger node, the trigger nodes with the node features in a certain proportion are injected into the attached nodes, and a back door graph is generated and used for attacking the rumor detector; according to the method, the performance of a propagation-based rumor detector is effectively destroyed, and the method can be transferred among different architectures.

The invention provides a federal learning backdoor attack defense method based on a multi-layer cooperative defense strategy, and belongs to the technical field of network data security. Aiming at the defects in the prior art that only single-point protection at a certain stage is covered in federated learning, systematic robust defense for a whole process is lacked, backdoor attack detection capability is weak, model recovery efficiency is low and the like, the invention provides a multi-layer defense framework penetrating through a whole period (training, aggregation and deployment) of federated learning. A special information flow channel, a global security evaluation system and a self-adaptive resource allocation strategy are established among the layers, and the three defense layers are organically combined into a cooperative combat whole. The cooperation mechanism not only improves the efficiency of a single defense layer, but also realizes' 1 + 1 + 1gt '; and 3 ''through the synergistic effect, an omnibearing and intelligent backdoor attack defense solution is provided for a federated learning system.

Various embodiments provide a robust backdoor attack on machine learning (ML)-based detection systems that can be applied to demonstrate and identify vulnerabilities thereof. In various embodiments, an artificial intelligence (AI)-based Trojan attack is generated and implanted inside a ML model trained for classification and / or detection tasks, and the AI-based Trojan attack can be triggered by specific inputs to manipulate the expected outputs of the ML model. Analysis of the behavior of an ML model having the AI-based Trojan implanted (and / or triggered) then enables identification of vulnerabilities of the ML model and further enables the design of ML models with improved security. Various embodiments of the present disclosure provide a fast and cost-effective solution in achieving 100% attack success rate that significantly outperforms adversarial attacks on ML models, thereby improving applicability and depth in testing ML-based detection systems.

The invention relates to the technical field of network security, in particular to a defense method and system for federated learning backdoor attacks, a server initializes a federated learning global model, and identifies at least one key layer easy to implant a backdoor in the global model; distributing the current global model parameters to a plurality of clients, receiving model update information uploaded by each client, extracting a multi-dimensional gradient feature vector of each client based on gradient information of each client in a key layer, calculating a mahalanobis distance anomaly score of each client relative to the gradient distribution of the whole client, and calculating the mahalanobis distance anomaly score of each client according to the mahalanobis distance anomaly score; and screening and rejecting the clients which are judged to be abnormal, aggregating model updates uploaded by the clients which are judged to be benign, and generating a new global model. Malicious updating is effectively isolated while the performance of the main task is ensured, and the robustness and safety of a global model are improved.

The invention discloses a text style backdoor defense method based on multi-granularity variant generation and style immunization. The method comprises the following steps: capturing a text style and content by combining explicit and implicit features; generating high-quality text variants on a plurality of granularities such as lexical, syntactic, style, context and the like by utilizing a large language model; carrying out label correction on the suspicious samples based on multi-dimensional risk assessment, and carrying out voting decision making by utilizing style neutralization variants and the like; and finally, through style invariant representation learning, style separation and style contrast training, the robustness of the model to style change is improved. According to the method, an explicit detection trigger is not needed, various attacks including style backdoors can be effectively defended, and through a systematic framework and a self-adaptive strategy, the safety and robustness of the model are remarkably improved while the normal performance of the model is ensured.

The invention discloses a backdoor attack detection method, device and equipment, a medium and a program product. The method comprises the following steps: acquiring a benign sample set and input data; transplanting the local features of the input data to each benign sample in the benign sample set to obtain a disturbance sample set; determining a prediction category of the disturbance sample set and a prediction category of the benign sample set based on the target classification model; and according to the prediction category of the benign sample set and the prediction category of the disturbance sample set, determining whether the input data is backdoor data. Local features of input data are transplanted to a benign sample to obtain a disturbance sample set by utilizing the strong correlation characteristic of model implanted backdoor attack and disturbance styles, backdoor attack detection is performed on the consistency of prediction categories of the benign sample set and the disturbance sample set based on a target classification model, the universality of a backdoor attack mode is high, and the accuracy of backdoor attack detection is high. The method is not constrained by an attack mode, and has relatively high detection accuracy for back door attacks irrelevant to input and back door attacks relevant to input.

The invention discloses a graph backdoor encoder defense method and device, and a readable storage medium. The method comprises the steps that at least two label-free enhanced data sets are generated based on a preset training data set, a to-be-processed encoder is trained based on the enhanced data sets to obtain a teacher encoder, and the training data set is a downstream data set or a subset of the downstream data set; defining identical attention operators in corresponding layers of the teacher encoder and the to-be-processed encoder, wherein the attention operators are used for outputting layer attention maps of corresponding encoder layers; calculating the distribution offset of the layer attention map of the teacher encoder and the layer attention map of the encoder to be processed in the same encoder layer; and fixing the parameters of the teacher encoder, and updating the parameters of the to-be-processed encoder based on the distribution offset. Compared with the prior art, through lightweight deployment, on the premise that the model precision is not reduced, the attack success rate of backdoor attack on graph self-supervised learning is effectively reduced, and the backdoor defense capability of the system is improved.

The invention provides a 3D point cloud defense method, device and equipment based on CLIP guidance, and a medium, relates to the technical field of point cloud defense, and aims to enhance the robustness and security of 3D point cloud data in the face of adversarial attack by using a contrast language-image pre-training model. According to the method, a new defense framework is provided by combining geometric and semantic features, triggers in backdoor attacks can be effectively identified and inhibited, meanwhile, the geometric structure of point cloud data is repaired, and the classification performance is improved. Specifically, the method includes projecting a 3D point cloud into a multi-view 2D depth map, extracting cross-modal features using a CLIP model, and performing semantically guided 3D reconstruction through a variational auto-encoder (VAE) to generate a repaired point cloud that retains key information and neutralizes perturbations. The method shows excellent performance in various complex scenes, and provides powerful support for the safety of the 3D vision technology in key application.

The invention discloses a self-learning defense method, device and equipment for backdoor attack of an artificial intelligence model of a power system, a storage medium and a program product, and the method comprises the steps: screening information in current power data through a prefilter, and determining whether abnormal information is screened out or not; when it is determined that the abnormal information is screened out, storing the current power data, and inputting the current power data into a data detection model to obtain a prediction category for the current power data; determining a tag for the current power data based on the prediction category; when the label is abnormal power data, training and updating the integrated learning model based on the abnormal power data; the updated integrated learning model is used for detecting power data input next time. Therefore, the safety of the model can be improved under the condition that the precision of the intelligent power grid artificial intelligence model is ensured.

The invention discloses a Non-IID federated learning backdoor attack defense method and system and a medium, and relates to the field of federated learning and network security. In order to solve the problems that an existing method depends on model parameters and is easy to avoid by a malicious client, and adaptive Non-IID scenes are poor, truncated singular value decomposition is executed through the client to extract first p left singular vectors, and the first p left singular vectors are uploaded to a server; the server constructs a matrix based on left singular vector cosine similarity and performs hierarchical clustering, and calculates a client similarity score and an aggregation weight by using a zoom dot product attention mechanism in combination with a left singular vector of the clean reference data set; the server distributes a global model, the client uploads parameters after local training, and the server weights and aggregates the model parameters in the cluster according to the weight and iteratively optimizes the model parameters. According to the method, the malicious client is identified from the data essential features, the malicious proportion does not need to be preset, the good client contribution and the data privacy are guaranteed while the backdoor attack is inhibited, the method is suitable for a Non-IID scene, and the model robustness and the main task performance are improved.

The invention provides a watermark processing method and device, watermark processing equipment, a program product and a medium, and relates to the technical field of privacy computing. The method comprises the following steps: acquiring first data which is requested to be input into a target model by a user through an application programming interface (API); aiming at the first data, judging whether watermarking processing needs to be carried out or not; when it is determined that watermark processing needs to be carried out, based on a backdoor function corresponding to the target model, an API response corresponding to the API request is generated according to the first data, and the backdoor function is constructed according to a decision function of the target model; the API response is output to the user, the first data is added to a trigger set corresponding to the target model, and the trigger set is used for verifying an alternative model of the target model. According to the scheme of the invention, the problem that the existing digital watermarking technology is difficult to effectively defend model extraction attacks initiated through an API (Application Program Interface) is solved.

The invention discloses a data privacy trusted computing method and system, and relates to the technical field of data security and privacy protection.The method comprises the steps that a data privacy trusted computing engine is established in a federal learning mode; performing backdoor defense on the data privacy trusted computing engine by adopting a multi-stage federated learning backdoor defense mode; the multi-stage federated learning backdoor defense mode comprises the steps of screening a model uploaded by a user by adopting a robust aggregation algorithm, detecting whether the model has a backdoor by adopting trigger reverse engineering and a backdoor judging device, and repairing the model with the backdoor by adopting a backdoor forgetting technology; and constructing a federated learning backdoor defense system according to processing results of a robust aggregation algorithm, trigger reverse engineering, a backdoor judger and a backdoor forgetting technology. According to the method, the technical problems of low heterogeneous data processing efficiency and insufficient backdoor defense are effectively solved by constructing a multi-modal federal learning architecture and a multi-level self-adaptive defense system, and reliable technical support is provided for data security sharing in the power industry.

Disclosure are a backdoor attack method and system for a classification task in a code model, the method includes: S1. collecting and preprocessing clean samples to obtain importance variable names; S2. classifying the variable names of the clean samples according to label categories to obtain a plurality of trigger sets; and selecting target labels from the clean samples; S3. performing score calculation on the variable names in the trigger sets corresponding to the target labels; replacing one importance variable name with the variable name having a maximum C score in the clean samples to obtain poisoned samples, and repeating the above process until the labels are changed into the target labels; and S4. randomly inserting the triggers in the poisoned samples into the clean samples to form negative samples; and performing an attack by using an attack model obtained based on the negative, poisoned and clean samples.

The invention provides a defense device and method for a physical backdoor attack of a biological information identification system, and belongs to the field of information security. The biological information identification system is used for collecting and authenticating a multi-mode biological signal. The defense device comprises a coupling path blocking module which is used for cutting off or weakening the conduction of malicious noise in a physical connection path from three aspects of active shielding, passive filtering and voltage stabilization so as to guarantee the stability of a front-end physical environment. And the physical domain signal purification module forms a closed loop from real-time signal monitoring, feature analysis and interference suppression, and reduces noise transmitted from a power supply circuit to the acquisition device. And the numeric field semantic verification module is used for verifying the collected multi-mode biological signal and judging whether an abnormity or a backdoor injection attack sign exists or not. And the system control and management module is used for realizing communication between external equipment and each module and making a decision according to a numeric field semantic verification result. According to the invention, the physical interference signal injected into the back door can be effectively blocked, and the safety and reliability of the biological information identification system are guaranteed.

A black box code search model backdoor attack method based on learnable discrete code transformation comprises the steps that a learnable backdoor generator is constructed, and malicious codes with backdoors are generated on the premise that internal parameters of a damaged model are not accessed through the learnable discrete code transformation. Firstly, an agent model capable of simulating victim model behaviors is trained through query-response data; secondly, on the proxy model, a discrete selection process is differentiable by using a re-parameterization sampling technology, and a backdoor generator is trained in combination with a multi-objective loss function so as to realize effectiveness and concealment of backdoor implantation; according to the method, the limitation of a black box is successfully bypassed by constructing the proxy model, and a new possibility is provided for an attacker. A backdoor generation process is integrated into a differentiable training framework through learnable discrete transformation, so that end-to-end learnability is realized, and an attack strategy can be automatically optimized.